Litestream - Tapedrive

Litestream streams a SQLite database’s changes to S3 storage and restores the database from there. It works against a Tapedrive bucket with the replica URL, the endpoint and your keys.

Before you start

You need a bucket and a credential. A bucket is a tape that has delegated writes to the gateway, and a credential is an access key pair the gateway operator issues you. Set up a bucket walks through both. Litestream writes continuously. Run it against your own gateway (gateway setup) rather than a public one.

Configure

litestream.yml

dbs:
  - path: /data/app.db
    replica:
      url: s3://my-bucket/app.db
      endpoint: https://s3.example.com
      access-key-id: TAPE4Q7ZK2M9XH3PD8AW
      secret-access-key: mZ9vQ3nL8kR2tW6yB4cX7dF1hJ5pS0aG3uE8iO2q

Everyday commands

# replicate continuously
litestream replicate -config litestream.yml

# restore to a new file
litestream restore -config litestream.yml -o restored.db /data/app.db

# confirm the restored database is sound
sqlite3 restored.db 'PRAGMA integrity_check'

# see what Litestream keeps in the bucket
litestream generations -config litestream.yml /data/app.db
litestream snapshots -config litestream.yml /data/app.db

With a writer inserting a row every 50 ms for five minutes, Litestream shipped every segment and the restored database had every row.

Good to know

The gateway acknowledges each upload immediately and writes it to the chain in the background. A 1 KB upload returns in about half a millisecond, so Litestream keeps up with a busy database.
A restore contains exactly what Litestream had shipped when you ran it.
Retention deletes and compaction work as on any S3 store. Each retention delete is a transaction on the chain.

Next

Deletions for what retention deletes cost on chain.
Uploads for how an upload reaches the chain.