A Datasette <-> Litestream plugin.

The plugin requires a recent alpha version of Datasette 1.0, whcih can be installed with:

pip install datasette==1.0a6

Then install this plugin in the same environment as Datasette.

datasette install datasette-litestream

To replicate my_database.db to S3, use the following configuration in your metadata.yaml file:

databases:
  my_database:
    plugins:
      datasette-litestream:
        replicas:
          - url: s3:https://my-bucket/my_database

Then make sure you export LITESTREAM_ACCESS_KEY_ID and LITESTREAM_SECRET_ACCESS_KEY with your S3 credentials (or AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY), then run with:

datasette my_database.db -m metadata.yaml

If you have multiple attached databases in Datasette and want to replicate all of them, the top-level all-replicate key can be used.

plugins:
  datasette-litestream:
    all-replicate:
      - s3:https://my-bucket/$DB_NAME

When all-replicate is used, a new replica URL is generated for each attached database. In this case, if you had a database named parking_tickets and another named city_budget, then datasette-litestream will replicate them to s3:https://my-bucket/parking_tickets and s3:https://my-bucket/city_budget.

This is done with "variables" that datasette-litestream replaces in the all-replicate URL. The supported variables are:

• $DB_NAME: The name of the Datasette database to replicate.
• $DB_DIRECTORY: The full parent directory that the SQLite database resides.
• $PWD: The current working directory of the Datasette process.

Some configuration in the metadata.yaml will be used to auto-generate the litestream.yml file under the hood. You can use this to customize the Litestream replication process.

The following are valid keys that are allowed when specifying top-level plugin configuration:

• all-replicate: A template replica URL used to replicate all attached Datasette databases, see aboce for details.
• metrics-addr: Defines the addr: Litestream option, which will expose a Prometheus endpoint at the given URL. Use which caution on public Datasette instances! When defined, the metrics info will appear on the datasette-litestream status page.
• access-key-id: An alternate way to provide a S3 access key (though the LITESTREAM_ACCESS_KEY_ID environment variable is preferred).
• secret-access-key: An alternate way to provide a S3 secret key (though the LITESTREAM_SECRET_ACCESS_KEY environment variable is preferred).

None of these keys are required.

Example:

plugins:
  datasette-litestream:
    all-replicate:
      - XXX
      - YYY
    metrics-addr: :5001
    access-key-id: $YOUR_KEY
    secret-access-key: $YOUR_SECRET

The following options are allowed on database-level plugin configuration.

• replicas
• monitor-interval
• checkpoint-interval
• min-checkpoint-page-count
• max-checkpoint-page-count

databases:
  my_database:
    plugins:
      datasette-litestream:
        replicas:
          - s3:https://...
          - file:https://...
        monitor-interval: XXX
        checkpoint-interval: XXX
        min-checkpoint-page-count: XXX
        max-checkpoint-page-count: XXX

See Litestream Database settings for more information.

To set up this plugin locally, first checkout the code. Then create a new virtual environment:

cd datasette-litestream
python3 -m venv venv
source venv/bin/activate

Now install the dependencies and test dependencies:

pip install -e '.[test]'

To run the tests:

pytest