Backups & restore

Back up and restore a self-hosted Future AGI instance: Postgres, ClickHouse, object storage and the secrets in .env, for Standalone and Distributed.

A self-hosted instance keeps its state in three stores and one file: Postgres for application data, ClickHouse for the observability records (spans and traces), object storage for uploads, exports and media, and .env for the secrets. Back up all four. Redis holds caches, locks and live updates, so there is nothing in it to back up.

Where object storage lives depends on the setup. In Standalone it runs inside the app container and keeps its files in the app-data volume; in Distributed it is the minio container. On Kubernetes, see Back up and restore for the Helm chart.

If you’ve already moved to managed data stores, your provider’s backup tooling replaces the steps below for those stores.

With the production overlay, add its --env-file and -f flags to every docker compose command on this page, as in Start Distributed with the production overlay. A plain docker compose stop there stops only the services of the Standalone file.

The named volumes

The Compose project name is futureagi unless COMPOSE_PROJECT_NAME in .env says otherwise, and every volume is prefixed with it. docker volume ls | grep futureagi lists them.

VolumeHoldsBack up?
futureagi_postgres-dataPostgres application dataYes
futureagi_clickhouse-dataClickHouse spans and tracesYes
futureagi_app-dataObject storage, Temporal’s SQLite database, and the collector’s spool and dead-letter file (/data/collector/dead_letter.jsonl)Yes
VolumeHoldsBack up?
futureagi_postgres-dataPostgres application dataYes
futureagi_clickhouse-dataClickHouse spans and tracesYes
futureagi_minio-dataMinIO objectsYes
futureagi_fi-collector-dataThe collector’s dead-letter file (/var/lib/fi-collector/dead_letter.jsonl) and its observation spoolDon’t delete it
futureagi_property-catalog-kafka-dataKafka topics of the observed-attribute catalogDon’t delete it
futureagi_redis-dataRedis cache, locks and live updatesNo
futureagi_peerdb-catalog-dataPeerDB replication catalog: its peers and mirrorsDon’t delete it: peerdb-init refuses to create a mirror again into ClickHouse tables that already hold data
futureagi_peerdb-minio-dataPeerDB staging objectsNo, transient

Note

The collector’s dead-letter file is not a cache. It holds spans that failed to reach ClickHouse, usually while ClickHouse was briefly down, kept so they can be replayed once it is back. Deleting it drops that telemetry for good, so don’t prune it with the rebuildable volumes.

An upgraded install made before Standalone became the default may also still have futureagi_rabbitmq-data and, if it once ran the Kafka catalog sequencer, futureagi_property-catalog-sequencer-data. Nothing mounts them any more: see Upgrading older installs for when to delete them.

Postgres

Postgres holds the application data, so back it up on a schedule. Use the custom format, and pass -T so docker compose exec doesn’t allocate a TTY and mangle the binary dump. The commands assume the default PG_USER and PG_DB, both futureagi.

docker compose exec -T postgres \
  pg_dump -U futureagi -d futureagi --format=custom \
  > backup-$(date +%F).dump

On Distributed, Temporal keeps its workflow state in the same futureagi database, so the dump carries it, and a restore rewinds it with the rest. Restore has the restore steps.

ClickHouse

ClickHouse holds data that exists nowhere else. The collector writes spans straight into it, so they are not recoverable from Postgres. Back it up on its own schedule with ClickHouse’s BACKUP command to S3, rather than copying files. Include property_catalog (PROPERTY_CATALOG_DATABASE), the separate database the attribute suggestions in trace filters are read from, next to default. ClickHouse refuses to write a backup to a path that already holds one, so give each run its own path:

docker compose exec -T clickhouse clickhouse-client --query \
  "BACKUP DATABASE default, DATABASE property_catalog TO S3('https://your-bucket.s3.amazonaws.com/ch-backup/$(date +%F)/', 'YOUR_ACCESS_KEY', 'YOUR_SECRET_KEY')"

If BACKUP reports that property_catalog does not exist, as on some installs made before Standalone became the default, leave it out of the command.

clickhouse-client inside the container signs in with the CH_PASSWORD that Compose hands the container, so the commands work whether or not your install has a ClickHouse password. The options of BACKUP and RESTORE are in ClickHouse’s Backup and Restore docs.

Warning

On Distributed, don’t rely on PeerDB init to rebuild ClickHouse. It restores the tables it mirrors from Postgres, not the spans the collector writes directly. ClickHouse needs a real backup on its own schedule.

Object storage

The stored files live in the futureagi_app-data volume, next to Temporal’s SQLite database and the collector’s dead-letter file. Back that volume up alongside Postgres. Stop the app container first so the copy is consistent:

docker compose stop app
docker run --rm -v futureagi_app-data:/data -v "$PWD":/backup alpine \
  tar czf /backup/app-data-$(date +%F).tgz -C /data .
docker compose up -d

The volume name starts with your Compose project name, futureagi unless COMPOSE_PROJECT_NAME says otherwise.

Mirror the MinIO buckets to any S3 endpoint with the MinIO client. MinIO’s API listens on 127.0.0.1:9005 on the Docker host; the user is MINIO_ROOT_USER (futureagi by default) and the password MINIO_ROOT_PASSWORD from .env:

mc alias set local http://localhost:9005 futureagi <MINIO_ROOT_PASSWORD>
mc alias set s3 https://s3.amazonaws.com <AWS_ACCESS_KEY> <AWS_SECRET_KEY>

mc mirror local/ s3/your-bucket/     # back up

Secrets

Back up .env (with the production overlay, deploy/.env.production and the root .env) with the data, and store the copy as carefully as the data: it holds the generated secrets. A restored database needs the same INTEGRATION_ENCRYPTION_KEY: integration credentials saved with one key cannot be read with another, and those integrations must be connected again. ./bin/uninstall --wipe-data deletes the volumes but keeps .env, so a reinstall reuses its secrets.

Restore

Restore Postgres, then ClickHouse from a backup taken at or after the same point, then object storage, with nothing else running. Restore the secrets file first if you lost it: the data needs the credentials it was created with.

Stop everything except the data stores

docker compose stop                          # every service
docker compose up -d postgres clickhouse     # only the data stores; Distributed: add minio

On Distributed, the collector writes to Postgres and Temporal keeps its tables there, so stopping only the backend and the workers is not enough: --clean would drop tables they hold open.

Restore Postgres

docker compose exec -T postgres \
  pg_restore -U futureagi -d futureagi --clean --if-exists \
  < backup-2026-04-22.dump

Restore ClickHouse into an empty store

RESTORE refuses tables that already hold rows, so restore into an empty ClickHouse. Recreating its volume deletes the ClickHouse data the install has now:

docker compose rm -sf clickhouse
docker volume rm futureagi_clickhouse-data
docker compose up -d clickhouse

Once docker compose ps clickhouse shows healthy, restore the backup from its dated path. Leave DATABASE property_catalog out if the backup does not hold it:

docker compose exec -T clickhouse clickhouse-client --query \
  "RESTORE DATABASE default, DATABASE property_catalog FROM S3('https://your-bucket.s3.amazonaws.com/ch-backup/2026-04-22/', 'YOUR_ACCESS_KEY', 'YOUR_SECRET_KEY')"

Restore object storage

Standalone, into the stopped app container’s volume:

docker run --rm -v futureagi_app-data:/data -v "$PWD":/backup alpine \
  sh -c 'find /data -mindepth 1 -delete && tar xzf /backup/app-data-2026-04-22.tgz -C /data'

Distributed, with the mc aliases from Object storage:

mc mirror s3/your-bucket/ local/

Start the stack

docker compose up -d

On Distributed, then check the PeerDB mirrors as PeerDB replication describes. With the production overlay, the start-up jobs only check, so before this command run step 5 of the first-install initialization in deploy/README.md again: it re-creates the observed-attribute catalog’s two ClickHouse users, which Standalone and Distributed without the overlay re-create on every start.

Dive deeper

Was this page helpful?

Questions & Discussion