Skip to main content

Backup and restore

A Fovea install has three pieces of persistent state. Each has a different cost-of-loss and a different backup cadence.

What to back up​

DataLives inCost of loss
Application databasePostgresAll annotation, persona, ontology, claim, and user state
Uploaded videosSTORAGE_PATH volumeVideos themselves; annotations on those videos still resolve
Model weightsHF_HOME volumeOne re-download per model at next use
Redis (queue state)Redis volumeAny currently-running job; user retries from the UI

Postgres is the only piece that justifies a full point-in-time backup story. The STORAGE_PATH volume is large but content- addressable; rsync to an object store on a slow cadence is enough. The model cache and Redis are recoverable from the network and from the user, respectively, and rarely need backing up at all.

Postgres backup​

The simplest correct backup is a nightly pg_dump against the running database, piped to compressed storage:

docker compose exec -T postgres pg_dump -U fovea fovea \
| gzip > backups/fovea-$(date +%Y-%m-%d).sql.gz

For point-in-time recovery, configure Postgres with WAL archiving and run a continuous-archiving tool like pgbackrest or wal-g against an object store. The schema is small enough that even hourly full dumps are cheap; per-row WAL is overkill for most installs.

Retention: keep at least 14 daily and 8 weekly dumps. Annotation data is the kind of thing operators sometimes notice was wrong a week ago.

Postgres restore​

Restore is the inverse of the dump. With the stack down:

docker compose up -d postgres
docker compose exec -T postgres dropdb -U fovea --if-exists fovea
docker compose exec -T postgres createdb -U fovea fovea
gunzip -c backups/fovea-2026-06-01.sql.gz \
| docker compose exec -T postgres psql -U fovea fovea
docker compose up -d

After a restore from a backup taken on an older release, run docker compose run --rm backend npx prisma migrate deploy to catch the database up to the migrations bundled in the running image. Prisma's migrate-deploy is forward-only and idempotent.

Video volume backup​

The STORAGE_PATH volume is a flat directory of opaque blob files. Snapshot-based filesystems (ZFS, Btrfs, EBS snapshots) are the right tool; for filesystems without snapshots, rsync to an object store works because uploaded videos are immutable once written:

rsync -av --delete \
/var/lib/fovea/videos/ \
s3://my-bucket/fovea/videos/

The same approach also covers thumbnails and any cached keyframe extractions the model service writes back to the volume.

Disaster recovery drill​

The dump file plus the video volume plus the env file is everything you need to rebuild the install. Run a restore drill quarterly:

  1. Stand up an empty stack on a separate host.
  2. Restore the most recent Postgres dump.
  3. Sync the video volume.
  4. Copy the production .env over with SESSION_SECRET and DATABASE_URL adjusted for the new host.
  5. Confirm a user can log in and load a previously-annotated video, that the annotations render, and that a re-run of summarization produces output.

The drill is the only way to catch backup gaps before you need them. Operators have hit at least two: forgetting to back up the STORAGE_PATH volume because Postgres "is the database", and backing up the volume but not the env file (which means no session secret to verify the existing sessions).