admin/dragonsstash
1
0
Fork 0
mirror of https://github.com/xCyanGrizzly/DragonsStash.git synced 2026-09-21 13:31:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: design database and file backups

Browse Source
This commit is contained in:
admin
2026-07-21 23:23:52 +02:00
parent 3b7202a662
commit 75c67b2036
1 changed files with 120 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/superpowers/specs/2026-07-21-database-and-file-backups-design.md
+120
View File
@@ -0,0 +1,120 @@
# Database and File Backup Design
**Date:** 2026-07-21
**Status:** Design approved for written-spec review
**Scope:** Disaster recovery for the Docker Compose deployment
## Problem
Dragon's Stash currently persists PostgreSQL in a Docker named volume and stores uploaded STL files in the `manual_uploads` volume. A Docker volume protects data from container recreation, but it is not an off-host backup. A host disk failure, accidental deletion, corruption, or ransomware event could destroy both the database and the files it references.
The Telegram worker and bot also persist authentication/session state in `tdlib_state` and `tdlib_bot_state`. These files are sensitive and losing them requires Telegram re-authentication.
## Goals
- Protect PostgreSQL data and uploaded STL files against loss of the application host.
- Store backups on a Synology NAS over an authenticated, host-restricted NFS share.
- Create one recoverable snapshot containing related database and file state.
- Retain 30 daily recovery points.
- Include Telegram worker and bot session state so a host restore does not require re-authentication.
- Provide a documented, repeatable restore process.
- Detect failed or corrupt backups instead of silently pruning the last good copy.
## Non-goals
- Building an in-app backup-management UI.
- Backing up temporary ZIP processing data in `tmp_zips`.
- Copying the raw `postgres_data` volume as the primary database backup.
- Providing protection against loss of the NAS itself. A later Synology Hyper Backup task can replicate this repository to another device or cloud destination.
## Selected approach
Use a Linux-host backup script, scheduled by a systemd timer, with Restic writing to an encrypted repository on a Synology NFS share.
This approach keeps the backup and restore workflow explicit, deduplicates large STL files across daily snapshots, and avoids tying recovery to PostgreSQL's internal data-directory layout. Restic's repository encryption also protects database contents, uploaded files, and Telegram session state if the NAS share is accessed directly.
## Storage layout
### Synology
Create a dedicated shared folder, for example `dragonsstash-backups`, with:
- NFS enabled only for the Docker host's fixed IP address.
- A restricted NFS export used only for this backup share, with host access limited to the Docker host's fixed IP address.
- No Internet exposure.
- Sufficient capacity for the repository plus growth and safety margin.
The Linux host mounts the share at a stable path such as `/mnt/dragonsstash-backups` using systemd-aware network mount options so a NAS outage does not block normal boot indefinitely.
### Restic repository
The repository lives below the mounted share. The Restic password is stored separately from the repository in a root-readable host secret file and must also be recorded in the operator's offline password-management system. Losing both the NAS and the only copy of the Restic password makes encrypted backups unrecoverable.
Each successful Restic snapshot contains:
- A PostgreSQL custom-format dump generated for that run.
- The contents of the `manual_uploads` Docker volume.
- The contents of the `tdlib_state` Docker volume.
- The contents of the `tdlib_bot_state` Docker volume.
- A small manifest with the backup timestamp, application image/version, and database migration state.
The `tmp_zips` volume is scratch space and is excluded.
## Backup flow
The systemd timer invokes one backup command at the chosen nightly time. The command:
1. Acquires an exclusive lock and refuses to run if another backup is active.
2. Verifies that the NFS mount is present, writable, and points to the expected backup directory.
3. Stops the `app`, `worker`, and `bot` services while leaving PostgreSQL running.
4. Creates a PostgreSQL custom-format dump from the running database.
5. Creates a manifest for the backup.
6. Runs one Restic backup over the dump and the read-only mounted Docker volumes.
7. Verifies that Restic created the snapshot successfully.
8. Applies the retention policy: keep the latest 30 daily snapshots, then prune unreferenced data.
9. Restarts all stopped services, whether the backup succeeded or failed.
The service-stop window ensures that application writes and TDLib session updates do not occur while the corresponding file volumes are being captured. A failed run must never trigger retention pruning.
## Restore flow
The restore tooling and documentation will support this sequence:
1. Stop `app`, `worker`, and `bot`.
2. Select and inspect a Restic snapshot.
3. Restore the PostgreSQL dump and volume contents to a staging location.
4. Preserve the current database and volumes or confirm that the operator intends to replace them.
5. Restore `manual_uploads`, `tdlib_state`, and `tdlib_bot_state` into their Docker volumes with the expected ownership and paths.
6. Restore the database from the custom-format dump into the configured PostgreSQL database.
7. Verify that `ManualUploadFile.filePath` values resolve to restored files and that the application health endpoint can connect to PostgreSQL.
8. Start the services and inspect logs for startup, migration, and worker/bot authentication errors.
The restore process must be safe to rehearse against a disposable Compose project without modifying the live deployment.
## Failure handling and verification
- A missing or read-only NAS mount fails the backup before services are stopped where possible.
- A lock prevents overlapping backups.
- A cleanup trap or equivalent guarantees service restart after errors.
- Backup failure produces a non-zero systemd result and a clear log entry.
- Retention pruning runs only after a verified successful snapshot.
- A snapshot listing and repository metadata check run after each backup.
- A full Restic integrity check and disposable restore test run monthly.
- The project documentation describes how to inspect the last successful snapshot and how to recover when the NAS is unavailable.
## Security considerations
- The NFS share is limited to the Docker host and is not exposed to the Internet.
- Restic encryption protects the repository at rest.
- PostgreSQL credentials, NAS credentials/rules, and the Restic password are never committed to the repository.
- Restore commands must avoid printing database passwords or the Restic password in logs.
- Telegram session volumes are included because they are operationally valuable, but they must be treated as secrets.
## Acceptance criteria
- A nightly systemd timer creates a Restic snapshot on the Synology share.
- A snapshot includes PostgreSQL, all uploaded STL files, and both Telegram session volumes.
- At least 30 daily recovery points are retained without duplicating unchanged STL content unnecessarily.
- A simulated host-loss restore reconstructs the database and uploaded files in a disposable Compose environment.
- A failed backup leaves services running and preserves the last known-good snapshot.
- The restore procedure is documented well enough for an operator to execute without reading the implementation.