# Scope correction implementation report ## Summary Implemented the approved backup scope correction for Dragon's Stash. Backups and restores now cover only: - PostgreSQL logical custom-format dump plus manifest/migration metadata. - `tdlib_state` worker Telegram session volume. - `tdlib_bot_state` bot Telegram session volume. The implementation no longer treats `manual_uploads`, completed local STL binaries, or `tmp_zips` as protected backup data. Future channel forwarding and archive/STL-content integrity auditing remain out of scope. ## Changed files - `docker-compose.yml` - Removed the backup service's read-only `manual_uploads:/data/uploads` mount. - Kept the normal operational app/worker `manual_uploads` mounts. - Kept both TDLib backup mounts. - `scripts/backup/container-entrypoint.sh` - Removed `/data/uploads` as a required mounted directory. - Removed uploads from the manifest `volumePaths`. - Removed the `source:uploads` Restic tag. - Removed uploads from the Restic source list. - Kept database dump, manifest, worker TDLib, and bot TDLib sources/tags. - `scripts/backup/restore.sh` - Removed restored uploads variables. - Removed uploads staging validation. - Removed manual uploads volume discovery, safety archive, replacement, and rollback. - Removed retained/manual upload file-path verification. - Removed temporary database verification that existed only for local upload file references. - Preserved guarded `restore-live` confirmation, backup mount/repository checks, service stop/start handling, safety PostgreSQL dump, TDLib volume safety archives, TDLib volume replacement/rollback, and `pg_restore --list`/`pg_restore --exit-on-error` validation. - `prisma/schema.prisma` - Removed `ManualUploadFile.retainedAt`. - `prisma/migrations/20260722100000_remove_retained_manual_files/migration.sql` - Added forward migration: `ALTER TABLE "manual_upload_files" DROP COLUMN IF EXISTS "retainedAt";` - Preserved the existing committed migration that added `retainedAt`. - `src/app/api/uploads/route.ts` - Removed `retainedAt: new Date()` from manual upload file creation. - `worker/src/manual-upload.ts` - Restored final best-effort cleanup of `/data/uploads/` using the older `path.join("/data/uploads", uploadId)` behavior. - `scripts/backup/README.md` - Rewrote backup set and restore rehearsal docs around PostgreSQL plus both TDLib volumes only. - Removed local STL file, retainedAt, upload path, retained file reference, and restored checksum checks. - Clarified that STL binaries remain in Telegram and recovery preserves database mappings/Telegram IDs. - Kept monthly `restic check --read-data` and disposable restore rehearsal runbook. - Explicitly left future channel forwarding and archive/STL-content integrity auditing out of scope. - `README.md` - Updated the production backup summary to name PostgreSQL logical dump plus Telegram session volumes as the protected set. - Clarified that `manual_uploads` and temporary ZIPs are excluded and STL binaries remain in Telegram. ## Verification - `git diff --check` - Passed. - `bash -n scripts/backup/container-entrypoint.sh scripts/backup/run-backup.sh scripts/backup/restore.sh` - Local `bash` failed because Windows only had the WSL shim and no installed WSL distribution. - Passed via Docker fallback: `docker run --rm --entrypoint bash -v E:\Projects\DragonsStash:/work:ro -w /work postgres:16-alpine -n scripts/backup/container-entrypoint.sh scripts/backup/run-backup.sh scripts/backup/restore.sh` - `npx prisma validate` - Passed. - `npm run build` - Passed. - `cd worker && npm run build` - Passed. - Focused backup/restore scope assertions - Backup shell paths assertion passed: no `manual_uploads`, `/data/uploads`, `retainedAt`, upload source tag, or upload-restore helper references in `scripts/backup/*.sh`. - Compose backup service assertion passed: no `manual_uploads`, `/data/uploads`, `retainedAt`, or `source:uploads` in the `backup` service block. - Active retainedAt assertion passed: no `retainedAt` in active Prisma schema, upload API, worker source, or backup shell scripts. - Active backup/restore upload-source assertion passed: no `manual_uploads`, `/data/uploads`, or `data/uploads` in backup/restore shell scripts. - Remaining expected matches are limited to normal operational app/worker upload mounts and paths, docs stating exclusions, and the historical add/drop migrations. ## Concerns - None for implementation scope. - Environment note: local Bash is unavailable because WSL has no installed distribution; Bash syntax was verified inside Docker instead. --- # Restore Path Scope Review-Finding Fix **Date:** 2026-07-22 ## Fix Addressed the Important restore finding by replacing both unrestricted `restic restore` calls in `scripts/backup/restore.sh` with a shared filtered restore wrapper. The wrapper restores only the backup source paths actually written by `scripts/backup/container-entrypoint.sh`: - `/staging/backup-*/database.dump` - `/staging/backup-*/manifest` and `/staging/backup-*/manifest/**` - `/data/tdlib-worker` and `/data/tdlib-worker/**` - `/data/tdlib-bot` and `/data/tdlib-bot/**` Added an explicit restored-tree guard that refuses unexpected restored content: top-level restored directories other than `staging` and `data`, direct `data/*` entries other than `tdlib-worker` and `tdlib-bot`, and direct `staging/backup-*/*` entries other than `database.dump` and `manifest`. This rejects old/broad snapshots that would otherwise restore `data/uploads`, temporary ZIP or database volume trees, or other unexpected volume content. Preserved the existing guarded live-restore confirmation, staging-directory validation, mount/repository checks, snapshot verification, custom PostgreSQL-dump validation, service stop/start lifecycle, health check, safety database dump, TDLib safety archives, and rollback of exactly the PostgreSQL database plus the two TDLib volumes. Added `scripts/backup/restore-path-assertions.sh`, a focused shell assertion harness that stubs Docker/Restic and verifies both staging and live restore use the expected include filters and that unexpected restored data-volume content is rejected explicitly. Addressed the Minor documentation gap in the root README backup section by stating that forwarding behavior and archive/STL-content integrity auditing are future work outside the backup scope. ## Verification - Red check before implementation: ```text & 'C:\Program Files\Git\bin\bash.exe' -lc 'scripts/backup/restore-path-assertions.sh' ASSERTION FAILED: database dump include filter missing ``` - Bash syntax check: ```text & 'C:\Program Files\Git\bin\bash.exe' -lc 'bash -n scripts/backup/container-entrypoint.sh scripts/backup/run-backup.sh scripts/backup/restore.sh scripts/backup/restore-path-assertions.sh' [passed with no output] ``` - Whitespace check: ```text git diff --check warning: in the working copy of '.superpowers/sdd/scope-correction-implementation-report.md', LF will be replaced by CRLF the next time Git touches it warning: in the working copy of 'README.md', LF will be replaced by CRLF the next time Git touches it warning: in the working copy of 'scripts/backup/restore.sh', LF will be replaced by CRLF the next time Git touches it [exit 0] ``` - Focused restore-path assertions: ```text & 'C:\Program Files\Git\bin\bash.exe' -lc 'scripts/backup/restore-path-assertions.sh' restore-path assertions passed ``` ## Concerns - None for implementation scope. - Git Bash was available and used for shell syntax/assertion checks, so Docker fallback was not needed for the final syntax verification. --- # Important Operational Findings Fix **Date:** 2026-07-22 ## Fix Addressed the two Important operational findings from final review: - `scripts/backup/run-backup.sh` - Backup wrapper restart failures now make an otherwise successful backup exit non-zero. - Existing non-zero backup failures remain preserved if service restart also fails. - Added `scripts/backup/run-backup-assertions.sh` to assert both exit-code cases with a fake Docker/Compose environment. - `scripts/backup/restore.sh` - `restore-live` now captures the managed services that were running before live restore using `docker compose --profile full ps --status running`. - Live restore stops only those previously running managed services. - Successful live restore starts only those previously running services, so the profile-gated optional `bot` is not started if it was not running. - Failure handling leaves services stopped and still rolls back only the PostgreSQL database plus both TDLib volumes. - App health wait now runs only when `app` was previously running. - Extended `scripts/backup/restore-path-assertions.sh` to assert subset stop/start behavior and skipped health checks when `app` was not running. Addressed the Minor staging-path documentation mismatch by aligning `.env.example` with the backup README example: `/var/lib/dragons-stash/backup-staging`. ## Verification - Red checks before implementation: - `docker run --rm -v "${PWD}:/work" -w /work ubuntu:24.04 bash scripts/backup/run-backup-assertions.sh` - Failed as expected: backup reported success when restart failed. - `docker run --rm -v "${PWD}:/work" -w /work ubuntu:24.04 bash scripts/backup/restore-path-assertions.sh` - Failed as expected: restore-live stopped the fixed `app worker bot` service set instead of the running subset. - Focused assertions after implementation: - `docker run --rm -v "${PWD}:/work" -w /work ubuntu:24.04 bash scripts/backup/run-backup-assertions.sh` - Passed. - `docker run --rm -v "${PWD}:/work" -w /work ubuntu:24.04 bash scripts/backup/restore-path-assertions.sh` - Passed. - `git diff --check` - Passed. - Docker Bash syntax check: - `docker run --rm -v "${PWD}:/work" -w /work ubuntu:24.04 bash -n scripts/backup/run-backup.sh scripts/backup/restore.sh scripts/backup/container-entrypoint.sh scripts/backup/restore-path-assertions.sh scripts/backup/run-backup-assertions.sh` - Passed. - `npx prisma validate` - Passed. - `npm run build` - Passed. - `npm run lint` - Failed on unrelated existing React lint issues in `src/` and mirrored `.worktrees/worker-improvements` files; no failures were in touched backup files. ## Concerns - Local `bash` is unavailable because the Windows `bash` command resolves to a WSL shim with no installed distribution; shell checks used Docker fallback. - Full `npm run lint` remains blocked by pre-existing unrelated lint errors.