dragonsstash/.superpowers/sdd/scope-correction-implementation-report.md

Scope correction implementation report

Summary

Implemented the approved backup scope correction for Dragon's Stash. Backups and restores now cover only:

• PostgreSQL logical custom-format dump plus manifest/migration metadata.
• tdlib_state worker Telegram session volume.
• tdlib_bot_state bot Telegram session volume.

The implementation no longer treats manual_uploads, completed local STL binaries, or tmp_zips as protected backup data. Future channel forwarding and archive/STL-content integrity auditing remain out of scope.

Changed files

• docker-compose.yml

  • Removed the backup service's read-only manual_uploads:/data/uploads mount.
  • Kept the normal operational app/worker manual_uploads mounts.
  • Kept both TDLib backup mounts.

• scripts/backup/container-entrypoint.sh

  • Removed /data/uploads as a required mounted directory.
  • Removed uploads from the manifest volumePaths.
  • Removed the source:uploads Restic tag.
  • Removed uploads from the Restic source list.
  • Kept database dump, manifest, worker TDLib, and bot TDLib sources/tags.

• scripts/backup/restore.sh

  • Removed restored uploads variables.
  • Removed uploads staging validation.
  • Removed manual uploads volume discovery, safety archive, replacement, and rollback.
  • Removed retained/manual upload file-path verification.
  • Removed temporary database verification that existed only for local upload file references.
  • Preserved guarded restore-live confirmation, backup mount/repository checks, service stop/start handling, safety PostgreSQL dump, TDLib volume safety archives, TDLib volume replacement/rollback, and pg_restore --list/pg_restore --exit-on-error validation.

• prisma/schema.prisma

  • Removed ManualUploadFile.retainedAt.

• prisma/migrations/20260722100000_remove_retained_manual_files/migration.sql

  • Added forward migration: ALTER TABLE "manual_upload_files" DROP COLUMN IF EXISTS "retainedAt";
  • Preserved the existing committed migration that added retainedAt.

• src/app/api/uploads/route.ts

  • Removed retainedAt: new Date() from manual upload file creation.

• worker/src/manual-upload.ts

  • Restored final best-effort cleanup of /data/uploads/<uploadId> using the older path.join("/data/uploads", uploadId) behavior.

• scripts/backup/README.md

  • Rewrote backup set and restore rehearsal docs around PostgreSQL plus both TDLib volumes only.
  • Removed local STL file, retainedAt, upload path, retained file reference, and restored checksum checks.
  • Clarified that STL binaries remain in Telegram and recovery preserves database mappings/Telegram IDs.
  • Kept monthly restic check --read-data and disposable restore rehearsal runbook.
  • Explicitly left future channel forwarding and archive/STL-content integrity auditing out of scope.

• README.md

  • Updated the production backup summary to name PostgreSQL logical dump plus Telegram session volumes as the protected set.
  • Clarified that manual_uploads and temporary ZIPs are excluded and STL binaries remain in Telegram.

Verification

• git diff --check

  • Passed.

• bash -n scripts/backup/container-entrypoint.sh scripts/backup/run-backup.sh scripts/backup/restore.sh

  • Local bash failed because Windows only had the WSL shim and no installed WSL distribution.
  • Passed via Docker fallback: docker run --rm --entrypoint bash -v E:\Projects\DragonsStash:/work:ro -w /work postgres:16-alpine -n scripts/backup/container-entrypoint.sh scripts/backup/run-backup.sh scripts/backup/restore.sh

• npx prisma validate

  • Passed.

• npm run build

  • Passed.

• cd worker && npm run build

  • Passed.

• Focused backup/restore scope assertions

  • Backup shell paths assertion passed: no manual_uploads, /data/uploads, retainedAt, upload source tag, or upload-restore helper references in scripts/backup/*.sh.
  • Compose backup service assertion passed: no manual_uploads, /data/uploads, retainedAt, or source:uploads in the backup service block.
  • Active retainedAt assertion passed: no retainedAt in active Prisma schema, upload API, worker source, or backup shell scripts.
  • Active backup/restore upload-source assertion passed: no manual_uploads, /data/uploads, or data/uploads in backup/restore shell scripts.
  • Remaining expected matches are limited to normal operational app/worker upload mounts and paths, docs stating exclusions, and the historical add/drop migrations.

Concerns

• None for implementation scope.
• Environment note: local Bash is unavailable because WSL has no installed distribution; Bash syntax was verified inside Docker instead.

---

Restore Path Scope Review-Finding Fix

Date: 2026-07-22

Fix

Addressed the Important restore finding by replacing both unrestricted restic restore calls in scripts/backup/restore.sh with a shared filtered restore wrapper. The wrapper restores only the backup source paths actually written by scripts/backup/container-entrypoint.sh:

• /staging/backup-*/database.dump
• /staging/backup-*/manifest and /staging/backup-*/manifest/**
• /data/tdlib-worker and /data/tdlib-worker/**
• /data/tdlib-bot and /data/tdlib-bot/**

Added an explicit restored-tree guard that refuses unexpected restored content: top-level restored directories other than staging and data, direct data/* entries other than tdlib-worker and tdlib-bot, and direct staging/backup-*/* entries other than database.dump and manifest. This rejects old/broad snapshots that would otherwise restore data/uploads, temporary ZIP or database volume trees, or other unexpected volume content.

Preserved the existing guarded live-restore confirmation, staging-directory validation, mount/repository checks, snapshot verification, custom PostgreSQL-dump validation, service stop/start lifecycle, health check, safety database dump, TDLib safety archives, and rollback of exactly the PostgreSQL database plus the two TDLib volumes.

Added scripts/backup/restore-path-assertions.sh, a focused shell assertion harness that stubs Docker/Restic and verifies both staging and live restore use the expected include filters and that unexpected restored data-volume content is rejected explicitly.

Addressed the Minor documentation gap in the root README backup section by stating that forwarding behavior and archive/STL-content integrity auditing are future work outside the backup scope.

Verification

• Red check before implementation:

  & 'C:\Program Files\Git\bin\bash.exe' -lc 'scripts/backup/restore-path-assertions.sh'
  ASSERTION FAILED: database dump include filter missing
  

• Bash syntax check:

  & 'C:\Program Files\Git\bin\bash.exe' -lc 'bash -n scripts/backup/container-entrypoint.sh scripts/backup/run-backup.sh scripts/backup/restore.sh scripts/backup/restore-path-assertions.sh'
  [passed with no output]
  

• Whitespace check:

  git diff --check
  warning: in the working copy of '.superpowers/sdd/scope-correction-implementation-report.md', LF will be replaced by CRLF the next time Git touches it
  warning: in the working copy of 'README.md', LF will be replaced by CRLF the next time Git touches it
  warning: in the working copy of 'scripts/backup/restore.sh', LF will be replaced by CRLF the next time Git touches it
  [exit 0]
  

• Focused restore-path assertions:

  & 'C:\Program Files\Git\bin\bash.exe' -lc 'scripts/backup/restore-path-assertions.sh'
  restore-path assertions passed
  

Concerns

• None for implementation scope.
• Git Bash was available and used for shell syntax/assertion checks, so Docker fallback was not needed for the final syntax verification.

---

Important Operational Findings Fix

Date: 2026-07-22

Fix

Addressed the two Important operational findings from final review:

• scripts/backup/run-backup.sh

  • Backup wrapper restart failures now make an otherwise successful backup exit non-zero.
  • Existing non-zero backup failures remain preserved if service restart also fails.
  • Added scripts/backup/run-backup-assertions.sh to assert both exit-code cases with a fake Docker/Compose environment.

• scripts/backup/restore.sh

  • restore-live now captures the managed services that were running before live restore using docker compose --profile full ps --status running.
  • Live restore stops only those previously running managed services.
  • Successful live restore starts only those previously running services, so the profile-gated optional bot is not started if it was not running.
  • Failure handling leaves services stopped and still rolls back only the PostgreSQL database plus both TDLib volumes.
  • App health wait now runs only when app was previously running.
  • Extended scripts/backup/restore-path-assertions.sh to assert subset stop/start behavior and skipped health checks when app was not running.

Addressed the Minor staging-path documentation mismatch by aligning .env.example with the backup README example: /var/lib/dragons-stash/backup-staging.

Verification

• Red checks before implementation:

  • docker run --rm -v "${PWD}:/work" -w /work ubuntu:24.04 bash scripts/backup/run-backup-assertions.sh
    • Failed as expected: backup reported success when restart failed.
  • docker run --rm -v "${PWD}:/work" -w /work ubuntu:24.04 bash scripts/backup/restore-path-assertions.sh
    • Failed as expected: restore-live stopped the fixed app worker bot service set instead of the running subset.

• Focused assertions after implementation:

  • docker run --rm -v "${PWD}:/work" -w /work ubuntu:24.04 bash scripts/backup/run-backup-assertions.sh
    • Passed.
  • docker run --rm -v "${PWD}:/work" -w /work ubuntu:24.04 bash scripts/backup/restore-path-assertions.sh
    • Passed.

• git diff --check

  • Passed.

• Docker Bash syntax check:

  • docker run --rm -v "${PWD}:/work" -w /work ubuntu:24.04 bash -n scripts/backup/run-backup.sh scripts/backup/restore.sh scripts/backup/container-entrypoint.sh scripts/backup/restore-path-assertions.sh scripts/backup/run-backup-assertions.sh
  • Passed.

• npx prisma validate

  • Passed.

• npm run build

  • Passed.

• npm run lint

  • Failed on unrelated existing React lint issues in src/ and mirrored .worktrees/worker-improvements files; no failures were in touched backup files.

Concerns

• Local bash is unavailable because the Windows bash command resolves to a WSL shim with no installed distribution; shell checks used Docker fallback.
• Full npm run lint remains blocked by pre-existing unrelated lint errors.