Merge branch 'feat/ranged-archive-listing' into main

# Conflicts:
#	.env.example
#	backup/Dockerfile
#	docker-compose.yml
This commit is contained in:
2026-07-31 04:24:58 +02:00
49 changed files with 7768 additions and 1627 deletions
@@ -0,0 +1,555 @@
# NAS Backup for Postgres + TDLib State Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add a `backup` container to the DragonsStash stack that takes daily, encrypted, deduplicated backups of the Postgres database and both TDLib state volumes, and ships them to a Synology NAS over NFS.
**Architecture:** A small Alpine-based image (restic + postgresql16-client + curl + dcron) runs as its own compose service. A crontab fires `backup.sh` daily at 03:00, which dumps Postgres, tars the TDLib volumes, hands both to `restic backup` against an NFS-backed Docker volume, prunes with `restic forget --keep-daily 14`, and reports success/failure to an Uptime Kuma push monitor. Matches the existing `worker`/`bot` pattern: build context in the repo's `docker-compose.yml`, prebuilt image in `/opt/stacks/DragonsStash/docker-compose.yml`, built and pushed by `.drone.yml`.
**Tech Stack:** Alpine 3.20, restic 0.16, postgresql16-client, dcron, bash, Docker Compose NFS volume driver.
## Global Constraints
- Retention: `restic forget --keep-daily 14 --prune` (14-day window, per approved spec).
- Schedule: daily backup at 03:00, weekly `restic check` at 04:00 Sunday.
- No Docker socket mount, no `privileged: true` — the backup container must not be able to control sibling containers.
- No host-level mount — NFS access only via Docker's native `driver_opts: type: nfs` volume, never `/etc/fstab`.
- Encryption and retention are restic's job — no hand-rolled `age`/`gpg`/`find -mtime` logic.
- TDLib volumes are tarred live (best-effort) — never pause `worker`/`bot` for the backup.
- Restore is a manual, documented procedure only — never scripted/automated.
**Required user input before Task 5 can run:** `NAS_HOST` and `NAS_EXPORT_PATH` (the Synology NFS share details) and a Kuma Push-monitor URL (`KUMA_PUSH_URL`, created manually in the existing Uptime Kuma instance, ~26h expected heartbeat interval). Tasks 14 need none of these and can proceed immediately; do not substitute placeholder values for them in Task 5 — stop and ask the user instead.
---
### Task 1: Backup image (Dockerfile + entrypoint)
**Files:**
- Create: `backup/Dockerfile`
- Create: `backup/entrypoint.sh`
- Create: `backup/crontab`
**Interfaces:**
- Produces: a buildable image tagged `dragonsstash-backup:test` locally, with `/entrypoint.sh` as `ENTRYPOINT`, `/backup.sh` present at the image root (written in Task 2 — this task only needs the `COPY` line and a placeholder-free stub isn't acceptable, so create an emptybody-but-real `backup/backup.sh` here containing just `#!/bin/bash` + `exit 0`, and Task 2 replaces its contents), `restic`, `pg_dump`/`pg_restore`, `curl`, `tar`, `bash`, `dcron` all on `PATH`.
- Consumes: nothing from earlier tasks.
- [ ] **Step 1: Write `backup/backup.sh` stub**
```bash
#!/bin/bash
set -euo pipefail
exit 0
```
- [ ] **Step 2: Write `backup/entrypoint.sh`**
```bash
#!/bin/bash
set -euo pipefail
if ! restic snapshots >/dev/null 2>&1; then
restic init
fi
exec crond -f -l 2
```
- [ ] **Step 3: Write `backup/crontab`**
```
0 3 * * * /backup.sh >> /proc/1/fd/1 2>&1
0 4 * * 0 restic check >> /proc/1/fd/1 2>&1
```
- [ ] **Step 4: Write `backup/Dockerfile`**
```dockerfile
FROM alpine:3.20
RUN apk add --no-cache restic postgresql16-client curl tzdata dcron tar bash
COPY backup/backup.sh /backup.sh
COPY backup/entrypoint.sh /entrypoint.sh
COPY backup/crontab /etc/crontabs/root
RUN chmod +x /backup.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
```
- [ ] **Step 5: Build the image**
Run: `cd /home/sam/Documents/DragonsStash && docker build -t dragonsstash-backup:test -f backup/Dockerfile .`
Expected: build completes with `Successfully tagged dragonsstash-backup:test` (or Buildkit's equivalent final `naming to docker.io/library/dragonsstash-backup:test done`), no errors.
- [ ] **Step 6: Verify the tools are present**
Run: `docker run --rm dragonsstash-backup:test restic version && docker run --rm dragonsstash-backup:test pg_dump --version`
Expected: `restic 0.16.x ...` and `pg_dump (PostgreSQL) 16.x` printed, both commands exit 0.
- [ ] **Step 7: Verify the entrypoint initializes an empty repo and starts cron**
```bash
mkdir -p /tmp/backup-repo-smoke
docker run -d --name backup-smoke \
-e RESTIC_REPOSITORY=/backups/restic-repo -e RESTIC_PASSWORD=smoketest \
-v /tmp/backup-repo-smoke:/backups \
dragonsstash-backup:test
sleep 2
docker logs backup-smoke
docker exec backup-smoke restic snapshots
docker rm -f backup-smoke
rm -rf /tmp/backup-repo-smoke
```
Expected: `docker logs` shows no errors (restic init ran silently); `restic snapshots` prints an empty snapshot list (repo exists, header row only, no error).
- [ ] **Step 8: Commit**
```bash
git add backup/Dockerfile backup/entrypoint.sh backup/backup.sh backup/crontab
git commit -m "Add backup service image (Dockerfile, entrypoint, crontab)"
```
---
### Task 2: `backup.sh` script
**Files:**
- Modify: `backup/backup.sh` (replace Task 1's stub with the real script)
**Interfaces:**
- Consumes: the image built in Task 1 (`dragonsstash-backup:test`), rebuilt after this change.
- Produces: `/backup.sh`, invoked by cron in Task 1's `crontab` and manually in Task 6's verification. Reads env vars `POSTGRES_USER`, `PGPASSWORD`, `POSTGRES_DB`, `RESTIC_REPOSITORY`, `RESTIC_PASSWORD`, `KUMA_PUSH_URL`. Assumes network hostname `dragonsstash-db:5432` for Postgres and mounts `/data/tdlib-worker`, `/data/tdlib-bot` (read-only) for TDLib state.
- [ ] **Step 1: Replace `backup/backup.sh` with the real script**
```bash
#!/bin/bash
set -euo pipefail
report_failure() {
curl -fsS "$KUMA_PUSH_URL" --get \
--data-urlencode "status=down" \
--data-urlencode "msg=$BASH_COMMAND failed" || true
}
trap report_failure ERR
DUMP_FILE=/tmp/dragonsstash.dump
TAR_FILE=/tmp/tdlib.tar.gz
pg_dump -h dragonsstash-db -U "$POSTGRES_USER" -d "$POSTGRES_DB" -Fc -f "$DUMP_FILE"
tar czf "$TAR_FILE" -C /data tdlib-worker tdlib-bot
restic backup "$DUMP_FILE" "$TAR_FILE"
restic forget --keep-daily 14 --prune
rm -f "$DUMP_FILE" "$TAR_FILE"
curl -fsS "$KUMA_PUSH_URL" --get \
--data-urlencode "status=up" \
--data-urlencode "msg=OK"
```
- [ ] **Step 2: Rebuild the image**
Run: `docker build -t dragonsstash-backup:test -f backup/Dockerfile .`
Expected: build succeeds.
- [ ] **Step 3: Stand up a scratch Postgres aliased as `dragonsstash-db`**
```bash
docker network create backup-test-net 2>/dev/null || true
docker run -d --name test-pg --network backup-test-net --network-alias dragonsstash-db \
-e POSTGRES_USER=dragons -e POSTGRES_PASSWORD=stash -e POSTGRES_DB=dragonsstash \
postgres:16-alpine
sleep 5
docker exec test-pg pg_isready -U dragons -d dragonsstash
```
Expected: `accepting connections`.
- [ ] **Step 4: Stand up a mock Kuma push endpoint**
```bash
mkdir -p /tmp/mock-kuma-root && touch /tmp/mock-kuma-root/push
docker run -d --name mock-kuma --network backup-test-net \
-v /tmp/mock-kuma-root:/srv -w /srv python:3-alpine \
python3 -m http.server 8000
sleep 1
```
- [ ] **Step 5: Prepare fake TDLib state and a local restic repo dir**
```bash
mkdir -p /tmp/backup-test/tdlib-worker /tmp/backup-test/tdlib-bot /tmp/backup-test/repo
echo "fake-session" > /tmp/backup-test/tdlib-worker/state.bin
echo "fake-session" > /tmp/backup-test/tdlib-bot/state.bin
```
- [ ] **Step 6: Initialize the test restic repo and run `backup.sh` (success path)**
```bash
docker run --rm --network backup-test-net \
-e RESTIC_REPOSITORY=/backups/restic-repo -e RESTIC_PASSWORD=testpassword \
-v /tmp/backup-test/repo:/backups \
--entrypoint restic dragonsstash-backup:test init
docker run --rm --network backup-test-net \
-e POSTGRES_USER=dragons -e POSTGRES_PASSWORD=stash -e PGPASSWORD=stash -e POSTGRES_DB=dragonsstash \
-e RESTIC_REPOSITORY=/backups/restic-repo -e RESTIC_PASSWORD=testpassword \
-e KUMA_PUSH_URL=http://mock-kuma:8000/push \
-v /tmp/backup-test/tdlib-worker:/data/tdlib-worker:ro \
-v /tmp/backup-test/tdlib-bot:/data/tdlib-bot:ro \
-v /tmp/backup-test/repo:/backups \
--entrypoint /backup.sh dragonsstash-backup:test
echo "exit code: $?"
```
Expected: exit code `0`, restic prints a line like `snapshot xxxxxxxx saved`, no error output.
- [ ] **Step 7: Verify the snapshot landed and contains both files**
```bash
docker run --rm -v /tmp/backup-test/repo:/backups \
-e RESTIC_REPOSITORY=/backups/restic-repo -e RESTIC_PASSWORD=testpassword \
--entrypoint restic dragonsstash-backup:test snapshots
docker run --rm -v /tmp/backup-test/repo:/backups \
-e RESTIC_REPOSITORY=/backups/restic-repo -e RESTIC_PASSWORD=testpassword \
--entrypoint restic dragonsstash-backup:test ls latest
```
Expected: `snapshots` shows exactly one entry; `ls latest` lists `/tmp/dragonsstash.dump` and `/tmp/tdlib.tar.gz`.
- [ ] **Step 8: Verify the failure path reports to Kuma**
```bash
docker run --rm --network backup-test-net \
-e POSTGRES_USER=dragons -e POSTGRES_PASSWORD=wrongpass -e PGPASSWORD=wrongpass -e POSTGRES_DB=dragonsstash \
-e RESTIC_REPOSITORY=/backups/restic-repo -e RESTIC_PASSWORD=testpassword \
-e KUMA_PUSH_URL=http://mock-kuma:8000/push \
-v /tmp/backup-test/tdlib-worker:/data/tdlib-worker:ro \
-v /tmp/backup-test/tdlib-bot:/data/tdlib-bot:ro \
-v /tmp/backup-test/repo:/backups \
--entrypoint /backup.sh dragonsstash-backup:test
echo "exit code: $?"
docker logs mock-kuma | tail -5
```
Expected: exit code nonzero (pg_dump auth failure trips `set -e`); `docker logs mock-kuma` shows a GET request line containing `status=down`.
- [ ] **Step 9: Clean up test resources**
```bash
docker rm -f test-pg mock-kuma
docker network rm backup-test-net
rm -rf /tmp/backup-test /tmp/mock-kuma-root
```
- [ ] **Step 10: Commit**
```bash
git add backup/backup.sh
git commit -m "Implement backup.sh: pg_dump + tdlib tar + restic backup/forget + Kuma reporting"
```
---
### Task 3: Wire the `backup` service into the repo's `docker-compose.yml`
**Files:**
- Modify: `docker-compose.yml`
- Modify: `.env.example`
**Interfaces:**
- Consumes: `backup/Dockerfile` (Task 1), `backup/backup.sh` (Task 2).
- Produces: a `backup` compose service buildable via `docker compose build backup`, and a `nas_backups` named volume other tasks (5) will mirror into the production compose file.
- [ ] **Step 1: Add the `backup` service to `docker-compose.yml`**
Insert after the existing `bot` service (before `db`):
```yaml
backup:
build:
context: .
dockerfile: backup/Dockerfile
pull_policy: never
environment:
- POSTGRES_USER=${POSTGRES_USER:-dragons}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-stash}
- PGPASSWORD=${POSTGRES_PASSWORD:-stash}
- POSTGRES_DB=${POSTGRES_DB:-dragonsstash}
- RESTIC_REPOSITORY=/backups/restic-repo
- RESTIC_PASSWORD=${RESTIC_PASSWORD:?Set RESTIC_PASSWORD in .env}
- KUMA_PUSH_URL=${KUMA_PUSH_URL:?Set KUMA_PUSH_URL in .env}
- TZ=${TZ:-Etc/UTC}
volumes:
- tdlib_state:/data/tdlib-worker:ro
- tdlib_bot_state:/data/tdlib-bot:ro
- nas_backups:/backups
depends_on:
db:
condition: service_healthy
restart: unless-stopped
deploy:
resources:
limits:
memory: 256M
networks:
- backend
```
- [ ] **Step 2: Add the `nas_backups` volume to the `volumes:` block**
```yaml
nas_backups:
driver_opts:
type: nfs
o: "addr=${NAS_HOST},rw,nfsvers=4,soft,timeo=100"
device: ":${NAS_EXPORT_PATH}"
```
- [ ] **Step 3: Document the new env vars in `.env.example`**
Append:
```
# Backup (NAS via NFS + restic)
NAS_HOST="" # Synology NAS IP or hostname reachable from this host
NAS_EXPORT_PATH="" # NFS export path, e.g. /volume1/dragonsstash-backups
RESTIC_PASSWORD="" # generate with: openssl rand -base64 32
KUMA_PUSH_URL="" # Uptime Kuma Push monitor URL (create the monitor first)
TZ="Etc/UTC"
```
- [ ] **Step 4: Validate the compose file parses**
Run (with dummy values so the `:?` guards don't fail parsing — `AUTH_SECRET` is required by the existing `app` service, not by this change, but `config` validates the whole file):
```bash
RESTIC_PASSWORD=dummy KUMA_PUSH_URL=http://dummy NAS_HOST=dummy NAS_EXPORT_PATH=/dummy AUTH_SECRET=dummy \
docker compose -f docker-compose.yml config --quiet
```
Expected: no output, exit code 0 (a syntax/interpolation error would print to stderr and exit nonzero).
- [ ] **Step 5: Validate the service actually builds through Compose**
Run:
```bash
RESTIC_PASSWORD=dummy KUMA_PUSH_URL=http://dummy NAS_HOST=dummy NAS_EXPORT_PATH=/dummy AUTH_SECRET=dummy \
docker compose -f docker-compose.yml build backup
```
Expected: build succeeds (reuses Task 1's image layers).
- [ ] **Step 6: Commit**
```bash
git add docker-compose.yml .env.example
git commit -m "Add backup service and nas_backups volume to docker-compose.yml"
```
---
### Task 4: CI — build and push the backup image
**Files:**
- Modify: `.drone.yml`
**Interfaces:**
- Consumes: `backup/Dockerfile` (Task 1).
- Produces: `git.samagsteribbe.nl/admin/dragonsstash-backup:latest` (and `:<short-sha>`), pushed on every push to `main`. Task 5's production compose file references this image tag.
- [ ] **Step 1: Add a `build-backup` step, mirroring `build-worker`/`build-bot`**
Insert after the existing `build-bot` step in `.drone.yml`:
```yaml
- name: build-backup
image: plugins/docker
depends_on: [clone]
settings:
repo: git.samagsteribbe.nl/admin/dragonsstash-backup
registry: git.samagsteribbe.nl
dockerfile: backup/Dockerfile
tags:
- latest
- "${DRONE_COMMIT_SHA:0:8}"
username:
from_secret: gitea_username
password:
from_secret: gitea_password
```
- [ ] **Step 2: Add `build-backup` to the `deploy` step's `depends_on`**
Change:
```yaml
- name: deploy
image: alpine
depends_on: [build-app, build-worker, build-bot]
```
to:
```yaml
- name: deploy
image: alpine
depends_on: [build-app, build-worker, build-bot, build-backup]
```
- [ ] **Step 3: Validate YAML syntax**
Run: `python3 -c "import yaml; yaml.safe_load(open('.drone.yml')); print('OK')"`
Expected: `OK` printed, no exception.
- [ ] **Step 4: Commit**
```bash
git add .drone.yml
git commit -m "Add build-backup CI step, include it in deploy dependencies"
```
---
### Task 5: Wire production (`/opt/stacks/DragonsStash`) — requires NAS + Kuma details from the user
**Do not start this task until the user has supplied `NAS_HOST` and `NAS_EXPORT_PATH` for the Synology NFS share, and has created an Uptime Kuma Push monitor (name it `dragonsstash-backup`, ~26h expected heartbeat interval) and shared its push URL. If any of these are missing, stop and ask — do not substitute placeholder values here, since this file drives the real deployment.**
**Files:**
- Modify: `/opt/stacks/DragonsStash/docker-compose.yml`
- Modify: `/opt/stacks/DragonsStash/.env`
**Interfaces:**
- Consumes: `git.samagsteribbe.nl/admin/dragonsstash-backup:latest` (published by Task 4's CI step once merged/pushed), the real `NAS_HOST`/`NAS_EXPORT_PATH`/`KUMA_PUSH_URL` values gathered above.
- Produces: a running `dragonsstash-backup` container on the production host, verified in Task 6.
- [ ] **Step 1: Generate `RESTIC_PASSWORD` and add all four new vars to `/opt/stacks/DragonsStash/.env`**
```bash
cd /opt/stacks/DragonsStash
printf '\n# Backup (NAS via NFS + restic)\nNAS_HOST="<value from user>"\nNAS_EXPORT_PATH="<value from user>"\nRESTIC_PASSWORD="%s"\nKUMA_PUSH_URL="<value from user>"\nTZ="Etc/UTC"\n' "$(openssl rand -base64 32)" >> .env
```
Replace the two `<value from user>` placeholders with the real NAS details and Kuma push URL before saving — this step cannot be completed with the literal placeholder text left in place.
- [ ] **Step 2: Add the `backup` service to `/opt/stacks/DragonsStash/docker-compose.yml`**
Insert after the existing `bot` service (before `db`):
```yaml
backup:
image: git.samagsteribbe.nl/admin/dragonsstash-backup:latest
container_name: dragonsstash-backup
restart: unless-stopped
environment:
- POSTGRES_USER=${POSTGRES_USER:-dragons}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-stash}
- PGPASSWORD=${POSTGRES_PASSWORD:-stash}
- POSTGRES_DB=${POSTGRES_DB:-dragonsstash}
- RESTIC_REPOSITORY=/backups/restic-repo
- RESTIC_PASSWORD=${RESTIC_PASSWORD:?Set RESTIC_PASSWORD in .env}
- KUMA_PUSH_URL=${KUMA_PUSH_URL:?Set KUMA_PUSH_URL in .env}
- TZ=${TZ:-Etc/UTC}
volumes:
- tdlib_state:/data/tdlib-worker:ro
- tdlib_bot_state:/data/tdlib-bot:ro
- nas_backups:/backups
depends_on:
db:
condition: service_healthy
deploy:
resources:
limits:
memory: 256M
networks:
- internal
```
- [ ] **Step 3: Add the `nas_backups` volume**
```yaml
nas_backups:
driver_opts:
type: nfs
o: "addr=${NAS_HOST},rw,nfsvers=4,soft,timeo=100"
device: ":${NAS_EXPORT_PATH}"
```
- [ ] **Step 4: Validate the production compose file parses with the real `.env`**
Run: `cd /opt/stacks/DragonsStash && docker compose config --quiet`
Expected: no output, exit code 0.
- [ ] **Step 5: Commit is not applicable here**`/opt/stacks/DragonsStash` is a deployed copy, not the git repo (confirm with `git -C /opt/stacks/DragonsStash status` — expect "not a git repository"). Skip committing; Task 6 deploys these file changes directly.
---
### Task 6: Deploy and verify
**Files:** none (operational task)
**Interfaces:**
- Consumes: everything from Tasks 15.
- Produces: a running, verified backup on the real NAS, and one completed restore drill.
- [ ] **Step 1: Confirm with the user before pushing/deploying**
Pushing to `main` triggers Drone CI to build all four images and deploy to the production host via SSH (`docker compose pull && docker compose up -d`). Confirm the user wants this to happen now before proceeding — this affects the live stack.
- [ ] **Step 2: Push to `main`**
```bash
cd /home/sam/Documents/DragonsStash
git push origin main
```
Expected: Drone pipeline runs `build-app`, `build-worker`, `build-bot`, `build-backup`, then `deploy`, all green. Check via the Drone UI or `drone build info admin/DragonsStash <build-number>` if the `drone` CLI is configured.
- [ ] **Step 3: Confirm the container is up on the production host**
```bash
ssh sam@192.168.68.68 "docker ps --filter name=dragonsstash-backup --format '{{.Names}}\t{{.Status}}'"
```
Expected: `dragonsstash-backup Up ...`.
- [ ] **Step 4: Trigger one manual backup run and confirm a snapshot lands on the NAS**
```bash
ssh sam@192.168.68.68 "docker exec dragonsstash-backup /backup.sh"
ssh sam@192.168.68.68 "docker exec dragonsstash-backup restic snapshots"
```
Expected: `backup.sh` exits 0; `restic snapshots` lists exactly one entry.
- [ ] **Step 5: Confirm the Uptime Kuma push monitor shows green**
Open the Uptime Kuma dashboard and check the `dragonsstash-backup` monitor's status is up with a recent heartbeat.
- [ ] **Step 6: Restore drill — prove the backup is actually restorable**
```bash
ssh sam@192.168.68.68 "docker exec dragonsstash-backup restic restore latest --target /tmp/restore-drill"
ssh sam@192.168.68.68 "docker exec dragonsstash-backup ls -la /tmp/restore-drill/tmp"
```
Expected: `/tmp/restore-drill/tmp/dragonsstash.dump` and `/tmp/restore-drill/tmp/tdlib.tar.gz` both present with nonzero size. Then, on a scratch Postgres (not the live `dragonsstash-db`), confirm the dump restores cleanly:
```bash
ssh sam@192.168.68.68 "docker run -d --rm --name restore-drill-pg --network dragonsstash_internal \
-e POSTGRES_USER=drill -e POSTGRES_PASSWORD=drill -e POSTGRES_DB=drill postgres:16-alpine"
ssh sam@192.168.68.68 "docker cp dragonsstash-backup:/tmp/restore-drill/tmp/dragonsstash.dump /tmp/dragonsstash.dump"
ssh sam@192.168.68.68 "docker cp /tmp/dragonsstash.dump restore-drill-pg:/tmp/dragonsstash.dump"
ssh sam@192.168.68.68 "docker exec -e PGPASSWORD=drill restore-drill-pg pg_restore -U drill -d drill --clean --if-exists /tmp/dragonsstash.dump"
ssh sam@192.168.68.68 "docker exec -e PGPASSWORD=drill restore-drill-pg psql -U drill -d drill -c '\\dt' | head -20"
ssh sam@192.168.68.68 "docker rm -f restore-drill-pg"
```
Expected: `pg_restore` completes without fatal errors; `\dt` lists the app's tables (e.g. `Package`, `User`, `TelegramLink`).
- [ ] **Step 7: Clean up drill artifacts**
```bash
ssh sam@192.168.68.68 "docker exec dragonsstash-backup rm -rf /tmp/restore-drill"
ssh sam@192.168.68.68 "rm -f /tmp/dragonsstash.dump"
```
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff