mirror of
https://github.com/xCyanGrizzly/DragonsStash.git
synced 2026-09-21 05:21:43 +00:00
fix: restrict backup restore paths
This commit is contained in:
@@ -86,3 +86,82 @@ The implementation no longer treats `manual_uploads`, completed local STL binari
|
|||||||
|
|
||||||
- None for implementation scope.
|
- None for implementation scope.
|
||||||
- Environment note: local Bash is unavailable because WSL has no installed distribution; Bash syntax was verified inside Docker instead.
|
- Environment note: local Bash is unavailable because WSL has no installed distribution; Bash syntax was verified inside Docker instead.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Restore Path Scope Review-Finding Fix
|
||||||
|
|
||||||
|
**Date:** 2026-07-22
|
||||||
|
|
||||||
|
## Fix
|
||||||
|
|
||||||
|
Addressed the Important restore finding by replacing both unrestricted
|
||||||
|
`restic restore` calls in `scripts/backup/restore.sh` with a shared filtered
|
||||||
|
restore wrapper. The wrapper restores only the backup source paths actually
|
||||||
|
written by `scripts/backup/container-entrypoint.sh`:
|
||||||
|
|
||||||
|
- `/staging/backup-*/database.dump`
|
||||||
|
- `/staging/backup-*/manifest` and `/staging/backup-*/manifest/**`
|
||||||
|
- `/data/tdlib-worker` and `/data/tdlib-worker/**`
|
||||||
|
- `/data/tdlib-bot` and `/data/tdlib-bot/**`
|
||||||
|
|
||||||
|
Added an explicit restored-tree guard that refuses unexpected restored content:
|
||||||
|
top-level restored directories other than `staging` and `data`, direct
|
||||||
|
`data/*` entries other than `tdlib-worker` and `tdlib-bot`, and direct
|
||||||
|
`staging/backup-*/*` entries other than `database.dump` and `manifest`. This
|
||||||
|
rejects old/broad snapshots that would otherwise restore `data/uploads`,
|
||||||
|
temporary ZIP or database volume trees, or other unexpected volume content.
|
||||||
|
|
||||||
|
Preserved the existing guarded live-restore confirmation, staging-directory
|
||||||
|
validation, mount/repository checks, snapshot verification, custom
|
||||||
|
PostgreSQL-dump validation, service stop/start lifecycle, health check, safety
|
||||||
|
database dump, TDLib safety archives, and rollback of exactly the PostgreSQL
|
||||||
|
database plus the two TDLib volumes.
|
||||||
|
|
||||||
|
Added `scripts/backup/restore-path-assertions.sh`, a focused shell assertion
|
||||||
|
harness that stubs Docker/Restic and verifies both staging and live restore use
|
||||||
|
the expected include filters and that unexpected restored data-volume content is
|
||||||
|
rejected explicitly.
|
||||||
|
|
||||||
|
Addressed the Minor documentation gap in the root README backup section by
|
||||||
|
stating that forwarding behavior and archive/STL-content integrity auditing are
|
||||||
|
future work outside the backup scope.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
- Red check before implementation:
|
||||||
|
|
||||||
|
```text
|
||||||
|
& 'C:\Program Files\Git\bin\bash.exe' -lc 'scripts/backup/restore-path-assertions.sh'
|
||||||
|
ASSERTION FAILED: database dump include filter missing
|
||||||
|
```
|
||||||
|
|
||||||
|
- Bash syntax check:
|
||||||
|
|
||||||
|
```text
|
||||||
|
& 'C:\Program Files\Git\bin\bash.exe' -lc 'bash -n scripts/backup/container-entrypoint.sh scripts/backup/run-backup.sh scripts/backup/restore.sh scripts/backup/restore-path-assertions.sh'
|
||||||
|
[passed with no output]
|
||||||
|
```
|
||||||
|
|
||||||
|
- Whitespace check:
|
||||||
|
|
||||||
|
```text
|
||||||
|
git diff --check
|
||||||
|
warning: in the working copy of '.superpowers/sdd/scope-correction-implementation-report.md', LF will be replaced by CRLF the next time Git touches it
|
||||||
|
warning: in the working copy of 'README.md', LF will be replaced by CRLF the next time Git touches it
|
||||||
|
warning: in the working copy of 'scripts/backup/restore.sh', LF will be replaced by CRLF the next time Git touches it
|
||||||
|
[exit 0]
|
||||||
|
```
|
||||||
|
|
||||||
|
- Focused restore-path assertions:
|
||||||
|
|
||||||
|
```text
|
||||||
|
& 'C:\Program Files\Git\bin\bash.exe' -lc 'scripts/backup/restore-path-assertions.sh'
|
||||||
|
restore-path assertions passed
|
||||||
|
```
|
||||||
|
|
||||||
|
## Concerns
|
||||||
|
|
||||||
|
- None for implementation scope.
|
||||||
|
- Git Bash was available and used for shell syntax/assertion checks, so Docker
|
||||||
|
fallback was not needed for the final syntax verification.
|
||||||
|
|||||||
@@ -147,8 +147,9 @@ logical dump plus the worker and bot Telegram session volumes in an encrypted
|
|||||||
Restic repository on a Synology NFS share. `manual_uploads` and temporary ZIP
|
Restic repository on a Synology NFS share. `manual_uploads` and temporary ZIP
|
||||||
processing data are excluded; STL binaries remain in Telegram, while the
|
processing data are excluded; STL binaries remain in Telegram, while the
|
||||||
database mappings and Telegram IDs are what recovery preserves for lookup and
|
database mappings and Telegram IDs are what recovery preserves for lookup and
|
||||||
delivery. See the [backup and recovery guide](scripts/backup/README.md) for
|
delivery. Forwarding behavior and archive/STL-content integrity auditing are
|
||||||
Synology setup, secrets, systemd installation, monitoring, retention, and
|
future work outside this backup scope. See the [backup and recovery guide](scripts/backup/README.md)
|
||||||
|
for Synology setup, secrets, systemd installation, monitoring, retention, and
|
||||||
guarded restore procedures.
|
guarded restore procedures.
|
||||||
|
|
||||||
### Seeding the Database
|
### Seeding the Database
|
||||||
|
|||||||
@@ -0,0 +1,181 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -Eeuo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
|
||||||
|
PROJECT_ROOT="$(cd -- "$SCRIPT_DIR/../.." && pwd -P)"
|
||||||
|
RESTORE_SCRIPT="$PROJECT_ROOT/scripts/backup/restore.sh"
|
||||||
|
TMP_ROOT="$(mktemp -d)"
|
||||||
|
|
||||||
|
cleanup() {
|
||||||
|
rm -rf -- "$TMP_ROOT"
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
|
||||||
|
fail() {
|
||||||
|
printf 'ASSERTION FAILED: %s\n' "$*" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
setup_fake_environment() {
|
||||||
|
local fake_bin="$TMP_ROOT/bin"
|
||||||
|
|
||||||
|
mkdir -p -- "$fake_bin" "$TMP_ROOT/mount" "$TMP_ROOT/staging"
|
||||||
|
printf 'not-secret\n' > "$TMP_ROOT/restic-password"
|
||||||
|
|
||||||
|
cat > "$fake_bin/docker" <<'EOF'
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
set -Eeuo pipefail
|
||||||
|
|
||||||
|
printf '%s\t' "$@" >> "$DRAGONS_STASH_DOCKER_CALLS"
|
||||||
|
printf '\n' >> "$DRAGONS_STASH_DOCKER_CALLS"
|
||||||
|
|
||||||
|
target=""
|
||||||
|
previous=""
|
||||||
|
safety_output_host=""
|
||||||
|
safety_archive=""
|
||||||
|
for argument in "$@"; do
|
||||||
|
if [[ "$previous" == "--target" ]]; then
|
||||||
|
target="$argument"
|
||||||
|
fi
|
||||||
|
if [[ "$previous" == "-v" && "$argument" == *":/safety-output" ]]; then
|
||||||
|
safety_output_host="${argument%:/safety-output}"
|
||||||
|
fi
|
||||||
|
if [[ "$argument" == /safety-output/pre-restore-*.tar ]]; then
|
||||||
|
safety_archive="${argument#/safety-output/}"
|
||||||
|
fi
|
||||||
|
previous="$argument"
|
||||||
|
done
|
||||||
|
|
||||||
|
if [[ "${1:-}" == "compose" && "${2:-}" == "config" && "${3:-}" == "--format" && "${4:-}" == "json" ]]; then
|
||||||
|
printf '{\n "name": "dragonsstash"\n}\n'
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "${1:-}" == "volume" && "${2:-}" == "ls" ]]; then
|
||||||
|
if [[ " $* " == *"com.docker.compose.volume=tdlib_state"* ]]; then
|
||||||
|
printf 'dragonsstash_tdlib_state\n'
|
||||||
|
elif [[ " $* " == *"com.docker.compose.volume=tdlib_bot_state"* ]]; then
|
||||||
|
printf 'dragonsstash_tdlib_bot_state\n'
|
||||||
|
fi
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ " $* " == *" exec -T db pg_dump "* ]]; then
|
||||||
|
printf 'custom dump\n'
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ " $* " == *" backup restore "* ]]; then
|
||||||
|
if [[ "$target" != /staging/* ]]; then
|
||||||
|
printf 'Unexpected restore target: %s\n' "$target" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
host_target="$BACKUP_STAGING_PATH/${target#/staging/}"
|
||||||
|
mkdir -p \
|
||||||
|
"$host_target/staging/backup-legacy/manifest" \
|
||||||
|
"$host_target/data/tdlib-worker" \
|
||||||
|
"$host_target/data/tdlib-bot"
|
||||||
|
printf 'custom dump\n' > "$host_target/staging/backup-legacy/database.dump"
|
||||||
|
printf '{}\n' > "$host_target/staging/backup-legacy/manifest/backup-manifest.json"
|
||||||
|
|
||||||
|
if [[ "${RESTORE_ASSERT_CREATE_UNEXPECTED:-0}" == "1" ]]; then
|
||||||
|
mkdir -p "$host_target/data/uploads" "$host_target/data/tmp-zips" "$host_target/data/postgres"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ -n "$safety_output_host" && -n "$safety_archive" ]]; then
|
||||||
|
printf 'archive\n' > "$safety_output_host/$safety_archive"
|
||||||
|
fi
|
||||||
|
|
||||||
|
exit 0
|
||||||
|
EOF
|
||||||
|
chmod +x "$fake_bin/docker"
|
||||||
|
|
||||||
|
cat > "$fake_bin/mountpoint" <<'EOF'
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
set -Eeuo pipefail
|
||||||
|
exit 0
|
||||||
|
EOF
|
||||||
|
chmod +x "$fake_bin/mountpoint"
|
||||||
|
|
||||||
|
cat > "$fake_bin/curl" <<'EOF'
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
set -Eeuo pipefail
|
||||||
|
exit 0
|
||||||
|
EOF
|
||||||
|
chmod +x "$fake_bin/curl"
|
||||||
|
|
||||||
|
export PATH="$fake_bin:$PATH"
|
||||||
|
export BACKUP_MOUNT_PATH="$TMP_ROOT/mount"
|
||||||
|
export BACKUP_STAGING_PATH="$TMP_ROOT/staging"
|
||||||
|
export BACKUP_RESTIC_PASSWORD_FILE="$TMP_ROOT/restic-password"
|
||||||
|
export BACKUP_REPOSITORY="/backup/restic"
|
||||||
|
}
|
||||||
|
|
||||||
|
run_restore_to_staging() {
|
||||||
|
local name="$1"
|
||||||
|
shift
|
||||||
|
|
||||||
|
DRAGONS_STASH_DOCKER_CALLS="$TMP_ROOT/docker-calls-$name.log" \
|
||||||
|
"$@" "$RESTORE_SCRIPT" restore-to-staging snapshot-scope "$TMP_ROOT/staging/$name"
|
||||||
|
}
|
||||||
|
|
||||||
|
run_restore_live() {
|
||||||
|
local name="$1"
|
||||||
|
shift
|
||||||
|
|
||||||
|
DRAGONS_STASH_DOCKER_CALLS="$TMP_ROOT/docker-calls-$name.log" \
|
||||||
|
"$@" "$RESTORE_SCRIPT" restore-live snapshot-scope --confirm-replace-live-data
|
||||||
|
}
|
||||||
|
|
||||||
|
assert_restore_call_is_filtered() {
|
||||||
|
local calls="$1"
|
||||||
|
local restore_call
|
||||||
|
|
||||||
|
restore_call="$(grep $'backup\trestore\tsnapshot-scope' "$calls" || true)"
|
||||||
|
[[ -n "$restore_call" ]] || fail "restore command was not invoked"
|
||||||
|
[[ "$restore_call" == *$'--include\t/staging/backup-*/database.dump'* ]] || fail "database dump include filter missing"
|
||||||
|
[[ "$restore_call" == *$'--include\t/staging/backup-*/manifest\t'* ]] || fail "manifest include filter missing"
|
||||||
|
[[ "$restore_call" == *$'--include\t/staging/backup-*/manifest/**'* ]] || fail "manifest subtree include filter missing"
|
||||||
|
[[ "$restore_call" == *$'--include\t/data/tdlib-worker\t'* ]] || fail "worker TDLib root include filter missing"
|
||||||
|
[[ "$restore_call" == *$'--include\t/data/tdlib-worker/**'* ]] || fail "worker TDLib subtree include filter missing"
|
||||||
|
[[ "$restore_call" == *$'--include\t/data/tdlib-bot\t'* ]] || fail "bot TDLib root include filter missing"
|
||||||
|
[[ "$restore_call" == *$'--include\t/data/tdlib-bot/**'* ]] || fail "bot TDLib subtree include filter missing"
|
||||||
|
[[ "$restore_call" != *"/data/uploads"* ]] || fail "restore includes uploads path"
|
||||||
|
}
|
||||||
|
|
||||||
|
assert_restore_uses_include_filters() {
|
||||||
|
local calls="$TMP_ROOT/docker-calls-safe.log"
|
||||||
|
|
||||||
|
setup_fake_environment
|
||||||
|
run_restore_to_staging safe bash >/dev/null
|
||||||
|
|
||||||
|
assert_restore_call_is_filtered "$calls"
|
||||||
|
}
|
||||||
|
|
||||||
|
assert_live_restore_uses_include_filters() {
|
||||||
|
local calls="$TMP_ROOT/docker-calls-live.log"
|
||||||
|
|
||||||
|
setup_fake_environment
|
||||||
|
run_restore_live live bash >/dev/null
|
||||||
|
|
||||||
|
assert_restore_call_is_filtered "$calls"
|
||||||
|
}
|
||||||
|
|
||||||
|
assert_unexpected_volume_content_is_rejected() {
|
||||||
|
local output="$TMP_ROOT/unexpected-output.log"
|
||||||
|
|
||||||
|
setup_fake_environment
|
||||||
|
if RESTORE_ASSERT_CREATE_UNEXPECTED=1 run_restore_to_staging unexpected bash >"$output" 2>&1; then
|
||||||
|
fail "restore accepted unexpected restored data volume content"
|
||||||
|
fi
|
||||||
|
grep -Eq 'Unexpected restored data volume content|Refusing restore' "$output" \
|
||||||
|
|| fail "restore rejected unexpected content without an explicit guard message"
|
||||||
|
}
|
||||||
|
|
||||||
|
assert_restore_uses_include_filters
|
||||||
|
assert_live_restore_uses_include_filters
|
||||||
|
assert_unexpected_volume_content_is_rejected
|
||||||
|
|
||||||
|
printf 'restore-path assertions passed\n'
|
||||||
@@ -5,6 +5,15 @@ readonly CONFIRM_REPLACE_LIVE_DATA="--confirm-replace-live-data"
|
|||||||
readonly STAGING_CONTAINER_ROOT="/staging"
|
readonly STAGING_CONTAINER_ROOT="/staging"
|
||||||
readonly BACKUP_CONTAINER_ROOT="/backup"
|
readonly BACKUP_CONTAINER_ROOT="/backup"
|
||||||
readonly -a LIVE_SERVICES=(app worker bot)
|
readonly -a LIVE_SERVICES=(app worker bot)
|
||||||
|
readonly -a RESTORE_INCLUDE_FILTERS=(
|
||||||
|
--include "/staging/backup-*/database.dump"
|
||||||
|
--include "/staging/backup-*/manifest"
|
||||||
|
--include "/staging/backup-*/manifest/**"
|
||||||
|
--include "/data/tdlib-worker"
|
||||||
|
--include "/data/tdlib-worker/**"
|
||||||
|
--include "/data/tdlib-bot"
|
||||||
|
--include "/data/tdlib-bot/**"
|
||||||
|
)
|
||||||
|
|
||||||
RESTORED_DUMP=""
|
RESTORED_DUMP=""
|
||||||
RESTORED_TDLIB_WORKER=""
|
RESTORED_TDLIB_WORKER=""
|
||||||
@@ -80,6 +89,13 @@ backup_restic() {
|
|||||||
docker compose --profile backup run --rm --no-deps backup "$@"
|
docker compose --profile backup run --rm --no-deps backup "$@"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
restore_snapshot_subset() {
|
||||||
|
local snapshot_id="$1"
|
||||||
|
local container_staging_dir="$2"
|
||||||
|
|
||||||
|
backup_restic restore "$snapshot_id" --target "$container_staging_dir" "${RESTORE_INCLUDE_FILTERS[@]}"
|
||||||
|
}
|
||||||
|
|
||||||
canonical_staging_root() {
|
canonical_staging_root() {
|
||||||
realpath -m -- "$BACKUP_STAGING_PATH"
|
realpath -m -- "$BACKUP_STAGING_PATH"
|
||||||
}
|
}
|
||||||
@@ -131,6 +147,48 @@ verify_custom_dump() {
|
|||||||
backup --list /restore/database.dump >/dev/null
|
backup --list /restore/database.dump >/dev/null
|
||||||
}
|
}
|
||||||
|
|
||||||
|
reject_unexpected_restored_content() {
|
||||||
|
local staging_dir="$1"
|
||||||
|
local backup_directory="$2"
|
||||||
|
local entry
|
||||||
|
local name
|
||||||
|
local -a unexpected_entries=()
|
||||||
|
|
||||||
|
while IFS= read -r -d '' entry; do
|
||||||
|
name="$(basename -- "$entry")"
|
||||||
|
case "$name" in
|
||||||
|
data|staging) ;;
|
||||||
|
*) unexpected_entries+=("$entry") ;;
|
||||||
|
esac
|
||||||
|
done < <(find "$staging_dir" -mindepth 1 -maxdepth 1 -type d -print0)
|
||||||
|
|
||||||
|
if [[ -d "$staging_dir/data" ]]; then
|
||||||
|
while IFS= read -r -d '' entry; do
|
||||||
|
name="$(basename -- "$entry")"
|
||||||
|
case "$name" in
|
||||||
|
tdlib-worker|tdlib-bot) ;;
|
||||||
|
*) unexpected_entries+=("$entry") ;;
|
||||||
|
esac
|
||||||
|
done < <(find "$staging_dir/data" -mindepth 1 -maxdepth 1 -print0)
|
||||||
|
fi
|
||||||
|
|
||||||
|
while IFS= read -r -d '' entry; do
|
||||||
|
name="$(basename -- "$entry")"
|
||||||
|
case "$name" in
|
||||||
|
database.dump|manifest) ;;
|
||||||
|
*) unexpected_entries+=("$entry") ;;
|
||||||
|
esac
|
||||||
|
done < <(find "$backup_directory" -mindepth 1 -maxdepth 1 -print0)
|
||||||
|
|
||||||
|
if ((${#unexpected_entries[@]})); then
|
||||||
|
printf 'Refusing restore: unexpected restored data volume content found under %s:\n' "$staging_dir" >&2
|
||||||
|
for entry in "${unexpected_entries[@]}"; do
|
||||||
|
printf ' - %s\n' "${entry#"$staging_dir"/}" >&2
|
||||||
|
done
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
validate_restored_tree() {
|
validate_restored_tree() {
|
||||||
local staging_dir="$1"
|
local staging_dir="$1"
|
||||||
local -a backup_directories=("$staging_dir"/staging/backup-*)
|
local -a backup_directories=("$staging_dir"/staging/backup-*)
|
||||||
@@ -142,6 +200,7 @@ validate_restored_tree() {
|
|||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
backup_directory="${backup_directories[0]}"
|
backup_directory="${backup_directories[0]}"
|
||||||
|
reject_unexpected_restored_content "$staging_dir" "$backup_directory"
|
||||||
RESTORED_DUMP="$backup_directory/database.dump"
|
RESTORED_DUMP="$backup_directory/database.dump"
|
||||||
manifest="$backup_directory/manifest/backup-manifest.json"
|
manifest="$backup_directory/manifest/backup-manifest.json"
|
||||||
RESTORED_TDLIB_WORKER="$staging_dir/data/tdlib-worker"
|
RESTORED_TDLIB_WORKER="$staging_dir/data/tdlib-worker"
|
||||||
@@ -169,7 +228,7 @@ restore_to_staging() {
|
|||||||
container_staging_dir="$(container_staging_directory "$staging_dir")"
|
container_staging_dir="$(container_staging_directory "$staging_dir")"
|
||||||
prepare_fresh_staging_directory "$staging_dir"
|
prepare_fresh_staging_directory "$staging_dir"
|
||||||
verify_snapshot "$snapshot_id"
|
verify_snapshot "$snapshot_id"
|
||||||
backup_restic restore "$snapshot_id" --target "$container_staging_dir"
|
restore_snapshot_subset "$snapshot_id" "$container_staging_dir"
|
||||||
validate_restored_tree "$staging_dir"
|
validate_restored_tree "$staging_dir"
|
||||||
printf 'Staging restore verified: %s\n' "$staging_dir"
|
printf 'Staging restore verified: %s\n' "$staging_dir"
|
||||||
}
|
}
|
||||||
@@ -327,7 +386,7 @@ restore_live() {
|
|||||||
LIVE_RESTORE_ACTIVE=1
|
LIVE_RESTORE_ACTIVE=1
|
||||||
docker compose --profile full stop "${LIVE_SERVICES[@]}"
|
docker compose --profile full stop "${LIVE_SERVICES[@]}"
|
||||||
create_safety_dump
|
create_safety_dump
|
||||||
backup_restic restore "$snapshot_id" --target "$container_staging_dir"
|
restore_snapshot_subset "$snapshot_id" "$container_staging_dir"
|
||||||
validate_restored_tree "$LIVE_STAGING_DIR"
|
validate_restored_tree "$LIVE_STAGING_DIR"
|
||||||
worker_volume="$(compose_volume_name "$project_name" tdlib_state)"
|
worker_volume="$(compose_volume_name "$project_name" tdlib_state)"
|
||||||
bot_volume="$(compose_volume_name "$project_name" tdlib_bot_state)"
|
bot_volume="$(compose_volume_name "$project_name" tdlib_bot_state)"
|
||||||
|
|||||||
Reference in New Issue
Block a user