mirror of
https://github.com/xCyanGrizzly/DragonsStash.git
synced 2026-09-21 13:31:42 +00:00
docs: amend 7z ranged design for encoded (LZMA) headers
Live spike showed start+end sparse reconstruction is insufficient for encoded-header 7z; fetch the mid-file packed header region as a 3rd region (parse PackInfo). Adds Task 4b. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1054,6 +1054,18 @@ Spot-check: pick one backfilled multipart RAR, compare its `package_files` count
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Task 4b: 7z encoded-header support (added 2026-07-27 after the Task 4 live spike)
|
||||||
|
|
||||||
|
The Task 4 deploy revealed the two-region (start+end) 7z reconstruction is rejected by `7z l`
|
||||||
|
for archives with an **encoded/LZMA-compressed header** — the packed header stream lives mid-file,
|
||||||
|
not at EOF. Fix: parse the encoded header's `PackInfo` and fetch that packed region as a third
|
||||||
|
sparse region. Full brief with exact code + tests: `.superpowers/sdd/task-4b-brief.md`. Adds
|
||||||
|
`read7zNumber` + `locate7zEncodedHeaderPack` to `sevenz-ranged.ts` and branches
|
||||||
|
`readSevenZListingRanged` on the next-header type (`0x01` plain → 2 regions; `0x17` encoded → 3
|
||||||
|
regions; else → null/fallback). Sequenced between Task 5 and Task 6; re-verified live at Task 8.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Self-Review
|
## Self-Review
|
||||||
|
|
||||||
- **Spec coverage:** 7z ranged read (Tasks 2-4) ✓; RAR walk incl. multipart (Tasks 5-7) ✓; sparse+CLI reconstruction (Task 1) ✓; dispatcher + downstream unchanged (Tasks 4, 8) ✓; size-capped full-download fallback + notification (Task 4) ✓; scanned + destination reads format-aware (Tasks 4, 8) ✓; spike via 7z-first-behind-fallback (Task 4 gate) ✓; unit tests + live verification (throughout) ✓; local deploy recipe (Tasks 4, 8) ✓.
|
- **Spec coverage:** 7z ranged read (Tasks 2-4) ✓; RAR walk incl. multipart (Tasks 5-7) ✓; sparse+CLI reconstruction (Task 1) ✓; dispatcher + downstream unchanged (Tasks 4, 8) ✓; size-capped full-download fallback + notification (Task 4) ✓; scanned + destination reads format-aware (Tasks 4, 8) ✓; spike via 7z-first-behind-fallback (Task 4 gate) ✓; unit tests + live verification (throughout) ✓; local deploy recipe (Tasks 4, 8) ✓.
|
||||||
|
|||||||
@@ -82,14 +82,29 @@ the end; the signature header stores the end header's location.
|
|||||||
1. Ranged-read `[0, 32)`; validate magic `37 7A BC AF 27 1C`. Read LE `uint64`
|
1. Ranged-read `[0, 32)`; validate magic `37 7A BC AF 27 1C`. Read LE `uint64`
|
||||||
`NextHeaderOffset` (byte 12) and `NextHeaderSize` (byte 20). End header is at absolute offset
|
`NextHeaderOffset` (byte 12) and `NextHeaderSize` (byte 20). End header is at absolute offset
|
||||||
`32 + NextHeaderOffset`, length `NextHeaderSize`.
|
`32 + NextHeaderOffset`, length `NextHeaderSize`.
|
||||||
2. Ranged-read `[32 + NextHeaderOffset, NextHeaderSize)`.
|
2. Ranged-read `[32 + NextHeaderOffset, NextHeaderSize)` — the "next header".
|
||||||
3. `listFromSparse` with regions `{0: sigHeader}` and `{32+NextHeaderOffset: endHeader}`, total
|
3. Branch on the next header's first byte (a 7z property id):
|
||||||
= file size, runner = `7z l`. `parse7zOutput` yields names+sizes (`crc32: null`, as today).
|
- **`0x01` (kHeader, plain/uncompressed header):** two regions suffice —
|
||||||
4. Return `null` on bad magic / read failure / CLI error.
|
`{0: sigHeader}` and `{32+NextHeaderOffset: endHeader}`.
|
||||||
|
- **`0x17` (kEncodedHeader, LZMA-compressed header):** the next header is only a *descriptor*
|
||||||
|
whose `PackInfo` points at a packed header stream stored **in the middle** of the file (not
|
||||||
|
at EOF). Parse the descriptor's `StreamsInfo → kPackInfo (0x06)` to read `PackPos` and the
|
||||||
|
`PackSize`s (7z variable-length "numbers"; sum them). Ranged-read the contiguous packed
|
||||||
|
region `[32 + PackPos, Σ PackSize)` and add it as a **third** sparse region. `7z l` then
|
||||||
|
decodes the header from that region.
|
||||||
|
- **anything else:** return `null` (→ fallback).
|
||||||
|
4. `listFromSparse` with the 2 or 3 regions, runner = `7z l`. `parse7zOutput` yields
|
||||||
|
names+sizes (`crc32: null`, as today). Return `null` on bad magic / read failure / CLI error.
|
||||||
|
|
||||||
`7z l` seeks to the end header (incl. decoding an LZMA-encoded header via the real binary) and
|
**Why the third region is required (spike finding, 2026-07-27):** the original two-region
|
||||||
never reads the packed-stream gap, so the sparse holes are untouched. All 7z placeholders are
|
(start+end) reconstruction was proven insufficient in a live test — `7z l` rejected it with
|
||||||
single-part.
|
"Cannot open the file as [7z] archive" because these archives use an *encoded* header whose
|
||||||
|
compressed bytes live in a packed stream in the file body (a sparse hole), not at EOF. The
|
||||||
|
`0x17` branch fetches exactly that packed region. `7z l` still never touches the file-data
|
||||||
|
packed streams (it only lists), so those gaps stay sparse. The `read7zNumber` reader (7z's
|
||||||
|
base-128-ish variable-length integer with a first-byte length mask) and the minimal
|
||||||
|
`kPackInfo` walk are the only new 7z binary parsing; `7z l` still does the actual file listing.
|
||||||
|
All 7z placeholders are single-part.
|
||||||
|
|
||||||
### RAR ranged listing
|
### RAR ranged listing
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user