feat(worker): retry old SkippedPackages + prefer specific topics over General

Three connected safeguards driven by user feedback after deploying the
incremental watermark and repost-detection fixes.

1. SkippedPackage retry pass (watermark pull-back)
   The auto-retry chain (d99a506 + watermark cap) only works for failures
   that occur AFTER the fix is deployed. Pre-existing SkippedPackages may
   sit below the current watermark — example from prod: secondary's
   "Turnbase Delivery Folder.7z" at msgId 37,109,104,640 vs watermark
   37,111,201,792. The auto-retry never sees it.

   Before scanning each channel/topic, we now query SkippedPackages with
   attemptCount < cap for that scope and pull the watermark back to
   (lowestSkippedMsgId - 1n) when needed. Both forum and non-forum
   branches handle this.

2. Topic scan order: specific topics first, General last
   In forum channels, files often appear in both a specific topic (e.g.,
   "Artisan Guild January 2022") AND in General. The first encounter
   created the Package and locked in the topic context. If we happened
   to scan General first, the Package recorded the less-informative
   topic.

   We now sort topics so General is processed last. New Packages get
   the more specific topic name as their context by default.

3. Backfill specific topic on existing Packages
   For Packages that were already created with General topic context,
   when findRepostedPackage matches and the current scan is in a more
   specific topic, update the existing Package's sourceTopicId (and
   creator, if it was derived from "General") to the more specific one.
   Audit log shows both old and new topic IDs.

The findRepostedPackage query also got an ORDER BY so it returns the
most-specific existing match (non-null sourceTopicId first) when
multiple Packages share the same filename + size in a channel — giving
the audit log richer context.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-23 09:02:54 +02:00
parent ff4e150544
commit 901f32ff41
2 changed files with 200 additions and 7 deletions

View File

@@ -207,7 +207,11 @@ export async function findRepostedPackage(
sourceChannelId: string,
fileName: string,
fileSize: bigint
): Promise<{ id: string; destMessageId: bigint | null } | null> {
): Promise<{
id: string;
destMessageId: bigint | null;
sourceTopicId: bigint | null;
} | null> {
return db.package.findFirst({
where: {
sourceChannelId,
@@ -215,7 +219,12 @@ export async function findRepostedPackage(
fileSize,
destMessageId: { not: null },
},
select: { id: true, destMessageId: true },
// Prefer the existing Package with the most specific (non-NULL)
// sourceTopicId, so when the user is re-scanning and the file already
// exists in a specific topic, the audit log shows the most informative
// match. NULLS LAST in DESC order achieves that for any non-null IDs.
orderBy: { sourceTopicId: { sort: "desc", nulls: "last" } },
select: { id: true, destMessageId: true, sourceTopicId: true },
});
}
@@ -702,6 +711,63 @@ export async function deleteSkippedPackage(
});
}
/**
* Find SkippedPackages for a given account+channel that are still eligible
* for auto-retry (attemptCount below the cap). Used at the start of a scan
* to pull the watermark back so we don't strand failed messages forever
* after the watermark has advanced past them.
*
* For non-forum channels, pass `topicId: null` to get rows with NULL topic.
* For forum channels, pass the topic ID to scope to that topic only.
*/
export async function getRetryableSkippedMessageIds(args: {
accountId: string;
sourceChannelId: string;
topicId: bigint | null;
cap: number;
}): Promise<bigint[]> {
const rows = await db.skippedPackage.findMany({
where: {
accountId: args.accountId,
sourceChannelId: args.sourceChannelId,
sourceTopicId: args.topicId,
attemptCount: { lt: args.cap },
},
select: { sourceMessageId: true },
orderBy: { sourceMessageId: "asc" },
});
return rows.map((r) => r.sourceMessageId);
}
/**
* Update a Package's source topic when a more specific topic context is
* discovered for the same content. Used when findRepostedPackage matches
* an existing Package whose topic is less specific (e.g., "General") than
* the topic we just encountered the file in.
*
* Also updates the creator if the new topic name is more informative than
* the existing creator (i.e., the existing creator was derived from a
* less-specific topic name like "General").
*/
export async function updatePackageTopicContext(
packageId: string,
newTopicId: bigint,
newTopicName: string | null
): Promise<void> {
await db.package.update({
where: { id: packageId },
data: {
sourceTopicId: newTopicId,
// Only overwrite creator if the new topic name is meaningful (non-empty,
// non-General). Keeps explicit creator values from filename or admin
// input intact.
...(newTopicName && newTopicName !== "General"
? { creator: newTopicName }
: {}),
},
});
}
export async function createOrFindPackageGroup(input: {
mediaAlbumId: string;
sourceChannelId: string;