Commit Graph
29 Commits
Author SHA1 Message Date
adminandClaude Opus 4.6 4baf5aad83 feat: capture media_album_id from TDLib messages during scanning
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 22:23:47 +01:00
adminandClaude Opus 4.6 ad7790c07b feat: add mediaAlbumId to TelegramMessage and TelegramPhoto interfaces
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 22:23:11 +01:00
adminandClaude Opus 4.6 e4398caebe feat: add PackageGroup schema for album-based file grouping
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 22:21:52 +01:00
adminandClaude Opus 4.6 6eb7129637 docs: add package grouping design spec and implementation plan
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 21:40:13 +01:00
adminandClaude Opus 4.6 d6386209be fix: improve download/upload reliability and fix FILE_PARTS_INVALID
- Add downloadStarted flag to prevent false "stopped unexpectedly" errors
  when TDLib emits initial updateFile before download is active
- Add 5-minute stall detection for both downloads and uploads
- Reduce max split part size from 2GiB to 1950MiB to stay under
  Telegram's internal upload part count limits
- Increase timeouts from max(10min, 15min/GB) to max(15min, 20min/GB)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 21:40:00 +01:00
adminandClaude Opus 4.6 fe28c31b9e fix: improve worker error handling and reliability
continuous-integration/drone/push Build is passing
1. Distinguish failure reasons: inspect error messages to label skipped
   packages as DOWNLOAD_FAILED, UPLOAD_FAILED, or EXTRACT_FAILED
   instead of catch-all DOWNLOAD_FAILED.

2. Detect orphaned uploads: before uploading, check if the same content
   hash already has a successful upload on the destination channel. Reuse
   the existing message ID instead of re-uploading (prevents duplicates
   when worker crashed between upload and DB write).

3. Increase timeouts: download from max(5min, GB*10min) to
   max(10min, GB*15min), upload from GB*10min to GB*15min.
   Prevents premature timeouts on slow connections.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 02:37:23 +01:00
adminandClaude Opus 4.6 55bdf3c890 feat: add migration for skipped_packages table
continuous-integration/drone/push Build is passing
Hand-written migration matching the SkippedPackage model in schema.prisma.
Will be applied automatically by docker-entrypoint.sh via `prisma migrate deploy`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 21:27:25 +01:00
adminandClaude Opus 4.6 5506c7d91b fix: resolve build errors in retry actions and package detail query
- Replace BigInt literal `1n` with `BigInt(1)` for ES target compatibility
- Add default matchedFileCount/matchedByContent to getPackageById return

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 16:40:14 +01:00
adminandClaude Opus 4.6 5a3550fa10 feat: add skipped/failed packages tab to STL files page
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 16:36:39 +01:00
adminandClaude Opus 4.6 ad3d42a997 feat: add skipped/failed packages table UI components
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 16:24:32 +01:00
adminandClaude Opus 4.6 dd0d246a77 feat: highlight matching files in package drawer when opened from search
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 16:22:50 +01:00
adminandClaude Opus 4.6 dcc1c97053 feat: add retry server actions for skipped/failed packages
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 16:22:31 +01:00
adminandClaude Opus 4.6 71c3228e44 feat: add query functions for listing skipped/failed packages
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 16:19:42 +01:00
adminandClaude Opus 4.6 094001f9f7 feat: show file match count badge in search results
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 16:19:40 +01:00
adminandClaude Opus 4.6 0faacc214b feat: return per-package file match counts from searchPackages
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 16:17:42 +01:00
adminandClaude Opus 4.6 d53e581623 feat: record skipped/failed archives in database for UI visibility
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 16:16:12 +01:00
adminandClaude Opus 4.6 780e6200d8 feat: add SkippedPackage model for tracking skipped/failed archives
Adds SkipReason enum (SIZE_LIMIT, DOWNLOAD_FAILED, EXTRACT_FAILED,
UPLOAD_FAILED) and SkippedPackage model with unique constraint on
(sourceChannelId, sourceMessageId). Reverse relations added to
TelegramAccount and TelegramChannel.

Note: Run `npx prisma migrate dev --name add-skipped-packages` when
database is available to create the migration.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 16:11:56 +01:00
adminandClaude Opus 4.6 9642adaba7 feat: raise default ingestion size limit from 4GB to 200GB
Multipart archives where individual parts fit under Telegram's 2GB limit
but total size exceeds 4GB were being silently skipped. These can now be
processed up to 200GB total, with each part uploading directly.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 16:01:41 +01:00
adminandClaude Opus 4.6 9bc9271f11 fix: auto-create READER links when enabling a source channel
continuous-integration/drone/push Build is passing
toggleChannelActive only flipped isActive but never created the
AccountChannelMap READER link needed by the worker. Channels enabled
via the toggle (rather than the channel picker) were invisible to the
scanner. Now auto-creates READER links for all active authenticated
accounts when a SOURCE channel is enabled.

Also ran a one-time DB fix to backfill READER links for the 14 active
channels that were missing them.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 23:29:52 +01:00
adminandClaude Opus 4.6 bd358a134b fix: add tags to new_package payload type assertion in bot
continuous-integration/drone/push Build is passing
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 23:07:22 +01:00
adminandClaude Opus 4.6 1425db8774 fix: use loadChats API and load chat folders for complete chat discovery
continuous-integration/drone/push Build is failing
- Switch from getChats pagination to loadChats (the TDLib-recommended
  API) which properly loads all chats into TDLib's cache and signals
  completion with a 404 error
- Discover and load chat folders via getChatFolders so chats in
  user-created folders are included
- Load from main + archive + all folders in both worker startup and
  getAccountChats channel discovery
- After loading, use getChats with high limit to retrieve all cached IDs
- This ensures private chats, 1-on-1 conversations, Saved Messages,
  basic groups, and archived/folder chats are all discoverable

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 22:38:49 +01:00
adminandClaude Opus 4.6 aef76828ef fix: support large accounts and archived chats in channel discovery
continuous-integration/drone/push Build is failing
- Increase getChats pagination from 50 pages (5K chats) to 500 pages
  (50K chats) to support accounts with many channels/groups
- Load from both chatListMain AND chatListArchive so older/archived
  chats are discovered and scannable
- Deduplicate chat IDs across both lists
- Worker startup also loads both lists before scanning

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 19:50:14 +01:00
adminandClaude Opus 4.6 29e95f780c feat: support all chat types in channel discovery and enrich bot messages
Channel Discovery:
- Remove channel/supergroup filter from getAccountChats — all chat types
  (private, groups, Saved Messages, etc.) are now discoverable as sources
- Detect and label the self-chat as "Saved Messages" via getMe
- Update channel picker dialog to accept any chat type string

Bot Rich Messages:
- Enhance package send preview with creator, file count, tags, and source
  channel info in MarkdownV2 caption
- Include tags in new_package subscription notifications
- Expand getPendingSendRequest to fetch richer package data

Performance:
- Reviewed pipeline for many-channel load — getChats pagination fix and
  per-channel getChat pre-load from prior commit address the main concerns
- Channels with no new messages skip in 2-3 API calls

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 18:27:48 +01:00
adminandClaude Opus 4.6 5fd341dfc4 feat: fix channel scanning bugs, add package tags, and kickstarters tab
Bug fixes:
- Fix channels not being scanned by paginating TDLib getChats (was only
  loading first batch, additional channels were unknown to TDLib)
- Add per-channel getChat pre-load as safety net before scanning
- Fix preview pictures not loading by checking previewData instead of
  previewMsgId for hasPreview flag
- Prevent previewMsgId from being set when preview download fails

Package Tags:
- Add tags Text[] column to Package with migration backfilling from
  channel categories
- Worker auto-inherits source channel category as initial tag
- Tag filter dropdown and Tags column in STL Files table
- Server actions for individual and bulk tag editing

Kickstarters Tab:
- New KickstarterHost, Kickstarter, and KickstarterPackage models
- Full CRUD with delivery status, payment status, host management
- Package linking (many-to-many with existing packages)
- Sidebar entry with Gift icon
- Table with search, filters, modal forms

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 18:17:44 +01:00
admin 2763de2711 Fix multiple issues 2026-03-07 21:33:40 +01:00
admin faaefd6c31 fix: correct User table name in bot migration FK
The User model has no @@map so its table is "User" (PascalCase),
not "users". The FK reference was wrong.
2026-03-03 21:47:15 +01:00
admin f42c51db3b fix: remove url from schema datasource (Prisma 7.x)
Prisma 7.x no longer supports 'url' in the schema datasource block.
The URL is provided exclusively via prisma.config.ts.
2026-03-03 21:45:05 +01:00
admin ed5ff3db43 fix: exclude bot/ from Next.js build and lint
tsconfig.json included **/*.ts but only excluded worker/**, not bot/**.
This caused next build to type-check bot source files whose dependencies
aren't installed in the root node_modules, breaking CI.

- Add bot/** to tsconfig.json exclude array
- Add bot/** and worker/** to eslint globalIgnores
2026-03-03 21:42:34 +01:00
admin 575ffdbc31 feat: Docker audit + Telegram bot service + send UI
Docker:
- Harden docker-compose.yml: parameterized DB creds, required AUTH_SECRET,
  health checks, resource limits, network isolation, removed exposed DB port
- Add profiles (telegram/bot/full) so base 'docker compose up' needs only AUTH_SECRET
- Fix docker-entrypoint.sh: AUTH_SECRET startup guard
- Fix Dockerfile: copy prisma.config.ts + dotenv into production image
- Update .env.example with all new variables
- Update .dockerignore

Telegram Bot Service (bot/):
- TDLib-based bot using bot token auth (not HTTP Bot API)
- Commands: /search, /latest, /package, /link, /unlink, /subscribe, /unsubscribe
- pg_notify listener for send requests (bot_send) and new packages (new_package)
- Subscription-based notifications when matching packages arrive
- Dockerfile with multi-stage build (bookworm-slim for glibc/TDLib)

API & Database:
- Prisma: TelegramLink, BotSendRequest, BotSubscription models + migration
- POST /api/telegram/bot/send - queue package delivery to linked TG account
- GET /api/telegram/bot/send/[id] - poll send request status
- Server actions: generateTelegramLinkCode, unlinkTelegram, getBotSendHistory
- Worker: emit pg_notify('new_package') after creating packages

Frontend:
- Settings: TelegramLinkCard for account linking via one-time code
- STL table + drawer: SendToTelegramButton with send dialog and status polling
- Telegram admin: Bot Sends tab with delivery history table
- Shared SendHistoryRow type

README: Updated with bot docs, profiles, config vars, project structure
2026-03-03 21:36:57 +01:00