GeiserX/Telegram-Archive v5.0.0

on GitHub

⚠️ Major Release - Real-time Sync & Media Path Changes

This release introduces real-time message sync, zero-footprint mass operation protection, and consistent media path naming. Migration scripts are provided for existing installations.

Added

Real-time Listener Mode

• ENABLE_LISTENER - Background listener for instant sync (no waiting for scheduled backup)
• LISTEN_EDITS - Apply text edits to backed up messages in real-time
• LISTEN_DELETIONS - Mirror deletions from Telegram (with protection, see below)
• LISTEN_NEW_MESSAGES - Save new messages immediately (default: true)
• LISTEN_NEW_MESSAGES_MEDIA - Download media in real-time (default: false)
• LISTEN_CHAT_ACTIONS - Track chat photo/title changes, member joins/leaves
• LISTEN_ALBUMS - Detect and group album uploads together

Zero-Footprint Mass Operation Protection

• Sliding-window rate limiter protects against mass edit/deletion attacks
• MASS_OPERATION_THRESHOLD - Operations before protection triggers (default: 10)
• MASS_OPERATION_WINDOW_SECONDS - Time window for counting operations (default: 30)
• When triggered, ALL pending operations are discarded - zero changes to your backup

Priority Chats

• PRIORITY_CHAT_IDS - Process these chats FIRST in all backup/sync operations
• Useful for ensuring important chats are always backed up before others

Viewer Enhancements

• WebSocket real-time updates - New messages appear instantly without refresh
• Infinite scroll - Cursor/keyset pagination for large chats
• Album grid display - Photo/video albums shown as grids like Telegram
• Compact stats dropdown - Stats moved to dropdown next to header
• Per-chat stats - Message count, media count, total size per chat
• "Real-time sync" indicator - Shows when listener is active
• SHOW_STATS - Hide stats dropdown for restricted viewers (default: true)

Web Push Notifications

• PUSH_NOTIFICATIONS - Notification mode: off, basic, full (default: basic)
  • off - No notifications at all
  • basic - In-browser notifications (tab must be open)
  • full - Persistent Web Push (works even when browser is closed!)
• Auto-generated VAPID keys - Stored in database, persist across restarts
• Subscription management - Subscriptions survive container restarts and updates
• Automatic cleanup - Expired subscriptions removed automatically
• Optional custom VAPID keys via VAPID_PRIVATE_KEY, VAPID_PUBLIC_KEY, VAPID_CONTACT

Migration Scripts

• scripts/migrate_media_paths.py - ⚠️ REQUIRED - Normalizes media folder names to use marked IDs
• scripts/update_media_sizes.py - ⚠️ REQUIRED - Populates file_size for accurate stats
• scripts/detect_albums.py - ⚠️ HIGHLY RECOMMENDED - Detect albums in existing backups for album grid display
• scripts/deduplicate_media.py - ⚠️ HIGHLY RECOMMENDED - Global deduplication using symlinks (saves disk space)
• scripts/restore_chat.py - Repost archived messages to Telegram

Changed

• Shared Telethon client - Backup and listener share connection (avoids session DB locks)
• WAL mode for session DB - Better concurrency for Telethon session
• Media folder naming - Groups/channels now use marked IDs (e.g., -35258041/ not 35258041/)
• Bulk SQL operations - Migration scripts use single queries per batch (10-100x faster)

Fixed

• Media 404s due to inconsistent folder naming (positive vs negative IDs)
• Audio files served with wrong Content-Type (now audio/ogg, audio/mp3, etc.)
• Stats calculation error with Decimal types (JSON serialization)
• Session DB locking when running backup and listener simultaneously

⚠️ Migration Required

If upgrading from v4.x with existing data:

1. Run migration scripts (inside Docker container):

   # 1. Normalize media paths (REQUIRED)
   docker run --rm -e DB_TYPE=postgresql ... python -m scripts.migrate_media_paths
   
   # 2. Update file sizes for accurate stats (REQUIRED)
   docker run --rm -e DB_TYPE=postgresql ... python -m scripts.update_media_sizes
   
   # 3. Detect albums for grid display (HIGHLY RECOMMENDED)
   docker run --rm -e DB_TYPE=postgresql ... python -m scripts.detect_albums
   
   # 4. Deduplicate media files (HIGHLY RECOMMENDED)
   docker run --rm -e DB_TYPE=postgresql ... python -m scripts.deduplicate_media

2. Update docker-compose.yml with new env variables (see README)

See Upgrading to v5.0.0 below for detailed instructions.

Related Issues

• Fixes #12 - Timezone-aware datetime sorting
• Fixes #20 - Real-time sync for edits/deletions
• Fixes #21 - Mass operation protection
• Fixes #22 - Media path consistency

📋 Full changelog: docs/CHANGELOG.md