datawhores/OF-Scraper 3.14

Release 3.14

on GitHub

🚀 Features & UI Overhauls

• Massive TUI Dashboard Upgrade: Implemented a new 4-column responsive header with tabbed navigation, providing detailed real-time trackers, keyboard shortcut legends, and improved table styling.
• Smart Cart Selection: Added 'New Only' selection logic and updated keybindings for selecting all cart items in the TUI.
• Unavailable Content Handling: Added config option to skip unavailable content from expired subscriptions, complete with a grace period customizable via an environment variable.
• Auto-Stray Media Cleaner: Improved automatic setting of scraping date, by integrating a background cleaner that detects and handles stray/ghost media items during scrapes.
• Enhanced User Status Logs: Split active and expiration status into distinct logs and added emoji indicators for quick visual account status checks.

⚡ Performance & Core Refactoring

• API Pagination & Chunking: Completely overhauled API pagination, chunking, and "ghost traps" for much more robust and reliable data retrieval.
• SQLite Optimizations: Tuned SQLite settings to dramatically improve concurrent access and memory mapping performance.
• Logging Hierarchy Overhaul: Refactored the internal logging hierarchy, changing default subprocess log levels to DEBUG to eliminate console spam on standard runs.
• Environment Constants: Refactored code to utilize constants in environment variables wherever possible for better configuration management.

🐛 Bug Fixes

• Download & Docker Stability: Fixed the "11/10" progress display error, and patched potential stalling issues caused by FFmpeg and after_download scripts.
• Backup Permissions (EPERM) Crash: Added a copyfile fallback to prevent permission crashes during database backups (PR #598).
• Cache & API Fixes: Resolved stale cache logic for check_modes, optimized empty API checks, and fixed an issue where the "previously downloaded" filter was incorrectly grabbing unlocked media.
• Variable & Attribute Errors: enforced strict string typing for arrow date handling.
• Timeline API: Fixed task retrieval and get_after_precheck logic in the timeline API.

🗄️ Database Changes

• Added the is_deleted column to messages/media and optimized unlocked media queries.