datawhores/OF-Scraper 3.13.5

Release 3.13.5

on GitHub

Date: July 13, 2025

This release delivers significant advancements in progress tracking, metadata management, and overall system robustness.

✨ New Features

• Comprehensive Progress System Overhaul

  • Introduced additional context providers for managing special sessions.
  • Added extra sleepers and configurable values for finer control over rate limits.
  • Implemented dedicated download and metadata request rate limiting.
  • Simplified Progress Tracking code for enhanced stability.

• Advanced Metadata Management

  • New properties for metadata status within the post class.
  • Enhanced media acquisition for metadata mode, alongside a revamped system for marking downloads and metadata as complete.
  • Added metadata status support for external scripts.
  • Integrated statistics tracking directly into metadata operations.

• Centralized Statistics Manager (StatsManager)

  • Introduced a new, comprehensive StatsManager for robust data collection.
  • Refactored and moved all existing managers into their own dedicated folder for better organization.
  • StatsManager is now utilized across download, check, and manual modes.
  • Extended statistics tracking to include 'like/unlike' actions.

• Improved Scripting & Automation

  • Integrated after_download_script functionality into text downloads.
  • Automatic detection of Python files for external scripts.
  • Introduced after_like_script to address liking-related issues.
  • General enhancements to after_download_script functionality.
  • Added support for logging script output, based on desired log level.

• Discord Integration

  • Discord integration now runs in a separate thread, improving overall application responsiveness.

• User Interface & Experience Enhancements

  • Current filters are now displayed automatically when setfilter is used.
  • last seen filter now includes a default value for convenience.

🐛 Bug Fixes

• Resolved an issue causing forced application closure.
• Fixed problems related to prompt menu model choices.
• Ensured statistics are correctly cleared on looping actions, preventing data inaccuracies.
• Corrected function usage within modelmanager.
• Standardized parameter names from action to actions.
• Addressed cases where posts might not be passed to after_download_script.
• Fixed incorrect URLs for protected content.
• Corrected issues with scrape labels.
• Ensured minimum sleep duration is respected.
• Fixed problems in both manual download and text download modes.
• Accurately filtered post IDs and media IDs.
• Resolved post check length filter issues.
• General fixes for manual mode operations.
• Standardized postid to post_id.
• Resolved issues with empty user lists.
• Synchronized user list keys for consistency.
• Fixed the switch list functionality.
• Ensured empty values are ignored in user list filters.
• Fixed various argument spelling and related configuration problems.
• Corrected last seen filters.
• Prevented special characters from being filtered out of the model list.
• Modified list prompt behavior.
• Fixed issues where arguments were not updated on change.
• Ensured JSON lists are converted to strings when required.
• Updated internal settings correctly after menu prompts.
• Resolved issues with marking download status.
• Fixed script naming issues.
• Implemented general fixes for scripts.

⚙️ Refactoring & Improvements

• Streamlined Git actions.
• Transitioned from hatch-vcs to version-pioneer for version management.
• Added a dedicated versioning script.
• Updated sensitive formatter for improved data handling.
• Updated Discord formatter.
• Consolidated paid scrape code with metadata functionality for reusability.
• Implemented queue clearing and process marking after each cycle.
• Adopted the new ModelManager API for various operations, including setting up model queues and managing resets on loop completion.
• Standardized naming conventions from .action to .actions.
• Improved script flow and ensured proper reset of processed models.
• Now displays all internal processed/unprocessed lists held within ModelManager.
• Removed stop code usage from texthandler.
• Added specific keys and additional inputs for arguments.
• Adjusted and reduced some sleep durations for better performance.
• Relocated models class to the of_classes folder for better structure.
• Utilized log levels for subprocess logging.
• Revamped check mode to support multiple users more effectively.
• Prevented redundant calls to after_download script.
• Utilized PostCollection in check mode and enhanced it with additional functions and attributes.
• Leveraged settings to read arguments.
• Centralized filepath management within the media class.
• Improved mark download status and created a specific after_download_action_script.
• Added manual search/add name functionality to ModelManager.