⚠️BREAKING CHANGES
- Stricter config validation logic. CDL will verify that every config value and CLI argument is valid before running
- All dedupe and hashing functions has been replaced.
xxh128is now the default hashing algorithm, replacingmd5- By default, all v5 hashes will be ignored for deduplication matches. You can re-enable md5 with
--add-md5-hash- It's no longer possible to keep the newest file while deduplicating.
- Every file previously downloaded is taken into account for deduplication. Files no longer need to currently exists on disk
- Removed
--sort-cdl-onlyoption
- The following arguments are deprecated and will be removed in a future version:
--download-folder--download-all-configs--sort-all-configs--sort-all-downloads--main-log-filename--last-forum-post-filename--unsupported-urls-filename--download-error-urls-filename--scrape-error-urls-filename
This update introduces the following changes:
- Add Python 3.13 support
- Add requests caching to speed up future download sessions for forums, Coomer, and Kemono.
- Update Kemono & Coomer crawlers to use API calls
- Update GoFile crawler to the new site design
- Update Bunkr crawler to handle new UUID filename format
- Rework entire deduplication and hashing logic. Now multiple hashing algorithms are supported
- Use Pydantic for config and CLI values validation
- Allow downloading duplicate files on the same run with
--ignore-history - Fix sorting functionality
- Allow running all configs sequentially using
--config ALL - Better cookies management system
- Added option to clear cookies for selected websites
- General improvements and bug fixes