🚀 Major Overhaul: Public Release Candidate
This release marks the transition from a personal script to a production-grade, public-ready application. It introduces a modular architecture, robust configuration options, and significant safety improvements for Docker environments.
✨ New Features
- CLI Support: Now supports command-line arguments!
--dry-run: Scan without saving to DB.--limit 10: Limit imports per site.--sites my_list.json: Load custom site lists.--no-cache: Force a fresh crawl.
- Smart Configuration: Automatically loads sites from
sites.json,.env, or falls back to a curated default list. - Visual Feedback: Added
tqdmprogress bars for a cleaner CLI experience (auto-hides in Docker logs). - Tandoor Support: Fully restored and optimized support for Tandoor alongside Mealie.
🛡️ Reliability & Safety
- Graceful Shutdown: Implemented
GracefulKillerto catch DockerSIGTERMsignals. The script now finishes its current loop and saves data before exiting, preventing data corruption during restarts. - Anti-Blocking (Jitter): Added randomized sleep timers ("Human Jitter") to
RateLimiterto avoid detection by aggressive firewalls. - Robust Parsing: Switched to
BeautifulSoupXML parsing for sitemaps, fixing issues with nested WordPress sitemaps and namespaces.
🐛 Fixes
- Fixed logging lag in Docker by forcing unbuffered output.
- Fixed crash when
sites.jsonwas missing. - Standardized version numbering.
🐳 Docker
- Updated image supports
linux/amd64andlinux/arm64(Raspberry Pi ready).