v1.1.0-beta.016 - perf & fix: Hybrid debounce metadata syncs and resolve issue year mismatches
This update resolves a massive N² performance bottleneck during batch imports and fixes year-variance filtering failures that prevented long-running series from downloading correctly.
🐛 The Problem
- Importing multiple issues within the same series triggered redundant API requests and intensive XML write operations for every individual issue.
- Massive batch downloads could cause a Disk I/O Avalanche where earlier issues in the batch were repeatedly unzipped and rewritten with ComicInfo.xml files as the batch progressed.
- Bulk series requests were previously hardcoding the parent series' start year into the search parameters for every individual issue within that series, causing them to fail strict year-variance checks when querying GetComics and Prowlarr.
✨ The Solution
- By utilizing a Hybrid Debounce in BullMQ (sliding time windows + completion cleanup flags), metadata operations are safely grouped per series for active batch imports. This guarantees straggler files aren't missed while preventing API bans and Disk I/O thrashing.
- The automation engine now dynamically looks up and applies the exact publication year for each specific issue during a bulk download, allowing issues that cross year boundaries to successfully bypass variance filters.
🛠️ Under the Hood
- Replaced the synchronous syncSeriesMetadata execution in src/lib/importer.ts with a delayed background queue job.
- Implemented a 1-minute rolling time window (Math.floor(Date.now() / 60000)) for METADATA_SYNC with a 60-second delay, and added removeOnComplete: true to prevent the Active Execution Blindspot.
- Applied a 15-minute rolling window (Math.floor(Date.now() / 900000)) and a 5-minute delay (delay: 300000) to the EMBED_METADATA queue additions in src/lib/metadata-fetcher.ts to prevent Disk I/O avalanches when writing XML files.
- Mocked @/lib/queue in tests/lib/importer.test.ts to prevent dynamic IORedis instantiations from hanging the test environment, and updated assertions to validate the new BullMQ deduplication logic.
- Added the looseCompareIssue local helper function to safely parse and match issue numbers without formatting collisions (e.g., treating 01 and 1 equally).
- Implemented a database lookup inside executeSearchAndDownload to retrieve the specific issue's releaseDate using the volumeId and metadataSource.
- Replaced the static year variable with the newly resolved dynamicYear when calling generateSearchQueries, GetComicsService.search, and ProwlarrService.searchComics.