Changes
Technical Infrastructure Release - IMDB dataset optimization and streaming improvements
Overview
Infrastructure improvements for IMDB data processing with pre-filtered datasets and optimized streaming utilities.
Key Updates
IMDB Dataset Optimization
- Pre-filtered ratings dataset: 52% size reduction by targeting types
- Automated daily updates: GitHub Actions workflow generates fresh datasets daily
- Self-hosted delivery: Switch to pre-processed datasets hosted in repository
Performance Improvements
- Memory optimization: Reduced streaming memory usage by ~50% through direct response streaming
- Enhanced error handling: Fixed abort cancellation and added fail-fast validation
- Data integrity: Preserved TSV field whitespace and improved error detection
Technical Changes
Streaming Utilities
- Direct response.body streaming instead of double-buffering
- Proper AbortError handling bypassing retry logic
- Enhanced validation for corrupted datasets
Infrastructure
- Daily dataset generation workflow with conflict prevention
- Improved PostgreSQL migration script
- Documentation reorganization and updates
Migration & Compatibility
No user action required - internal optimizations only.
Full Changelog: v0.5.11...v0.5.12
Docker Image: lakker/pulsarr:latest
Upgrade Path: Standard Docker image update
- feat: impliment pre-filtered ratings dataset and runner to update daily (#548) @jamcalli
- Develop (#547) @jamcalli
- Chore/docs update (#546) @jamcalli
Docker
docker pull lakker/pulsarr:0.5.12