What's Changed
Improvements
- Removed redundant batch configuration to rely on FireCrawl library's built-in functionality
- Simplified batch processing logic by leveraging library's native implementation
- Optimized parallel processing and rate limiting handling
- Reduced code complexity and potential configuration conflicts
Technical Details
- Removed custom
CONFIG.batch
settings:- Removed
maxParallelOperations
configuration - Removed
delayBetweenRequests
configuration
- Removed
- Simplified batch operation processing to use library's built-in batch handling
- Updated server startup logging to remove batch configuration references
- Maintained credit usage tracking and error handling functionality
Benefits
- More reliable batch processing using library's battle-tested implementation
- Better handling of rate limits and concurrent operations
- Reduced maintenance overhead
- Improved code maintainability