Added
- Pagination support with customizable page size (25 default, max 100)
- Added filtering capabilities for crawl requests by UUID, URL, status, and date
- Added filtering for crawl results by URL and creation date
- Support for multiple output formats in download endpoint (markdown, json)
- New serializer for full crawl results with prefetching option
- New common pagination module for consistent pagination across the API
Changed
- Enhanced API documentation with detailed query parameters
- Improved filtering capabilities with advanced filters (contains, startswith, greater than, less than)
- Updated API endpoints to support prefetched result data
- Modified download endpoint to return zip files with formatted content
- Updated crawl status checking to support prefetched results