Summary:
Enhanced ingestion process, added new tests, and improved logging and configuration handling across the codebase.
Key points:
- Added POSTGRES_VECS_COLLECTION to .env.example and ci.yml.
- Updated Python version to 3.12.4 in ci.yml.
- Added Ollama installation and server setup steps in ci.yml.
- Modified Dockerfile to use multi-stage builds and updated CMD to use uvicorn.
- Added CLIENT_MODE to compose.yaml.
- Updated pyproject.toml to ignore specific warnings and set asyncio_mode to auto.
- Added new exceptions R2RDocumentProcessingError and R2RException in r2r/base/abstractions/exception.py.
- Updated r2r/base/abstractions/document.py to include DocumentStatus and set initial status to processing.
- Updated document ingestion process to mark progressive status throughout ingestion process.
- Refactored ingestion and update document methods in r2r/main/services/ingestion_service.py.
- Added new tests for ingestion service in tests/test_ingestion_service.py.
- Added async fixtures and cleanup tasks in multiple test files to handle asyncio event loops and task cleanup.
- Updated embedding providers to handle retries and backoff in r2r/providers/embeddings/ollama/ollama_base.py.
- Added new end-to-end tests in tests/test_end_to_end.py.
- Refactored logging tests to include async fixtures and handle multiple logging providers in tests/test_logging.py.