New Features
- Add media properties generation (#3620) by @obulat
- Project Proposal: Rekognition data incorporation (#3948) by @AetherUnbound
Improvements
- Get change in record counts during data refresh from Elasticsearch (#4067) by @mjtung
- Report reingestion errors in aggregate (#4074) by @stacimc
- Reduce visual footprint of DAG Alerts in Slack (#4071) by @nicoepp
- Refactor: Remove
get_media_type()redundant override in providers with a single media type (#4061) by @zaharoian - Add elasticsearch concurrency tags for Airflow (#3921) by @stacimc
Internal Improvements
- Update catalog to non-legacy environment names and point to new airflow.openverse.org (#4053) by @sarayourfriend
- Update dependency apache-airflow to v2.9.0, Python to 3.12 (#4159) by @openverse-bot
- Implementation plan: Ingestion server removal (#4026) by @stacimc
- Reduce visual footprint of DAG Alerts in Slack (#4071) by @nicoepp
- Fix: Escape space in
just catalog/testdirectory injection (#4060) by @zaharoian - Clarify Batched Update DAG docs with use cases for failure recovery (#4041) by @stacimc
- Improve testing import behavior for the catalog (#4029) by @AetherUnbound
- Update dependency tldextract to v5.1.2 (#4010) by @openverse-bot
- Update dependency flaky to v3.8.1 (#3997) by @openverse-bot
- Use a
make_insert_queryfunction in test_sql.py (#3983) by @kuephi - Update dependency apache-airflow to v2.8.4 [SECURITY] (#3974) by @openverse-bot
Bug Fixes
- Fix date range sent to Slack from reingestion workflows (#4093) by @BaileyMcKelway
- Update Science Museum ingester with API changes (#4105) by @stacimc
- Fill out query params for Stocksnap DAG, allow restarts (#4102) by @AetherUnbound
- Wait for iNaturalist load completion before compiling statistics (#4104) by @AetherUnbound
- Fix: Escape space in
just catalog/testdirectory injection (#4060) by @zaharoian - Filter out duplicates from
raw_tagsin the catalog v2 (#4014) by @krysal - Increase Wikimedia request timeout (#4004) by @AetherUnbound