Release Highlights
- Business Glossary: Phase 1 is feature complete. Full support for UI viewing and API-based edits, no support for UI edits.
- Users and Groups: Just-in-time User and Group provisioning on login (SSO/OIDC), basic Group pages with membership information
- New Integrations: Redash
Notable Changes
- GraphQL and REST API-s are now both served by datahub-metadata-service (new name for gms). Frontend is now a proxy. Container names are not changed.
- Kafka source will no longer tokenize on
.in the topic name. This will result in a flat browse experience in UI. - Airflow lineage emission will only populate specific properties of Tasks and DAGs to limit bloat and avoid leaking environment variables.
- Schema history feature turned off in UI based on feedback from the community. Will re-emerge in a future release!
- Mongodb collections with extremely wide schemas will have schema fields sampled to keep UI responsive.
- Full changelog below.
ChangeLog
- #3156 @swaroopjagadish fix(frontend): replacing broken link for default avatar
- #3154 @swaroopjagadish fix(frontend): fixing broken link to default avatar
- #3153 @swaroopjagadish feat(ingest): adding maxSchemaSize to mongodb source
- #3150 @saxo-lalrishav fix(business-glossary): business glossary visual changes
- #3142 @greysond fix(metadata-service): actually load keys from keystore for elastic connections
- #3110 @frsann feat(ingestion): bring your own SQL parser
- #3146 @jjoyce0510 fix(react): refactoring hasKeySchema computation
- #3145 @swaroopjagadish deps(ingest): upgrade to pick up acryl-pyhive changes
- #3144 @sgomezvillamor fix(profiles): prevent NoneType exception when profiling empty datasets
- #3140 @swaroopjagadish fix(glossary): Make terms searchable and browseable
- #3139 @swaroopjagadish fix(deps): Adding min version to python-dateutil to guard against isoparse failures
- #3135 @dexter-mh-lee fix(kafka): Change consumer id of mae/mce processor
- #3137 @swaroopjagadish fix(airflow): only emit specific keys for airflow lineage properties
- #3131 @jjoyce0510 feat(graphql): migrating GraphQL API to metadata-service (nee GMS)
- #3082 @jjoyce0510 feat(sso): Just-In-Time User & Group Provisioning on SSO Login (oidc)
- #3129 @saxo-lalrishav feat(business-glossary): Business glossary relationship UI
- #3113 @dexter-mh-lee feat(ingest): Add custom browse paths for kafka sources and remove browse lowercase filter
- #2918 @taufiqibrahim feat(ingest): adding redash source
- #3103 @saxo-lalrishav feat(business-glossary): glossary term relationship graphql changes
- #3015 @jjoyce0510 refactor: remove unused gms code, frontend endpoints part 2/4
- #3094 @jjoyce0510 feat(group ui): Basic group search membership in UI
- #3012 @Shikha-Trivedi-Saxo feat(business-glossary): Glossary term relationship backend
- #3049 @neojunjie feat(frontend): logout with oidc
- #3099 @gabe-lyons fix(schema-version): temporarily hide schema version tab
- #3048 @saxo-lalrishav feat(business-glossary): added field level glossary terms
- #3095 @shirshanka fix(ingest): increasing default ingestion REST timeout to 30 seconds
- #3096 @dexter-mh-lee fix(upgrade): Fix MAE consumer and upgrade's dependency issue
- #3092 @jensenity fix(postgres): fix postgres setup to handle existing database