Disclaimers!
- Ingesting Chart Inputs was broken in a PR that got into this release. This will be fixed in v0.8.23. If you plan to ingest Charts / Dashboards, we recommend skipping this version and upgrading to v0.8.23 directly.
Release Highlights:
- Support for mapping DBT meta properties of a dataset to metadata operations, such as add_owner, add_term, add_tag etc.
- Java REST emitter library to programmatically generate metadata events from Java-based clients such as from Spark jobs.
- Data freshness indication via Last Updated Timestamp.
- Improvements to data profiling performance and lineage extraction
What's Changed
- feat(snowflake-usage): Generate email address if not exists by @treff7es in #3791
- feat(java datahub-client): add Java REST emitter by @MugdhaHardikar-GSLab in #3781
- fix(docker): Fix path to elastic definition in dev docker compose by @MikeSchlosser16 in #3808
- feat(nocode): Add get entities v2 endpoint that can get without snapshot by @dexter-mh-lee in #3738
- docs(modeling): Add a link to MXE page inside the Metadata Modeling page by @pramodbiligiri in #3765
- docs(fix): fix broken reference by @RyanHolstien in #3814
- feat(java-emitter): improvements to builder API-s, moving spark-linea… by @swaroopjagadish in #3819
- fix(ingestion): Make url an optional field of the DefaultConfig for business glossary by @rslanka in #3817
- fix(ingest): Handle string redshift type by @treff7es in #3811
- feat(gms): add schema registry support for tls in gms by @MikeSchlosser16 in #3804
- Add table, changed formatting and wording by @dannylee8 in #3802
- feat(mae/mcl): Make ingestAspect produce both MCLs and MAEs by @dexter-mh-lee in #3737
- docs(confluent): Add new topic names by @anshbansal in #3825
- (feat)(glossary): Increase number of autocomplete results shown to 25 by @aditya-radhakrishnan in #3821
- feat(sql-parser): Replacing sqlmetadata sql parser lib with sqlineage parser lib by @treff7es in #3806
- feat(profiler): using approximate queries for profiling by @treff7es in #3752
- docs: improve docs for kafka configuration by @abiwill in #3828
- test(fixEbeanEntityServiceTest): fix bug on verification for EbeanEntityService by @RyanHolstien in #3829
- fix(ingest): ignore custom connectors for Glue ingestion by @kevinhu in #3805
- fix(java-emitter): check for null callback by @swaroopjagadish in #3830
- feat(dbt-meta): add support for dbt meta mapping by @swaroopjagadish in #3832
- fix(ingestion): Fix the datetime parsing issue in the metabase source. by @rslanka in #3831
- feat(removeGMA): remove all dependencies on gma libraries by @RyanHolstien in #3835
- perf(ingest): changes to improve ingest performance a bit by @anshbansal in #3837
- fix(azure AD): fix problem with missing key causing failures in ingestion by @anshbansal in #3824
- docs: fix typo by @anshbansal in #3848
- docs(cli): fix wrong heading, add link to release notes by @anshbansal in #3700
- feat(ci): split metadata-ingestion ci to streamline build by @swaroopjagadish in #3854
- fix(dbt): fix warning due to struct type not being mapped by @anshbansal in #3846
- fix(ingest): bigquery-usage - fix remove_extras to remove all partitions by @gfalcone in #3842
- fix(ingestion): handle database=None for dbt ingestion by @iasoon in #3851
- feat(ingest): last updated - show last updated for sql usage sources by @aditya-radhakrishnan in #3845
- feat(lineage): allow for expanding of lineage node titles in the lineage explorer by @gabe-lyons in #3856
New Contributors
- @MikeSchlosser16 made their first contribution in #3808
- @pramodbiligiri made their first contribution in #3765
- @aditya-radhakrishnan made their first contribution in #3821
- @abiwill made their first contribution in #3828
- @gfalcone made their first contribution in #3842
- @iasoon made their first contribution in #3851
Full Changelog: v0.8.21...v0.8.22