Theme: Activity Feeds, DBT integration, Storage Location and Metabase, Druid, MLFlow connectors
Activity Feeds
- Enables users to view a summary of the metadata change events.
- The most recent changes are listed at the top.
- Entities (tables, dashboards, team names) are clickable.
- Displays 'All Changes' and 'My Changes'.
- A feed has been provided for data you are 'Following'.
UX Improvements
- New and improved UX has been implemented for search and its resulting landing pages.
- The Explore page has an updated UI. Introduced a 3-column layout to present the search results and several visual element improvements to promote readability and navigation of the search results.
- Users can filter by database name on the Explore page.
- Support has been added to view the recent search terms.
- Major version changes for backward incompatible changes are prominently displayed.
DBT Integration
- DBT integration enables users to see what models are being used to generate tables.
- DBT models have been associated with Tables and are no longer treated as a separate Entity on the UI.
- Each DBT model can produce a Table, and all the logic and metadata is now being captured as part of it.
- DBT model is accessible from a tab in the table view.
- Storage Location
UI supports StorageLocation.
- Can extract the location information from Glue.
- The ingestion framework gets the details from Glue to publish the location information (such as S3 bucket, HDFS) for the external tables.
- The table results view now displays the table location in addition to the service.
Elasticsearch
- SSL-enabled Elasticsearch (including self-signed certs) is supported.
- Automatically runs an indexing workflow as new entities are added or updated through ingestion workflows.
- Metadata indexing workflow is still available; in case indexes need to be cleanly created from the ground up.
New Connectors
- Metabase, an open-source BI solution
- Apache Druid has been added to integrate Druid metadata.
- MLflow, an open-source platform for the machine learning lifecycle
- Apache Atlas import connector, imports metadata from Atlas installations into OpenMetadata.
- Amundsen import connector, imports metadata from Amundsen into OpenMetadata.
- AWS Glue - improved to extract metadata for tables and pipelines. Now uses the AWS boto session to get the credentials from the different credential providers, and also allows the common approach to provide credentials. Glue ingestion now supports Location.
- MSSQL now has SSL support.
##Lineage View Improvements
- Improved discoverability
- Implemented incremental loading to lineage.
- Users can inspect the Table’s columns.
- Users can expand entity nodes to view more upstream or downstream lineage.
- Helps data consumers and producers to assess the impact of changes to the data sources.
Other Features
- Admins can now access the user listing page, filter users by team, and manage admin privileges
- Updated the Redoc version to support search in the API, as well as to add query parameter fields fixes.
- 'Entity Name' character size has been increased to 256.
- Upgraded Log4j to 2.15