Theme: Lineage, Data Reliability, Pipelines, Complex Types
Lineage
- Schema and API support for Lineage
- UI integration to show the lineage for Pipelines and Tables
- Ingestion support for Airflow to capture lineage
Data Reliability
- UI Integration for Data profiler
- Unique , Null proportion
- See how the table data grows through interactive visualization
New Entities: Pipeline
- Add Apache Airflow as a pipeline service
- Ingest all of your pipeline's metadata into OpenMetadata
- Explore and Search Pipelines
- Add description, tags, ownership and tier your pipelines
Complex Data types
- Schema and API support to capture complex data types
- Ingestion support to capture complex data types from Redshift, BigQuery, Snowflake and Hive
- UI Support for nested complex data types, users can add description, tags to nested fields
New Connectors
- Trino connector
- Redash connector
- Amazon Glue - In progress
User Interface
- UI now completely built on top of JsonSchema generated code
- Expand complex data types and allow users to update descriptions and tags
- Pipeline Service and Details page
- Pipeline Explore & Search integration
- Search results will show if the query matches description or column names, description
Full Changelog: 0.4.0-pre...0.5.0