MarquezProject/marquez 0.30.0

Marquez 0.30.0

on GitHub

Added

• Proposals: add proposal for OL facet tables #2076 @wslulciuc
  Adds the proposal Optimize query performance for OpenLineage facets.
• UI: display column lineage of a dataset #2293 @pawel-big-lebowski @tito12
  Adds a JSON preview of column-level lineage of a selected dataset to the UI.
• UI: Add soft delete option to UI #2343 @tito12
  Adds option to soft delete a data record with a dialog component and double confirmation.
• API: split lineage_events table to dataset_facets, run_facets, and job_facets tables. 2350, 2355, 2359
  @wslulciuc, @pawel-big-lebowski
  Performance improvement storing and querying facets.
  Migration procedure requires manual steps if database has more than 100K lineage events.
  We highly encourage users to review our migration plan.
• Docker: add new script for stopping Docker #2380 @rossturk
  Provides a clean way to stop a deployment via docker-compose down.
• Docker: seed data for column lineage #2381 @rossturk
  Adds some ColumnLineageDatasetFacet JSON snippets to docker/metadata.json to seed data for column-level lineage facets.

Fixed

• API: validate RunLink and JobLink #2342 @pawel-big-lebowski
  Fixes validation of the ParentRunFacet to avoid NullPointerExceptions in the case of empty run sections.
• Docker: use docker-compose.web.yml as base compose file #2360 @wslulciuc
  Fixes the Marquez HTTP server set in docker/up.sh so the script uses docker-compose.web.yml with overrides for dev set via docker-compose.web-dev.yml.
• Docs: update copyright headers #2353 @merobi-hub
  Updates the headers with the current year.
• Chart: fix Helm chart #2374 @perttus
  Fixes minor issues with the Helm chart.
• Spec: update dataset version API spec #2389 @phixme
  Adds limit and offset to the openapi.yml spec file as query parameters.