OpenLineage/OpenLineage 0.25.0

OpenLineage 0.25.0

on GitHub

Added

• Spark: add Spark/Delta merge into support #1823 @pawel-big-lebowski
  Adds support for merge into queries.

Fixed

• Spark: fix JDBC query handling #1808 @nataliezeller1
  Makes query handling more tolerant of variations in syntax and formatting.
• Spark: filter Delta adaptive plan events #1830 @pawel-big-lebowski
  Extends the DeltaEventFilter class to filter events in cases where rewritten queries in adaptive Spark plans generate extra events.
• Spark: fix Java class cast exception #1844 @Anirudh181001
  Fixes the error caused by the OpenLineageRunEventBuilder when it cast the Spark scheduler's ShuffleMapStage to boolean.
• Flink: include missing fields of Openlineage events #1840 @pawel-big-lebowski
  Enriches Flink events so that missing eventTime, runId and job elements no longer produce errors.