github OpenLineage/OpenLineage 0.21.1
OpenLineage 0.21.1

latest releases: 1.5.0, 1.4.1, 1.4.0...
9 months ago


  • Clients: add DEBUG logging of events to transports #1633 by @mobuchowski
    Ensures that the DEBUG loglevel on properly configured loggers will always log events, regardless of the chosen transport.
  • Spark: add CustomEnvironmentFacetBuilder class #1545 by New contributor @Anirudh181001
    Enables the capture of custom environment variables from Spark.
  • Spark: introduce the new output visitors AlterTableAddPartitionCommandVisitor and AlterTableSetLocationCommandVisitor #1629 by New contributor @nataliezeller1
    Adds visitors for extracting table names from the Spark commands AlterTableAddPartitionCommand and AlterTableSetLocationCommand. The intended use case is a custom transport for the OpenMetadata lineage API.
  • Spark: add column lineage for JDBC relations #1636 by @tnazarew
    Adds column lineage information to JDBC events with data extracted from query by the SQL parser.
  • SQL: add Linux-aarch64 native library to Java SQL parser #1664 by @mobuchowski
    Adds a Linux-ARM version of the native library. The Java SQL parser interface had only Linux-x64 and MacOS universal binary variants previously.


  • Airflow: get table database in Athena extractor #1631 by New contributor @rinzool
    Changes the extractor to get a table's database from the table.schema field or the operator default if the field is None.


  • dbt: add dbt seed to the list of dbt-ol events #1649 by New contributor @pohek321
    Ensures that dbt-ol test no longer fails when run against an event seed.
  • Spark: make column lineage extraction in Spark support caching #1634 by @pawel-big-lebowski
    Collect column lineage from Spark logical plans that contain cached datasets.
  • Spark: add support for a deprecated config #1586 by @tnazarew
    Maps the deprecated spark.openlineage.url to spark.openlineage.transport.url.
  • Spark: add error message in case of null in url #1590 by @tnazarew
    Improves error logging in the case of undefined URLs.
  • Spark: collect complete event for really quick Spark jobs #1650 by @pawel-big-lebowski
    Improves the collecting of OpenLineage events on SQL complete in the case of quick operations.
  • Spark: fix input/outputs for one node LogicalRelation plans #1668 by @pawel-big-lebowski
    For simple queries like select col1, col2 from my_db.my_table that do not write output,
    the Spark plan contained just a single node, which was wrongly treated as both
    an input and output dataset.
  • SQL: fix file existence check in build script for openlineage-sql-java #1613 by @sekikn
    Ensures that the build script works if the library is compiled solely for Linux.


  • Airflow: remove JobIdMapping and update macros to better support Airflow version 2+ #1645 by @JDarDagran
    Updates macros to use OpenLineageAdapter's method to generate deterministic run UUIDs because using the JobIdMapping utility is incompatible with Airflow 2+.

Don't miss a new OpenLineage release

NewReleases is sending notifications on new releases.