- Client: allow setting client's endpoint via environment variable
Enables setting this endpoint via environment variable because creating the client manually in Airflow is not possible.
- Flink: expand Iceberg source types
Adds support for
FlinkIcebergTableSourcefor Flink Iceberg lineage.
- Spark: add debug facet
An extra run facet containing some system details (e.g., OS, Java, Scala version), classpath (e.g., package versions, jars included in the Spark job), SparkConf (like openlineage entries except auth, specified extensions, etc.) and LogicalPlan details (execution tree nodes' names) are added to events emitted. SparkConf setting
spark.openlineage.debugFacet=enabledneeds to be set to include the facet. By default, the debug facet is disabled.
- Spark: enable Nessie REST catalog
Adds support for Nessie catalog in Spark.