MarquezProject/marquez 0.21.0

Marquez 0.21.0

on GitHub

Added

• Add MDC to the LoggingMdcFilter to include API method, path, and request ID @fm100
• Add Postgres sub-chart to Helm deployment for easier installation option @KevinMellott91
• GitHub Action workflow to validate changes to Helm chart @KevinMellott91

Changed

• Upgrade from Java11 to Java17 @ucg8j
• Switch JDK image from alpine to temurin enabling Marquez to run on multiple CPU architectures @ucg8j

Fixed

• Error when running Marquez on Apple M1 @ucg8j

Removed

• The /api/v1-beta/lineage endpoint @wslulciuc

• The marquez-airflow lib. has been removed, Please use the openlineage-airflow library instead. To migrate to using openlineage-airflow, make the following changes @wslulciuc:

  # Update the import in your DAG definitions
  -from marquez_airflow import DAG
  +from openlineage.airflow import DAG

  # Update the following environment variables in your Airflow instance
  -MARQUEZ_URL
  +OPENLINEAGE_URL
  -MARQUEZ_NAMESPACE
  +OPENLINEAGE_NAMESPACE

• The marquez-spark lib. has been removed. Please use the openlineage-spark library instead. To migrate to using openlineage-spark, make the following changes @wslulciuc:

  SparkSession.builder()
  - .config("spark.jars.packages", "io.github.marquezproject:marquez-spark:0.20.+")
  + .config("spark.jars.packages", "io.openlineage:openlineage-spark:0.2.+")
  - .config("spark.extraListeners", "marquez.spark.agent.SparkListener")
  + .config("spark.extraListeners", "io.openlineage.spark.agent.OpenLineageSparkListener")
    .config("spark.openlineage.host", "https://api.demo.datakin.com")
    .config("spark.openlineage.apiKey", "your datakin api key")
    .config("spark.openlineage.namespace", "<NAMESPACE_NAME>")
  .getOrCreate()