delta-io/delta v4.4.1

Delta Lake 4.4.1

on GitHub

We are pleased to announce the release of Delta Lake 4.4.1, a patch release on top of Delta Lake 4.4.0. This release improves maintenance safety for catalog-managed tables and fixes snapshot construction for tables that use coordinated commits.

Bug Fixes

• [Spark] Safer maintenance for catalog-managed tables: Delta extends existing safeguards on catalog-managed tables to prevent dangerous data and metadata cleanup operations. Automatic maintenance operations are skipped so writes can complete successfully without failure.
• [Spark] S3A Fast Listing: Fixed an issue that could produce incorrect snapshots. Users can enable fast listing with delta.enableFastS3AListFrom again.

Delta Spark

Delta Spark 4.4.1 is built for Apache Spark 4.2.0, Apache Spark 4.1.0, and Apache Spark 4.0.1. As with Apache Spark, Maven artifacts are published for Scala 2.13.

• Maven artifacts for Spark 4.2.0:
  • delta-spark_4.2_2.13
  • delta-connect-client_4.2_2.13
  • delta-connect-common_4.2_2.13
  • delta-connect-server_4.2_2.13
• Maven artifacts for Spark 4.1.0:
  • delta-spark_4.1_2.13
  • delta-connect-client_4.1_2.13
  • delta-connect-common_4.1_2.13
  • delta-connect-server_4.1_2.13
• Maven artifacts for Spark 4.0.1:
  • delta-spark_4.0_2.13
  • delta-connect-client_4.0_2.13
  • delta-connect-common_4.0_2.13
  • delta-connect-server_4.0_2.13
• Backward-compatibility artifacts without a Spark version in the name default to Spark 4.2.0:
  • delta-spark_2.13
  • delta-connect-client_2.13
  • delta-connect-common_2.13
  • delta-connect-server_2.13
• Other Maven artifacts:
  • delta-contribs_2.13
  • delta-storage
  • delta-storage-s3-dynamodb
• Python artifact:
  • delta-spark

The fixes and behavior changes in this release are:

• Safer maintenance for catalog-managed tables (#7603): Catalog-managed tables require the catalog to control destructive maintenance. Delta now rejects unsupported direct data cleanup, data reorganization, and transaction-log cleanup before changing the table. This release blocks metadata cleanup, column-mapping removal, and manual symlink-format manifest generation on top of those that are already blocked. Delta skips automatic log cleanup, auto compaction, and automatic symlink-format manifest generation on these tables so that the original write can complete. Tables that are not catalog-managed keep their existing behavior.
• Correct staged-commit handling in filesystem listings (#7648): An S3A Fast List setup can return files from _delta_log/_staged_commits. Delta now ignores these files unless the commit coordinator has ratified the commit. This prevents an orphaned staged commit from appearing as a committed table version and prevents duplicate versions when staged and backfilled copies both appear. Ratified commits remain readable before backfill completes.

Delta Kernel

Delta Kernel is a set of Java libraries for building Delta connectors that read and write Delta tables without requiring each connector to implement the Delta protocol directly.

• Maven artifacts:
  • delta-kernel-api
  • delta-kernel-defaults
  • delta-kernel-unitycatalog

There are no source changes to Delta Kernel in this patch release.

Delta UniForm

Delta UniForm's delta-iceberg module keeps Apache Iceberg metadata synchronized with Delta commits, allowing Iceberg readers to query Delta tables without duplicating data.

• Maven artifact:
  • delta-iceberg_2.13

Note: In Delta 4.4, delta-iceberg_2.13 supports Spark 4.1 and is not compatible with Spark 4.2.

There are no source changes to Delta UniForm in this patch release.

Delta Sharing

Delta Sharing is a Spark DataSource that supports batch, streaming, change data feed, and time-travel reads on tables shared through the Delta Sharing protocol.

• Maven artifact for Spark 4.2.0:
  • delta-sharing-spark_4.2_2.13
• Maven artifact for Spark 4.1.0:
  • delta-sharing-spark_4.1_2.13
• Maven artifact for Spark 4.0.1:
  • delta-sharing-spark_4.0_2.13
• Backward-compatibility artifact without a Spark version in the name defaults to Spark 4.2.0:
  • delta-sharing-spark_2.13

There are no source changes to Delta Sharing in this patch release.

Delta Flink

The Kernel-based delta-flink connector remains experimental. Delta 4.4.1 supports Apache Flink 2.0.2, 2.1.3, 2.2.1, and 2.3.0.

• Maven artifacts:
  • delta-flink_2.0
  • delta-flink_2.1
  • delta-flink_2.2
  • delta-flink_2.3

There are no source changes to Delta Flink in this patch release.

For the complete commit history, see the comparison between v4.4.0 and v4.4.1.