What's Changed
- feat(storage): make native compaction safe for a live catalog by @adubovikov in #947
- feat(storage): support hours as a retention unit alongside days by @lukeescude in #948
- chore: update Go from 1.26.5 to 1.26.6 by @github-actions in #949
The default compaction engine is still duckdb — nothing changes unless you opt in. engine=native is now safe to try: every catalog write goes through ducklake_add_data_files, so DuckLake alone allocates snapshot and file IDs (this is what corrupted catalogs before). Merging is done in Go, outside DuckDB's memory_limit, and a partition is left alone rather than merged when it is unsafe or not worth it: source row groups larger than max_row_group_bytes (default 256MB uncompressed), row-level delete files in that partition, inlined data, hive-style data paths, or a column type that survives the parquet round trip differently than the catalog declares it. min_age_sec now actually works and defaults to 300s so a busy writer still consolidates during the day.
Real-file measurement: Homer rows are wide (~7KB), so a 10.5k-row row group is ~1.5GB uncompressed and merging three of them peaked at ~10GB RSS. max_row_group_bytes is the guard against exactly that — see docs/OOM.md before raising it.
Retention can now be expressed in hours via retention_unit: "hours", which reinterprets the existing retention_days values. Default behavior is unchanged. Note that an hours cutoff falls mid-day and creates row-level delete files, which makes the native engine skip that one partition until retention removes the whole day.
Full Changelog: 11.0.320...11.0.321