Release Notes

v0.15.0 is an integrity release built on a format break, and a speed release on top of it.

Decompression gains up to 24 % at levels 3-5 and 9-14 % at level 7 over v0.14.x; compression gains up to 39 % at levels 3-7. Level 3 now decodes 2.03x faster than LZ4 on Apple M2 (1.75x in v0.14), at a smaller size.

Format v9 redoes every integrity field: checksums that leave no header bit unprotected, block checksums over the decoded bytes, an optional archive digest, a seek table that validates itself group by group. On top of it, ZXC archives become multi-frame: cat a.zxc b.zxc is a valid archive, and the seekable reader spans every frame.

⚠️ Breaking change (format and ABI). The container format is now v9, and dictionaries move to .zxd version 2. v0.15.0 cannot read v8 archives or version-1 dictionaries, and v0.14.x cannot read what v0.15.0 writes (ZXC_ERROR_BAD_VERSION). The soname moves to libzxc.so.5: seekable block counts are 64-bit (#438), so programs linked against libzxc.so.4 must be rebuilt.

Migrating v8 -> v9 is the same one-time transcode as before:

zxc-old -dc old.zxc | zxc-new -z -c > new.zxc

A dictionary is migrated with its archives: retrain it with the new build, or convert it byte for byte to keep its dict_id. Full guide, including the CLI output changes and the new Windows archive layout: docs/MIGRATION.md.

ZXC format v9

The compressed data is unchanged: same blocks, same parser, same entropy coders. The break is in what protects them and in how a reader finds its way. (#452)

• Multiplicative header checksums (#432). The XOR/shift construction left part of the compressed size unprotected; the same 2 bytes now cover every header bit.
• Block checksums cover the decoded bytes, seeded by position (#434). A block moved or duplicated inside an archive no longer verifies. The whole-archive hash gives way to an optional 8-byte archive digest folded from the block checksums.
• A seek table that validates itself (#436, #438). Groups of one 64-bit anchor and 64 block sizes, checked one group at a time and loaded lazily, so opening a large archive reads only what a range needs. Block counts are 64-bit.
• HAS_SEEK_TABLE header flag (#448). The table is announced instead of probed for. In v0.14, about one footer in 65 536 parsed as a seek-table header and failed an intact archive with ZXC_ERROR_IO; that ambiguity is gone.
• A variable-length footer (#489). Source size and compressed frame size are stored on the bytes they need, behind the optional digest, with a last byte giving their lengths: 3 to 17 bytes, plus 8 with the digest. ZXC_FILE_FOOTER_SIZE is replaced by ZXC_FILE_FOOTER_MIN_SIZE / ZXC_FILE_FOOTER_MAX_SIZE.
• One valid form for the EOF and SEK headers. Reserved bytes must be zero there and decoders check them, so a lying flag reads as corrupt data rather than a truncated table.
• .zxd dictionaries version 2, signed with the new header checksum.

Multi-frame archives (#490)

A ZXC file is now one or more frames back to back. After each footer comes the end of the input, the next frame's magic, or an error.

• Concatenation just works. cat a.zxc b.zxc > ab.zxc decodes to both sources in order, through the buffer, context, FILE* and push-stream APIs, and the size queries add the frames up. Frames keep their own block size, checksum flag and dictionary.
• Seekable across frames. When every frame carries a seek table, zxc_seekable_open spans them all (up to 2^20 frames), with a dictionary per frame.
• Frame info API (#489). zxc_get_frame_info(), zxc_get_last_frame_info() and their FILE* versions report a frame's sizes and flags; the compressed frame size in each footer lets a reader walk the frames back from the end.
• Wrappers. Go, Rust and Python readers decode concatenations as one stream by default, with an opt-out (Multistream(false), .multistream(false), multistream=False) for a source that stays open after one frame.
• CLI. zxc -d and -t decode every frame. zxc -l gains Frames and Seekable columns and reports enabled / disabled / mixed; the JSON keys changed (see MIGRATION).

Performance

Silesia (202 MB), single-threaded, v0.14.x -> v0.15.0 on the same reference hardware.

Decompression:

Compression:

In absolute terms, level 3 decodes at 9.7 GB/s on Apple M2 and 8.3 GB/s on Zen 5, and level 1 at 13.5 GB/s on M2. Full tables and effective throughput: README and WHITEPAPER.

v0.14.x figures are the v0.14.0 run (lzbench 2.3.1): v0.14.1 writes byte-identical output and its decode path only changed for the block API. v0.15.0 was measured with lzbench 2.4.1.

Long GLO matches split into inline sequences at levels 3-5 (#441)

Long matches leave the decoder's slow path: the encoder splits them into sequences the inline copy handles, which is where most of the level 3-5 gain comes from. The trade is paid in ratio, at those three levels only:

Decoder and encoder kernels

• Wild-copy loops (#439) and GHI batch register pressure (#460) tightened in the decoder's hot loops.
• PivCo Huffman tree simplified (#451).
• Faster LZ77 parsing (#466): lazy evaluation exits early, and the optimal parser carries a match over instead of re-evaluating it, skipping redundant comparisons and DP relaxations.

Parallel FILE* decompression (#495)

From a regular file, workers read their own blocks with positioned reads and small blocks travel in batches, so the reader no longer copies every byte. Concatenated frames of the same geometry share one engine instead of restarting threads and buffers per frame. Pipes and sockets still decode block by block, so a live stream is never held back.

API

• Zero-allocation in-place decompression (#461): zxc_decompress_inplace_dctx() decodes in place through a reusable context, with no allocation per call.
• Seekable archives from the context API (#487): the compression context can write a seek table.
• Frame info (#489): zxc_get_frame_info(), zxc_get_last_frame_info(), zxc_stream_get_frame_info(), zxc_stream_get_last_frame_info(), zxc_frame_info_size().
• 64-bit seekable block counts (#438): zxc_seekable_get_num_blocks() and zxc_seek_table_size() take and return uint64_t (the ABI break above).
• Linux kernel support (#454): the library builds as a kernel module, scalar paths only, with its own CI workflow.

Wrappers

• Python (#505): zxc.compress() and zxc.Cctx take seekable=True, as stream_compress() already did.
• Go (#499): Compress and CompressTo honour WithSeekable(true); they returned an archive without a seek table. CompressTo accepts empty input and writes a minimal frame, as Compress does.

Reliability & Security

• In-place decompression (#450): strict read/write separation; a write could overtake unread input under a forged archive.
• Type-safe stream callback (#449): the processor callback no longer goes through an incompatible function-pointer cast (UBSan -fsanitize=function).
• MSVC 19.51 ARM64 (#458): worked around a miscompile of the seek-table size fold.
• Go seekable reader (#486): a panic in a user reader is recovered and reported as an error instead of crashing across the C trampoline.
• WASM (#459): a failed heap allocation throws instead of returning a null pointer.
• Frame validation and dictionary edge cases: truncated or forged footers, stale FILE* error indicators and dictionaries that no frame needs are handled consistently across the buffer, stream and seekable paths.
• Fuzzing (#463): a block-API fuzzer and coverage of the dictionary Huffman table; OSS-Fuzz line coverage of the v9 decoder went from 0.9 % to 84.8 % once the corpus was rebuilt.

CLI

• zxccat (#503): an alias for zxc -dc that decompresses every file it is given, or standard input, to standard output, one after the other: zxccat logs-*.zxc | grep ERROR. Installed next to unzxc on POSIX systems.
• -c appends to a >> redirect (#503): decompressing to standard output reopened it and truncated a file it should have appended to.
• zxcless (#506): pages files in less(1), decompressing .zxc archives on the fly (through LESSOPEN, less 451 or later); other files and standard input display unchanged. With it, -f now copies input that is not a ZXC archive unchanged when decompressing to standard output.
• zxcgrep (#507): grep(1) on .zxc archives, regular files and standard input, with grep's options and exit statuses; a damaged archive is reported as an error. Both scripts are installed on POSIX systems.
• Man page installed (#504): zxc.1 is installed with the CLI, with unzxc.1 and zxccat.1 as aliases (POSIX systems).
• -r stays inside the tree (#498, #500): recursive mode no longer follows symbolic links, junctions or mount points; it reports each one, and reports a directory it cannot read instead of skipping it silently. Paths are built without a fixed MAX_PATH buffer.
• --progress and TTY detection are honoured in benchmark mode (#464).
• A file named like a command verb is no longer taken for a mode (#494).
• Multi-frame decoding and listing, as above.

Build & Packaging

• Windows archives built with MinGW-w64 on the UCRT (#498): GCC on x86_64, Clang on ARM64, nothing to install. They ship libzxc.dll with its MinGW import library libzxc.dll.a, the static libzxc.a, and libzxc.def, from which MSVC users build an import library (lib /def). The MSVC static lib/zxc.lib of 0.14.x is gone; see MIGRATION.
• Warnings as errors in CI (#497, #498). Every compile job builds with -DCMAKE_COMPILE_WARNING_AS_ERROR=ON (Meson: -Dwerror=true) on GCC 9-16, Clang 12-22, MSVC (now at /W4) and MinGW; the library adds -Wconversion -Wsign-conversion on 64-bit targets (GCC 10+ and Clang). CMake 3.21 is now the minimum, the first version that knows C17.
• LTO is off by default (#498): a static library built with it only links with the same compiler. -DZXC_ENABLE_LTO=ON turns it back on.
• Linux kernel module build and CI workflow (#454).
• Stream output is flushed and write errors reported before returning (#453).
• Toolchains refreshed, Clang 21-22 added, glibc floor checked in CI (#465); apt caching for the multiarch workflows (#482); wrapper tests run on pull requests (#501).
• SPDX headers and copyright notices standardized (#483).

Documentation

• FORMAT: the v9 integrity fields, the variable-length footer, concatenated frames and the one valid form of the EOF and SEK headers.
• MIGRATION: the v8 -> v9 transcode, dictionary conversion, the CLI output changes and the Windows archive layout.
• Documentation structure reworked (#485); Doxygen markup corrected (#508).

Acknowledgements

Thanks to @Vladexy88x, who found and fixed four bugs in this release: unflushed stream output that hid write errors (#453), a failed WASM heap allocation that went unreported (#459), reader panics crossing the Go seekable trampoline (#486), and Go Compress / CompressTo dropping WithSeekable (#499); and who added the seekable option to the Python compress() and Cctx (#505).

Changelog

• api: Implement variable-length file footer and frame info API (#489)
• api: Enable seekable archives in compression context API (#487)
• api: Add support for zero-allocation in-place decompression (#461)
• api: Introduces kernel zxc support (#454)
• api: Introduce format v9 and .zxd dictionary v2 (#452)
• api: Announce the seek table with a HAS_SEEK_TABLE header flag (#448)
• api: Support 64-bit block counts for seekable archives (#438)
• api: Validate the seek table per group of 64 blocks and load it lazily (#436)
• api: Checksum decompressed blocks, seeded by position, and add an archive digest (#434)
• api: Replace xor/shift header checksums with multiplication-based hashing (#432)
• perf: Optimizes parallel stream decompression I/O (#495)
• perf: Optimizes LZ77 compression performance (#466)
• perf: Reduce register pressure in GHI batch decompression (#460)
• perf: Optimizes wild copy decompression loops (#439)
• perf: Simplify the PivCo Huffman tree representation (#451)
• perf: Split long GLO matches into inline sequences at L3-L5 (#441)
• cli: Add zxcgrep utility for searching compressed files (#507)
• cli: Add zxcless utility and enhances decompression pass-through (#506)
• cli: Add zxccat alias for concatenated decompression (#503)
• cli: Install the man page with the CLI (#504)
• cli: Unify path concatenation for directory walking (#500)
• cli: prevent mode misinterpretation of files named like command verbs (#494)
• cli: Address frame validation and dictionary handling edge cases
• cli: Honour --progress and tty detection in benchmark mode (#464)
• fix: Stop -c from truncating a >> redirect (#503)
• fix: Make the stream processor callback type-safe (#449)
• fix: Enforce strict read/write separation in in-place decompression (#450)
• fix: Work around an MSVC 19.51 ARM64 miscompile of the SEK size fold (#458)
• fix: Throw on failed WASM heap allocation (#459)
• build: Replace #error with static_assert for compile-time validation (#509)
• build: Release Windows with MinGW-w64, build MSVC at /W4 (#498)
• build: Trigger wrapper tests on pull requests (#501)
• build: Enable compiler warnings as errors in CI (#497)
• build: Bump source-map-js in /wrappers/nodejs (#496)
• build: Standardizes SPDX headers and copyright notices (#483)
• build: Bump scikit-build-core in /wrappers/python (#475)
• build: Bump setuptools-scm in /wrappers/python (#473)
• build: Bump cibuildwheel from 4.2.0 to 4.2.1 in /wrappers/python (#474)
• build: Bump vitest from 4.1.11 to 5.0.2 in /wrappers/nodejs (#472)
• build: Bump build from 1.6.0 to 1.6.1 in /wrappers/python (#477)
• build: Bump cc from 1.4.4 to 1.5.1 in /wrappers/rust (#470)
• build: Bump thiserror from 2.0.20 to 2.0.21 in /wrappers/rust (#471)
• build: Implement apt package caching for multiarch workflows (#482)
• build: Rename lzbench directory to reflect lz+entropy integration
• build: Refresh toolchains, add Clang 21-22 and a glibc floor check (#465)
• build: Remove temporary Docker downgrade workaround (#462)
• build: Add Linux kernel module support and its CI workflow (#454)
• build: Flush stream output and report write errors (#453)
• build: Prevent auxv header inclusion when ZXC_ONLY_DEFAULT is defined (#447)
• doc: Correct Doxygen markup in header files (#508)
• doc: Update documentation structure and content (#485)
• misc: Reject blocks with trailing bytes in RLE sections (#511)
• misc: Wrapper Python: add seekable to compress() and Cctx (#505)
• misc: Wrapper Go - honor WithSeekable in Compress/CompressTo, accept empty input (#499)
• misc: Wrapper Go - recover reader panics in the Go seekable trampoline (#486)
• misc: Supports multi-frame ZXC archives (#490)
• misc: Bump github/codeql-action/init from 4.37.9 to 4.38.2 (#476)
• misc: Bump actions-rust-lang/setup-rust-toolchain (#467)
• misc: Bump github/codeql-action/upload-sarif (#478)
• misc: Bump oss-fuzz-base/base-builder in /.clusterfuzzlite (#479)
• misc: Bump meson from 1.12.0 to 1.12.1 in /.github/requirements (#480)
• misc: Bump ninja from 1.13.0 to 1.13.2 in /.github/requirements (#481)
• misc: Bump orhun/git-cliff-action from 4.8.0 to 4.9.1 (#468)
• misc: Bump actions/cache from 5.0.5 to 6.1.0 (#469)
• misc: Add block API fuzzer and dictionary huffman table coverage (#463)

Full Changelog: v0.14.1...v0.15.0