lz4/lz4 v1.9.3

LZ4 v1.9.3

on GitHub

LZ4 v1.9.3 is a maintenance release, offering more than 200+ commits to fix multiple corner cases and build scenarios. Update is recommended. Existing liblz4 API is not modified, so it should be a drop-in replacement.

Faster Windows binaries

On the build side, multiple rounds of improvements, thanks to contributors such as @wolfpld and @remittor, make this version generate faster binaries for Visual Studio. It is also expected to better support a broader range of VS variants.
Speed benefits can be substantial. For example, on my laptop, compared with v1.9.2, this version built with VS2019 compresses at 640 MB/s (from 420 MB/s), and decompression reaches 3.75 GB/s (from 3.3 GB/s). So this is definitely perceptible.

Other notable updates

Among the visible fixes, this version improves the _destSize() variant, an advanced API which reverses the logic by targeting an a-priori compressed size and trying to shove as much data as possible into the target budget. The high compression variant LZ4_compress_HC_destSize() would miss some important opportunities in highly compressible data, resulting in less than optimal compression (detected by @hsiangkao). This is fixed in this version. Even the "fast" variant receives some gains (albeit very small).
Also, the corresponding decompression function, LZ4_decompress_safe_partial(), officially supports a scenario where the input (compressed) size is unknown (but bounded), as long as the requested amount of data to regenerate is smaller or equal to the block's content. This function used to require the exact compressed size, and would sometimes support above scenario "by accident", but then could also break it by accident. This is now firmly controlled, documented and tested.

Finally, replacing memory functions (malloc(), calloc(), free()), typically for freestanding environments, is now a bit easier. It used to require a small direct modification of lz4.c source code, but can now be achieved by using the build macro LZ4_USER_MEMORY_FUNCTIONS at compilation time. In which case, liblz4 no longer includes <stdlib.h>, and requires instead that functions LZ4_malloc(), LZ4_calloc() and LZ4_free() are implemented somewhere in the project, and then available at link time.

Changes list

Here is a more detailed list of updates introduced in v1.9.3 :

• perf: highly improved speed in kernel space, by @terrelln
• perf: faster speed with Visual Studio, thanks to @wolfpld and @remittor
• perf: improved dictionary compression speed, by @felixhandte
• perf: fixed LZ4_compress_HC_destSize() ratio, detected by @hsiangkao
• perf: reduced stack usage in high compression mode, by @Yanpas
• api : LZ4_decompress_safe_partial() supports unknown compressed size, requested by @jfkthame
• api : improved LZ4F_compressBound() with automatic flushing, by Christopher Harvie
• api : can (de)compress to/from NULL without UBs
• api : fix alignment test on 32-bit systems (state initialization)
• api : fix LZ4_saveDictHC() in corner case scenario, detected by @IgorKorkin
• cli : compress multiple files using the legacy format, by Filipe Calasans
• cli : benchmark mode supports dictionary, by @rkoradi
• cli : fix --fast with large argument, detected by @picoHz
• build: link to user-defined memory functions with LZ4_USER_MEMORY_FUNCTIONS
• build: contrib/cmake_unofficial/ moved to build/cmake/
• build: visual/* moved to build/
• build: updated meson script, by @neheb
• build: tinycc support, by Anton Kochkov
• install: Haiku support, by Jerome Duval
• doc : updated LZ4 frame format, clarify EndMark