Cyan4973/xxHash v0.8.1

on GitHub

xxHash v0.8.1 is a general clean up of the code base, following the stabilization of xxh3 and xxh128 in v0.8.0.
There are a few welcomed evolutions and improvements, but for the most part, this release consists of fixes for multiple corner cases and scenarios, that shall improve usability of libxxhash and xxhsum across a wide range of platforms.
Stable API entry points have not changed, all entry points labelled "stable" will continue to work as intended in this release and future ones.

Improved performance

While the "big picture" is unchanged, there are a few notable improvements.
XXH3 / XXH128 feature a large speed improvement in streaming mode, which is particularly sensible for gcc and MSVC (clang was already in good shape), by as much as +40%, making streaming speed essentially on par with single-shot mode when ingesting large quantities of data.

XXH64 and even XXH32 feature improved latency performance for small inputs of random sizes. Perhaps as importantly, their binary size is smaller.

New capabilities

There is a new experimental XXH3 variant, named _withSecretandSeed(). In a nutshell, it combines seed for small inputs, with secret for large inputs.
The main driver for this variant is a wish to skip the delay from secret's transparent generation when using _withSeed() variant with large inputs, resulting in measurable performance drop for "not so large" sizes (< 1 KB) (note: this delay is insensible for "large" inputs, such as > 256 KB). Coupled with new function XXH3_generateSecret_fromSeed(), which generates the same secret as the one generated internally when using the _withSeed() variant, it results in exactly the same return values, while skipping the secret generation stage, thus improving speed.

Experimental XXH3_generateSecret() has been extended to allow generation of secret of any size (though respecting the specification's minimum size). It's generally recommended to use this generator to ensure a source of "high entropy" for the secret.

On the CLI front, a highly demanded xxhsum feature was an ability to generate XXH3 checksum values. This is achieved in v0.8.1, using the --tag format, which ensures that XXH3 results cannot be confused with (default) XXH64 ones, even though they feature the same 64-bit width.

Detailed changelist

• perf : much improved performance for XXH3 streaming variants, notably on gcc and msvc
• perf : improved XXH64 speed and latency on small inputs
• perf : small XXH32 speed and latency improvement on small inputs of random size
• perf : minor stack usage improvement for XXH32 and XXH64
• api : new experimental variants XXH3_*_withSecretandSeed()
• api : updated XXH3_generateSecret(), can now generate secret of any size (>= XXH3_SECRET_SIZE_MIN)
• cli : xxhsum can now generate and check XXH3 checksums, using command -H3
• build: can build xxhash without XXH3, with new build macro XXH_NO_XXH3
• build: fix xxh_x86dispatch build with MSVC, by @apankrat
• build: XXH_INLINE_ALL can always be used safely, even after XXH_NAMESPACE or a previous XXH_INLINE_ALL
• build: improved PPC64LE vector support, by @mpe
• install: fix pkgconfig, by @ellert
• install: compatibility with Haiku, by @Begasus
• doc : code comments made compatible with doxygen, by @easyaspi314
• misc : XXH_ACCEPT_NULL_INPUT_POINTER is no longer necessary, all functions can accept NULL input pointers, as long as size == 0
• misc : complete refactor of CI tests on Github Actions, offering much larger coverage, by @t-mat
• misc : xxhsum code base split into multiple specialized units, within directory cli/, by @easyaspi314