apache/datasketches-cpp 5.3.0

Apache Release DataSketches-5.3.0

on GitHub

Apache DataSketches C++ 5.3.0

This release focuses on correctness and cross-language compatibility with datasketches-java, plus hardening of deserialization against malformed or truncated input. There are no intentional breaking API changes. Please review the serialization and behavior changes below if you exchange sketch images with other languages or rely on edge-case behavior.

Serialization and compatibility changes

• Theta: a compact sketch holding a single item now writes the single-item flag, matching Java's serialized form (#526)
• Theta/Tuple: an empty compact sketch now reports a zero seed hash (#518), and a-not-b ignores the seed hash of an empty B (#517)
• Theta: sketches with lg_k = 4 are now allowed, matching Java (#514)
• HLL: serialization flag bit 0x20 is no longer written as a "full size" flag. Java reads that bit as its union-rebuild flag, so the two languages disagreed about what an image meant (#521)
• HLL: finished repairing the deferred KxQ rebuild so union results no longer depend on merge order, and pinned the estimator constants (#522). Also fixed a union estimate/serialization regression caused by the lazy KxQ rebuild (#512)
• Frequent Items: is_empty() is now correct after a purge clears every counter (#527). Emptiness is decided from the preamble longs, and a new reset() method was added (#529)

New features

• Theta: update_theta_sketch::compact() takes an optional trim flag that reduces the result to at most the nominal size k (#524)
• Count-Min: new count_min_sketch::serialize_to (#509)
• t-digest: const_iterator over the centroids (#458)

Bug fixes

• t-digest: fixed the rank and quantile tails (#531) and kept the stored min and max when merging (#525)
• t-digest: const_iterator no longer returns a dangling reference (#474), infinite values are rejected (#471), and a bug fix was ported from Java (#462)
• REQ: fixed get_RSE() (#497) and added a test for it (#498)
• VarOpt: only positive weights are accepted, following the original paper and matching Java (#495)
• Quantiles: deserialize() no longer aborts with SIGABRT on certain inputs (#478)
• Binomial bounds: fixed a division by zero (#470)
• Theta: compressed-iterator cleanup to align with Java (#453)
• Added missing headers for GCC 15 (#500), and missing <cstdint> and <iterator> includes in the HLL and CPC internal headers (#530)
• Fixed an error message in the CPC compressor for empty window data (#482)

Deserialization hardening

• Header fields and sizes are now validated before use when deserializing CPC, Count-Min, Theta (serial version 4) and VarOpt (#538)
• Compact Theta checks the buffer size before reading preamble fields (#536)
• Compact Tuple bounds-checks each key read (#511)
• HLL CouponList validates the coupon count (#503)
• Bloom filter validation is stricter when deserializing from bytes (#516)

Malformed or truncated input that earlier versions might have read past now throws an exception.

Performance

• Count-Min: removed an allocation from get_hashes (#506) and added a micro-benchmark (#507)
• push_back replaced with emplace_back in several places (#486)

Code quality, tests, build and docs

• clang-tidy configuration and fixes, and consistent braces across KLL, HLL, FI, CPC and Count-Min (#465, #479, #483, #487, #488, #490, #491)
• New tests and coverage: KLL long-sketch compatibility (#481), REQ exact mode from serialized bytes (#508), error-message checks with REQUIRE_THROWS_WITH (#455)
• CI: Apache RAT license-header audit (#523), updated code coverage workflow (#493, #496), branch protection (#454), cross-language tests against the current Java toolchain (#456, #463, #468), Count-Min test snapshots in binary mode (#505), removal of a stale Python CMake hook (#510)
• Docs: README refresh (#472), link to datasketches-rust (#501), cross-language compatibility notes for string sketches (#492), typo fixes (#459), .gitattributes updates (#504)
• Refactoring: use get_preamble_longs (#467)

Contributors

Thanks to everyone who contributed to this release: @AlexanderSaydakov, @alexey-milovidov, @ankane, @cv4g, @devillove084, @FluorineDog, @geonove, @jaideeppyne, @jihuayu, @leerho, @MaheshGPai, @PointKernel, @proost, @SavicStefan, @SYaoJun, @tisonkun

Full Changelog: 5.2.0...5.3.0