Changes from 3.1.5 to 3.2.0
New features
-
Opt-in cross-process locking for disk-based frames, via a sidecar
lock file (.b2lock,flock/LockFileEx) plus a generation
counter that detects mutations other handles made even when the on-disk
length is unchanged. Enable it per-handle (blosc2_stdio_params.locking,
or the equivalent for other I/O backends) or fleet-wide with the
BLOSC_LOCKINGenvironment variable. Advisory: every handle touching
the container must opt in. Not supported over NFS. -
blosc2_schunk_lock()/blosc2_schunk_unlock(): a bracket API to
hold the exclusive frame lock across several operations, making a
multi-step mutation atomic to other locked handles instead of only each
individual call. No-ops on a handle without locking enabled.
b2nd_resize()andb2nd_set_slice_cbuffer()now bracket their own
multi-chunk sequences this way too, so a locked reader never observes a
half-applied resize or slice write, and two overlapping slice writers no
longer interleave at chunk granularity. -
Growth-SWMR (single writer, multiple readers): a reader handle
opened on a disk-based frame or b2nd array now follows shape/length
growth made through another handle (typically another process) on its
next access, without reopening. New publicb2nd_refresh()reader-side
API, andexamples/b2nd/example_growth_swmr.cdemonstrating the
contract. Works with or without locking; only the on-disk length is
polled without it, so growth is virtually always noticed but a
same-length mutation may not be until the next one that changes it. -
Stale-handle coherence fixes: a handle whose cached view of an
on-disk frame goes stale (another handle rewrote it) now re-syncs before
trusting cached state, closing gaps inblosc2_vlmeta_exists()and in
the counters (nbytes/cbytes/nchunks) that
blosc2_schunk_append_chunk()/blosc2_schunk_insert_chunk()apply their
deltas to. Also fixed:- A race between opening a frame and a concurrent writer growing or
otherwise mutating it (blosc2_schunk_open_offset_udio()could
returnNULLon a transient "frame length exceeds file boundary"
read instead of retrying it). - A torn read during the (unlocked) staleness poll used to hard-error
instead of keeping the cached view and retrying on the next poll, as
intended. - A leak, and a free of an uninitialized pointer in
blosc2_schunk_copy(), on an unreadable-frame staleness poll (hit
legitimately by a freshly created, not-yet-flushed file-backed
schunk).
- A race between opening a frame and a concurrent writer growing or
API/ABI notes
- Appends
change_ticktoblosc2_schunkstruct andlast_tickto
b2nd_array_t, and addslockingtoblosc2_stdio_paramsand the new
BLOSC2_ERROR_LOCKerror code. No existing field or function signature
was altered, but this breaks the ABI as two public structs have an
additional field; the shared library'sSOVERSIONis bumped accordingly
(8 to 9).
Notes
- Advisory locking is not a replacement for a real lock manager: it only
protects a container if every handle on it (across every process)
enables it. It does not work on network filesystems (NFS), and is
incompatible with memory-mapped I/O. - See
plans/high-level-formats-locking.mdand
plans/todo-locking-swmr.mdin the source tree for the full design
rationale and known follow-ups.