github Blosc/c-blosc2 v3.2.0
Release 3.2.0

latest releases: v3.3.2, v3.3.1, v3.3.0...
one month ago

Changes from 3.1.5 to 3.2.0

New features

  • Opt-in cross-process locking for disk-based frames, via a sidecar
    lock file (.b2lock, flock/LockFileEx) plus a generation
    counter that detects mutations other handles made even when the on-disk
    length is unchanged. Enable it per-handle (blosc2_stdio_params.locking,
    or the equivalent for other I/O backends) or fleet-wide with the
    BLOSC_LOCKING environment variable. Advisory: every handle touching
    the container must opt in. Not supported over NFS.

  • blosc2_schunk_lock() / blosc2_schunk_unlock(): a bracket API to
    hold the exclusive frame lock across several operations, making a
    multi-step mutation atomic to other locked handles instead of only each
    individual call. No-ops on a handle without locking enabled.
    b2nd_resize() and b2nd_set_slice_cbuffer() now bracket their own
    multi-chunk sequences this way too, so a locked reader never observes a
    half-applied resize or slice write, and two overlapping slice writers no
    longer interleave at chunk granularity.

  • Growth-SWMR (single writer, multiple readers): a reader handle
    opened on a disk-based frame or b2nd array now follows shape/length
    growth made through another handle (typically another process) on its
    next access, without reopening. New public b2nd_refresh() reader-side
    API, and examples/b2nd/example_growth_swmr.c demonstrating the
    contract. Works with or without locking; only the on-disk length is
    polled without it, so growth is virtually always noticed but a
    same-length mutation may not be until the next one that changes it.

  • Stale-handle coherence fixes: a handle whose cached view of an
    on-disk frame goes stale (another handle rewrote it) now re-syncs before
    trusting cached state, closing gaps in blosc2_vlmeta_exists() and in
    the counters (nbytes/cbytes/nchunks) that
    blosc2_schunk_append_chunk()/blosc2_schunk_insert_chunk() apply their
    deltas to. Also fixed:

    • A race between opening a frame and a concurrent writer growing or
      otherwise mutating it (blosc2_schunk_open_offset_udio() could
      return NULL on a transient "frame length exceeds file boundary"
      read instead of retrying it).
    • A torn read during the (unlocked) staleness poll used to hard-error
      instead of keeping the cached view and retrying on the next poll, as
      intended.
    • A leak, and a free of an uninitialized pointer in
      blosc2_schunk_copy(), on an unreadable-frame staleness poll (hit
      legitimately by a freshly created, not-yet-flushed file-backed
      schunk).

API/ABI notes

  • Appends change_tick to blosc2_schunk struct and last_tick to
    b2nd_array_t, and adds locking to blosc2_stdio_params and the new
    BLOSC2_ERROR_LOCK error code. No existing field or function signature
    was altered, but this breaks the ABI as two public structs have an
    additional field; the shared library's SOVERSION is bumped accordingly
    (8 to 9).

Notes

  • Advisory locking is not a replacement for a real lock manager: it only
    protects a container if every handle on it (across every process)
    enables it. It does not work on network filesystems (NFS), and is
    incompatible with memory-mapped I/O.
  • See plans/high-level-formats-locking.md and
    plans/todo-locking-swmr.md in the source tree for the full design
    rationale and known follow-ups.

Don't miss a new c-blosc2 release

NewReleases is sending notifications on new releases.