GitoxideLabs/gitoxide gix-pack-v0.76.0

gix-pack v0.76.0

on GitHub

Bug Fixes

• reject incomplete delta traversals and overlapping offsets

  A Report describes indexed delta components that cannot be reached from
  any base object yet pass delta-tree verification. Require every indexed item
  to have been inspected before returning success, using the existing object
  counter. Legitimate forward references remain supported.

  Return corruption errors for duplicate or overlapping index offsets and for
  an entry at or past the pack end, instead of panicking. The reported decoder
  cycle and allocation problem is handled by the fix.

  Three regressions failed before the fix: an unresolved cycle was accepted,
  while duplicate offsets and an invalid pack end panicked. Coverage includes
  self and multi-node components with and without a valid root, plus a valid
  forward delta base.

  Git reference: builtin/index-pack.c::conclude_pack() at
  d38352cd43ab9745686d697872408bc3249a153f rejects unresolved deltas.

• reject cyclic delta chains and bound chain allocations

  A report describes delta base cycles that never terminate in header
  lookups and keep allocating during object decoding. Track a constant-space
  cycle checkpoint in both loops, without relying on the untrusted pack
  object count or rejecting valid forward references.

  Account for heap-backed delta-chain metadata in the existing per-allocation
  limit and reserve it fallibly, including when delta payloads are empty.
  The regressions failed before the fix and cover self, multi-entry, mixed
  REF/OFS cycles, empty deltas, and valid forward chains.

  Git reference: t/t5309-pack-delta-cycles.sh at
  d38352cd43ab9745686d697872408bc3249a153f rejects cycles while allowing
  forward delta bases.

• map packs read-only on Windows so large ones can be opened
  On Windows if you memory map with copyonwrite (which is what map_copy_read_only does) it charges against the commit limit for that. As such, when opening extremely large packs, i.e. 40GB, This can fail with an out of memory error: ERROR_COMMITMENT_LIMIT (os error 1455, "The paging file is too small for this operation to complete")

  The fix is to memory map it plainly. Which is otherwise pretty much the same as it is on other operating systems for read-only memory mapping.

Bug Fixes (BREAKING)

• validate multi-pack-index references on access

  Looked at everything in detail and added error handling in callbacks
  that didn't have them. Reduced the scope of this commit to not overlap
  with odb-parallelism branch which makes many changes additionally.

  Many of the newly added tests are very specific about error handling,
  one might call them overkill, but I left them in for good measure.
  Further, errors.rs in gix-odb I just skimmed the test titles off
  as everything else would be too time consuming - better have them than
  not have them I thought, particularly to support odb-paralellism.

  A report describes object lookup panics caused by unchecked pack IDs
  and large-offset ordinals in otherwise structurally valid MIDX files.

  Validate these references when reading an object entry, before indexing
  pack arrays or dereferencing the optional LOFF (Large Offsets) table.
  Bound LOFF ordinals by the chunk itself, not by the remaining file.
  Keep MIDX loading limited to structural validation: an eager scan of
  every offset entry adds linear startup work and faults in memory-mapped
  pages even when a command needs only a handful of objects. Git likewise
  checks these references on access.

  Make pack_id_and_pack_offset_at_index() return gix_error::Result
  and iter() yield fallible entries. Propagate corruption errors through
  object and header lookup, delta-base resolution, integrity verification,
  and CLI entry listing. ID-only iteration remains available without valid
  offset references. Preserve literal high-bit 32-bit offsets when LOFF
  is absent.

  Return Result<Option<_>> from Find::location_by_oid() and
  Find::pack_offsets_and_oid(): absence and lookup failure are different
  outcomes, and callers must be able to distinguish them. Propagate errors
  through forwarding implementations, object counting, pack-entry iterator
  construction, and thin-pack base lookup rather than treating corruption
  as a missing object or a reason to recompress. Report a pack that becomes
  unavailable during construction as an error instead of panicking.

  Retain failed index loads separately from missing files so repeated
  lookups cannot silently turn corruption into absence. Load pending
  indices before retrying failures, retry all failed slots, and reconcile
  disk state when refreshing. This keeps independent valid packs usable
  and permits recovery after repair or removal without endless refresh
  loops on persistent failures.

  Regressions cover boundary and extreme invalid references, lazy loading,
  valid LOFF ordinals and literal offsets, integrity verification,
  corruption propagation through object lookup and pack generation, and
  repeated, shared, concurrent, repaired, and removed index-load failures.

  Git reference: d38352cd43ab9745686d697872408bc3249a153f,
  midx.c::midx_for_pack() and nth_midxed_offset().

Commit Statistics

• 15 commits contributed to the release over the course of 13 calendar days.
• 13 days passed between releases.
• 4 commits were understood as conventional.
• 0 issues like '(#ID)' were seen in commit messages

Commit Details