github GitoxideLabs/gitoxide git-pack-v0.16.0
git-pack v0.16.0

2 years ago

New Features

  • zero-objects check for index and multi-index integrity validation
  • support for fan-checking in index and multi-index integrity verification
  • introduce type for entry indices within an index or multi-index
    That way it's a littl emore descriptive than a bare u32.
  • git_pack::Find::try_find_cached(…, pack_cache)
    With this method it's easier to bypass local caches and control
    the cache oneself entirely.
  • A simplified version of the Find trait
    It's meant for the next generation of object db handles which keep a
    local cache of all the details of the actual object database.

Bug Fixes

  • support Rust 1.52

  • Allow resolution of in-pack ref-deltas
    This finally allows delta tree caches to be used on typical small packs
    returned by GitHub.

  • don't try to short-cut detection of large offsets when writing index files
    The code incorrectly assumed that the input is sorted by offsets, with
    the largest offset being last, even though by all means that's not the
    case.

  • Avoid the dashmap being cloned for each thread
    Intead, share it by reference, it's sync after all.

    This issue was introduced when switching to a Send + Clone model,
    instead of Send + Sync, to allow thread-local caches in database
    handles of all kinds.

  • linked::Store now assures unique IDs across compound stores

Changed (BREAKING)

  • Improve method signatures of cache::Tree::*

  • cleanup and unify verify_integrity() method signature
    Previously they used many different ways of handling their parameters
    despite all boiling down to calling the same 'index::File::traverse()`
    method.

    This allows for more reuse of Options structs and generally makes
    clearer how these optinos are used.

  • index::File::traverse() now returns an Outcome struct instead of tuple of 3 fields

  • remove Option<impl Progress> in favor of impl Progress

  • multi-index integrity check; use integrity::Outcome for various integrity checks

  • remove unnecessary Arc around should_interrupt flag

  • remove Sha1 mentions in index::verify::Mode::* variants
    The hash is repository defined and not hard-coded

  • introduce index::File::verify_integrity(…, pack: Option<PackContext>, …), replacing tuple
    This allows for more documentation on what input is required there and
    generally makes for an easier to use API.

  • rename oid::try_from() to try_from_bytes(), add from_bytes_unchecked()
    This change was done in the name of consistency, as from_bytes() is
    used in many other git-* crates

  • Change accessors named hash_kind() to object_hash() for consistency

  • consistently use object_hash instead of hash_kind

  • data::Entry::from_read() now takes a hash lengths as parameter
    That way ref-deltas can be interepreted without hard-coding SHA1

  • data::Entry::from_bytes(…, hash_len) takes new parameter
    The hash-len tells it how to interpret ref-delta objects, which
    store the complete hash of the base object.

    This is now entirely configurable.

  • index::File::at() with git_hash::Kind parameter
    It will allow to assume different hashes even in the index file format
    which isn't yet capable of storing this information.

  • data::File::at() and Bundle::at() now have hash_kind parameter
    It's used to configure the kind of hash to assume when reading packs and
    indices.

  • remove make_object_cache parameter from git_pack::data::output::count::objects()
    It now is an implementation detail of the Find trait.

  • move bundle::Location to data::entry::Location
    The latter place best describes its purpose.

  • remove pack-cache from Find::try_find(…)
    With the new architecture this can be an implementation detail without
    forcing it to be Sync.

  • move git_pack::data::Object to git_object::Data, massively alter git_odb::Find trait
    This will break a lot, but has to happen to prepare these traits for the
    next generation of object databases.

Commit Statistics

  • 146 commits contributed to the release over the course of 45 calendar days.
  • 55 days passed between releases.
  • 33 commits where understood as conventional.
  • 6 unique issues were worked on: #260, #266, #279, #287, #293, #67

Thanks Clippy

Clippy helped 8 times to make code idiomatic.

Commit Details

view details
  • #260
    • linked::Store now assures unique IDs across compound stores (b605c1f)
  • #266
    • remove unused dependencies (c800fdd)
    • upgrade dashmap to latest version (52d4fe5)
    • refactor (b88f253)
    • refactor (52a4dcd)
    • Make single-threaded programs possible to use with git-repository (dde5c6b)
    • Use new odb in place of the old one and it works (8ad25c5)
    • Make find::Entry self-contained (ad36fb9)
    • Remove iterator access in favor of fully owned data (62d3f10)
    • Adjust pack-create to changes in git-pack (12db899)
    • remove make_object_cache parameter from git_pack::data::output::count::objects() (3f05fea)
    • :Find implementation for Rc (f48630b)
    • MultiPackIndex compatible pack::Find trait definition (5fa1a9d)
    • git_pack::Find::try_find_cached(…, pack_cache) (58c2edb)
    • refactor (3310d8f)
    • remove pack-cache from Find::try_find(…) (ebc7f47)
    • fix docs (1bb4253)
    • move git_pack::data::Object to git_object::Data, massively alter git_odb::Find trait (2290d00)
    • A simplified version of the Find trait (e25f4ea)
    • Remove CRC32 check entirely as it doesn't seem to be important in the big picture (22d35bd)
    • Notes about multi-pack indices in the current data::entry::location (7eff6bf)
    • Add 'contains()' method to Find (dfdd6fb)
    • move bundle::Location to data::entry::Location (82b9b33)
    • Use existing git_features facilities (ed0c266)
  • #279
    • Add a less thorough and faster way of verifying multi-indices (7517482)
    • refactor (91e6d38)
    • Allow interrupting multi-index creation more often (f223ecb)
    • also test pack-creation with multi-index repo (235a27a)
    • better multi-pack verification progress (2e16f13)
    • Handle large multi-pack indices correctly (4f6b030)
    • Fix progress and handling of large of multi-pack index offsets (5dc1f81)
    • add missing docs (4137327)
    • write progress for multi-pack writing (1bea1d4)
    • adapt to changes in git-features (542c0df)
    • progress for chunk writing (50fde01)
    • multi-pack index writing complete with large-offset support (f7d5c7f)
    • write pack-ids and offsets (bfc8069)
    • Add chunk for oids (565a7ae)
    • Write the fanout table (6a68ed7)
    • refactor (93dc660)
    • Write multi-index header along with path-names chunk (2fc6751)
    • Sketch all the chunk-write API and use it from multi-index write (5457761)
    • Add frame for writing a multi-pack index (9ce1e7f)
    • index::File::traverse() now returns an Outcome struct instead of tuple of 3 fields (2cf7727)
    • refactor (c361ee3)
    • multi-index verification now matches that of git itself (3a76a28)
    • zero-objects check for index and multi-index integrity validation (b80dec2)
    • support for fan-checking in index and multi-index integrity verification (56fc99f)
    • More detailed multi-index verification (8f9a55b)
    • Add remaining docs for multi-index (10a24c1)
    • even nicer printing (d2bea27)
    • docs for multi_index::chunk (73fbc91)
    • nicer printing of index verification results (e3dfa12)
    • very first experimental support for multi-pack index verification (bb35c69)
    • refactor (eafdff4)
    • remove Option<impl Progress> in favor of impl Progress (bf04644)
    • multi-index iteration (1c99903)
    • Access pack-indices and pack-offsets of multi-pack indices (c2a6918)
    • multi-index integrity check; use integrity::Outcome for various integrity checks (6829e5e)
    • oid lookup for multi-pack indices (254f618)
    • remove unnecessary Arc around should_interrupt flag (d851bed)
    • Add basic oid by multi-index file index (a54f552)
    • remove Sha1 mentions in index::verify::Mode::* variants (c2679a0)
    • introduce index::File::verify_integrity(…, pack: Option<PackContext>, …), replacing tuple (80b120d)
    • rename oid::try_from() to try_from_bytes(), add from_bytes_unchecked() (79dc0d5)
    • multi-index verify checksum (853d468)
    • Change accessors named hash_kind() to object_hash() for consistency (2ef9a84)
    • Adapt to changes in git-hash (754a663)
    • Remove unnecessary Default implementation for user of Tree::traverse (9da20e9)
    • fix docs (ce044ef)
    • remove unnecessary Default bound for data in Tree nodes (d548f72)
    • adjust to changes in git-hash (9bf25cc)
    • consistently use object_hash instead of hash_kind (b76f6be)
    • introduce type for entry indices within an index or multi-index (28e3ea8)
    • replace bare u32 data::Id typedef (e0b8636)
    • adjust to changes in git-hash (ca35246)
    • Adjust to changes in git-hash and git-pack (0cae25b)
    • data::Entry::from_read() now takes a hash lengths as parameter (629412b)
    • Adjust to changes in git-odb (710780c)
    • data::Entry::from_bytes(…, hash_len) takes new parameter (851dc2c)
    • refactor (7331e99)
    • adjust to changes in git-hash (07aa1bc)
    • :File uses its hash_len parameter (e6ff1a8)
    • index::File::at() with git_hash::Kind parameter (db8c8c4)
    • data::File::at() and Bundle::at() now have hash_kind parameter (e6a3c9f)
    • remove unnecessary dev-depednency (b71ea6a)
    • adapt to changes in git-hash (82fec95)
    • Calculate trailer offset instead of storing it (bf62067)
    • Make pessimistic size-estimation instead of an optimistic one (69f1d2a)
    • refactor (8b8b4c5)
    • refactor (8c9c7fc)
    • Adapt to changes in git-chunk (44ea5c3)
    • refactor (ac46765)
    • Adapt to latest changes to git-chunk (743d696)
    • Provide multi-index checksum (a363de9)
    • update changelog prior to release (6ae49e3)
    • completely validate and parse multi-index file (e7e40c3)
    • read and validate index names contained in the multi-pack index (24a9790)
    • read and validate fanout chunk (3ca04e3)
    • Read all mandatory and optional chunks (99023bb)
    • Load chunk index of midx file (fac8efa)
    • frame for git-chunk crate to share among git-pack and git-commitgraph (b2d2ae2)
    • basic midx header parsing (edf02ae)
    • First pieces of header parsing; allow to respect multi-index desired hash kind in git-odb (1a2a049)
    • frame for instantiation of multi-pack-index (5e085ec)
  • #287
    • way nicer progress messages for repo verification (4b4f9f8)
    • upgrade to prodash 17 (47860b7)
    • refactor (831397c)
    • Allow resolution of in-pack ref-deltas (84ade1d)
    • refactor (38426a1)
    • Test to reproduce ref-delta forward references and the issue it poses for index traversal (7db7195)
    • Very rough version of repository verification (80a4a7a)
    • refactor (6c06659)
    • Improve method signatures of cache::Tree::* (a79a7fb)
    • cleanup and unify verify_integrity() method signature (91d0476)
  • #293
    • fix docs, again (7b2ab26)
    • fix build (e3977fe)
    • Use InOrderIter from git-features (7721b5f)
    • Basic IEOT parsing (35bdee4)
    • Assure we are right about the leb64 buffer needed for a 64 bit int (7558844)
    • Adapt to changes in git-features: use var-int decoding from there (52e3c6f)
    • remove byteorder from git-pack (4122306)
    • git-pack uses memmap2 instead of filebuffer (d9011c7)
  • #67
    • Avoid the dashmap being cloned for each thread (6d3f52d)
    • Properly count total objects during pack creation (bcb3d37)
  • Uncategorized
    • Release git-actor v0.8.0, git-config v0.1.10, git-object v0.17.0, git-diff v0.13.0, git-tempfile v1.0.4, git-chunk v0.3.0, git-traverse v0.12.0, git-pack v0.16.0, git-odb v0.26.0, git-packetline v0.12.3, git-url v0.3.5, git-transport v0.15.0, git-protocol v0.14.0, git-ref v0.11.0, git-repository v0.14.0, cargo-smart-release v0.8.0 (8f57c29)
    • Release git-features v0.19.1, git-actor v0.8.0, git-config v0.1.10, git-object v0.17.0, git-diff v0.13.0, git-tempfile v1.0.4, git-chunk v0.3.0, git-traverse v0.12.0, git-pack v0.16.0, git-odb v0.26.0, git-packetline v0.12.3, git-url v0.3.5, git-transport v0.15.0, git-protocol v0.14.0, git-ref v0.11.0, git-repository v0.14.0, cargo-smart-release v0.8.0 (d78aab7)
    • Release git-hash v0.9.1, git-features v0.19.1, git-actor v0.8.0, git-config v0.1.10, git-object v0.17.0, git-diff v0.13.0, git-tempfile v1.0.4, git-chunk v0.3.0, git-traverse v0.12.0, git-pack v0.16.0, git-odb v0.26.0, git-packetline v0.12.3, git-url v0.3.5, git-transport v0.15.0, git-protocol v0.14.0, git-ref v0.11.0, git-repository v0.14.0, cargo-smart-release v0.8.0, safety bump 4 crates (373cbc8)
    • Fix git-pack changelog to be stable (fd5b616)
    • prepar changelogs for cargo-smart-release release (8900d69)
    • support Rust 1.52 (42e0487)
    • Release git-bitmap v0.0.1, git-hash v0.9.0, git-features v0.19.0, git-index v0.1.0, safety bump 9 crates (4624725)
    • thanks clippy (d8925f5)
    • thanks clippy (5a68d2f)
    • thanks clippy (1e051dc)
    • don't try to short-cut detection of large offsets when writing index files (ba92cc0)
    • refactor (e7fbd9f)
    • thanks clippy (7bd3ad3)
    • thanks clippy (533a532)
    • Release git-chunk v0.2.0, safety bump 4 crates (b792fab)
    • thanks clippy (35cf46f)
    • refactor (0032223)
    • Merge branch 'oknozor-feat/traversal-sort-by-committer-date' (6add377)
    • make fmt (066f3ff)
    • thanks clippy (4ca9e07)
    • thanks clippy (7dd2313)

Don't miss a new gitoxide release

NewReleases is sending notifications on new releases.