borgbase/vykar v0.20.0

on GitHub

A feature release introducing repository format v3 with BLAKE3 chunk and pack IDs, alongside memory-use reductions across backup and check, an SSH stack upgrade, and several correctness fixes.

Repository format v3 (BLAKE3)

• vykar init now creates format v3 repositories, whose chunk and pack IDs are BLAKE3 instead of BLAKE2b. There is deliberately no user-facing choice — no --hash flag and no YAML key. The repository version implies the algorithm, and RepoFormat::from_version is the single gate, so an unknown future version is refused outright rather than silently given v3 semantics.

• Existing v2 repositories keep BLAKE2b for life and stay fully readable and writable. Frozen v2 fixtures in the test suite assert this on every run.

• A repository created by this release cannot be opened by older binaries. A fleet sharing one repository must upgrade together before running vykar init.

• Servers: PROTOCOL_VERSION is bumped to 2 and declared only for BLAKE3 repositories, so a new client talking to a pre-BLAKE3 vykar-server gets a clean 400 rather than a false whole-repository corruption report from verify-packs, or repack writing packs at BLAKE2b-named keys. MIN_PROTOCOL_VERSION stays at 1 — new clients keep working against old servers for v2 repositories. The REST backend verifies upload integrity with the repository's hash and probes server capability first.

• vykar info reports the repository format and hash algorithm.

• Only chunk and pack IDs follow the format. The TOFU fingerprint, pin/check-state file names, PathHash, the index-cache checksum and the hard-link fingerprint remain BLAKE2b for every version by design — each site now says so, and the documentation carries a table of every digest with the reason it is compatible as-is.

• The chunk-ID key is now zeroized: ChunkHasher no longer derives Copy, so worker threads take it by reference instead of each duplicating secret material that was never wiped.

Performance

• Backup peak heap down ~21% on a 768-file corpus (median 113.1 MiB to 89.2 MiB across three heaptrack runs). StreamCDC's buffer is now capped by the enforced read limit, shrinking four reader buffers from 32 MiB to 8 MiB. Chunk boundaries are preserved.

• check peak RSS down ~24% on a 49 GiB corpus (192.1 to 147.0 MiB; check --verify-data 329.2 to 284.5 MiB), by releasing the server-verification pack map before the local scan.

• Backup buffers retain less capacity. Prepared chunks are shrunk after packing when at least half the reserved worst-case capacity is unused, and pack buffers now allocate room for the header so the blob crossing the target no longer forces Vec to double a buffer that is about to be flushed.

• glibc allocator tuning at CLI startup (M_ARENA_MAX=2, M_MMAP_THRESHOLD=1 MiB). glibc's dynamic mmap threshold ratchets to its 32 MiB ceiling early in a backup, after which multi-MiB chunk and pack buffers come from an arena that is never trimmed back. Either knob is skipped if you already set the matching MALLOC_* variable or GLIBC_TUNABLES entry. musl builds use mimalloc and are unaffected.

Bug fixes

• S3 uploads no longer abort on a recoverable mid-part connection reset. ureq 3.4.1 changed how a peer reset arriving mid-request-body on a pooled connection is reported — it now surfaces as InvalidInput rather than a connection-reset kind, which vykar classified as permanent and gave up on. That is the #151 scenario: a transient S3 blip failing the whole backup.

• Config parse errors no longer echo file snippets. A YAML error near a credential — including values expanded from an env_file — could print adjacent secrets. Error locations are retained.

• vykar.example.yaml now loads as shipped. Literal ${DB_USER}-style placeholders inside a commented-out command_dumps block still went through environment expansion, which runs over the whole file before YAML parsing, so copying the example and running vykar failed immediately with environment variable 'DB_USER' is not set. An integration test now asserts the shipped file parses.

• Odd chunker parameters are rejected with actionable guidance rather than producing subtly different boundaries. FastCDC 5 requires even sizes; existing snapshots in repositories with odd stored parameters stay readable and restorable.

• SSH host-key handling drops an unused async.

Dependencies

• SFTP: russh 0.62 → 0.63.2 and russh-sftp 2.4 → 3.0, picking up GHSA-p8qx-h547-fjw9 and GHSA-47hw-gvq5-r2gm plus hardening of PKCS#8 parsing, agent unlock comparison and Curve25519 parameter sanitisation. check_server_key changed signature; host certificates are now rejected explicitly rather than flattened to their public key, which would have pinned a certified host key without ever validating the signing CA.

• serde_yaml replaced with serde-saphyr 1.2. serde_yaml has been unmaintained since March 2024 and parses every config file vykar reads; serde-saphyr is pure Rust over the YAML-1.2 saphyr parser and avoids the transpiled-C unsafe-libyaml its forks still carry. strict_booleans restores serde_yaml 0.9's scalar rules exactly, so label: no keeps meaning the string no rather than becoming a boolean.

• blake2/argon2 to RustCrypto 0.11/0.6; fastcdc, zstd, xorf, base64, quick-xml, lz4_flex, croner and comfy-table updated. FastCDC 4 is gone from both lockfiles.

Documentation

• Rationale for BLAKE3, the v2/v3 version mapping, and why there is no stored hash field (#177).
• restore --pattern path semantics.
• Updated benchmarks for BLAKE3 chunk IDs.

Updating

To update to the latest version, run:

curl -fsSL https://vykar.borgbase.com/install.sh | sh

See the quickstart guide for other install options.

Downloads

SHA256 checksums are attached to this release.