kunobi-ninja/kache v0.27.0

v0.27.0: warm new worktrees, self-cleaning target directories

on GitHub

This release is for machines that run many checkouts at once, such as coding agents working in git worktrees. A new worktree now starts with its dependencies already built, and the daemon removes target directories and build units nobody uses any more. Kache also gains a native Google Cloud Storage remote, and it no longer lists S3 buckets unless you ask it to.

Upgrading

The first build after upgrading is cold. The cache key version goes from 31 to 32 (#1222), so no entry written by 0.26.x matches any more, locally or on a remote. A shared remote used by both 0.26.x and 0.27.0 clients holds two sets of entries that never serve each other, so upgrade CI runners and developer machines together. kache gc --stale-schema drops the old local entries. Kache never deletes remote objects, so the old ones stay until your bucket's lifecycle rule expires them.

The local index migrates to generation 7 the first time 0.27.0 opens it (#1198, #1306). The first GC housekeeping pass afterwards also rebuilds the file_hashes table in a smaller form (#1228).

Defaults that change what Kache does on your machine or your remote:

S3 readers now need only s3:GetObject. Without s3:ListBucket, S3 answers a missing key with 403, and Kache counts that refused read as a miss. The catch: a policy that denies the prefix, or an expired token on an existence check, also reads as a miss, and the build compiles instead of failing. Run kache doctor to check the remote (see below). If you want the old behaviour, set remote_key_listing = true and grant s3:ListBucket. Listing only feeds the planner's last-resort candidates for crates that no build manifest, shard or local history covers. kache sync still lists and still needs s3:ListBucket.

The experimental daemon-assisted local hit path is gone (#1259). [cache] local_hit_daemon, KACHE_LOCAL_HIT_DAEMON and KACHE_LOCAL_HIT_TIMEOUT_MS are now ignored. A config file that still sets the key loads as before.

Worth rerunning once after upgrading:

• kache init on Unix now offers to pace test binaries in every project (#1263). Existing installs get it only by rerunning init.
• kache install-shims and kache daemon install now record a path that survives upgrades under Homebrew, Nix and mise (#1307, #1318). Shims installed earlier may point into a versioned directory and break when that version is removed. kache doctor now reports a broken farm, and rerunning install-shims repairs it.

If you run kache-service, update it before the clients: 0.27.0 uploads build timeline schema 7, which older services reject (#1172).

New

Target directories that clean up after themselves

• The daemon removes target directories whose workspace was deleted and that no build has used for a day (#1267). It checks at most once an hour on a quiet machine, and only directories that builds through Kache recorded. A directory must still be the Cargo target directory that was recorded, with Cargo's CACHEDIR.TAG. It is never removed when it holds Cargo.toml or .git, or while a build holds its lock. The daemon log names each directory it removes. Like kache clean, the removal does not stop at a mount point inside the target directory. cache.auto_clean_idle_targets_days = N also removes targets idle for N days; that is off by default.
• Inside target directories that stay, the same check removes build units no build has used for 30 days (#1296). Kache tells used from unused by access times: the first check marks each unit, so nothing goes until 30 days after that. Nothing is removed on noatime or network filesystems.
• kache targets lists every tracked target directory with its size, idle time and what deleting it frees (#1237).
• kache clean --orphans selects the targets of deleted worktrees at any age, and no clean mode deletes a target a running build holds (#1234). kache clean also measures custom and cross-target profiles and finds worktrees under .worktrees/ and .claude/worktrees/ (#1235).

New checkouts start warm

The first build in a new worktree used to restore every dependency from the cache, one compiler call at a time. Now the daemon copies the registry and git units of the new checkout's Cargo.lock from the most recently used target directory built by the same rustc, and Cargo compiles only the workspace (#1292, #1306). Workspace members and path dependencies are never copied. It covers the host debug profile and needs a running daemon. A --target-dir passed on Cargo's command line is invisible to the wrapper, so seeding lands in <workspace>/target unused.

Pacing tests and compiles

• kache test-runner makes test binaries take slots from the same pool compiles use (#1179), and kache init can now turn it on for every project through CARGO_TARGET_<TRIPLE>_RUNNER (#1263). A project's own runner (sudo -E, an emulator) still runs the binary. When the test runner cannot tell which runner Cargo would have used, the test fails with the reason. A lone cargo test keeps a quarter of the pool free for compiles, so it runs fewer test threads than before.
• When the machine or its cgroup is short of memory, compiles run one after another until pressure eases (#1299). Kache reads the macOS memory pressure level and Linux memory PSI. Compiles started inside a test still take no permit, so this does not guarantee a compile runs alone on the machine.

Remote cache

• A native Google Cloud Storage remote, type = "gcs", using Application Default Credentials, including GKE workload identity (#1295). No HMAC keys needed.
• pull_request_prefix lets pull request jobs write to a prefix of their own, so a second push reuses what the first one built (#1297). They still read the main prefix first, and protected branches never read the pull request prefix. The bucket policy is the boundary; the CI docs show the IAM split.
• kache doctor now contacts the remote and names the cause of a failure: expired credentials, a wrong key, clock skew, the bucket's real region, or a missing bucket (#1289). It also fails a plain http:// endpoint on another machine.
• CI jobs that share a lockfile merge their build manifests and shards instead of overwriting each other (#1261, #1274). Stores without conditional writes, including GCS for manifests, can still lose entries when two jobs finish at the same moment.
• Packs and entry manifests upload create-only, so a second uploader leaves the first copy alone (#1288).
• After a dependency bump, prefetch borrows the manifests of the last three committed lockfiles (#1291). Shallow clones skip this.

Cargo 1.100

Kache understands Cargo 1.100's per-unit build layout, already the default on nightly, in keys, build-script caching, macOS debug info and cleanup (#1222, #1240). If Cargo writes to a layout Kache does not recognise, it says so once per build.

Opt-in: hermetic build-script runs

KACHE_BUILD_SCRIPT_HERMETIC=1 (Unix, not as root) runs cached build scripts with OUT_DIR at one path named by their inputs, so worktrees share one read-only copy through a symlink (#1243). It fixes the case where a script writes a value derived from its OUT_DIR path, such as its length or a hash, which the regular cache cannot detect and serves from the first checkout. GC and kache purge remove shared runs no target directory links to (#1252). It stays off by default.

Faster

• Builds in a second checkout reuse more of the first one's work. Input records for registry units (#1191), workspace members (#1202) and vendored crates (#1304) now serve other checkouts, which cut warm other-checkout builds by 23 to 44% on our benchmarks. Records also travel through the remote, so a fresh CI runner skips most of rustc's dep-info pre-pass (#1204).
• Registry proc macros that bake an empty OUT_DIR path into their output (expander, the wasmtime component macros) now hit in every checkout (#1203). A second-checkout cargo check -p polkadot went from 220 misses to 6. Remote hits for these units need the same cache directory path on both machines.
• Build scripts that bundle C libraries, such as libz-sys and libgit2-sys, hit in another checkout (#1238, #1246).
• Restores copy less. On Linux, proc macros and build-script binaries share the store's inode (#1187). On arm64 macOS, a restored binary's signature is checked once per blob (#1196). The daemon stages large executables before the build needs them (#1248).
• Adaptive incremental mode now switches on in a normal edit loop (#1322). It used to need two misses of a unit less than 60 seconds apart and gave up after 30 idle seconds. Now any build of a unit, a cache hit included, followed by an edit seeds incremental state, and the unit keeps skipping the cache for up to 15 idle minutes. From the second edit on, rebuilds of an edited crate ran at about Cargo's own speed in our test (42 s down to 16 s). The first edit after a restored build still compiles in full.
• The scheduler sizes a permit slot by the machine's memory instead of a fixed 512 MiB, so large links run side by side on machines with memory to spare (#1323). On Linux the size is bounded by the cgroup's memory.max. Memory-pressure pacing still applies.
• C and C++ misses cost less: the daemon takes over the store put, and header hashes and include listings are memoised (#1184, #1185, #1249, #1250).

Fixed

Correctness:

• A native archive rebuilt in place could restore a stale binary when it reached the output through a dependency, #[link(kind = "static")], a link modifier or -C link-arg (#1179, #1182). Each route is now keyed, or the unit is not cached. An archive in a system library directory that reaches an output without an -l naming it is still not keyed.
• A C compile handed to the daemon late could end up neither stored nor uploaded (#1316). A daemon that stalls for more than 5 seconds between claiming the key and reading the hand-off can still drop one.
• kache doctor --verify and --repair no longer abort on cached build-script entries (#1247).
• trybuild UI tests pass under Kache (#1189).
• Kache's own progress and warning lines no longer come back when Cargo replays a fresh unit's stderr (#1197).
• kache cargo build no longer leaks its build-dir override into nested Cargo builds started from a build.rs (#1188).
• Two Kache installs with shim farms on one PATH no longer run each other's shims in a loop, and on macOS automatic GC runs again when Kache was started through a shim (#1179).
• C++ compiles with -faligned-new (rust-rocksdb) are cached (#1245).

Cache maintenance:

• C and C++ misses that compile before keying take a scheduler permit again. Since 0.26.0 they skipped it, which could oversubscribe the machine when make -j and cargo -j ran side by side (#1286).
• Automatic GC keeps entries the remote delivered in the last 6 hours, so a CI job does not lose its warm set between cargo commands (#1198). kache gc still evicts them.
• Size GC stops fighting bytes that target directories still hold, and no longer counts APFS snapshot-held blocks as held by targets (#1227, #1194, #1253). It also waits for the index write lock instead of skipping evictions (#1183).
• file_hashes rows are pruned 30 days after they were written (#1214).
• [cache.volumes] shards are swept against their own budget (#1262).
• Event log rotation keeps the build in progress, so kache report counts large builds fully (#1213).
• Prefetch discovery through cargo metadata has a 3-second budget and backs off after failures (#1210).

Daemon:

• The daemon starts again in runtime directories that fit its socket but not the longer control endpoint that 0.26.3 added, such as deep self-hosted CI work directories (#1217).
• A daemon that fails to start says why and where its log is (#1175, #1177, #1226).
• On Unix, Ctrl-C in a build no longer stops the daemon (#1219). On Windows, a daemon auto-started under Cargo still stops with the build; kache daemon install avoids that.
• On macOS, kache daemon start loads a launchd job that was booted out instead of failing (#1224).

Platform

• macOS: cached test binaries no longer print dsymutil warnings, and their dSYMs carry debug info again, a regression from 0.26.0 (#1168). Links without Rust debuginfo no longer embed the checkout path of bundled C objects, and a proc macro built fresh in two checkouts is byte-identical, so dependents hit in the second one (#1199, #1200).
• Windows: build scripts and proc macros linked through Firefox's Python cargo-linker are cached, and hosts with two MSVC toolsets no longer refuse to key a link (#1231, #1233).
• Nix: Kache builds with nixpkgs' own rustc on aarch64 Linux, so the nixpkgs package can drop its NIX_LDFLAGS=-lc workaround (#1254).
• New crate: kache-shims, published with Kache's version, holds shim install and path selection (#1307).
• Helm chart: service.appProtocol sets the Service port's appProtocol for an HTTP/2 Ingress (#1223).

Full changelog: v0.26.3...v0.27.0