Stability: the CUBLAS_STATUS_NOT_INITIALIZED restart loop is fixed
If your rig has been looping with these two lines, this release is for you:
PoM[gpuN]: a walk still holds the miner after 30s — releasing anyway
PoM[gpuN]: device miner build failed: CublasError(CUBLAS_STATUS_NOT_INITIALIZED)
What was wrong. On a transient GPU fault the miner recovers by uninstalling the card's walk and
rebuilding it. Uninstall waits for the last handle to the walk to drop before the table's VRAM is
released — but the grind loop was still holding one of those handles while it asked for the teardown.
That wait could never succeed. It burned its full 30-second timeout, gave up ("releasing anyway"), and
left the old walk table resident. The rebuild then had to fit a second copy of the model onto
the card, and on any GPU without a spare model's worth of VRAM cuBLAS cannot initialise — so the
rebuild failed, and failed again, until the miner was restarted.
This has been present since the v4 relaunch. It is why v0.11.13 rigs fell over every few hours, and
why v0.11.15 (which doubled the default batch, so bigger buffers and longer launches on already-tight
cards) hit it within minutes.
What changed. The handle is now released before recovery, so the table is actually freed.
Measured against a real injected fault: recovery went from never completing to a successful rebuild
12 seconds later, with no cuBLAS error.
Three further hardening changes around the same path:
- The walk's offset buffers are freed on uninstall. They lived in per-device caches that were never
cleared, so every rebuild had to fit a fresh table around ~100 MiB of stale buffers. - The miner never rebuilds on top of a table that was not released — that allocation cannot succeed
on a tight card. The table is quarantined instead and reclaimed as soon as the stuck walk lets go,
so one card idles briefly rather than the whole process restarting and taking healthy GPUs with it. - The higher starting batch introduced in v0.11.15 now applies only when the card has VRAM headroom
(≥1.5 GB free). Tight cards keep the smaller SM-derived batch until the autotune measures them.
No throughput cost: RTX 3070 1.35 MH/s, RTX 5080 2.98–3.02 MH/s, shares accepted, zero faults.
Also in this release
- macOS / Metal builds again. The Apple build had been broken since v0.11.9: shared code calls
into the CUDA backend under configurations that include Metal, and several functions had no Metal
counterpart. Parity restored. KERYX_FAULT_INJECT_GPU=<ordinal>is wired up again. It existed but was never called after the v4
relaunch, which is why this recovery path had no way to be tested on a healthy rig.