michaelfeil/infinity v3.0.0-alpha.4

Infinity v3.0.0-alpha.4

on GitHub

Alpha release of Infinity v3.

Changes since alpha.3:

• Add LibTorch UltraBERT embeddings and sequence/token classification support (#697).
• Support OneJev block attention with shared prefixes, with packed execution and gated chunked DeltaNet improvements (#708).
• Improve Qwen vision memory use, BF16 execution, fused attention, and configured dtype handling in LibTorch and Candle (#703–#706).
• Default backend selection to a compiled backend, preferring LibTorch when available (#710).
• Add opt-in billing response headers through INFINITY_BILLING_HEADERS (#700).
• Add an experimental persistent LibTorch prefix cache, disabled by default. Known regressions remain; this port does not enable it for production (#712).
• Remove heuristic transient-memory admission from LibTorch (#709).
• Remove SM75/Turing builds and FlashAttention v1; CUDA inference requires SM80/Ampere or newer (#711).
• Correct the scoped OCI license metadata for v3 images.

Images:

docker pull ghcr.io/michaelfeil/infinity_v3:nvidia-3.0.0-alpha.4
docker pull ghcr.io/michaelfeil/infinity_v3:cpu-3.0.0-alpha.4

The NVIDIA image uses LibTorch; the CPU image uses Candle. Images publish asynchronously through the tag-triggered Infinity v3 Docker workflow. Wait for its build and image-start checks to pass before pulling the new tags.

LibTorch guide · Experimental prefix cache · Licensing and component notices · Full changelog