michaelfeil/infinity v3.0.0-alpha.8

Infinity v3.0.0-alpha.8

on GitHub

Changes since alpha.7:

• Count each distinct OneJev block-attention prefix once per request, plus every question suffix, in JSON input usage and billing headers. A 1,000-token shared prefix plus ten 100-token suffixes reports 2,000 input tokens (#732).
• Replace INFINITY_TORCH_PREFIX_CACHE_BYTES with INFINITY_TORCH_PREFIX_CACHE_SIZE, using explicit units such as 4GiB or 4GB. 0 disables the cache; bare nonzero values fail startup (#731).

CPU and NVIDIA image builds run asynchronously. Check the workflow for publication and startup-check results:

docker pull ghcr.io/michaelfeil/infinity_v3:nvidia-3.0.0-alpha.8
docker pull ghcr.io/michaelfeil/infinity_v3:cpu-3.0.0-alpha.8

Image builds · Licensing · Full changelog