Infernix windows-2026.10.09.2

AI disclaimer: Infernix was developed with AI (mostly Claude Opus 5.5). It is hobby work and
is likely to be neither complete nor entirely accurate.

Highlights

• Qwen3.8-Flash-Next Uncensored: runs
  Qwen3.8-Flash-Next-Uncensored-NVFP4-Infernix,
  a conversion of
  orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4.
  Its routed experts are weight-only 4-bit (W4A16), served by new exact expert routes on the CPU and
  on the GPU (integer tensor cores for prefill); the dense layers stay bit-exact FP8. It needs the
  52 GB PLE n-gram volume from the
  NVIDIA conversion's repository
  (the same file: the abliteration left the n-gram table unchanged).
• One prefix cache: every model now runs on the hybrid prefix cache (content-addressed 64-token
  KV blocks and sparse state snapshots, on the GPU and in one pinned Host pool). The original
  continuation/checkpoint cache and its code are removed.
• Fix for #1: a request that recomputed
  already-cached blocks pinned two pages per block, so with --kv-capacity equal to --max-context
  the pool filled and the server answered 503 to every later request. A recomputed block now takes
  over its unused cached copy. The capacity error that remains possible now fails only the requests
  holding lanes on Qwen3.8-Flash-Next, and the server keeps serving.

Breaking changes

• --use-original-prefix-caching and --device-state-slots are removed. --device-snapshot-slots
  sizes the Device snapshot slots; --host-context-mib sizes the prefix cache's Host pool and is
  rejected with --no-prefix-reuse.
• The JSONL request log is schema 27. context_cache reports selections (root, prefix), state
  operations, partial_tail_cow_pages and Device occupancy. The Host-context, transfer, capture
  and spill fields, revoked_checkpoints, the snapshot recovery route and the checkpoint
  reuse path are gone.
• Prometheus: infernix_checkpoint_selections_total is now infernix_prefix_selections_total;
  the infernix_host_context_* gauges and infernix_context_transfer_* counters are removed.
• OpenAI Responses no longer derives an Engine session key from response IDs: reuse is by content.

Behaviour notes

• An exact repeat of a very short prompt (under one 64-token block) is no longer served from the
  cache, and after a generation shorter than 64 tokens (a short tool call, say) the next turn
  resumes from the turn's generation opener rather than the end of the reply. The prefix cache
  keeps a snapshot only a block or more past the previous one; infernix-serve already used this
  cache by default, so served behaviour is unchanged.

Downloads

• infernix-2026.10.09.2-windows-x64.zip: infernix-serve.exe with its FFmpeg, curl and zlib
  DLLs, the licence of every component it contains or loads (licenses/) and
  THIRD_PARTY_NOTICES.md.
• ffmpeg-9.0.1-source.zip: not needed to run Infernix. It is provided for licence
  compliance: the FFmpeg DLLs in the release zip are LGPL-2.1-or-later, and this is their
  corresponding source.

Requirements: an RTX 50-series GPU (sm_120a), an NVIDIA driver of 580 or later (CUDA 13), and the
Visual C++ Redistributable for Visual Studio 2015-2026. No CUDA toolkit is needed. Linux users build
from source (see the README).

Verification and known gaps

Tested on one RTX 5090 (PCIe Gen5 x8), Windows 11, CUDA 13.4, on this release's build:

• All 211 unit tests pass.
• Qwen3.8-27B real tests pass (NInfer's official NVFP4 artifact; a Q6 DFlash2 artifact for the
  DFlash2 and n-gram tests): hybrid prefix cache, the prefix-cache scenarios, preemption (with and
  without MTP), tool calling (graph, replay, cancel, MTP), grammar and constraints, readout, causal
  scoring, native transactions, DFlash2 and the n-gram lifecycle.
• Qwen3.8-Flash-Next real tests pass on the Uncensored artifact (prefix cache, preemption, decide,
  expert-read fault recovery, VRAM pressure) and on Dense8 (prefix cache, decide, VRAM pressure).
• The release zip, unpacked with only System32 on PATH, serves both the Dense8 and the
  Uncensored Flash-Next artifacts: model list, chat, streaming, a tool call, Anthropic Messages and
  Responses.

Known gaps:

• Prefill speed (Qwen3.8-27B, prefix cache disabled): against the build before the cache change,
  1,024-token prompts measured −0.5 %, 4,096-token prompts −1.8 % and 16,384-token prompts −0.2 %
  (infernix_bench, two runs per arm in A/B/B/A order, six repetitions each). A profile shows the
  same kernels, launch counts and API calls on both builds; the difference sits in the duration of
  unchanged GEMM kernels, and its cause is not identified. Decode measured −0.7 % (27B, DFlash2) to
  +1.8 % (Flash-Next).
• N-gram concurrency real test: ngram_concurrent fails on this release (a copy-drafting lane
  ends its turn after 40 tokens). It also fails on the previous build, at a later check, so it is
  not a regression from this release; it is a test still to be reworked.
• Device-only prefix cache: with no Host pool, a request that branches from a cached prefix and
  continues past it can replace the snapshot the parent conversation would resume from, so the
  parent's next turn recomputes from an earlier point. Outputs are unaffected; only reuse is lost.

Support

If you would like to help pay for the Claude Code subscription Infernix is developed with, you can
sponsor me on GitHub. Please also consider supporting
Neroued, who created NInfer.