Infernix windows-2026.11.01

AI disclaimer: Infernix was developed with AI (mostly Claude Opus 5.5). It is hobby work and
is likely to be neither complete nor entirely accurate.

Highlights

• Infernix Launcher (beta). infernix-launcher.exe is a web UI that runs and supervises
  infernix-serve for you, and one OpenAI- and Anthropic-compatible endpoint (port 1234 by
  default) that loads whichever model a request names:

  • a dashboard with live speed, memory and system gauges and the server's console;
  • a model library that scans your folders, downloads artifacts from Hugging Face (pause, resume
    and SHA-256 checks) and keeps launch settings per model, as named configurations you can
    import from and export to .bat/.sh scripts;
  • a Will it fit? check before loading (infernix-serve --plan), and load failures explained
    with suggested changes (VRAM, RAM, a port in use);
  • model switching that lets running requests finish first, model routes (for example
    claude-* to a configuration of your choice) and a pin for the loaded model;
  • a watchdog that restarts a crashed or locked-up server (it never ends an idle one);
  • a chat playground with thinking, images, Markdown, maths and code highlighting, plus usage
    statistics, named API keys and an event log;
  • setup snippets for Claude Code, Codex, Zed, Qwen Code, Hermes and the OpenAI and Anthropic
    SDKs.

  Everything it remembers lives in one infernix-launcher.json beside the executable. See
  docs/launcher.md.

• Qwen3.5-122B-A10B on one 32 GB GPU. The 60.75 GiB of routed experts live in host RAM (or on
  the SSD) behind a VRAM expert cache, as for Flash-Next. Vision, MTP and DFlash2 speculation and
  serving all work.

  • Ornith-1.5-122B-NVFP4-Infernix:
    about 64-97 tok/s plain decode depending on the text and 6,500 tok/s prefill at 32K;
    --spec mtp --draft-tokens auto decodes 14 % faster at one request, and a decode split added
    late in the cycle gives another 8-9 % on real code (3 % plain).
  • Ornith-1.5-122B-UNCENSORED-NVFP4-Infernix:
    the refusal-removed export with its per-row FP8 projections, shared experts and head converted
    exactly; same prefill and decode as the original at the same settings.

• NVIDIA's Qwen3.8-Flash-Next-FP4-Local
  (Infernix conversion):
  the quantization-aware-distilled FP4 checkpoint bit-exact (NVFP4 experts, MXFP8 dense, its own
  NVFP4 n-gram volume, vision MLPs and merger as exact NVFP4), with 16-bit expert activations as
  NVIDIA serves it. Perplexity 3.4865 against the FP8 conversion's 3.4854.

• A DFlash2 drafter for Qwen3.8-Flash-Next
  (Qwen3.8-Flash-Next-DFlash2-Infernix,
  776 MB, works with every conversion): Inco AI's drafter in 8 bits, 8-11 % faster than the
  model's MTP drafter (26 % on sampled code), with VRAM left for 330 more cached experts.

• Speculative decoding on by default. With no --spec, the server runs --spec auto --draft-tokens auto: the Engine picks the drafter measured fastest for the artifact (DFlash2,
  MTP or DFlash) and reports its choice at startup. --spec off decodes plainly.

• Exact and reproducible routes, and checked quality.

  • --lossless computes every operation of the model correctly rounded, so the output depends
    only on the request and the artifact. It covers generation and scoring for Qwen3.5 and
    Qwen3.8-Flash-Next, with CUDA Graph decode.
  • --reproducible (which replaces --disable-fast-paths) is deterministic and request-invariant
    on every artifact: other requests, concurrency, cache contents and memory pressure do not change
    a request's output.
  • The vocabulary head computes its logits in FP32 on every route, closer to the lossless
    reference (--logits-dtype bf16 gives the old rounding for comparisons with other engines).
  • New certification tooling in the repository (infernix-certify, the IQS quality standard)
    measures KL, top-1 and TV against these references, so a fast path only goes on by default
    when it stays inside the quality budget.

Key improvements

• Qwen3.8-Flash-Next speed and memory:
  • tensor-core kernels for the narrow W4A16 and W4A4 expert calls, and exact tensor-core GGUF
    (IQ3_S) experts;
  • the exclusive expert RAM layout is now the default: RAM holds only the experts VRAM does not,
    so the experts' RAM plan falls from 48.0 to 42.4 GiB on IQ3_S and from 64.4 to 60.3 GiB on
    Dense8, at the same decode speed and the same output bits (--expert-ram-layout full keeps a
    copy of every expert);
  • an adaptive expert cache across the VRAM and RAM pools, with cross-pool swaps;
  • one-pass prefill expert calls with the SSD tier, a shorter route chain and a faster GGUF output
    projection.
• Qwen3.5 family: automatic DFlash widths, batch rounds and n-gram window; MTP auto drafting
  on offloaded experts; gated-delta-net verification fused into its record kernel; Gumbel-max
  speculation for sampled requests.
• Serving:
  • infernix-serve --plan FILE sizes a start on this machine without loading the model (exit 0
    fits, 3 does not, 1 no plan);
  • --any-model-name serves OpenAI requests whatever model name they give;
  • --config reads TOML as well as JSON, for infernix and infernix-serve;
  • --log-file PATH writes plain timestamped log records beside the console;
  • pooled HTTP connections stay alive between agent turns (45 s, --keep-alive-timeout-sec) and
    retire gracefully;
  • the server refuses a port already in use instead of sharing it;
  • /metrics shows request labels and live prompt progress, and /metrics?format=json adds the
    session statistics of the console panel;
  • thinking budgets always leave the answer room: a thinking request closes its thinking
    early enough to leave the answer a quarter of its output limit, now also under
    --default-thinking-budget and for a request's own budget at or above its limit. Before, a
    server budget of 16,384 with the default max_tokens of 8,192 could think to the limit and
    never answer; it now thinks for at most 6,144 tokens there, and the full 16,384 when a client
    asks for 32,768. A client budget below its limit stands as sent;
  • Codex: /v1/responses accepts the hosted web_search declaration Codex sends;
  • Anthropic thinking: {"type": "enabled"} works without budget_tokens;
  • tool calling copes with damaged parameter openers and multi-line edits in array and object
    parameters, keeps calls quoted in Markdown code as text, and returns tool results in call order;
  • chat reasoning comes back as both reasoning_content and reasoning, and client stop strings
    match only the answer, not the thinking or tool calls;
  • launch output keeps what users need and moves engine detail to the debug level; the
    sysmem-fallback startup error prints the readings it compared.
• Fixes:
  • a request paused inside its prompt and restored from a missing snapshot no longer fails every
    later request;
  • WSL2 startups with offloaded experts no longer abort;
  • a failed device or pinned allocation no longer leaves a stale CUDA error that aborted the next
    launch, and device faults are reported instead of trapping;
  • DFlash drafters keep their own unscaled RoPE under the target's YaRN;
  • the MoE router logits are rounded to BF16, as the model does;
  • the default CPU expert team leaves four logical CPUs to the engine;
  • in the launcher, the chat's Loaded model setting works (the request named no model, which
    the server refused), and a launcher started by another program on a console without a window
    now stops its server in order instead of waiting out the stop timeout.
• And many more smaller fixes, test-suite speedups, Ninja builds on Windows and documentation
  updates (a new 122B guide, a refreshed README and model cards).

Supported models

Each repository's model card gives its files, requirements, recommended command, quality and
speed; the README's Models section lists them
all.

• Qwen3.8-Flash-Next (experts offloaded to RAM or the SSD):
  NVIDIA NVFP4 Dense8
  (recommended), NVIDIA NVFP4 bit-exact,
  GSQ-RCO IQ3_S,
  FP8,
  NVIDIA FP4-Local (new),
  Uncensored NVFP4
  and Swift-1.5,
  each with the DFlash2 drafter (new)
  and any of the three PLE n-gram volumes (FP8 recommended).
• Qwen3.5-122B-A10B (experts offloaded): Ornith-1.5-122B-NVFP4
  and Ornith-1.5-122B-UNCENSORED-NVFP4 (both new).
• Qwen3.8-27B: NVIDIA NVFP4 (recommended),
  Quasar NVFP4,
  Uncensored NVFP4,
  Swift NVFP4, and NInfer's
  official NVFP4 and
  groupwise artifacts.
• Qwen3.6: NInfer's official 27B,
  27B NVFP4 and
  35B-A3B artifacts.
• Your own checkpoints of these architectures convert with python -m tools.convert (Hugging
  Face, NVIDIA ModelOpt including block-FP8 and per-row-scaled FP8, Quasar and GGUF sources).

Benchmarks

The README's tables were re-measured for this release on one RTX 5090 (PCIe Gen5 x8),
2026-10-30/31, on main at fb491903 (NVIDIA's NVFP4 weights) and 6b066a54 (the others),
against each engine's latest version, every engine native on Windows under the same 84 GiB RAM
budget:

Activation precision differs: the other engines round to 8 bits what Infernix keeps at 16. The
README's Benchmarks section gives the
settings, the notes on Basalt's n-gram reads and Strata's tail skip, and every run. Those runs
predate the launcher and the serving changes at the end of the cycle; they were not repeated on the
tagged build.

Behaviour notes

• Breaking: --disable-fast-paths is renamed --reproducible.
• Breaking: --ram-headroom-mib is replaced by --ram-free-mib, the RAM that Infernix's
  weights, experts, n-gram table and caches leave free for other programs (8 GiB by default).
• Breaking: Qwen3.8-Flash-Next's expert RAM layout is exclusive by default
  (--expert-ram-layout full keeps a copy of every expert in RAM, as before).
• Speculative decoding is on by default (--spec auto --draft-tokens auto); --spec off turns it
  off. --spec auto refuses a fixed --draft-tokens and --spec-min-p: name the drafter
  (--spec mtp, --spec dflash2, ...) to set those.
• The FP32 vocabulary-head logits change the default output slightly; --logits-dtype bf16
  restores the old rounding.
• A thinking request whose budget (the server's or its own, at or above its limit) would take the
  answer's room now thinks for less, so its output can differ from the previous release.
• The serve request log is schema 31 (28 in the previous release).

Downloads

• infernix-2026.11.01-windows-x64.zip: infernix-serve.exe and infernix-launcher.exe with
  their FFmpeg, curl, zlib and libjpeg-turbo DLLs (jpeg62.dll is new in this release), the
  maintained chat templates, the licence of every component they contain or load (licenses/,
  including every npm package bundled in the launcher's web UI) and THIRD_PARTY_NOTICES.md.
  Unpack it to a folder and run infernix-launcher.exe, or infernix-serve.exe directly as
  before.
• ffmpeg-9.0.1-source.zip: not needed to run Infernix. It is provided for licence
  compliance: the FFmpeg DLLs in the release zip are LGPL-2.1-or-later, and this is their
  corresponding source.

Requirements: an RTX 50-series GPU (sm_120a), an NVIDIA driver of 580 or later (CUDA 13), and the
Visual C++ Redistributable for Visual Studio 2015-2026. No CUDA toolkit is needed. Linux users build
from source (see the README).

Verification and known gaps

Verified on an RTX 5090 at PCIe Gen5 x8, Windows 11, VS 2026 / CUDA 13.4:

• The full CTest suite (360 tests) on the release candidate (986c95c4), with the real-model tests
  on NInfer's official NVFP4 27B and Qwen3.8-Flash-Next Dense8: 353 passed. The 5 that need the
  35B-A3B (its MoE and DFlash tests) pass on it, and the 2 remaining failures were tests not
  updated for an earlier change, fixed in 16db1d9f and passing. 10 real tests skip without their
  extra reference inputs.
• The changes after the candidate (the launcher fixes, the thinking budget and the packaging) were
  checked with the affected tests (23 serve and launcher tests), the launcher's check against its
  stand-in server (102 of 102) and real runs on the 27B: the chat page, the five thinking-budget
  cases, and an orderly unload in 2.3 s.
• The release zip, unpacked with only System32 on PATH: the launcher loaded the official NVFP4 27B
  through the zip's infernix-serve, answered an OpenAI chat that named no model and an Anthropic
  Messages request, and unloaded in order in 2.4 s. Both executables report build
  windows-2026.11.01.
• Through the launcher, chat included, build 5336e291 (the release's code before the two
  packaging fixes) served Qwen3.8-Flash-Next IQ3_S, and build 330f8101 (before the thinking-budget
  change) served FP4-Local.

Known gaps:

• The full suite was not re-run on the final tag; the commits after the candidate were checked
  with the tests they affect.
• Qwen3.8-Flash-Next was not run from the release zip itself.
• Linux was not built or tested for this release.

Support

If you would like to help pay for the Claude Code subscription Infernix is developed with, you can
sponsor me on GitHub. Please also consider supporting
Neroued, who created NInfer.