github Wallawalla47/Infernix windows-2026.11.20
Infernix windows-2026.11.20

59 minutes ago

Infernix windows-2026.11.20

AI disclaimer: Infernix was developed with AI (mostly Claude Opus 5.5). It is hobby work and
is likely to be neither complete nor entirely accurate.

Highlights

  • NVIDIA's Qwen3.8-27B-NVFP4-QAD, now the recommended 27B. NVIDIA's quantization-aware
    distillation of the 27B stores the GDN projections of 38 of its 48 GDN layers in NVFP4, where its
    NVFP4 release keeps every GDN projection in FP8. Two conversions are published. The bit-exact one
    (Infernix only) keeps every NVFP4 and FP8 code and every FP32 weight and input scale as NVIDIA stores
    them (NVFP4 as nvfp4_mul: NVIDIA's scale a multiplier, not a rounded divisor) and runs them as NVIDIA
    serves them: 4-bit activations at the NVFP4 layers and static 8-bit ones at the FP8 layers in prompts,
    16-bit in decode and verification. The NInfer form (NInfer and Infernix) rounds the per-tensor scales
    into NInfer's stored forms, with a Q8 head and an FP8 embedding. Perplexity on the quick corpus (INT8
    KV, default route): bit-exact 4.3373, NInfer form 4.3423, against 4.3472 for my earlier NVIDIA NVFP4
    conversion (bit-exact against it: paired −0.0023 nats per token, 95 % interval −0.0044 to −0.0003; the
    NInfer form's difference is not significant). Against that conversion (RTX 5090, INT8 KV, one request,
    pp2048 + tg256), the bit-exact one prefills 1.06-1.12× as fast, decodes 1.12× plainly (90.4 tok/s),
    1.08× with MTP and 1.04× with DFlash2 and the proposal head (1.05× at two requests); the bench text is
    very predictable (DFlash2 accepts about 9 of a 15-draft block), so the speculative figures are the
    upper end. A DFlash2 round of 5-16 columns now materializes an NVFP4 GDN layer's projection instead
    of running the fused small-T kernel (a cold [16384, 5120] call of 16 columns 53.9 instead of 92.8 µs),
    which makes the QAD conversions decode 7.5-9.6 % faster with DFlash2. The weights take 17.0 GiB of
    VRAM, 19.4 GiB with the DFlash2 drafter and the proposal head. The README's recommended 27B command
    and the launcher catalog now name the bit-exact conversion, which needs this release or later (older
    builds refuse its nvfp4_mul weights).
  • 16- and 8-bit activations on the NVFP4 27Bs, for precision comparisons. --a16-activations,
    which only Flash-Next accepted before, now runs every text projection of NVIDIA's NVFP4 Qwen3.8-27B
    artifacts (the QAD bit-exact conversion among them) with 16-bit activations. --a8-activations, which
    those artifacts refused, now runs every NVFP4 and FP8 projection of the NVFP4 27Bs (NVIDIA's exports
    and NInfer's official NVFP4, Quasar and Uncensored artifacts) with 8-bit activations at every width,
    decode included: NVFP4 on int8 codes with an FP32 scale per 32 values, FP8 on one E4M3 term; the head
    and the drafters keep their routes. On the QAD bit-exact conversion (quick corpus, INT8 KV) perplexity
    is 4.2869 at 16 bits and 4.2871 at 8 bits (4.2868 before the 8-bit fast paths), against 4.3373 by
    default. Both read prompts much more
    slowly (RTX 5090, INT8 KV, each with its fast paths, before the ring below): pp16384 ran at 15,486
    tok/s by default, 3,002 at 16 bits and 3,956 at 8 bits, and plain decode at 90.4, 90.4 and 86.1 tok/s.
    The 8-bit prefill tiles have since moved to a decode-warp ring, with the same bits; in that change's
    own A/B (a separate session): prefill +5.1 to +9.1 % at 128-1,024 tokens, pp4096 3,974 to 4,287 and
    pp16384 3,875 to 4,167 tok/s. Each flag has its own
    fast-path registrations, those measured faster under it (QAD bit-exact, INT8 KV): under
    --a16-activations among others batched-decode (plain decode +14.5 % at one request, +12.2 % at
    four), idle-wide-prefill (pp16384 +4.7 %) and free-cache-resume (continuing-turn time to first
    token 0.13-0.33 s to 0.04-0.06 s); under --a8-activations among others idle-wide-prefill (+6.5 %).
    --draft-tokens auto drafts five MTP tokens under both flags (+37 % and +35 % against three), and
    with DFlash2 verifies seven fixed drafts: adaptive-dflash won by 38-43 % on the bench's repeating
    text but lost on five real chat prompts (seven fixed drafts +3.2 % summed under --a16-activations,
    its worst prompt −11.6 %; +16.4 % under --a8-activations, +4.6 % without the short question, whose
    answers differ; the other prompts' texts were not compared), and its sign changed with the workload (the code and docs slices: seven fixed drafts −2.2 %
    and +3.9 % under --a16-activations, −3.5 % and −2.8 % under --a8-activations).
  • Flash-Next's last-layer trim now keeps the untrimmed call's bits. windows-2026.11.18 ran a
    prefill's last-layer MoE over only the columns the prompt reads and listed it as the same output. It
    was not on every artifact: a dense shared expert's projections pick their routes by the call's width,
    and a routed expert job of more than 8 entries takes its bank's wide route, so over the kept columns
    alone the first token's logits moved by up to 0.023 on NVIDIA Dense8 and IQ3_S, 0.038 on recipe A and
    0.001 on recipe C, and on the default route's fast paths by up to 0.052 on FP4-Local
    (expert-a4-residual, INT8 KV) and 3.1e-3 on IQ3_S (ggml-bf16-prefill). The trimmed layer now runs
    its dense shared expert over the whole call and gives each kept entry the route its job takes in the
    untrimmed call. Trimmed and untrimmed prefills of 9 to 16,384 columns, pair calls and two-chunk walks
    are equal on NVFP4-QAD Dense8, NVIDIA Dense8, IQ3_S, FP4-Local, recipe A and recipe C on the exact
    routes, and on the default fast paths of IQ3_S and FP4-Local with all seven KV formats and with
    --a8-activations, of NVFP4-QAD Dense8, and of NVFP4-QAD with 16-bit expert activations; through the
    Engine, 256 log-probabilities of prompts of 9 to 17,000 tokens equal the untrimmed ones on FP4-Local,
    IQ3_S (default route) and NVIDIA Dense8 (reproducible route). The speed stays (RTX 5090, default route,
    INT8 KV): against no trim, pp512 / pp2048 / pp16384 FP4-Local +1.72 / +1.89 / +0.51 %, IQ3_S +2.3 /
    +2.24 / +0.90 % (pp512 leaving out a no-trim run that read 20 % slow in every repetition; +6.6 % with
    it) and NVIDIA Dense8 +1.45 / +1.90 / +0.44 %; against the trim on the kept columns' own routes −0.10
    to +0.26 %, within the runs' spread. Running the dense shared expert over the whole call measured
    −0.04 % at pp2048 and +0.21 % at pp100 on recipe A against windows-2026.11.18's one-column trim. The
    official FP8 artifact was not checked (its 114 GiB of pinned weights do not load in that test); its
    BF16 shared expert is recipe A's.
  • Fixes for video prompts, shared media, VRAM warnings and the launcher.
    • A video's frames kept the template's outer <|vision_start|> / <|vision_end|> pair around their
      own: two extra tokens a video, and every later position shifted (Qwen3.5 and Flash-Next, every
      route). A video's frames now replace the whole wrapper, as the reference processor does.
    • Requests that send the same image or video share one preparation; when the request that started it
      failed for its own reasons (cancelled, a queue timeout, its own vision budget), every request
      waiting on it failed with that error. Each now prepares the media again under its own request; an
      undecodable file is still one shared error.
    • #68: on a GPU that drives a display, every
      start warned "64 MiB of the expert frames were placed in system memory" while the expert cache grew
      the same frames 30 s later. Windows moved the budget during the step, which read as a whole chunk
      missing. On Windows and WSL2 a chunk now counts as spilled only when this process's own device-memory
      use did not rise by it; elsewhere the drop in free VRAM decides. With --log-level debug the warning
      is followed by its readings.
    • Launcher: an update from Settings → Updates never ran its hand-over script from a data folder
      whose path holds a space and one of & < > ( ) @ ^ | (such as C:\Users\Jo Bloggs (work)); a
      browser page on another origin that sent an API key was refused by the browser, since the CORS
      preflight's Access-Control-Allow-Headers: * does not cover Authorization (it is now named); the
      management API took any host beginning with 127. (such as 127.evil.com) for loopback, so a
      DNS-rebinding page there passed its Host and Origin checks; on a non-English Windows a download error
      in the system code page made every later save of the launcher's memory file fail; and an Unload, or
      another load, sent while a model was starting stopped nothing, and the model started anyway.

Key improvements

  • Qwen3.5 family (27B, 35B-A3B, 122B-A10B):
    • fast paths registered on the default route where they measured faster (RTX 5090, ABBA, pp2048 +
      tg256): fp16-gdn-state with BF16 KV on the NVFP4 27B (plain decode +0.81 / +0.32 %, MTP +0.64 /
      +0.68 %; that plain run's prefill read 1.48 / 0.61 % slower, the MTP run's 0.09 / 0.31 % faster) and
      the groupwise 27B (+0.42 / +0.60 %), q5-ksplit-mma with BF16 KV on the groupwise 27B (MTP +0.12 /
      +0.09 %), and gumbel-sampling on the NVFP4 27B's MTP rounds now also with FP8, NVFP4, K8V4, VQ2 and
      K4V2 KV (sampling reads no KV, so it was not timed there). gumbel-sampling on the groupwise 27B's
      sampled MTP measured 0.31 % slower and is not registered;
    • the NVFP4 27B's drafting defaults were re-timed on the QAD bit-exact conversion (INT8 KV, proposal
      head; chat prompts, code, sampled docs and copy-heavy rewrites at one and two requests), and nothing
      changed: --draft-tokens auto and --ngram-draft-tokens auto won or tied every workload but
      two-request copies, and fixed lengths of 5-15 drafts, MTP fixed or adaptive and seven drafts with
      automatic trees all lost or tied. Batched copies of 63 drafts now copy 8.6 % faster than 31 (1,553
      against 1,430 tok/s) but cost code 1.1 % and the sampled docs 1-5 % at two requests; over the
      workloads that is within the spread, and I kept 31. The chat prompts' texts were not compared.
  • Serving:
    • Responses: include: null is accepted as absent, as OpenAI's schema allows (it was refused);
      /v1/responses/input_tokens refuses a tool_choice that create refuses (an undeclared tool,
      required without a callable tool) instead of counting it; a streamed turn cancelled after a hosted
      tool search call opened, before the search ran, no longer throws while its response is built; an
      allowed_tools entry naming a tool declared as the other kind (custom against function) is a 400
      invalid_tool_choice instead of a duplicate declaration; a loaded namespace's description must be a
      string or null;
    • a turn that runs server tools (web search, fetch) reported only its last generation's n-gram and
      tree counters, per-position acceptance, draft lengths, round steps and expert cache counts; it now sums
      them over its generations, as it did the rest;
    • Anthropic: a system message whose role, or a content block whose type, is not a string answered
      500; it is a 400;
    • two requests rendering the same PDF page at once counted its image twice against the PDF cache's
      budget;
    • when the statistics reporter's first sample failed, the next interval reported the cumulative
      counters as one interval's work (a throughput spike in the log and in /metrics);
    • the Claude Code system-prompt rewrite erased an # Environment heading inside a fenced example.
  • Launcher:
    • while a model loads, count_tokens, input_tokens, compact, decide and systemone sent with
      "stream": true got an empty event stream: they now wait and relay the server's JSON. A generation
      route whose server answers 200 with a whole body ends its stream with an upstream_not_a_stream
      error event instead of nothing;
    • a download cancelled just as it failed ended "failed" and kept its partial files;
    • web UI: a text setting the server refused (port 70000, an empty listen address) kept the refused
      text in its box, and Enter then blur committed twice; a download whose n-gram volume or drafter
      request failed left the model queued, and a retry queued it twice (the dialog now names what was and
      was not queued); 119.6 s read as "1m 60s"; route patterns of many * took minutes to match while
      typing (they now use the server's linear rule); datasets/, spaces/ and models/ URLs were taken
      for model repositories; chat options show the valid range under an invalid temperature or max tokens;
      "Use this folder" is disabled while a folder lists or after a failed listing; the API key dialog
      resets on each open, and the chat page creates one draft per visit.
  • Conversion and tools:
    • the recipe qwen3_8_27b_nvfp4_nvidia_exact (new) imports NVIDIA's mixed NVFP4/FP8 27B exports
      bit-exactly: NVFP4 as nvfp4_mul with NVIDIA's multiplier and input scale, FP8 as
      fp8_e4m3fn_row_f32 with its static input scale, the NVFP4 head likewise, every BF16 tensor kept;
      only the drafters are re-quantized. An independent word check of the published artifact found no
      difference;
    • qwen3_8_27b_nvfp4_nvidia (NInfer's form) takes each matrix's form from its stored encoding, so it
      also reads the QAD export, and stores the vocabulary head in grouped Q8 instead of row FP8: on the
      QAD checkpoint 4.3423 instead of 4.3537 (−0.0026 nats per token, 95 % interval −0.0033 to −0.0019),
      for 0.5-0.7 % of NInfer's plain decode speed (the perplexity measured in Infernix, the speed in
      NInfer). This also changes what the recipe
      makes from the older NVIDIA checkpoint; its published artifact is unchanged;
    • --graft dflash2=ARTIFACT (or dflash=) copies a drafter byte for byte from another artifact after
      checking its bindings and configuration against the new backbone, and --source mtp=FILE supplies an
      MTP stem the checkpoint lacks (the QAD export omits mtp.fc and three MTP norms; they come from
      nvidia/Qwen3.8-27B-NVFP4, whose MTP decoder layer is byte-identical);
    • convert refuses --ngram-reuse for a Qwen3.5 source (it was ignored); inspect --compare no
      longer raises on an artifact whose entries carry no activation policy; rename checks the conversion
      report's new name before it moves anything (a stale report there left the artifact renamed and its
      report not); a GGUF F32/F16/BF16 tensor's shape must match up to size-1 dimensions (element counts
      alone let a transposed tensor bind), and GGUF metadata gguf-py cannot decode raises with the field's
      name instead of reading as absent;
    • tools.flash_next.verify_artifact requires the n-gram volume (--no-ngram skips it, and the report
      says so) and no longer passes an artifact of which nothing was compared;
    • the Python tools read and write text as UTF-8 (sixteen calls used the Windows code page);
      the benchmark tools idle_gap_ttft (without a measured gap-0 pair it read an excess of 0.0 ms) and
      basalt_parity (when nothing was measured) exit 1 instead of passing, and the expert-cache replay's
      --engine-log parser reads the link split of Flash-Next's NVFP4 W4A16 banks and the 122B (it found
      none and staged every miss).
  • Fixes:
    • an orderly stop answered paused requests only after its cleanup, which saves an attached prefix
      cache file and can take seconds for a large Host tier; they are now answered with the others. A
      throw while failing the requests at a stop ended the process;
    • Flash-Next: a Replay that ran chunk-major reported only its last call's tokens as replayed (the FP8
      artifact's paused requests replayed 1,341 tokens and reported 313-335) and did not age the other
      prefilling lanes in the scheduler. This was the FP8 preemption test's known gap in
      windows-2026.11.18;
    • offloaded experts: a failed SSD read or copy as a streamed chunk began left the expert stream busy
      and its read slot held for the next chunk; it now fails that chunk as a recoverable error and the
      next one streams. A stream whose construction failed part-way leaked its CUDA stream and events;
    • --no-fast-path adaptive-dflash (or adaptive-mtp) with --draft-tokens auto stopped the Engine at
      startup; the startup notice now says the policy is off by --no-fast-path;
    • a Flash-Next n-gram volume whose header records 0-byte rows reached a division by zero; it is
      refused;
    • a Flash-Next call that is refused no longer leaves work on the stream or resets the workspace first,
      and a few kernels gained checks of operands or grid sizes that no current call reaches (the
      offloaded MoE's routing operands, the conv add's and the sampler's 65,535-row grids, the Q4 ring
      linear's padded K), and the narrow expert kernels report a refused shared-memory setting at the call.
  • And many more: refactors with no behaviour change (the launcher's routes and the prefix cache index
    split into files by concern, the three streaming routes share how a stream ends, the two server-side
    tool protocols run on one loop, the Qwen3.5 decode rounds share their identical steps, one helper raises every
    kernel's dynamic shared memory, and the unused first GPU narrow expert route is gone), new tests (a
    routed sparse-MoE fixture covering every expert and job-tile residue, a resident MoE call past 65,535
    routed entries, a guard that Flash-Next's decode and verification expert calls keep their fork route,
    every way a server-side tool loop or a stream ends over loopback HTTP, the launcher web UI's pure
    helpers under npm test, and the chunked GDN prefill's bits on the chunk grid), a faster test suite
    (the non-real entries 36.6 to 21.3 min at -j 8, the Qwen3.5 real entries about 21 % shorter, and the
    Flash-Next wide-plan scenarios in one process, 513.6 to 252.9 s on QAD Dense8), and documentation
    corrections (four option descriptions, among them --kv-capacity auto with offloaded experts, which is
    --max-context; max_tool_calls, which bounds the hosted tool_search calls; the NInfer v2 refusal's
    upgrade hint now names Python 3.11).

Fixes from the 2026-11-19 audit

  • The --a16-activations fast-path registrations above also reached, by default, the 16-bit 27Bs of
    other MLP gate/up formats (the qwen3_8_27b_q6 recipe's Q6, a Q8_0 GGUF import), which have the same
    heads and activation precision, and gave them the NVFP4 27B's drafting constants; none of that was
    measured on them. The registry's configuration now includes the MLP gate/up format (NVFP4, groupwise
    Q4 or another), so those artifacts select what they did in windows-2026.11.18. This never shipped:
    the registrations are new in this release. The startup's configuration line names the format
    ("24 heads, 4-bit activations, int8 KV, NVFP4 gate/up"). On every local 27B (NInfer's official,
    NVIDIA, Quasar, Uncensored and the QAD bit-exact NVFP4 conversions, by default and under both flags,
    and the groupwise 27B) and the 35B-A3B the selected fast paths are unchanged.
  • --cors: infernix-serve's preflight answer listed only the headers the APIs define, so a browser
    page using an official JavaScript SDK, which adds headers of its own, failed its preflight. The list
    now ends in * (Authorization stays named: a wildcard does not cover it). The launcher allows the
    same list and its api-key.
  • A request for no output tokens (OpenAI max_tokens: 0, Responses max_output_tokens: 0) was
    answered even after the server began stopping or the Engine failed: an empty 200 instead of 503. The
    Engine now refuses it as unavailable, as it does every other request (tested at the Engine; the 503 is
    the routes' existing answer to that error and was not tested over HTTP).
  • The A/B switch INFERNIX_TMP_CONTINUING_PROMPT is checked at startup: a malformed value used to fail
    the Engine at its first offloaded prompt, and a value such as 0.5x was accepted.
  • The FP8 GDN input projection's workspace query refuses the two-term 8-bit route on weights with a
    static input scale, as its dispatch does (no shipped configuration asks for it).
  • Documentation: a fast-path option that does not exist (--disable-fast-paths), three broken links,
    stale notes about a removed test switch.

Supported models

Each repository's model card gives its files, requirements, recommended command, quality and
speed; the README's Models section lists them
all.

Benchmarks

The README's speed tables are from 2026-10-30/31 (one RTX 5090 at PCIe Gen5 x8, on builds of that cycle).
They were not repeated for this release, so they include neither the speedups above nor the costs noted
with them. The figures in these notes are each change's own measurement, on the build and artifact it
names.

Behaviour notes

  • Output bits on the default and --reproducible routes change for:

    • video prompts, whose token ids and positions change on every route (two tokens fewer a video), on
      Qwen3.5 and Flash-Next;
    • Flash-Next prompts whose last prefill call trimmed its last layer: the first token is back to the
      untrimmed call's bits on NVIDIA Dense8, IQ3_S, recipe A and recipe C (and, by its BF16 shared expert,
      presumably the official FP8 artifact: not measured). With W4A4 experts (NVIDIA's NVFP4 conversions,
      and NVFP4-QAD at its default activations) the trimmed call now takes the wide routes on the
      reproducible route too, so its first token can take other bits than before; none of NVIDIA Dense8's
      11 test prompts did.

    Default route only: the trimmed last layer on FP4-Local (expert-a4-residual, INT8 KV) and IQ3_S
    (ggml-bf16-prefill), whose first token now equals the untrimmed call's; DFlash2 rounds of 5-16
    columns on artifacts with NVFP4 GDN layers (the reproducible route already materialized them at every
    width); and the new registrations of fp16-gdn-state and q5-ksplit-mma with BF16 KV and of
    gumbel-sampling with the KV formats below INT8 (a seeded sampled request's text changes, its
    distribution does not). Both routes stay deterministic and request-invariant.

  • --lossless: no lossless kernel's arithmetic changed; the QAD 27B's dense nvfp4_mul weights
    gain their lossless factor. Its output still changes for video prompts, whose prompt tokens change on
    every route.

  • Changed defaults:

    • the recommended 27B (README and launcher catalog) is NVIDIA's NVFP4-QAD bit-exact conversion instead
      of the NVIDIA NVFP4 NInferV3 artifact;
    • on the NVFP4 27Bs under --a16-activations or --a8-activations, --draft-tokens auto verifies
      seven fixed DFlash2 drafts and drafts five MTP tokens;
    • the fast paths fp16-gdn-state (BF16 KV, the NVFP4 and groupwise 27Bs), q5-ksplit-mma (BF16 KV,
      the groupwise 27B) and gumbel-sampling (FP8, NVFP4, K8V4, VQ2 and K4V2 KV, the NVFP4 27B's MTP
      rounds) are selected there by default; --no-fast-path NAME leaves one out;
    • the recipe qwen3_8_27b_nvfp4_nvidia writes a grouped Q8 vocabulary head instead of a row FP8 one.
  • New, renamed and removed options:

    • --a16-activations is no longer refused outside Flash-Next: on NVIDIA's NVFP4 Qwen3.8-27B
      artifacts it runs every text projection at 16 bits, with a launch warning;
    • --a8-activations is now accepted by the NVFP4 Qwen3.8-27B artifacts (it was refused);
    • converter: the recipe qwen3_8_27b_nvfp4_nvidia_exact, --graft COMPONENT=ARTIFACT and
      --source mtp=FILE are new; --ngram-reuse is refused for a Qwen3.5 source;
    • tools.flash_next.verify_artifact: --ngram is required unless --no-ngram (new) is given;
    • no server or CLI option was renamed or removed.
  • Files and formats:

    • a dense nvfp4_mul matrix can now be stored in block_scale_k16_m128x4_v1 (the QAD 27B bit-exact
      conversion); earlier builds refuse such an artifact;
    • an n-gram volume whose header records 0-byte rows is refused with its own message;
    • the Python tools read and write their text files as UTF-8.
  • Protocol behaviour:

    • Responses: include: null is accepted; /v1/responses/input_tokens answers a 400 tool_choice for a
      choice create refuses; an allowed_tools entry naming a tool as the other kind is a 400
      invalid_tool_choice; a loaded namespace's description other than a string or null is a 400;
    • Anthropic: a system message's non-string role or a content block's non-string type is a 400 (500
      before);
    • a turn that runs server tools reports every counter summed over its generations;
    • launcher: a route that never streams answers its JSON and status during a model load whatever
      stream says, a whole-body success answer to a waiting stream ends it with an
      upstream_not_a_stream error event;
    • CORS preflights (infernix-serve --cors and the launcher) name the APIs' request headers
      (Authorization among them) and allow any other by *; the launcher adds its api-key.

Downloads

  • infernix-2026.11.20-windows-x64.zip: infernix-serve.exe and infernix-launcher.exe with their
    FFmpeg, curl, zlib and libjpeg-turbo DLLs, pdfium.dll (PDF documents; PDFium's build without V8 and
    XFA, release chromium/8152), the maintained chat templates, the licence of every component they
    contain or load (licenses/, including PDFium's under licenses/pdfium and every npm package bundled
    in the launcher's web UI) and THIRD_PARTY_NOTICES.md. Unpack it to a folder and run
    infernix-launcher.exe, or infernix-serve.exe directly. An earlier launcher can install it from
    Settings → Updates, which keeps infernix-launcher.json, logs and chats.
  • ffmpeg-9.0.1-source.zip: not needed to run Infernix. It is provided for licence
    compliance: the FFmpeg DLLs in the release zip are LGPL-2.1-or-later, and this is their
    corresponding source (unchanged since the previous release).

The zip is 241.3 MB (windows-2026.11.18: 238.6 MB). Its DLLs and THIRD_PARTY_NOTICES.md are
byte-identical to the previous release's; the two executables changed.

Requirements: an RTX 50-series GPU (sm_120a), an NVIDIA driver of 580 or later (CUDA 13), and the
Visual C++ Redistributable for Visual Studio 2015-2026. No CUDA toolkit is needed. Linux users build
from source (see the README).

Verification and known gaps

Everything below ran on one RTX 5090 (PCIe Gen5 x8, driver 621.24) under Windows 11, built with
CUDA 13.4, on the release commit and the build that is in the zip.

  • The full test suite passed with no failures. The 333 non-real CTest entries passed (25 min, 8 at a
    time); one, a tokenizer check, skips without a tokenizer directory passed by hand. The 85 real-model
    entries passed (86 min, one at a time) on the 27B NVFP4 (NInfer's official artifact), Flash-Next
    NVFP4-QAD Dense8 with the DFlash2 drafter and the n-gram volume, the 35B-A3B and Ornith-1.5-122B; six
    skip by design: five tools that run only on inputs passed by hand, and the 4-bit-activation wide-plan
    scenarios, which need an artifact that offers them. Every Flash-Next entry passed again on NVIDIA's
    NVFP4 Dense8 (30 entries, 26 run and 4 skip by design, 55 min; windows-2026.11.18 counted 48
    entries because each wide-plan scenario had its own, where the same scenarios now run in two entries), and the four entries FP4-Local changes most passed on NVIDIA
    FP4-Local, the 4-bit wide-plan scenarios among them. The Python suites passed (507 tests, 1 skipped
    because it needs a transformers 5.5 environment, plus the reproducibility digest's 3), and the
    launcher's end-to-end check passed all 119 of its checks.
  • The audit fixes: each fix's new check failed with the old behaviour restored and passes with the
    fix. The fast-path selection was compared on real artifacts against the previous build: on NInfer's
    official, NVIDIA, Quasar, Uncensored and QAD bit-exact NVFP4 27Bs (QAD also under both activation
    flags, NVIDIA under --a16-activations with INT8 KV), the groupwise 27B (BF16 and INT8 KV) and the
    35B-A3B, every artifact selects the same fast paths. The 122B (no dense MLP, so the change cannot
    reach its configuration) ran in the suite but was not compared before and after.
  • Linux: the release commit built in WSL Ubuntu (CUDA 13.4, GCC 13.3) without warnings, and the
    tests the audit fixes added or changed built with GCC without warnings and passed. The CLI ran the
    27B NVFP4 with DFlash2, and the server answered an OpenAI Chat request ("Bonjour") and an Anthropic
    Messages request with a PDF, finding its code word.
  • The release zip, unpacked with only System32 on PATH: its infernix-serve.exe (build
    windows-2026.11.20, the bytes the suite tested) served the new recommended 27B, NVIDIA's NVFP4-QAD
    bit-exact conversion. It answered an OpenAI Chat request ("red, yellow, blue"), an Anthropic Messages
    request ("Bonjour") and an Anthropic request with a PDF, finding its code word through pdfium.dll.
    Its infernix-launcher.exe reported the same build and served its page.
  • Not checked for this release: updating to this release from an earlier launcher, and the README's
    speed tables (see Benchmarks).

Known gaps:

  • --a16-activations and --a8-activations on the NVFP4 27Bs are precision options: they read prompts
    about 5× and 3.7× slower than the default at 16K tokens (the 8-bit figure after the ring above;
    separate sessions). adaptive-dflash's round costs were not measured under
    either flag, and the decision for seven fixed drafts rests on five chat prompts whose texts were
    compared only on the short question (identical under --a16-activations, different under
    --a8-activations).
  • The re-timing of the NVFP4 27B's drafting defaults did not compare the chat prompts' texts.
  • The QAD 27B's speculative speeds are from bench text that DFlash2 drafts very well (about 9 of a
    15-draft block accepted); treat them as the upper end.
  • The last-layer trim was not checked on the official FP8 Flash-Next artifact.
  • gumbel-sampling on the KV formats below INT8 was registered without timing there, and its
    registration still rests on a distribution test of low power.
  • The #68 spill rule reads this process's own device-memory use under WSL2 too, untested there.
  • The README's 27B section recommends --prefill-chunk 4096 in one command and the default chunk in
    its prose, which measured 4096 slower; to be settled with a measurement.
  • The 16-bit 27Bs of other gate/up formats (Q6 recipe, Q8_0 GGUF) run with no fast path registered;
    none was measured on them.
  • fp16-gdn-state with BF16 KV on the NVFP4 27B: the plain decode run's prefill read 1.48 / 0.61 %
    slower (the MTP run's 0.09 / 0.31 % faster); not investigated.
  • Failure paths without a test: an unload in the window of a model's start, a failed first statistics
    sample, a refused shared-memory setting of the narrow expert kernels, a CUDA stream or event that
    fails to create in the expert stream, and the stop's error path.
  • Still open from windows-2026.11.18 (nothing in this release addresses them):
    screened-lm-head on QAD Dense8 choosing a token 0.1 nats below the best in one synthetic prompt;
    Dense8's small prefill cost from the CPU-served W4A4 misses' single walk, whose cause is not
    established;
    out-of-memory recovery tested by injection only; PDFs (real-world, URL, encrypted, OpenAI file inputs)
    not tried served; Anthropic tool changes not checked against a real Claude Code capture;
    --restore-reasoning not checked with real clients; the launcher's GPU choice not checked on a
    machine with two GPUs; the unexplained lower CPU-team rates in one series of FP4-Local starts.

Support

If you would like to help pay for the Claude Code subscription Infernix is developed with, you can
sponsor me on GitHub. Please also consider supporting
Neroued, who created NInfer.

Don't miss a new Infernix release

NewReleases is sending notifications on new releases.