Infernix windows-2026.11.20
AI disclaimer: Infernix was developed with AI (mostly Claude Opus 5.5). It is hobby work and
is likely to be neither complete nor entirely accurate.
Highlights
- NVIDIA's Qwen3.8-27B-NVFP4-QAD, now the recommended 27B. NVIDIA's quantization-aware
distillation of the 27B stores the GDN projections of 38 of its 48 GDN layers in NVFP4, where its
NVFP4 release keeps every GDN projection in FP8. Two conversions are published. The bit-exact one
(Infernix only) keeps every NVFP4 and FP8 code and every FP32 weight and input scale as NVIDIA stores
them (NVFP4 asnvfp4_mul: NVIDIA's scale a multiplier, not a rounded divisor) and runs them as NVIDIA
serves them: 4-bit activations at the NVFP4 layers and static 8-bit ones at the FP8 layers in prompts,
16-bit in decode and verification. The NInfer form (NInfer and Infernix) rounds the per-tensor scales
into NInfer's stored forms, with a Q8 head and an FP8 embedding. Perplexity on the quick corpus (INT8
KV, default route): bit-exact 4.3373, NInfer form 4.3423, against 4.3472 for my earlier NVIDIA NVFP4
conversion (bit-exact against it: paired −0.0023 nats per token, 95 % interval −0.0044 to −0.0003; the
NInfer form's difference is not significant). Against that conversion (RTX 5090, INT8 KV, one request,
pp2048 + tg256), the bit-exact one prefills 1.06-1.12× as fast, decodes 1.12× plainly (90.4 tok/s),
1.08× with MTP and 1.04× with DFlash2 and the proposal head (1.05× at two requests); the bench text is
very predictable (DFlash2 accepts about 9 of a 15-draft block), so the speculative figures are the
upper end. A DFlash2 round of 5-16 columns now materializes an NVFP4 GDN layer's projection instead
of running the fused small-T kernel (a cold [16384, 5120] call of 16 columns 53.9 instead of 92.8 µs),
which makes the QAD conversions decode 7.5-9.6 % faster with DFlash2. The weights take 17.0 GiB of
VRAM, 19.4 GiB with the DFlash2 drafter and the proposal head. The README's recommended 27B command
and the launcher catalog now name the bit-exact conversion, which needs this release or later (older
builds refuse itsnvfp4_mulweights). - 16- and 8-bit activations on the NVFP4 27Bs, for precision comparisons.
--a16-activations,
which only Flash-Next accepted before, now runs every text projection of NVIDIA's NVFP4 Qwen3.8-27B
artifacts (the QAD bit-exact conversion among them) with 16-bit activations.--a8-activations, which
those artifacts refused, now runs every NVFP4 and FP8 projection of the NVFP4 27Bs (NVIDIA's exports
and NInfer's official NVFP4, Quasar and Uncensored artifacts) with 8-bit activations at every width,
decode included: NVFP4 on int8 codes with an FP32 scale per 32 values, FP8 on one E4M3 term; the head
and the drafters keep their routes. On the QAD bit-exact conversion (quick corpus, INT8 KV) perplexity
is 4.2869 at 16 bits and 4.2871 at 8 bits (4.2868 before the 8-bit fast paths), against 4.3373 by
default. Both read prompts much more
slowly (RTX 5090, INT8 KV, each with its fast paths, before the ring below): pp16384 ran at 15,486
tok/s by default, 3,002 at 16 bits and 3,956 at 8 bits, and plain decode at 90.4, 90.4 and 86.1 tok/s.
The 8-bit prefill tiles have since moved to a decode-warp ring, with the same bits; in that change's
own A/B (a separate session): prefill +5.1 to +9.1 % at 128-1,024 tokens, pp4096 3,974 to 4,287 and
pp16384 3,875 to 4,167 tok/s. Each flag has its own
fast-path registrations, those measured faster under it (QAD bit-exact, INT8 KV): under
--a16-activationsamong othersbatched-decode(plain decode +14.5 % at one request, +12.2 % at
four),idle-wide-prefill(pp16384 +4.7 %) andfree-cache-resume(continuing-turn time to first
token 0.13-0.33 s to 0.04-0.06 s); under--a8-activationsamong othersidle-wide-prefill(+6.5 %).
--draft-tokens autodrafts five MTP tokens under both flags (+37 % and +35 % against three), and
with DFlash2 verifies seven fixed drafts:adaptive-dflashwon by 38-43 % on the bench's repeating
text but lost on five real chat prompts (seven fixed drafts +3.2 % summed under--a16-activations,
its worst prompt −11.6 %; +16.4 % under--a8-activations, +4.6 % without the short question, whose
answers differ; the other prompts' texts were not compared), and its sign changed with the workload (the code and docs slices: seven fixed drafts −2.2 %
and +3.9 % under--a16-activations, −3.5 % and −2.8 % under--a8-activations). - Flash-Next's last-layer trim now keeps the untrimmed call's bits. windows-2026.11.18 ran a
prefill's last-layer MoE over only the columns the prompt reads and listed it as the same output. It
was not on every artifact: a dense shared expert's projections pick their routes by the call's width,
and a routed expert job of more than 8 entries takes its bank's wide route, so over the kept columns
alone the first token's logits moved by up to 0.023 on NVIDIA Dense8 and IQ3_S, 0.038 on recipe A and
0.001 on recipe C, and on the default route's fast paths by up to 0.052 on FP4-Local
(expert-a4-residual, INT8 KV) and 3.1e-3 on IQ3_S (ggml-bf16-prefill). The trimmed layer now runs
its dense shared expert over the whole call and gives each kept entry the route its job takes in the
untrimmed call. Trimmed and untrimmed prefills of 9 to 16,384 columns, pair calls and two-chunk walks
are equal on NVFP4-QAD Dense8, NVIDIA Dense8, IQ3_S, FP4-Local, recipe A and recipe C on the exact
routes, and on the default fast paths of IQ3_S and FP4-Local with all seven KV formats and with
--a8-activations, of NVFP4-QAD Dense8, and of NVFP4-QAD with 16-bit expert activations; through the
Engine, 256 log-probabilities of prompts of 9 to 17,000 tokens equal the untrimmed ones on FP4-Local,
IQ3_S (default route) and NVIDIA Dense8 (reproducible route). The speed stays (RTX 5090, default route,
INT8 KV): against no trim, pp512 / pp2048 / pp16384 FP4-Local +1.72 / +1.89 / +0.51 %, IQ3_S +2.3 /
+2.24 / +0.90 % (pp512 leaving out a no-trim run that read 20 % slow in every repetition; +6.6 % with
it) and NVIDIA Dense8 +1.45 / +1.90 / +0.44 %; against the trim on the kept columns' own routes −0.10
to +0.26 %, within the runs' spread. Running the dense shared expert over the whole call measured
−0.04 % at pp2048 and +0.21 % at pp100 on recipe A against windows-2026.11.18's one-column trim. The
official FP8 artifact was not checked (its 114 GiB of pinned weights do not load in that test); its
BF16 shared expert is recipe A's. - Fixes for video prompts, shared media, VRAM warnings and the launcher.
- A video's frames kept the template's outer
<|vision_start|>/<|vision_end|>pair around their
own: two extra tokens a video, and every later position shifted (Qwen3.5 and Flash-Next, every
route). A video's frames now replace the whole wrapper, as the reference processor does. - Requests that send the same image or video share one preparation; when the request that started it
failed for its own reasons (cancelled, a queue timeout, its own vision budget), every request
waiting on it failed with that error. Each now prepares the media again under its own request; an
undecodable file is still one shared error. - #68: on a GPU that drives a display, every
start warned "64 MiB of the expert frames were placed in system memory" while the expert cache grew
the same frames 30 s later. Windows moved the budget during the step, which read as a whole chunk
missing. On Windows and WSL2 a chunk now counts as spilled only when this process's own device-memory
use did not rise by it; elsewhere the drop in free VRAM decides. With--log-level debugthe warning
is followed by its readings. - Launcher: an update from Settings → Updates never ran its hand-over script from a data folder
whose path holds a space and one of& < > ( ) @ ^ |(such asC:\Users\Jo Bloggs (work)); a
browser page on another origin that sent an API key was refused by the browser, since the CORS
preflight'sAccess-Control-Allow-Headers: *does not coverAuthorization(it is now named); the
management API took any host beginning with127.(such as127.evil.com) for loopback, so a
DNS-rebinding page there passed its Host and Origin checks; on a non-English Windows a download error
in the system code page made every later save of the launcher's memory file fail; and an Unload, or
another load, sent while a model was starting stopped nothing, and the model started anyway.
- A video's frames kept the template's outer
Key improvements
- Qwen3.5 family (27B, 35B-A3B, 122B-A10B):
- fast paths registered on the default route where they measured faster (RTX 5090, ABBA, pp2048 +
tg256):fp16-gdn-statewith BF16 KV on the NVFP4 27B (plain decode +0.81 / +0.32 %, MTP +0.64 /
+0.68 %; that plain run's prefill read 1.48 / 0.61 % slower, the MTP run's 0.09 / 0.31 % faster) and
the groupwise 27B (+0.42 / +0.60 %),q5-ksplit-mmawith BF16 KV on the groupwise 27B (MTP +0.12 /
+0.09 %), andgumbel-samplingon the NVFP4 27B's MTP rounds now also with FP8, NVFP4, K8V4, VQ2 and
K4V2 KV (sampling reads no KV, so it was not timed there).gumbel-samplingon the groupwise 27B's
sampled MTP measured 0.31 % slower and is not registered; - the NVFP4 27B's drafting defaults were re-timed on the QAD bit-exact conversion (INT8 KV, proposal
head; chat prompts, code, sampled docs and copy-heavy rewrites at one and two requests), and nothing
changed:--draft-tokens autoand--ngram-draft-tokens autowon or tied every workload but
two-request copies, and fixed lengths of 5-15 drafts, MTP fixed or adaptive and seven drafts with
automatic trees all lost or tied. Batched copies of 63 drafts now copy 8.6 % faster than 31 (1,553
against 1,430 tok/s) but cost code 1.1 % and the sampled docs 1-5 % at two requests; over the
workloads that is within the spread, and I kept 31. The chat prompts' texts were not compared.
- fast paths registered on the default route where they measured faster (RTX 5090, ABBA, pp2048 +
- Serving:
- Responses:
include: nullis accepted as absent, as OpenAI's schema allows (it was refused);
/v1/responses/input_tokensrefuses atool_choicethat create refuses (an undeclared tool,
requiredwithout a callable tool) instead of counting it; a streamed turn cancelled after a hosted
tool search call opened, before the search ran, no longer throws while its response is built; an
allowed_toolsentry naming a tool declared as the other kind (customagainstfunction) is a 400
invalid_tool_choiceinstead of a duplicate declaration; a loaded namespace'sdescriptionmust be a
string or null; - a turn that runs server tools (web search, fetch) reported only its last generation's n-gram and
tree counters, per-position acceptance, draft lengths, round steps and expert cache counts; it now sums
them over its generations, as it did the rest; - Anthropic: a system message whose
role, or a content block whosetype, is not a string answered
500; it is a 400; - two requests rendering the same PDF page at once counted its image twice against the PDF cache's
budget; - when the statistics reporter's first sample failed, the next interval reported the cumulative
counters as one interval's work (a throughput spike in the log and in/metrics); - the Claude Code system-prompt rewrite erased an
# Environmentheading inside a fenced example.
- Responses:
- Launcher:
- while a model loads,
count_tokens,input_tokens,compact,decideandsystemonesent with
"stream": truegot an empty event stream: they now wait and relay the server's JSON. A generation
route whose server answers 200 with a whole body ends its stream with anupstream_not_a_stream
error event instead of nothing; - a download cancelled just as it failed ended "failed" and kept its partial files;
- web UI: a text setting the server refused (port 70000, an empty listen address) kept the refused
text in its box, and Enter then blur committed twice; a download whose n-gram volume or drafter
request failed left the model queued, and a retry queued it twice (the dialog now names what was and
was not queued); 119.6 s read as "1m 60s"; route patterns of many*took minutes to match while
typing (they now use the server's linear rule);datasets/,spaces/andmodels/URLs were taken
for model repositories; chat options show the valid range under an invalid temperature or max tokens;
"Use this folder" is disabled while a folder lists or after a failed listing; the API key dialog
resets on each open, and the chat page creates one draft per visit.
- while a model loads,
- Conversion and tools:
- the recipe
qwen3_8_27b_nvfp4_nvidia_exact(new) imports NVIDIA's mixed NVFP4/FP8 27B exports
bit-exactly: NVFP4 asnvfp4_mulwith NVIDIA's multiplier and input scale, FP8 as
fp8_e4m3fn_row_f32with its static input scale, the NVFP4 head likewise, every BF16 tensor kept;
only the drafters are re-quantized. An independent word check of the published artifact found no
difference; qwen3_8_27b_nvfp4_nvidia(NInfer's form) takes each matrix's form from its stored encoding, so it
also reads the QAD export, and stores the vocabulary head in grouped Q8 instead of row FP8: on the
QAD checkpoint 4.3423 instead of 4.3537 (−0.0026 nats per token, 95 % interval −0.0033 to −0.0019),
for 0.5-0.7 % of NInfer's plain decode speed (the perplexity measured in Infernix, the speed in
NInfer). This also changes what the recipe
makes from the older NVIDIA checkpoint; its published artifact is unchanged;--graft dflash2=ARTIFACT(ordflash=) copies a drafter byte for byte from another artifact after
checking its bindings and configuration against the new backbone, and--source mtp=FILEsupplies an
MTP stem the checkpoint lacks (the QAD export omitsmtp.fcand three MTP norms; they come from
nvidia/Qwen3.8-27B-NVFP4, whose MTP decoder layer is byte-identical);convertrefuses--ngram-reusefor a Qwen3.5 source (it was ignored);inspect --compareno
longer raises on an artifact whose entries carry no activation policy;renamechecks the conversion
report's new name before it moves anything (a stale report there left the artifact renamed and its
report not); a GGUF F32/F16/BF16 tensor's shape must match up to size-1 dimensions (element counts
alone let a transposed tensor bind), and GGUF metadata gguf-py cannot decode raises with the field's
name instead of reading as absent;tools.flash_next.verify_artifactrequires the n-gram volume (--no-ngramskips it, and the report
says so) and no longer passes an artifact of which nothing was compared;- the Python tools read and write text as UTF-8 (sixteen calls used the Windows code page);
the benchmark toolsidle_gap_ttft(without a measured gap-0 pair it read an excess of 0.0 ms) and
basalt_parity(when nothing was measured) exit 1 instead of passing, and the expert-cache replay's
--engine-logparser reads the link split of Flash-Next's NVFP4 W4A16 banks and the 122B (it found
none and staged every miss).
- the recipe
- Fixes:
- an orderly stop answered paused requests only after its cleanup, which saves an attached prefix
cache file and can take seconds for a large Host tier; they are now answered with the others. A
throw while failing the requests at a stop ended the process; - Flash-Next: a Replay that ran chunk-major reported only its last call's tokens as replayed (the FP8
artifact's paused requests replayed 1,341 tokens and reported 313-335) and did not age the other
prefilling lanes in the scheduler. This was the FP8 preemption test's known gap in
windows-2026.11.18; - offloaded experts: a failed SSD read or copy as a streamed chunk began left the expert stream busy
and its read slot held for the next chunk; it now fails that chunk as a recoverable error and the
next one streams. A stream whose construction failed part-way leaked its CUDA stream and events; --no-fast-path adaptive-dflash(oradaptive-mtp) with--draft-tokens autostopped the Engine at
startup; the startup notice now says the policy is off by--no-fast-path;- a Flash-Next n-gram volume whose header records 0-byte rows reached a division by zero; it is
refused; - a Flash-Next call that is refused no longer leaves work on the stream or resets the workspace first,
and a few kernels gained checks of operands or grid sizes that no current call reaches (the
offloaded MoE's routing operands, the conv add's and the sampler's 65,535-row grids, the Q4 ring
linear's padded K), and the narrow expert kernels report a refused shared-memory setting at the call.
- an orderly stop answered paused requests only after its cleanup, which saves an attached prefix
- And many more: refactors with no behaviour change (the launcher's routes and the prefix cache index
split into files by concern, the three streaming routes share how a stream ends, the two server-side
tool protocols run on one loop, the Qwen3.5 decode rounds share their identical steps, one helper raises every
kernel's dynamic shared memory, and the unused first GPU narrow expert route is gone), new tests (a
routed sparse-MoE fixture covering every expert and job-tile residue, a resident MoE call past 65,535
routed entries, a guard that Flash-Next's decode and verification expert calls keep their fork route,
every way a server-side tool loop or a stream ends over loopback HTTP, the launcher web UI's pure
helpers undernpm test, and the chunked GDN prefill's bits on the chunk grid), a faster test suite
(the non-real entries 36.6 to 21.3 min at-j 8, the Qwen3.5 real entries about 21 % shorter, and the
Flash-Next wide-plan scenarios in one process, 513.6 to 252.9 s on QAD Dense8), and documentation
corrections (four option descriptions, among them--kv-capacity autowith offloaded experts, which is
--max-context;max_tool_calls, which bounds the hostedtool_searchcalls; the NInfer v2 refusal's
upgrade hint now names Python 3.11).
Fixes from the 2026-11-19 audit
- The
--a16-activationsfast-path registrations above also reached, by default, the 16-bit 27Bs of
other MLP gate/up formats (theqwen3_8_27b_q6recipe's Q6, a Q8_0 GGUF import), which have the same
heads and activation precision, and gave them the NVFP4 27B's drafting constants; none of that was
measured on them. The registry's configuration now includes the MLP gate/up format (NVFP4, groupwise
Q4 or another), so those artifacts select what they did in windows-2026.11.18. This never shipped:
the registrations are new in this release. The startup's configuration line names the format
("24 heads, 4-bit activations, int8 KV, NVFP4 gate/up"). On every local 27B (NInfer's official,
NVIDIA, Quasar, Uncensored and the QAD bit-exact NVFP4 conversions, by default and under both flags,
and the groupwise 27B) and the 35B-A3B the selected fast paths are unchanged. --cors:infernix-serve's preflight answer listed only the headers the APIs define, so a browser
page using an official JavaScript SDK, which adds headers of its own, failed its preflight. The list
now ends in*(Authorizationstays named: a wildcard does not cover it). The launcher allows the
same list and itsapi-key.- A request for no output tokens (OpenAI
max_tokens: 0, Responsesmax_output_tokens: 0) was
answered even after the server began stopping or the Engine failed: an empty 200 instead of 503. The
Engine now refuses it as unavailable, as it does every other request (tested at the Engine; the 503 is
the routes' existing answer to that error and was not tested over HTTP). - The A/B switch
INFERNIX_TMP_CONTINUING_PROMPTis checked at startup: a malformed value used to fail
the Engine at its first offloaded prompt, and a value such as0.5xwas accepted. - The FP8 GDN input projection's workspace query refuses the two-term 8-bit route on weights with a
static input scale, as its dispatch does (no shipped configuration asks for it). - Documentation: a fast-path option that does not exist (
--disable-fast-paths), three broken links,
stale notes about a removed test switch.
Supported models
Each repository's model card gives its files, requirements, recommended command, quality and
speed; the README's Models section lists them
all.
- Qwen3.8-Flash-Next (experts offloaded to RAM or the SSD):
NVIDIA NVFP4-QAD Dense8
(recommended), NVIDIA NVFP4-QAD bit-exact,
NVIDIA NVFP4 Dense8
(the one benchmarked), NVIDIA NVFP4 bit-exact,
GSQ-RCO IQ3_S,
FP8,
NVIDIA FP4-Local,
Huihui abliterated FP4-Local,
Uncensored NVFP4
and Swift-1.5,
each with the DFlash2 drafter
and any of the four PLE n-gram volumes (FP8 recommended; the two FP4-Local conversions recommend
FP4-Local's own). The NVFP4-QAD conversions need windows-2026.11.18 or later. - Qwen3.5-122B-A10B (experts offloaded): Ornith-1.5-122B-NVFP4
and Ornith-1.5-122B-UNCENSORED-NVFP4. - Qwen3.8-27B: NVIDIA NVFP4-QAD bit-exact
(new, recommended; Infernix only, needs this release or later),
NVIDIA NVFP4-QAD in NInfer's form (new),
NVIDIA NVFP4,
Quasar NVFP4,
Uncensored NVFP4,
Swift NVFP4, and NInfer's
official NVFP4 and
groupwise artifacts. - Qwen3.6: NInfer's official 27B,
27B NVFP4 and
35B-A3B artifacts. - Your own checkpoints of these architectures convert with
python -m tools.convert(Hugging
Face, NVIDIA ModelOpt including block-FP8 and per-row-scaled FP8, Quasar and GGUF sources). The new
recipeqwen3_8_27b_nvfp4_nvidia_exactimports NVIDIA's NVFP4 and NVFP4-QAD 27B exports bit-exactly,
andqwen3_8_27b_nvfp4_nvidianow also reads the QAD export, in NInfer's form.
Benchmarks
The README's speed tables are from 2026-10-30/31 (one RTX 5090 at PCIe Gen5 x8, on builds of that cycle).
They were not repeated for this release, so they include neither the speedups above nor the costs noted
with them. The figures in these notes are each change's own measurement, on the build and artifact it
names.
Behaviour notes
-
Output bits on the default and
--reproducibleroutes change for:- video prompts, whose token ids and positions change on every route (two tokens fewer a video), on
Qwen3.5 and Flash-Next; - Flash-Next prompts whose last prefill call trimmed its last layer: the first token is back to the
untrimmed call's bits on NVIDIA Dense8, IQ3_S, recipe A and recipe C (and, by its BF16 shared expert,
presumably the official FP8 artifact: not measured). With W4A4 experts (NVIDIA's NVFP4 conversions,
and NVFP4-QAD at its default activations) the trimmed call now takes the wide routes on the
reproducible route too, so its first token can take other bits than before; none of NVIDIA Dense8's
11 test prompts did.
Default route only: the trimmed last layer on FP4-Local (
expert-a4-residual, INT8 KV) and IQ3_S
(ggml-bf16-prefill), whose first token now equals the untrimmed call's; DFlash2 rounds of 5-16
columns on artifacts with NVFP4 GDN layers (the reproducible route already materialized them at every
width); and the new registrations offp16-gdn-stateandq5-ksplit-mmawith BF16 KV and of
gumbel-samplingwith the KV formats below INT8 (a seeded sampled request's text changes, its
distribution does not). Both routes stay deterministic and request-invariant. - video prompts, whose token ids and positions change on every route (two tokens fewer a video), on
-
--lossless: no lossless kernel's arithmetic changed; the QAD 27B's densenvfp4_mulweights
gain their lossless factor. Its output still changes for video prompts, whose prompt tokens change on
every route. -
Changed defaults:
- the recommended 27B (README and launcher catalog) is NVIDIA's NVFP4-QAD bit-exact conversion instead
of the NVIDIA NVFP4 NInferV3 artifact; - on the NVFP4 27Bs under
--a16-activationsor--a8-activations,--draft-tokens autoverifies
seven fixed DFlash2 drafts and drafts five MTP tokens; - the fast paths
fp16-gdn-state(BF16 KV, the NVFP4 and groupwise 27Bs),q5-ksplit-mma(BF16 KV,
the groupwise 27B) andgumbel-sampling(FP8, NVFP4, K8V4, VQ2 and K4V2 KV, the NVFP4 27B's MTP
rounds) are selected there by default;--no-fast-path NAMEleaves one out; - the recipe
qwen3_8_27b_nvfp4_nvidiawrites a grouped Q8 vocabulary head instead of a row FP8 one.
- the recommended 27B (README and launcher catalog) is NVIDIA's NVFP4-QAD bit-exact conversion instead
-
New, renamed and removed options:
--a16-activationsis no longer refused outside Flash-Next: on NVIDIA's NVFP4 Qwen3.8-27B
artifacts it runs every text projection at 16 bits, with a launch warning;--a8-activationsis now accepted by the NVFP4 Qwen3.8-27B artifacts (it was refused);- converter: the recipe
qwen3_8_27b_nvfp4_nvidia_exact,--graft COMPONENT=ARTIFACTand
--source mtp=FILEare new;--ngram-reuseis refused for a Qwen3.5 source; tools.flash_next.verify_artifact:--ngramis required unless--no-ngram(new) is given;- no server or CLI option was renamed or removed.
-
Files and formats:
- a dense
nvfp4_mulmatrix can now be stored inblock_scale_k16_m128x4_v1(the QAD 27B bit-exact
conversion); earlier builds refuse such an artifact; - an n-gram volume whose header records 0-byte rows is refused with its own message;
- the Python tools read and write their text files as UTF-8.
- a dense
-
Protocol behaviour:
- Responses:
include: nullis accepted;/v1/responses/input_tokensanswers a 400tool_choicefor a
choice create refuses; anallowed_toolsentry naming a tool as the other kind is a 400
invalid_tool_choice; a loaded namespace'sdescriptionother than a string or null is a 400; - Anthropic: a system message's non-string
roleor a content block's non-stringtypeis a 400 (500
before); - a turn that runs server tools reports every counter summed over its generations;
- launcher: a route that never streams answers its JSON and status during a model load whatever
streamsays, a whole-body success answer to a waiting stream ends it with an
upstream_not_a_streamerror event; - CORS preflights (
infernix-serve --corsand the launcher) name the APIs' request headers
(Authorizationamong them) and allow any other by*; the launcher adds itsapi-key.
- Responses:
Downloads
infernix-2026.11.20-windows-x64.zip:infernix-serve.exeandinfernix-launcher.exewith their
FFmpeg, curl, zlib and libjpeg-turbo DLLs,pdfium.dll(PDF documents; PDFium's build without V8 and
XFA, release chromium/8152), the maintained chat templates, the licence of every component they
contain or load (licenses/, including PDFium's underlicenses/pdfiumand every npm package bundled
in the launcher's web UI) andTHIRD_PARTY_NOTICES.md. Unpack it to a folder and run
infernix-launcher.exe, orinfernix-serve.exedirectly. An earlier launcher can install it from
Settings → Updates, which keepsinfernix-launcher.json, logs and chats.ffmpeg-9.0.1-source.zip: not needed to run Infernix. It is provided for licence
compliance: the FFmpeg DLLs in the release zip are LGPL-2.1-or-later, and this is their
corresponding source (unchanged since the previous release).
The zip is 241.3 MB (windows-2026.11.18: 238.6 MB). Its DLLs and THIRD_PARTY_NOTICES.md are
byte-identical to the previous release's; the two executables changed.
Requirements: an RTX 50-series GPU (sm_120a), an NVIDIA driver of 580 or later (CUDA 13), and the
Visual C++ Redistributable for Visual Studio 2015-2026. No CUDA toolkit is needed. Linux users build
from source (see the README).
Verification and known gaps
Everything below ran on one RTX 5090 (PCIe Gen5 x8, driver 621.24) under Windows 11, built with
CUDA 13.4, on the release commit and the build that is in the zip.
- The full test suite passed with no failures. The 333 non-real CTest entries passed (25 min, 8 at a
time); one, a tokenizer check, skips without a tokenizer directory passed by hand. The 85 real-model
entries passed (86 min, one at a time) on the 27B NVFP4 (NInfer's official artifact), Flash-Next
NVFP4-QAD Dense8 with the DFlash2 drafter and the n-gram volume, the 35B-A3B and Ornith-1.5-122B; six
skip by design: five tools that run only on inputs passed by hand, and the 4-bit-activation wide-plan
scenarios, which need an artifact that offers them. Every Flash-Next entry passed again on NVIDIA's
NVFP4 Dense8 (30 entries, 26 run and 4 skip by design, 55 min; windows-2026.11.18 counted 48
entries because each wide-plan scenario had its own, where the same scenarios now run in two entries), and the four entries FP4-Local changes most passed on NVIDIA
FP4-Local, the 4-bit wide-plan scenarios among them. The Python suites passed (507 tests, 1 skipped
because it needs a transformers 5.5 environment, plus the reproducibility digest's 3), and the
launcher's end-to-end check passed all 119 of its checks. - The audit fixes: each fix's new check failed with the old behaviour restored and passes with the
fix. The fast-path selection was compared on real artifacts against the previous build: on NInfer's
official, NVIDIA, Quasar, Uncensored and QAD bit-exact NVFP4 27Bs (QAD also under both activation
flags, NVIDIA under--a16-activationswith INT8 KV), the groupwise 27B (BF16 and INT8 KV) and the
35B-A3B, every artifact selects the same fast paths. The 122B (no dense MLP, so the change cannot
reach its configuration) ran in the suite but was not compared before and after. - Linux: the release commit built in WSL Ubuntu (CUDA 13.4, GCC 13.3) without warnings, and the
tests the audit fixes added or changed built with GCC without warnings and passed. The CLI ran the
27B NVFP4 with DFlash2, and the server answered an OpenAI Chat request ("Bonjour") and an Anthropic
Messages request with a PDF, finding its code word. - The release zip, unpacked with only System32 on PATH: its
infernix-serve.exe(build
windows-2026.11.20, the bytes the suite tested) served the new recommended 27B, NVIDIA's NVFP4-QAD
bit-exact conversion. It answered an OpenAI Chat request ("red, yellow, blue"), an Anthropic Messages
request ("Bonjour") and an Anthropic request with a PDF, finding its code word throughpdfium.dll.
Itsinfernix-launcher.exereported the same build and served its page. - Not checked for this release: updating to this release from an earlier launcher, and the README's
speed tables (see Benchmarks).
Known gaps:
--a16-activationsand--a8-activationson the NVFP4 27Bs are precision options: they read prompts
about 5× and 3.7× slower than the default at 16K tokens (the 8-bit figure after the ring above;
separate sessions).adaptive-dflash's round costs were not measured under
either flag, and the decision for seven fixed drafts rests on five chat prompts whose texts were
compared only on the short question (identical under--a16-activations, different under
--a8-activations).- The re-timing of the NVFP4 27B's drafting defaults did not compare the chat prompts' texts.
- The QAD 27B's speculative speeds are from bench text that DFlash2 drafts very well (about 9 of a
15-draft block accepted); treat them as the upper end. - The last-layer trim was not checked on the official FP8 Flash-Next artifact.
gumbel-samplingon the KV formats below INT8 was registered without timing there, and its
registration still rests on a distribution test of low power.- The #68 spill rule reads this process's own device-memory use under WSL2 too, untested there.
- The README's 27B section recommends
--prefill-chunk 4096in one command and the default chunk in
its prose, which measured 4096 slower; to be settled with a measurement. - The 16-bit 27Bs of other gate/up formats (Q6 recipe, Q8_0 GGUF) run with no fast path registered;
none was measured on them. fp16-gdn-statewith BF16 KV on the NVFP4 27B: the plain decode run's prefill read 1.48 / 0.61 %
slower (the MTP run's 0.09 / 0.31 % faster); not investigated.- Failure paths without a test: an unload in the window of a model's start, a failed first statistics
sample, a refused shared-memory setting of the narrow expert kernels, a CUDA stream or event that
fails to create in the expert stream, and the stop's error path. - Still open from windows-2026.11.18 (nothing in this release addresses them):
screened-lm-headon QAD Dense8 choosing a token 0.1 nats below the best in one synthetic prompt;
Dense8's small prefill cost from the CPU-served W4A4 misses' single walk, whose cause is not
established;
out-of-memory recovery tested by injection only; PDFs (real-world, URL, encrypted, OpenAI file inputs)
not tried served; Anthropic tool changes not checked against a real Claude Code capture;
--restore-reasoningnot checked with real clients; the launcher's GPU choice not checked on a
machine with two GPUs; the unexplained lower CPU-team rates in one series of FP4-Local starts.
Support
If you would like to help pay for the Claude Code subscription Infernix is developed with, you can
sponsor me on GitHub. Please also consider supporting
Neroued, who created NInfer.