v2.0 ships a new DeepSeek-V4.1-Flash-1017 build, the closest to native 1017 of the five 1017 packs we measured, and a new engine that serves it faster than TensorFold on code and structured output on the same two DGX Sparks. The engine also makes v1.2's 2.7 bpw build faster, ahead of TensorFold on all three prompts.
Weights: sudoingx/DeepSeek-V4.1-Flash-1017-EXL3-2.8bpw, 204 GB, plus 1017's two Engram shards from DeepSeek's repo (the same ones v1.2 uses). Experts at 2.80 bpw with a cap of 3.10 bpw per decoded token, dense linears at K5 with wq_b at K4, the head at K5, the drafter with its own K4 head, and a draft-depth model fitted to the pack.
Quality against native 1017, scored by DeepSeek's reference implementation with each pack's own tensors loaded:
- KLD 0.0198 on our panel and 0.0213 on a second panel of native 1017 replies, the lowest of the five 1017 packs we measured: HeNryous 2.75 bpw 0.0201 / 0.0216, object-oriented-architects 2.7 bpw 0.0232 / 0.0249, our v1.2 0.0243 / 0.0262, LibertAIDAI 2.9 bpw 0.0304 / 0.0322. Top-1 agreement 95.94%, level with HeNryous's 95.96%.
- HumanEval+ 97.0 / 89.6, MBPP+ 90.7 / 77.8, IFEval prompt strict 90.8, each within noise of native 1017 (paired p 0.61 to 1.00).
- On 300 IFEval prompts with native replies HeNryous's pack is a little closer: 0.0974 against 0.0980.
Speed, kit_bench.py set b, single stream, DSpark on, median of 9 reps a prompt, against TensorFold with object-oriented-architects' 1017 2.7 bpw pack on the same boxes with the same client:
- The 2.8 bpw build, one session with alternating fresh boots: code 103.8 vs 98.5 tok/s (+5.3%), structured 144.1 vs 141.1 (+2.1%), prose 57.8 vs 58.2 (-0.6%). A second session (the release gates): 103.7 vs 97.7 (+6.1%), 144.4 vs 140.4 (+2.8%), 57.8 vs 57.5 (+0.5%).
- v1.2's 2.7 bpw build on the v2.0 engine, the second session: 106.4 / 61.8 / 149.1 tok/s, +8.9% / +7.5% / +6.2% over TensorFold (v1.2's own engine: 100.7 / 59.2 / 141.3).
- Against TensorFold's newest DeepSeek TP2 commit (
af26d42b, its v0.6 release plus two fixes), a third session with alternating fresh boots: code 103.9 vs 98.4 (+5.6%), structured 143.4 vs 140.5 (+2.0%), prose 57.8 vs 57.9 (-0.3%); v1.2's build 106.2 / 62.0 / 147.6 (+7.9% / +7.0% / +5.0%). The first two sessions used8e0966e2, the commit lucataco's 1017 recipe pins.
Engine (image dsv41-serve:v5): the draft depth from the confidence head plus each draft's logit margin and top-1 probability through a small model fitted to the pack; the drafter's own head; greedy verify from a vocabulary-shard argmax; Engram's projection row-parallel on the RDMA ring; a row GEMM for the attention inputs; five drafter-phase changes; every SWA metadata build of a decode step in one launch; the critical-first L2 prefetch order; RoPE tables and the indexer's prefill buffer sized to the serve (1.4 GiB a rank, after HeNryous's patches). Patches 0023-0038.
Upgrading from v1.x: stop the serve (scripts/launch/dsv41_tp2.sh down), run git pull and ./install.sh (images only, the wheels are kept), download the 2.8 bpw weights and up with MODEL_DIR on the new folder; the launcher's defaults are this build's. v1.2's weights on v2.0: DOCKER_ENV="VLLM_DSPARK_ROUND_MS=27.3,31.3,34.7,38.1,41.1,43.8".
Also in v2.0:
data/: the plan, KLD on three panels for five 1017 packs, the suites, the same-box and gate files, the round-cost table, the depth model's log, and the native-1017 panel.- FINDINGS 24-29: compare engines in one session, where the confidence head is weak, targeted bit moves cost several times their predictions, small kernels under PDL, an L2 discard that changed outputs.
scripts/dspark/: the depth model's fitter.
Limits: after load the 2.8 bpw build leaves 6 / 6 GiB free with vision on, 4 / 4 after a 128K prompt; the 256K needle rung has not run on it (needles pass at 8K, 32K, 64K and 128K). The 2-hour agent soak has not run.