github samuelfaj/distill v2.0.46
Distill v2.0.46

3 hours ago

Jev retrieval with utility-model extraction

Distill now uses the utility model to narrow evidence after successful Jev selection in session-scoped stored-output search. Jev selects relevant chunks; the utility selects literal source lines. Distill copies those lines from the originals and keeps source paths and original line numbers for recovery. Existing history stays unchanged.

The default utility chain now tries openrouter/free, then nvidia/nemotron-3.5-lightning:floor, then inclusionai/ling-3.0-flash:floor, in that order. An explicitly configured utility model still takes priority. Worker models remain available for delegated work; this extraction step does not require running the worker.

Unavailable, invalid, empty or failed utility extraction retains the original Jev evidence. Existing session isolation, secret filtering, disable controls and exact-source recovery remain in place. Jev remains the production decision model.

Measured pilot

The paired retrieval experiment made 72 requests across three synthetic tasks and cold/warm turns. It compared GPT-6-Astra with Jev alone, Jev plus a Luna worker, and Jev plus a Qwen utility. All alternatives used the same literal formatting.

  • Jev alone: $0.2943 per 12 answers; 16/20 evidence groups retained.
  • Jev plus Luna: $0.2338, 20.6% less; 14/20 evidence groups retained.
  • Jev plus Qwen: $0.2361, 19.8% less; 16/20 evidence groups retained.

All three produced 8/12 grounded task-complete answers. Qwen preserved every evidence group supplied by Jev; Luna dropped two. Jev itself missed four groups, which extraction could not recover. Costs include Jev, the main model, helpers, cache reads and cache writes. Additional extraction increased measured sequential latency. This is a small retrieval pilot, not a full-session or equal-quality guarantee.

Price-only projections

Repricing the same measured token profile for Opus 5.5 gives approximately $9.69 per 1,000 similar retrieval queries with Jev alone, $7.79 with Ling utility, $7.85 with Haiku 5.5 extraction, or $9.25 with Sonnet 5.5 extraction. Each total includes remaining Opus work and Jev. Helper selections, token counts and main-model cache behavior are held constant for comparison. These are estimates, not benchmarks of those models. Ling's estimate uses the currently advertised lowest-price Novita endpoint; discounts and routing availability can change.

The free router, Nemotron, Ling, Sonnet and Haiku quality and latency were not benchmarked. The price-only Ling projection does not predict which model the default chain will serve. The previous Ling Flash VL attempt returned HTTP 429; that is a different model from the new Ling Flash default. Production integration uses the existing utility transport and safety controls rather than the experiment's Python transport.

See method and retained measurements and retrieval boundaries.

Verification

Thirty stored-output Rust tests, ten shared utility-transport tests, the default-chain regression, and 58 offline benchmark checks passed. Focused independent review found no concrete correctness or security defect in the extraction flow. The full repository suite was not run. No paid benchmark was rerun for this release.

Don't miss a new distill release

NewReleases is sending notifications on new releases.