MakazhanAlpamys/Soup v0.71.17 on GitHub

The serve-side and RAFT tail of the retrieval/multi-tenant work — validated live on SmolLM2-135M (RTX 3050).

What's New

Serve a trained MoLE (#259) — soup serve --mole <dir> loads the base + N frozen task LoRAs + the trained gate and blends them per token at decode (non-streaming + SSE). A task=moe_lora_routing run now writes a self-describing mole_manifest.json next to mole_gate.pt. Transformers-only; mutually exclusive with --bank/--steer/--adapters/--speculative-decoding. Base comes from --base or the manifest.
Per-request multi-tenant vector banks (#260) — soup serve --bank resolves the active VeRA/VB-LoRA user per request (X-User-Id) via a contextvars.ContextVar, so concurrent requests on a threaded server never race on shared state. The streaming path re-selects in-context. Live: two users get distinct steered outputs; an absent/unknown id self-clears to the clean baseline (no cross-request leak).
Epoch-aware RAFT document shuffle (#253) — data.raft_epoch_shuffle: true re-permutes the golden + distractor documents each training epoch (per-epoch salt) so the model can't latch onto a fixed citation slot. epoch=0 reproduces the legacy order exactly.
soup diagnose --citation-style / --shuffle-seed (#254) — the live citation failure-mode probe now takes the citation style (bracket / inline / footnote) and the RAFT shuffle seed so the golden [doc-N] ids line up with what the model saw at train time.

MoLE train() now returns the initial_loss/final_loss/total_steps/duration_secs keys the generic train handler reads, so soup train task=moe_lora_routing completes cleanly (previously raised KeyError after writing the gate).

pip install --upgrade soup-cli

MoLE manifest + mole_gate.pt loads are cwd-contained, symlink-rejected, and size-capped; the gate loads with torch.load(weights_only=True).
The per-request ContextVar is genuinely thread-isolated (FastAPI sync endpoints run in the anyio threadpool, which copies the caller's context per request) — no cross-request user-vector leak, including the streaming path.

The MoLE serve blend recomputes the sequence each decode step (no KV cache), so soup serve --mole is best for small / demo models.