What's New
Seven surfaces that shipped schema-only in earlier releases are now live, validated
end-to-end on tiny models (SmolLM2-135M + a locally-built tiny Llama, RTX 3050 4 GB):
| # | Feature | What you can do |
|---|---|---|
| #221 | VeRA / VB-LoRA multi-tenant serving | soup serve --bank bank.json [--bank-strength S] — serve N personas at KB-per-user (shared projection + per-user vectors); pick the active user per request via the X-User-Id header.
|
| #222 | MoLE per-token adapter routing | task: moe_lora_routing + mole_task_adapters: [...] trains a gating network over N frozen task LoRAs (mole_top_k, mole_temperature); only the router trains, saved as mole_gate.pt.
|
| #145 | Sequence-level distillation | task: distill + distill_mode: sequence trains the student on the teacher's generated continuations (cross-tokenizer-friendly hard-label KD), alongside the existing token logit-KL.
|
| #146 | Classifier LoRA | task: classifier|reranker|cross_encoder + a lora: section → frozen encoder + LoRA adapter classifier.
|
| #84 | Mixture-of-Depths | use_mod: true (mod_capacity_factor) routes only the top-k tokens through each block. Llama / Qwen / Mistral.
|
| #148 | LLaMA Pro block expansion | expand_layers + freeze_trainable_layers interleaves zero-init identity decoder blocks and trains only the new ones. Llama / Qwen / Mistral.
|
| #158 | LongLoRA S² | use_longlora: true installs shifted-sparse attention on the Q/K projections for long-context tuning. Llama / Mistral / Qwen / Phi.
|
apply_bank_to_serve (#221) and build_gating_kernel (#222) now return live objects (a
LoadedVectorBank and a torch.nn.Module router) instead of the v0.67.0 deferred-stub
NotImplementedError.
Install / Upgrade
pip install -U soup-cli # light: CLI + config + data tools
pip install -U 'soup-cli[train]' # + the training stack (torch, transformers, peft, trl)Security
- #221 vector-bank serving: the active user is selected per request via
X-User-Id; an
unknown / absent id is a zero-delta no-op (plain base model) — no cross-request persona
leak. The--bankpath is cwd-contained + null-byte-validated;--bank-strengthmagnitude
is capped at 100. - No new external network or subprocess surface. The MoLE trainer runs every frozen
forward undertorch.no_grad()+.detach()(only the gate carries gradient).
Known Limitations
- Sequence-level KD is hard-label —
distill_mode: sequencetrains on the teacher's
generated tokens (SeqKD-style), not its full per-step distribution; it is mutually
exclusive with the v0.70 cross-tokenizer ULD logit path (rejected at trainer setup). - #221 active-user is per-bank-instance state —
soup serve --bankassumes single-worker
/ serialized handling; a multi-worker uvicorn deployment could race concurrentX-User-Id
requests. Single-process serving (the default) is correct. The hook applies at one layer
(last decoder layer by default). - #222 MoLE trains the gate only and is training-only — the base + every task LoRA stay
frozen,compute_lossruns N+1 forwards per step (step time scales with N), and there is
no serve-time MoLE path yet (the trainedmole_gate.pt+ frozen adapters would need a
custom serve wiring). - #148 / #84 / #158 architecture allowlists — block expansion + MoD are gated to
Llama / Qwen / Mistral, LongLoRA to Llama / Mistral / Qwen / Phi; unsupported bases warn +
skip (expansion / MoD) or are rejected at the schema gate (LongLoRA, plus FlashAttention-3 /
ring-attention). Pick one of MoD / LLaMA Pro / LongLoRA per run. - #146 classifier LoRA trains the classification head too — the
score/classifier
head is freshly initialised for a newnum_labels, so a classifier-LoRA run updates the
adapter and the head, not the adapter alone.
Full history: CHANGELOG.md