github MakazhanAlpamys/Soup v0.71.13
v0.71.13 — Prompt-compile family (live)

latest releases: v0.75.2, v0.75.1, v0.75.0...
4 months ago

Four commands that shipped as deferred stubs in v0.68.0 are now real, validated end-to-end on tiny models (real DPO train on SmolLM2-135M + real Ollama teacher distillation, RTX 3050 4 GB).

What's New

  • soup local-rl train --once (#229) — harvest your latest 👍/👎 feedback into DPO pairs and train a real DPO/KTO/ORPO round via soup train (argv list, no shell). A state table tracks the last train so a re-run with no new feedback skips, and a run with fewer than --min-pairs (default 10) skips. Without --once it renders a systemd .service/.timer + launchd .plist nightly scheduler scaffold into --scheduler-dir. New flags: --once, --min-pairs, --output/-o, --scheduler-dir, --hour, --minute.
  • soup distill-prompt (#226) — call a teacher (Ollama / Anthropic / vLLM) once per trace and write a real distillation dataset: sft/kl → {messages}, preference → {prompt, chosen, rejected}. New flags: --provider, --base-url, --temperature, --max-rows.
  • soup compile (#225) — DSPy / GEPA / TextGrad prompt-program optimisation — and soup compile-tools (#227) — TextGrad / GEPA tool-schema optimisation (reuses Agent Forge spec parsing) — both behind the new [compile] extra with a friendly ImportError when absent. --plan-only renders the plan and exits 0.

Install / Upgrade

pip install -U soup-cli
pip install -U 'soup-cli[compile]'   # for soup compile / compile-tools (dspy-ai + textgrad + gepa)

Security

  • systemd / launchd injection defence (#229) — local-rl and the scheduler renderers reject \n / \r in the model id and shell-quote every ExecStart argument, so a crafted model id cannot inject extra unit directives.

Fixed

  • The nightly soup train YAML now renders output as a plain string (schema-valid) instead of a {dir: ...} mapping; a regression test validates the rendered config against SoupConfig. (Caught by the step-6 smoke — a real DPO train on SmolLM2-135M.)

Known Limitations

  • #225 / #227 real-library run not validated in-tree — the live DSPy / TextGrad / GEPA dispatcher + injectable seam + a genuine missing-dep ImportError path all ship, but the actual optimiser run is exercised only when pip install 'soup-cli[compile]' is present (the libs are not in the test env to avoid a pydantic/litellm conflict). The orchestration runs end-to-end on real input up to the honest [compile] gate.
  • distill-prompt teacher is called once per trace, sequentially — no batching; throughput is provider-bound.
  • local-rl train is single-round — --once trains one DPO/KTO/ORPO round from the current thumbs; multi-round refinement is the separate soup iterative-dpo surface.
  • scheduler scaffold is render-only — write_scheduler_files writes the unit files but never runs systemctl / launchctl; the printed instructions show how to install them.

Full notes: CHANGELOG.md

Don't miss a new Soup release

NewReleases is sending notifications on new releases.