MakazhanAlpamys/Soup v0.53.5

v0.53.5 — Adaptive Training (BETA → stable)

on GitHub

Closes every v0.48.0 BETA deferral with live wiring, plus the deepseek-v3-reasoning recipe.

What's New

• Dynamic curriculum is live — new DynamicCurriculumCallback (HF TrainerCallback): per-step bucket stats, recompute every N steps, multi-rank all_reduce(SUM) coordination, atomic curriculum_history.jsonl append on rank 0. Visualise via soup runs curriculum-curve <run_id>.
• Curriculum-aware on every transformer trainer — curriculum_dynamic: true schema gate widened from {sft, pretrain} to all 13 transformer-backend wrappers (DPO / GRPO / KTO / ORPO / SimPO / IPO / BCO / RewardModel / Embedding / PPO / Distill included). MLX backend still rejected with a distinct error.
• soup data mix --live runs real proxy soup train subprocesses for each Bayesian-search candidate. Argv-list invocation (no shell), per-candidate timeout min(budget/probes, 30 min), tracker-SQLite parse for eval_loss, atomic tmp-YAML cleanup. Requires --base-yaml <path> under cwd.
• scikit-optimize Bayesian optimiser — when scikit-optimize is installed, the mix optimiser now drives skopt.Optimizer(GP) through the existing OptimizerProtocol. Silent fallback to the Dirichlet sampler when absent.
• MixOptimizationReport.elapsed_seconds excludes failed candidates — headline elapsed-time now sums only successful candidate wall-clock, so a single timed-out proxy can no longer inflate an otherwise quick run.
• deepseek-v3-reasoning recipe — GRPO + reasoning template on deepseek-ai/DeepSeek-V3 with reward_fn=accuracy,format + math verifiable domain.

Install / Upgrade

pip install --upgrade soup-cli

Optional Bayesian backend:

pip install scikit-optimize

Security

• DynamicCurriculumCallback.output_dir runs through is_under_cwd + null-byte / oversize / non-string rejection before any filesystem touch; rank-0 guard via _is_rank_zero defends against multi-rank double-write; tempfile.mkstemp + os.replace atomic JSONL append mirrors the v0.48.0 write_mix_recipe policy.
• proxy_run_for_weights uses argv-list subprocess.run (NO shell) — defends against shell injection via crafted dataset / base-yaml paths. Weights validated through simplex + finite + bool-rejected + per-element [0, 1] bounds. base_yaml_path cwd-contained + null-byte-rejected before tmp YAML rendering. timeout_seconds ∈ [60, 30*60]. SOUP_DB_PATH env-override propagated to the child for per-candidate DB isolation.
• attach_curriculum_callback uses is None guard (not falsy shortcut) so a schema-bypass curriculum_dynamic=0 surfaces loudly rather than silently no-op.
• 7935 → 7998 tests (+63 net); four review agents (python / code / security / tdd) ran sequentially; every CRITICAL → LOW finding fixed.

Known Limitations

• _pick_bucket is step-mod round-robin (BETA); loss-percentile / curriculum-metric routing tracked for a follow-up patch.
• validate_distributed_curriculum helper still requires external callers to attest rank_coordinated=True; the new callback wires all_reduce internally so the schema invocation passes.
• proxy_run_for_weights is single-shot — concurrent proxy runs would race on ~/.soup/experiments.db; use SOUP_DB_PATH for per-candidate isolation.
• scikit-optimize is an optional dep, not bundled — silent fallback to Dirichlet sampler when absent.
• MixOptimizationReport.elapsed_seconds is a behaviour change — operators printing the headline elapsed must note it now excludes failed-candidate time (per-candidate wall_clock_seconds retains the per-trial timing).

🤖 Generated with Claude Code