MakazhanAlpamys/Soup v0.25.0 on GitHub

What's New in v0.25.0 — "Beyond the Wrapper"

Soup goes from being an aggregator to being the smartest fine-tuning tool on the planet. Eight new capabilities close every competitive gap vs LLaMA-Factory/Axolotl/Unsloth, anchored by an Autopilot that picks task, quantization, PEFT, batch size, LR, epochs, max length, and perf flags for you from just (model, data, goal).

✨ Autopilot — the killer feature

soup autopilot \
  --model meta-llama/Llama-3.1-8B-Instruct \
  --data my_chats.jsonl \
  --goal chat \
  --gpu-budget 24GB

You give 3 inputs, Soup analyzes your dataset, model, and hardware, then generates an optimal soup.yaml with every decision justified. Forgetting detection and checkpoint intelligence are enabled by default.

🍏 Apple Silicon MLX backend

pip install 'soup-cli[mlx]'
soup train --config soup.yaml  # backend: mlx, task: sft

First CLI fine-tuning tool with MLX support for M1–M4 chips.

🔧 Tool-calling / agentic fine-tuning

data:
  format: tool-calling

New tool-calling format with detection, normalization, synth data template, init template (soup init --template tool-calling), and eval scoring (tool_call_match, tool_call_name_match, tool_call_args_subset).

🎯 RLVR — RL from Verifiable Rewards

task: grpo
training:
  reward_fn: verifiable
  verifiable_domain: math  # or code / json_schema

Deterministic reward signals for math (regex-extracted numeric answers, no eval()), code (subprocess sandbox with RLIMIT_AS + RLIMIT_CPU on POSIX, ephemeral cwd, concurrency cap), and JSON schema conformance.

🧠 Training intelligence

training:
  forgetting_detection: true
  checkpoint_intelligence: true
  early_stop_on_regression: true

Auto-detects catastrophic forgetting during training via mini benchmarks (MMLU, common sense, instruction-following) and tracks the best checkpoint by quality — not just loss. Smart pruning refuses symlinks and non-checkpoint dirs.

🔬 VeRA + OLoRA PEFT

lora:
  use_vera: true    # OR use_olora: true

Smaller memory footprint (VeRA) and better convergence (OLoRA via QR init), mutually exclusive with DoRA.

📈 Data augmentation

soup data augment --input data.jsonl --strategy rephrase --count 3
soup data augment --input data.jsonl --strategy translate --lang ru,zh,es
soup data augment --input data.jsonl --strategy style --styles formal,casual

🎨 14 new recipes

Llama 4 Scout 17B (sft/dpo/grpo/tools), Qwen 3 14B/32B, Qwen 3 8B GRPO, Gemma 3 12B SFT, Gemma 3 27B DPO, DeepSeek V3 MoE, plus 3 MLX SFT recipes (llama3.1-8b-sft-mlx, qwen3-8b-sft-mlx, gemma3-9b-sft-mlx). 43 total recipes.

Stats

2313 tests passing (+183 over v0.24.3)
86 test files (+8 new)
43 ready-made recipes (+14)
16 built-in templates (+1 tool-calling)
Every CRITICAL/HIGH/MEDIUM/LOW review finding addressed

Install / Upgrade

pip install --upgrade soup-cli

For Apple Silicon:

pip install --upgrade 'soup-cli[mlx]'

Security

code_exec_reward is a best-effort subprocess sandbox. Python-level socket patching can be bypassed by os.system/subprocess/ctypes. A one-time warning panel on first use tells you to run inside a container or VM with no network interface for strong isolation.
math_verify_reward uses strict regex + float() — no eval() on model output.
Tool-call arguments are parsed via json.loads only — no eval, no remote code.
Every new CLI path (soup data augment, soup autopilot) has path-traversal protection via resolve() + relative_to(cwd).
All new config fields are bounded via Pydantic ge=/le=/Literal constraints.

See .claude/CLAUDE.md for the full v0.25.0 security changelog.

MakazhanAlpamys/Soup v0.25.0 v0.25.0 — Beyond the Wrapper on GitHub