MakazhanAlpamys/Soup v0.71.21

v0.71.21 — Precision & rollout lift (BETA, hw-gated)

on GitHub

What's New

v0.71.21 — Precision & rollout lift (BETA, hardware-gated). Five long-deferred stubs go live (#141, #124, #125, #228, #97):

• Multi-turn agent rollouts for GRPO (#125) — training.rollout_backend: openenv + training.rollout_func: my_module:my_fn runs a live multi-turn rollout at the start of GRPO training; the {prompt, answer?} rows your function returns replace the prompt dataset (extra keys stripped, message-list prompts deep-copied, non-string answers rejected loudly). ART / RULER / NeMo-Gym adapters ship behind honest dependency gates with an injectable runner seam. Validated by a real GRPO + openenv rollout train on SmolLM2-135M.
• FP8 attention + NVFP4 training (#141) — training.fp8_attention: true converts the attention projections (q/k/v/o + fused variants) to torchao float8 training (Hopper-gated); training.nvfp4: true applies torchao NVFP4 quantization (Blackwell-gated). Unsupported hardware degrades to a clear advisory; a conversion failing partway raises an honest "model may be PARTIALLY converted" error.
• vLLM sleep mode for RL (#124) — training.vllm_sleep_mode: true puts the vLLM engine on standby between GRPO rollouts (vLLM ≥ 0.7), freeing VRAM for the training step; the GRPO trainer threads the flag into TRL's GRPOConfig when the installed TRL exposes the hook.
• Apple-adapter conversion is live (#228) — soup apple-adapter <dir> --direction hf-to-mlx | mlx-to-hf converts PEFT LoRA safetensors ↔ mlx-lm adapters with a numerically-equal round trip (both matrices transpose; rank / scale / dropout / num_layers carried; legacy adapters.npz still read; optional Merkle-root signing). The *-to-apple directions stay upstream-gated until Apple publishes the FoundationModels adapter spec.
• Llama-4 expert delinearization is live (#97) — soup delinearize-llama4 <src> --target <out> reshapes fused [E*din, dout] expert weights to [E, din, dout] shard-by-shard with atomic writes and copies the JSON sidecars so the target stays loadable. Expert count defaults from config.json; --num-experts overrides.

Also fixed: safetensors.numpy.save silently mangles non-contiguous (transposed) arrays — the apple-adapter writer now makes every array C-contiguous first.

Install / Upgrade

pip install -U soup-cli            # core (light, no torch)
pip install -U 'soup-cli[train]'   # + torch/transformers/peft/trl training stack

Security

No new security-sensitive surfaces. rollout_func imports operator-named code — same trusted-input policy as data.prompt_strategy; do not point it at untrusted modules on shared CI hosts. All new file I/O (apple-adapter, delinearize) is cwd-contained with symlink rejection and size caps.

Known Limitations

• fp8_attention / nvfp4 end-to-end runs are hardware-gated — the torchao converters ship live and gates + dispatch are fully tested via capability probes and fake-module injection, but a real FP8/NVFP4 training step needs a Hopper/Blackwell GPU + torchao (not on the RTX 3050 dev box).
• vLLM sleep-mode live engine cycle is hardware-gated — vLLM does not install on Windows; validated against fake engines and the real installed TRL 0.19.1 GRPOConfig signature. When TRL ships the vllm_enable_sleep_mode kwarg, the existing probe picks it up without a Soup change.
• art / ruler / nemo_gym rollout adapters are BETA-gated — friendly ImportError when missing, honest not-yet-validated error when present; _EXTERNAL_ROLLOUT_RUNNERS lets operators inject a real runner today. openenv (module:fn) is the fully live path.
• rollout_func imports operator-named code — trusted-input policy, documented at the schema field.
• apple-adapter *-to-apple directions stay upstream-gated — exit 3 with a tracking URL.
• bf16 adapters upcast to float32 through the conversion (numpy has no bf16) — documented precision note.
• delinearize-llama4 validated on a stub, not a real 400B checkpoint — reshape math + sidecar copy + exit codes exercised on a real bf16 safetensors fixture; a full Llama-4 checkpoint exceeds the dev box.

Tests: 13874 → 14084 (+210 in tests/test_v07121.py — 207 passed + 3 POSIX-skipped on Windows). Full suite: 13967 passed, 117 skipped. ruff clean.