MakazhanAlpamys/Soup v0.53.0

v0.53.0 — Quant Menu II (UD GGUFs + KV cache + NVFP4 + LF parity)

on GitHub

Schema-only release. Live wiring deferred to v0.53.1 (mirrors v0.50.0 / v0.51.0 / v0.52.0 stub-then-live pattern).

What's New

• Unsloth Dynamic 2.0 GGUF ladder. 14-entry closed allowlist (UD-Q{8..2}_K_XL + UD-IQ{4_XS, 3_M, 3_XXS, 2_M, 2_XS, 2_XXS, 1_M, 1_S}). New validate_calibration_data_path shape validator gates --calibration-data <jsonl> for v0.53.1 CLI dispatch. O(1) lowercase index for case-insensitive lookup.
• IQ + Apple/ARM GGUF flavours. 12-entry IQ family (IQ1/2/3/4 — including IQ4_NL non-linear) + 10-entry Apple/ARM-friendly set (Q4_0_4_4 / Q4_NL / Q5_K_M / etc.).
• KV cache types. New training.kv_cache_type: q8_0 | bf16 | f16 | fp8. FP8 is Hopper-only; MLX backend rejected at config load. The other three pass through every backend in v0.53.0.
• FP8 attention + NVFP4 + native unsloth_bnb_4bit. Three new TrainingConfig bools. fp8_attention=true requires quantization_aware: fp8 + non-MLX. nvfp4=true gated to non-MLX + modality: text (Blackwell SM ≥ 12 check is runtime-only). unsloth_bnb_4bit=true requires backend='unsloth' + quantization='4bit'.
• LF / Axolotl quant parity. bnb_4bit_use_double_quant (requires quantization='4bit'), llm_int8 (explicit 8bit assertion — distinct from v0.41.0 load_in_8bit aliasing), quantize_ref_model (extends Quant Menu to the ref model on DPO/IPO/SimPO/ORPO/BCO/KTO/GRPO/PPO/preference), quantize_reward_model (PPO + reward_model tasks).
• Advanced save formats. soup merge --save-format {4bit, 4bit_forced} (single BNB-4bit merged checkpoint without dequant → merge → requant cycle) and soup export --format torchao --quant-config <yaml> (closed allowlist: Int4WeightOnly / Int8DynActInt4 / Float8DynActFloat8 / NVFP4 — case-sensitive PyTorch class names).
• +157 net new tests (7453 → 7610) across 154 tests in test_v0530.py. Five review agents (python / code / security / tdd / verification) ran; every CRITICAL / HIGH / MEDIUM / LOW finding fixed or documented.

Install / Upgrade

```bash
pip install -U soup-cli
```

Or via GHCR:

```bash
docker pull ghcr.io/makazhanalpamys/soup:v0.53.0
```

Security

Every new validator follows the project's established hardening policy:

• Closed allowlists wrapped in `frozenset` (`UD_GGUF_FORMATS`, `IQ_GGUF_FORMATS`, `APPLE_ARM_GGUF_FORMATS`, `KV_CACHE_TYPES`, `MERGE_SAVE_FORMATS`, `TORCHAO_PTQ_SCHEMES`).
• All metadata registries wrapped in `MappingProxyType` so they cannot be mutated at runtime.
• String validators reject non-string / bool / empty / null-byte / oversize / unknown with case-insensitive normalisation (matches v0.41.0 / v0.51.0 policy).
• `validate_torchao_scheme` is INTENTIONALLY case-sensitive (PyTorch class names) — asymmetry documented at both validators.
• `validate_calibration_data_path` + `validate_quant_config_path` are shape-only this release; their docstrings name the exact controls a v0.53.1 CLI dispatch contributor MUST add (`os.path.realpath` + `os.path.commonpath` cwd containment, `os.lstat` + `stat.S_ISLNK` symlink rejection before `open()`, existence check, `yaml.safe_load`-only for quant configs).
• All 7 new bool fields share a `field_validator(mode='before')` that rejects bool-as-int with explicit `TypeError("v0.53.0 flag must be bool")` — `None` falls through to Pydantic's `default=False` rather than silently coercing.
• Cross-validator ordering picks the most actionable error: `fp8_attention`'s `quantization_aware='fp8'` prerequisite check fires BEFORE the MLX gate.
• `requires_hopper` reads from `_KV_CACHE_METADATA` spec (single source of truth).
• Extended ref-task set (`{dpo, ipo, simpo, orpo, bco, kto, preference, grpo, ppo}`) so `quantize_ref_model=true` on GRPO / KTO / PPO no longer silently passes.

Known Limitations

1. Every live wiring is deferred to v0.53.1 — `export_advanced_gguf` (llama.cpp imatrix), `apply_kv_cache_type` (serve backend route), `apply_fp8_attention` / `apply_nvfp4` (torchao swap + Blackwell quant prep), `merge_4bit` (BNB-4bit merged checkpoint writer), `export_torchao` (`torchao.quantize_` + `save_pretrained`) all raise `NotImplementedError` with explicit `v0.53.1` markers. Schema accepts every new flag; trainer / export / serve dispatch is gated by the validators above so misconfigured runs fail loudly at config load.
2. Calibration-data + quant-config path I/O is shape-only this release — both validators enforce non-empty / null-byte-free / ≤4096-char shape; cwd-containment + `os.lstat + S_ISLNK` TOCTOU rejection MUST be added at CLI dispatch time in v0.53.1 (controls documented at validator docstrings).
3. `kv_cache_type` MLX permissive policy — only `fp8` is rejected on MLX (Hopper-only); `q8_0` / `bf16` / `f16` pass through every backend. v0.53.1 wiring may need to narrow further once the runtime serve path lands.
4. Hopper SM-capability check is runtime-only — schema accepts `kv_cache_type='fp8'` and `fp8_attention=true` without probing GPU capability. The SM ≥ 9.0 check fires at trainer / serve construction time.
5. NVFP4 + Blackwell — schema accepts `nvfp4=true` for any CUDA build; the actual Blackwell (SM ≥ 12.0) check is runtime-only.
6. `bnb_4bit_use_double_quant` only gated against `quantization` — does not check `quantization_aware`. A user setting both `bnb_4bit_use_double_quant=true` + `quantization_aware='fp8'` will be caught by the existing v0.28.0 cross-validator that already rejects `quantization=4bit + quantization_aware=fp8`.
7. `llm_int8` is an assertion, not a rewriter — unlike v0.41.0 `load_in_8bit` (which rewrites `quantization → 8bit`), `llm_int8=true` requires the user to ALSO set `quantization: 8bit`; mismatch raises loudly. Design choice: assertion ↔ alias surfaces are intentionally kept disjoint.

🤖 Generated with Claude Code