MakazhanAlpamys/Soup v0.71.14

v0.71.14 — Export QA + serve KV-cache + FSDP consolidate

on GitHub

Closes the doable tail of the export-QA + deferred-stub family on the RTX 3050 4 GB dev budget. Two deferred-stub runtimes go live, plus an ONNX export QA pass.

What's New

• soup merge-sharded-fsdp-weights is live (#96). consolidate_shards() lifts the v0.44.0 plan-only stub: lazy-torch + safetensors.torch.save, TOCTOU-hardened (enforce_under_cwd_and_no_symlink on the output path and every shard, torch.load(weights_only=True), atomic write), a 16 GiB/shard cap, and duplicate-key handling (conflicting shape → error; same shape → keep-first + warn). New --plan-only flag; --yes removed. Validated by merging two real FSDP shards on an RTX 3050.
• soup serve --kv-cache-type is live for the transformers backend (#140). apply_kv_cache_type() → KvCacheRuntime: bf16/f16 set the model compute dtype; q8_0 wires the transformers quantized KV cache (cache_implementation="quantized", hqq backend) with a friendly install advisory when no quant backend is present; fp8 is Hopper-gated. Early CLI validation rejects bad combos with exit 2.
• ONNX export QA (#71). tiny-GPT2 round-trips cleanly; TinyLlama-1.1B is host-RAM-bound at the onnx.load external-data step (not a Soup bug) — documented in tests/qa/v07114_qa.md.

Install / Upgrade

pip install --upgrade soup-cli

Security

• consolidate_shards rejects shards/output outside CWD and any symlink in the path (TOCTOU), loads with weights_only=True (no arbitrary pickle exec), and surfaces a renamed non-torch .bin as a clean validation error.
• KV-cache backend probe no longer raises when optimum is absent — the q8_0 path degrades to a friendly advisory instead of crashing.

Known Limitations

• GGUF export QA (#70), AWQ/GPTQ export QA (#72), CUDA+llama.cpp QA doc (#144), and HF Hub push/Spaces QA (#74) are infra-blocked on the RTX 3050 4 GB / Windows budget (need llama.cpp toolchain, maintained Windows quant wheels, or private HF creds) — kept open with the infra-blocked label.
• fp8 KV cache is Hopper-only in transformers; rejected on pre-(9,0) GPUs (use vLLM there).
• q8_0 KV cache needs hqq (or optimum-quanto) installed; otherwise the CLI exits 2 with an install hint. bf16/f16 need no extra dep.

Full changelog: https://github.com/MakazhanAlpamys/Soup/blob/main/CHANGELOG.md