MakazhanAlpamys/Soup v0.71.19

v0.71.19 — Quant-menu + multipack hardening

on GitHub

Two long-standing limitations closed, both validated on Windows + RTX 3050.

What's New

• #81 — Quant Menu for vision/audio models. The full train-time Quant Menu (gptq / awq / hqq:Nbit / aqlm / eetq / mxfp4 / fp8) now applies to modality: vision and modality: audio, not just text. The modality != "text" rejection is gone; _setup_vision_transformers / _setup_audio_transformers thread the unified build_quantization_config_for_loader (a strict superset of the inline 4bit/8bit blocks they replaced). The mlx-backend gate is retained.
• #80 — Multipack sharding under multi-GPU. multipack: true now shards the FFD-packed bins correctly across ranks under FSDP / DeepSpeed ZeRO / DDP — get_train_dataloader routes the DataLoader through accelerator.prepare when num_processes > 1, so accelerate's BatchSamplerShard round-robins whole bins to each rank (preserving the packing). The single-GPU path is byte-for-byte unchanged.

Install / Upgrade

pip install --upgrade soup-cli            # core (light, no PyTorch)
pip install --upgrade 'soup-cli[train]'   # + torch/transformers/peft/trl

Security

No new security surface. Both changes are internal: a dropped schema gate (#81) and a DataLoader routing fix (#80). Existing path-containment, bool-as-int, and lazy-import policies are preserved.

Known Limitations

• Multi-GPU multipack sharding is mocked-tested, not live-validated. There is no multi-GPU box in the dev budget (single RTX 3050). The single-GPU path is live-validated; the distributed path is exercised via a mocked accelerator (asserting prepare(loader) is called and the bin seed is identical across ranks). A real 2+-GPU FSDP / DeepSpeed ZeRO run remains an INFRA-BLOCKED QA follow-up.
• Vision/audio Quant Menu inherits upstream model-class constraints. The unified loader emits the right config for any modality, but a given vision/audio checkpoint only loads under a format its AutoModelForVision2Seq / AutoModelForSpeechSeq2Seq class + installed kernel actually supports (e.g. autoawq for awq); an unsupported pairing surfaces as a loud from_pretrained error. The 4bit/8bit BNB path is the broadly-supported default.

Tests: 13770 → 13807 (+37). Full suite green across ubuntu/windows/macos × Python 3.10–3.12.