MakazhanAlpamys/Soup v0.37.0

v0.37.0 — Multipack

on GitHub

What's New

Multipack — First-Fit-Decreasing bin-packing sampler. Closes the throughput gap with Axolotl on uneven-length chat fine-tuning data.

• MultipackBatchSampler — pure-Python FFD packer (no numba dep) groups variable-length samples into bins approaching batch_size × max_seq_length, eliminating padding waste. Two modes: real_batches=True (list-of-bins, collator stacks) and real_batches=False (Axolotl's micro-batch-as-flat-sequence trick). Deterministic across DDP ranks.
• Loud-fail architecture allowlist — 18 supported architectures (Llama 3.x, Qwen 2/3, Mistral, Gemma 2/3, Phi 3/4, DeepSeek V2/V3, Mixtral, Falcon, StableLM, SmolLM2). Unknown architecture raises at config-load — critical fix vs Axolotl's silent-miss footgun.
• neat_packing 4D attention mask — block-diagonal segment-aware mask (B, 1, S, S) for backends without FlashAttention. select_packing_strategy auto-routes between FA varlen and 4D mask. Composes with v0.28.0 packing_cross_doc_attn_mask.
• JinjaTemplateAnalyzer — parse-only AST walker discovers message[...] fields (role, content, tool_calls, name, weight, train) the chat template references. Used by v0.36.0 per-message training masks. 128KB cap, null-byte rejection, never renders the template.
• Schema gates — multipack and packing mutually exclusive. v0.37.0 ships for sft / pretrain only on the transformers backend; preference / RLHF / MLX backends get distinct error messages.

Install / Upgrade

pip install --upgrade soup-cli

# soup.yaml
training:
  multipack: true
  packing: false   # mutually exclusive with multipack

Security

• _MAX_FFD_ITEMS = 1_000_000 cap (FFD is O(N²) worst-case — defence against adversarial dataset DoS)
• _MAX_MASK_ELEMENTS = 2³¹ cells (~8GB float32 cap on 4D mask allocation)
• _MAX_BOUNDARY_SEGMENTS = 1_000_000 cap on tag_sub_sequences
• _MAX_TEMPLATE_BYTES = 128 KB on Jinja analyzer input
• Generator-input materialisation in FFD packer prevents silent empty-bin output
• bool rejection on every numeric input (matches v0.30.0+ project policy)
• JinjaTemplateAnalyzer is parse-only — Environment.parse never renders, so a crafted soup.yaml cannot trigger SSRF / filesystem reads
• validate_multipack_architecture raises on null bytes, non-string, empty input

Known Limitations

• Live HF Trainer sampler-swap deferred to v0.37.1. v0.37.0 ships the schema gate, allowlist validator, and build_multipack_sampler_for_lengths helper. Wiring of _get_train_sampler per-trainer follows in a patch (mirrors the v0.27.0 MII stub-then-live pattern). Today, setting multipack: true validates and constructs the sampler but does not yet re-route HF Trainer's DataLoader.
• Multipack is sft / pretrain only in v0.37.0. Preference / RLHF trainers (DPO/GRPO/KTO/ORPO/SimPO/IPO/PPO/RewardModel/Embedding) and the MLX backend are explicitly rejected at config-load with distinct messages.
• License changed from MIT to Apache-2.0 (already applied 2026-04-21). Downstream redistributors must retain NOTICE per §4(d).

Stats

• +125 tests (4249 → 4374)
• +5 test files (116 → 121)
• 5 review-agent waves clean before tag (python / code / security / tdd / verification-loop)