MakazhanAlpamys/Soup v0.71.34

v0.71.34 — adapter algebra + LISA

on GitHub

What's New

Adapter algebra + LISA — compose fine-tunes like vectors, and train big-model quality on a small-model memory budget.

soup adapters arithmetic — task-vector algebra over LoRA adapters

Apply task arithmetic (arXiv:2212.04089) to LoRA deltas — add, scale, and NEGATE trained behaviours into one merged adapter:

soup adapters arithmetic "coder + 0.5*math - toxic" \
  --adapter coder=./coder-lora --adapter math=./math-lora \
  --adapter toxic=./toxic-lora -o ./blended

The math is done at the effective-delta level: a LoRA contributes ΔW = B·A, so the coefficient is applied so that ΔW scales linearly — subtracting an adapter actually negates its delta (a naïve element-wise sum would scale by c², making negation a no-op). Same-base + same-rank required (mixed ranks refused); the v0.71.2 backdoor-scan gate runs on every input.

LISA — Layerwise Importance Sampling (#267)

Full-fine-tuning quality at LoRA-like memory (arXiv:2403.17919): every N steps LISA re-activates a small random set of decoder layers (embeddings + head always trainable) and freezes the rest.

task: sft
training:
  quantization: none
  lisa_enabled: true
  lisa_num_layers: 2
  lisa_interval_steps: 20

sft + transformers + text + quantization: none; mutually exclusive with LoRA features and the other freeze mechanisms.

Install / Upgrade

pip install --upgrade soup-cli   # or: pip install 'soup-cli[train]'

Security

• read_adapter_base explicitly rejects a symlinked adapter_config.json (O_NOFOLLOW is a no-op on Windows) and re-validates cwd containment.
• Untrusted base_model_name_or_path strings are C0/ESC-stripped before reaching the terminal.
• Hand-written expression parser (no eval) with length / term-count / coefficient-magnitude caps; cwd-contained, symlink-rejecting paths throughout.

Known Limitations

• Element-wise (same-rank) arithmetic only — exact on the diagonal (single-adapter negate/scale is exact) but with cross-terms for multi-adapter sums; exact concatenation+SVD for MIXED-rank adapters is deferred (mixed ranks are refused, not approximated).
• LISA memory win is mechanism-proven, not dramatically demonstrated at 135M — at small scale full-FT also fits 4 GB; its magnitude at 7B+ is unproven on the dev box.
• LISA is sft + text + transformers only — pretrain-family / other modalities are follow-ups.
• Directional task-arithmetic quality (does subtracting a toxic adapter reduce toxicity on a real eval?) is proven mechanically (ΔW negates) but not benchmarked against a safety suite — proof-of-mechanism.

Live-validated on Windows + RTX 3050 (SmolLM2-135M-Instruct): LISA trains end-to-end; 2·a produces an adapter whose ΔW is exactly 2× the original's.