MakazhanAlpamys/Soup v0.71.16

v0.71.16 — Knowledge edit depth

on GitHub

Deeper, safer surgical knowledge editing + a wider LongLoRA allowlist. Closes #251, #252, #250, #147. Validated on real gpt2 + SmolLM2-135M (RTX 3050 4 GB).

What's New

• GPT-2 models are now editable (#251). soup edit set (ROME / MEMIT / AlphaEdit) handles the GPT-2 Conv1D (mlp.c_proj / transformer.h) weight layout alongside Llama — a transpose-aware rank-1 update, AlphaEdit null-space projection, and MEMIT band dim-check, with PEFT-wrapped models unwrapped automatically. Real gpt2 smoke: target probability 0.005 → 0.9996.
• Covariance-preconditioned ROME (#250). soup edit set --method rome --cov-corpus <jsonl|txt> estimates the key covariance C = E[k kᵀ] + λI over a stats corpus and applies the genuine ROME closed form u = C⁻¹ k* — spreading the rank-1 update mass to reduce collateral interference. Falls back to C = I with no corpus; the exact post-condition down(k*) += delta is preserved either way. The corpus loader is cwd-contained, symlink-rejected, and size/line-capped; --cov-corpus is rejected (fail-loud) for any method other than rome.
• Atomic edit-governor count (#252). Two concurrent soup edit set runs on the same base model can no longer lose an increment — save_state re-reads the persisted count and merges this run's delta under the cross-process lock (baseline-delta, mirroring namespace_pin). Verified: 3 + 2 edits → 5.
• Mixtral joins the LongLoRA allowlist (#147). use_longlora: true now accepts Mixtral-8x7B / 8x22B — a bare mistral token never matched the MoE variant (m-i-x vs m-i-s), so a dedicated is_mixtral_model helper + MixtralAttention forward-override entry was added (the attention is the standard separate-QKV shell; the MoE lives in the MLP).

Install / Upgrade

pip install -U soup-cli            # light CLI (no PyTorch)
pip install -U 'soup-cli[train]'   # + training stack

Security

• The --cov-corpus loader is cwd-contained, rejects symlinks (raw-path lstat + O_NOFOLLOW, with the S_ISREG check on the raw fd before fdopen), and is byte/line-capped (64 MiB / 100k lines); error messages use the basename only.
• The covariance-path rank-1 update guards against a non-finite denom (math.isfinite) so a degenerate/singular C raises a clean error instead of corrupting weights.

Known Limitations

• MEMIT distributes the residual so its per-fact edit is weaker than ROME / AlphaEdit — use --method rome for a one-shot fact.
• Norm-blowup thresholds are tuned for larger models; on a ~135M model a single ROME edit can trip the default blowup_threshold, so the governor refuses the next edit. Raise NormBlowupPolicy.blowup_threshold for tiny models.
• --cov-corpus is ROME-only by design (covariance preconditioning is the ROME closed form); it is rejected for memit / alphaedit / grace.

Full notes: CHANGELOG.md