MakazhanAlpamys/Soup v0.59.0

v0.59.0 — Governance & Provenance

on GitHub

Governance & Provenance. Every Soup run is procurement-ready. ML-BOMs, SLSA-3 attestations, EU AI Act Annex XI/XII auto-docs, HIPAA/SOC2 audit log, SR 11-7 reproducibility receipts. The compliance story no SaaS can tell because the operator is the only one who sees weights, dataset, eval, and cost together. Pure orchestration on top of v0.26 Registry + v0.34 cost tracker + v0.56 diagnose — no new training code, just structured exporters that emit the formats procurement demands.

What's New

• soup bom emit --format cyclonedx|spdx|both — CycloneDX 1.6 ML-BOM + SPDX 2.3 + AI-profile dual emitter from any RegistryEntry. Base-model SHA, config SHA, data SHA, parent lineage, license-chain (SPDX id), and SLSA-style material list. Atomic write under cwd containment + os.lstat symlink rejection (TOCTOU-safe).
• soup attest emit --stage train --subject <name> --sha <64hex> — in-toto v1 Statement wrapping a SLSA-3 provenance v1 predicate. Stage allowlist (extract / train / eval / export / publish). Sigstore + ed25519 signing arrive in v0.59.1; the schema + atomic-write surface ships now so CI can integrate.
• soup train --annex-xi <out.md> — EU AI Act Annex XI Section 1+2 (technical documentation) + Annex XII Article 53(1)(d) (public training summary) markdown renderer. Top-10 domains, modality breakdown, FLOPs / kWh / CO₂. Markdown-active chars in operator-controlled fields are escape-neutralised — defends against forged-heading / Markdown-link injection in downstream PDF/HTML renderers.
• soup audit-log tail / rotate — append-only JSONL audit at ~/.soup/audit.jsonl (override via SOUP_AUDIT_LOG_PATH, containment-checked to $HOME / $CWD / $TMPDIR). Splunk/ELK ingestion-ready. PII redaction across every string field via the v0.40.3 _SECRET_RE policy. POSIX O_NOFOLLOW + 0o600 perms; rotation at 100 MiB with symlink rejection at the backup path.
• soup train --repro-receipt <out.json> — SR 11-7-style receipt: seeds (torch / numpy / python), Python version, OS + arch, Soup version, kernel versions (CUDA / cuDNN / NCCL — best-effort from torch when available), GPU model + driver.
• CO₂ + energy schema — EnergyMeasurement frozen dataclass with PUE adjustment, electricityMap SSRF-hardened endpoint validator (scheme allowlist + loopback-only HTTP + private-IP rejection — full parity with v0.51.0 validate_hub_endpoint). CodeCarbon hook lands in v0.59.1.
• Shared atomic_write_text helper — all four v0.59 atomic writes (BOM / attest / Annex / repro) now go through one paths.atomic_write_text — single-source-of-truth so a future contributor cannot accidentally drop the symlink check (mirrors v0.40.6 / v0.53.5 centralisation policy).

Install / Upgrade

```bash
pip install --upgrade soup-cli
soup version # should print soup v0.59.0
```

Security

3 HIGH + 5 MEDIUM + 4 LOW fixes across four review waves before tag:

• HIGH: audit-log lstat-before-write TOCTOU (no lexists race); POSIX O_NOFOLLOW on log append; redaction extended from args-only to every string field (command / host_id / operator_id); SOUP_AUDIT_LOG_PATH env override containment to $HOME / $CWD / $TMPDIR; BOM artifact size_bytes validation (TypeError surface fix); BomEntry.attach_energy type-hint via TYPE_CHECKING; default_log_path exported as public symbol; duplicated seeds validation removed.
• MEDIUM: Annex markdown injection escape (|[](){}!<> + newline / CR / tab in every operator-controlled field); shared paths.atomic_write_text helper centralising four duplicated TOCTOU patterns; energy.measure_run_energy inconsistent-return-type fixed; urlsplit IPv6 hostname stripping cleanup; SLSA startedOn / finishedOn separable via invocation mapping; _format_flops bool/NaN/Inf guard.

Known Limitations

1. Sigstore + ed25519 signing deferred to v0.59.1. UNSIGNED backend live; --sign sigstore / --sign ed25519 raise NotImplementedError with explicit marker.
2. Live CodeCarbon hook deferred to v0.59.1. measure_run_energy returns None even with codecarbon installed. The SSRF-hardened electricityMap validator ships now so v0.59.1 wiring is additive.
3. PDF rendering deferred to v0.59.1. soup train --annex-xi emits Markdown; reportlab integration later.
4. Soup Can manifest v3 attestations field deferred to v0.59.1. Attest emit is a standalone surface for now; embedding under a manifest-version bump is the v0.59.1 deliverable.
5. Audit-log auto-instrumentation NOT shipped. Every command does NOT yet auto-emit a line via the Typer top-level callback; only the reader + write API + redaction + rotation are live.
6. Top-10 domain auto-population NOT shipped. AnnexXIData.top_domains accepts an operator-supplied tuple but --annex-xi currently passes empty. Auto-extraction from the data.train JSONL manifest is the v0.59.1 deliverable.
7. --annex-xi + --repro-receipt fire only on LOCAL_RANK=0. Single-node multi-GPU correct; multi-node RANK-aware guard tracked separately.

Test count: 9193 → 9294 (+101 net). Four review waves: 0 CRITICAL + 8 HIGH + 12 MEDIUM + 4 LOW resolved.