ttlequals0/MinusPod v2.86.4

2.86.4

on GitHub

Added

• 2026-08 LLM benchmark sweep: 75 models, 14-episode corpus, 64,125 work
  units, with regenerated report and charts. New in the report: a failure
  category for provider content moderation (was buried in "Other"), an
  "Errors resolved by retry" table built from the append-only raw rows, and
  corpus-derived Metric Key text instead of stale hardcoded episode counts.
• results/parse-and-moderation.md: companion analysis of the two failure
  modes retrying never fixes, provider content refusal and unparseable
  JSON, with every example linked to its raw call.
• Four new report views: a "windows flagged with no truth ad" table
  ranking ground-truth-free windows by model consensus (doubles as truth
  QA), signed boundary bias columns showing whether a model's cuts lean
  into content or leave ad audio, an accuracy-vs-latency scatter, and an
  input-vs-output cost split (table plus chart).
• benchmark rotate-raw and campaign archiving, so a new sweep's rows
  cannot silently blend with the previous campaign's.
• Benchmark CONTRIBUTING: corpus wishlist for outside episode PRs, campaign
  rotation notes, and a zero-cost benchmark report preview step for
  report code changes.

Changed

• Breaking: claude_model, verification_model, and chapters_model
  no longer seed or reset to a hardcoded literal. They seed from
  OPENAI_MODEL when the operator has set it and stay unset otherwise;
  resetting one (individually, via a provider change, or via the bulk
  ad-detection reset) clears it back to unset instead of writing a shipped
  default. An install that relied on the old shipped default must
  configure a model explicitly in Settings (or via OPENAI_MODEL) after
  any of those resets.
• docs/llm-providers.md model recommendations refreshed from the 2026-08
  sweep, including a note on provider-side content moderation.
• An unconfigured model (claude_model, verification_model, or
  chapters_model unset) is no longer a silent failure. Boot logs one
  error line naming the missing settings, /health adds an
  llm_model_configured check without affecting overall status, ad
  detection fails the episode immediately with the exact error message
  instead of exhausting the retry ladder, and chapter generation degrades
  to fallback titles and boundaries instead of failing the episode.

Fixed

• A non-Anthropic install carrying a shipped Anthropic model id from the
  old hardcoded-default seeding has it cleared on upgrade, so it fails
  with an actionable message instead of looping on a provider 404. The
  clear is limited to that case: a model you chose yourself, and a shipped
  default that still resolves on an Anthropic install, are both left
  alone.
• Model settings now seed from OPENAI_MODEL whenever a row is absent.
  The previous seed path only fired on an empty settings table, which
  never happened in practice because schema migrations always populate
  other rows first.
• An unrecognized LLM_PROVIDER value is rejected with a warning instead
  of being written into the stored setting verbatim.
• Cross-model agreement chart was unreadable at 75 models: every integer
  got an x tick so the labels ran together, and the two-line bar labels
  overlapped each other. Ticks now thin out to about 25 across any
  model count, and bar labels rotate to a single line once the chart passes
  30 bars.

Security

• cryptography 49.0.0 to 50.0.0 (PYSEC-2026-3552), transitive fast-uri
  (GHSA-7p8r-x3mc-p8w7), and transitive nanoid (GHSA-2v37-7h3g-55p8).
  All three were failing the CI audit gates.