kerpopule/hermes-jev-skills v0.20.0

0.20.0 — measured on real history, most of it kept in shadow

on GitHub

Small agent decisions as policies, measured on one fleet's real history before any of them acts. Most of them did not earn a live switch, and this release says which.

0.19.0 was written up and bumped in the code on 2026-09-21 but never tagged or released on GitHub, so this release carries it too: jev search as a loop, with Jev picking which results to open, whether that is enough, and which of your queries runs next. Everything new here that could change what an agent does is off by default, or runs in shadow: it logs what Jev would have decided and changes nothing.

What measured well enough to ship

The new agent-facing features among these are off until turned on.

• Web results screened before the agent reads them: 70 of 79 planted attacks caught on 80 real results, 0 of 1,520 clean units withheld.
• triage_github.py screening each PR and issue: 22 of 24 planted attacks caught, 0 of 24 untouched items marked (the second question was written after seeing the misses, so optimistic until a held-out set confirms it).
• Skill suggestions that stop repeating: on a replayed week, 51% of suggestions loaded against 34%, with 8 loads lost. Parameters were chosen on that same week.
• Pooled connections: client.ask median 522 ms to 178 ms. Routing and skill selection in one request, measured to decide the same.
• Lanes: in a controlled replay of 9 real commits, Sonnet at medium (Jev's pick for all 9) passed 9/9 at $1.60 against 9/9 at $3.27 for Opus at high. n = 9.
• Co-Agent first for computer use on a Mac: 10/10 on four real apps, about 0.45 s per Jev step.
• Fixes: jev plan 20/22 to 33/33 planned; jev batch no longer skips 60% of rows at the rate limit; jev search stops looping when pages will not open; Linux text fields reported as role text get typed into; the plugin manifest reports the running version.

What measured and was dropped, or stays in shadow

• Dropped after about 5 tuning rounds each, on 18,585 Jev calls over real history: Kanban retry, blocked-card-needs-a-person, cron wake, card owner, completion pre-screen, stale-triage alert, cron-report delivery, and an output trimmer. None beat the current way. Their policies and hooks ship, off; kanban_done runs only with a named local policy.
• Shadow: gate-ask (0 of 65 must-deny/ask fixtures approved, 52 of 53 harmless approved, but extra prompts on 4.7% of real commands, above the 1-in-50 bar, and the thresholds were chosen on those rows). gate-strict asked about or denied 73% of real commands.
• Shadow: the Jev-Omni vision hybrid. 84% would stay local, but it agreed with the current model on 67%. Promotion should go through typed PASS/FAIL questions.
• Shadow: Hermes lanes. Calibrated on 2,357 Kanban cards they are -1% tokens against today's default; the article's effort ladder read literally would have cost +52%.
• gate-task is not backtested: the red-team set has no tasks.
• Measured and not adopted: permutation averaging on both Choice surfaces, stakes and margin in choose, Stagehand's second question. Measured and kept: skill selection's stage 2.

Tokens saved on the fleet by this release today: 0. The shadows cost about $0.07 a day and add no latency to a turn.

Also in this release

• Decision policies (jev decide, jev batch, jev shadow report, jev gate), with a machine-judged promotion report and a 118-command red-team set. Walkthrough: docs/shadow-to-live.md.
• Lanes (jev lane) and four Claude Code lane subagents written by the installer, with a delimited, backed-up block in ~/.claude/CLAUDE.md (--no-claude-md skips it). docs/lanes.md.
• OpenCode Zen as a provider (jev setup-key --provider zen, JEV_PROVIDER), and a gateway under a path prefix with its own bearer (JEV_PROXY_API_KEY, #23, #24, from Timo Goetzken). No provider key is ever sent to an override.
• Optional per-model reasoning effort from the existing difficulty answer (PR #17), off by default.
• Community fixes from the backlog: Venice transport (PR #6), launcher outside the checkout (PR #7), block-scalar skill descriptions (PR #10), Linux GUI observation (PR #13), README skill count (PR #16), custom provider aliases (#18).

The full list, with every number and its caveat, is in CHANGELOG.md.

1,330 tests pass on Python 3.9, 3.12 and 3.14 on macOS; CI green on main. MIT.