github kerpopule/hermes-jev-skills v0.17.0
0.17.0 — the bugs outside readers found

latest releases: v0.25.2, v0.25.1, v0.25.0...
one month ago

Three releases' worth of work, and most of what is in here was found by people reading the code rather than by anyone running it.

Two bugs an outside reader found and reproduced

A cached routing decision could send a turn to a model that cannot hold it. The cache key left out the context size, and a cache hit returns before the model is picked — so the context-fit check and the "a large context never switches down" guard were both skipped. A repeated short instruction seen first in a small context and again once the session had grown was answered from the first: a 300,000-token turn routed to a model with a 32,000-token window. The key now carries a coarse context bucket, and a fingerprint of the pools, so editing routing.json also takes effect without restarting.

Compaction batched by turn count, so a non-Latin transcript lost whole batches and said ok. Forty turns of Japanese encode to about 120,000 characters against a 60,000 limit. Every batch failed, and "ok" if calls let one good batch hide the rest. Batches are packed by encoded size now, and a run where some failed reports partial and names the turns left unjudged.

Both reported by @MrJev in #3, with reproductions. jev ask returning http_400 for the list question shape its own help advertises was reported by @tontontimiro in #2 and fixed in 0.16.0.

Guards that read the wrong copy

The risk-word floor and the "does this look sensitive" check both ran on the clipped copy of a long turn. A destructive instruction in the middle of a long paste tripped neither: it routed to the cheapest tier and its text was sent as ordinary text. Clipping is for what Jev reads, never for what we check.

Jev through OpenRouter — one key instead of two

jev setup-key --provider openrouter reaches Jev through OpenRouter's Decisions API. Same request, same answers, same model: 433 ms against 569 ms direct on one measured decision. If both keys are present TypeSafe wins, so an existing install never silently reroutes.

The idea and the first implementation are @Barba2k2's. The mechanism changed — the original prompted a chat model for JSON, and a chat model reports a confidence it invented, which would have quietly moved every threshold in this repo.

We measured our own headline feature and it lost

0.14.0 shipped an eval that tested the claim that Jev-selected handoffs are better. On seven real sessions and 104 recall questions, a handoff written from the Jev digest recalled 37.5% against 48.1% for one written from the plain transcript of the same size. What won was the whole dialogue at a 1,200-word budget: 58.7%, and 75.0% with one search of the old session. The Jev pre-pass is off by default now. The eval is in the repo so you can run it on your own sessions.

For people who want to contribute

SECURITY.md, issue and PR templates, and a CONTRIBUTING.md that opens with the two traps that have caught everyone here, including the author: a green run locally is not a green run on CI, and Jev's confidence is calibrated while a chat model's is not. scripts/triage_github.py reads the open PRs and issues nightly and writes a report — it posts nothing.

Contributors are named in the changelog, kept as commit author or co-author, and named in NOTICE if their project was ported from.


825 tests, on Ubuntu and macOS, Python 3.9 and 3.13. MIT.

Don't miss a new hermes-jev-skills release

NewReleases is sending notifications on new releases.