alexgreensh/token-optimizer v5.11.15

v5.11.15 — UTF-8/locale session detection on non-UTF-8 hosts

on GitHub

Fixes

UTF-8 / locale session detection (the headline)

On a non-UTF-8 active encoding (Windows ANSI code pages, a non-UTF-8 LANG/LC_ALL, or PYTHONUTF8=0), Python's std streams, default open(), and subprocess(text=True) decoding fell back to that charset. Session paths or transcript content containing non-ASCII text (Hebrew, CJK, accented names) then raised UnicodeDecodeError/UnicodeEncodeError and aborted the hook before it could identify the Claude session.

• New utf8_io.py: enforce_utf8_io() (reconfigures stdio to UTF-8/replace) and reexec_in_utf8_mode() (re-execs once under python -X utf8 only when the locale is genuinely non-UTF-8; the Windows UTF-8 code page cp65001 is treated as UTF-8 so those users don't re-exec needlessly; loop-guarded).
• hooks/run.py injects PYTHONUTF8=1 + PYTHONIOENCODING=utf-8 into every dispatched child.
• measure.py re-execs then reconfigures stdio first thing, closing the whole class for direct invocation.
• hook_io.py reads stdin from the raw buffer and decodes UTF-8 on POSIX and Windows.
• Explicit encoding="utf-8" added to hot-path subprocess/open() calls.

Platform parity: the fix is centralized in the shared Python scripts, so Claude Code, Codex, Copilot, and Hermes are all covered (every platform that spawns measure.py now injects PYTHONUTF8, and measure.py self-re-execs as a backstop). The TypeScript platforms (OpenClaw, OpenCode, VS Code) were audited and are immune: Node reads UTF-8 regardless of locale.

CLA bot: "signed but still red"

Granted job-scoped actions: write so the post-sign re-run can clear the check (it was failing with "Resource not accessible by integration", leaving PRs red even after signing). Workflow-level permissions set to deny-by-default.

Codex: symlinked config/hooks + accurate memory report (#70, closes #71)

Accept symlinked Codex config/hooks when the resolved target stays under $HOME, write through the symlink on atomic writes, stop failing codex-doctor on a generated hook template, and measure Codex memory from state_*.sqlite instead of legacy ~/.codex/memories.

Housekeeping

• Synced the stale openclaw/openclaw.plugin.json (2.4.7 to 2.4.12) to match its package and last release.