MohamedAbdallah-14/unslop v0.5.4

v0.5.4 — reasoning-trace sanitizer + real DivEye reading

on GitHub

Highlights

Closes two research gaps previously marked partial in docs/research/IMPLEMENTATION_TRACE.md.

New: --strip-reasoning — reason privately, humanize publicly (Cat 06 / 19)

Strips agent reasoning traces before the humanizer runs. Six shapes:

• <thinking> (Claude), <think> (DeepSeek-R1, Qwen), <analysis>, <reasoning>, <scratchpad>, <plan>
• Markdown ## Reasoning, ## Thought Process, ## Plan, ## Analysis, ## Scratchpad sections

Opt-in. File mode writes the stripped content to <stem>.reasoning.md as an audit sidecar.

unslop --deterministic --strip-reasoning notes.md

New: --surprisal-variance — real DivEye reading (Cat 15)

Loads a small causal LM (distilgpt2 default, ~330MB) and computes real per-token surprisal stats. Supersedes the stylometry proxies shipped in v0.5.3, which remain as the zero-dep fallback.

unslop --surprisal-variance --file sample.md
# {"surprisal_stdev": 1.42, "surprisal_cv": 0.38, ...}

Heavy deps (torch, transformers) import lazily. SurprisalUnavailable when unavailable. UNSLOP_SKIP_SURPRISAL=1 forces the skip path in CI.

Test-coverage hole filled

The CLI itself had no direct tests since v0.5.0. Not anymore:

• tests/unslop/test_cli.py — 21 tests
• tests/unslop/test_detect.py — 30 tests (the safety gate, previously untested)
• tests/unslop/test_reasoning.py — 19 tests
• tests/unslop/test_surprisal.py — 9 unit + 2 opt-in end-to-end

Total: 474 tests pass. ruff and mypy --strict clean. 92.0% AI-ism reduction on the nine-fixture benchmark holds.

Full changelog: CHANGELOG.md