yvgude/lean-ctx v3.8.13 on GitHub

Added

ctx_explore — delegated, deterministic repository exploration (gitlab #907).
A new multi-turn explorer — MCP tool #78 plus a lean-ctx explore CLI — that
answers "where does X live / how does Y work" in a single call instead of the
agent's usual read→grep→read loop. It seeds with BM25 lexical retrieval, expands
along a bounded graph BFS grounded in the hit set, then selects citations by
coverage, returning byte-stable path:start-end ranges (with a citation-only
mode for minimal token spend). Wired into the tool registry, the standard and
read-only tool profiles, and the heavy-index warm-need; eval_harness gains a
SearchArm::Explore (output-token metric plus new "exploration" queries in
rust/eval/search-suite.ndjson) so A/B runs can compare explore vs hybrid vs
bm25 on recall/MRR/tokens. Output is a deterministic function of repo content
(#498).
Codex ChatGPT subscription auth now routes through the proxy (#568).
Completes the #554 fix: instead of skipping config when a Codex ChatGPT login
is detected (which left subscription users at 0 savings), install_codex_env
now writes a mode-specific config. ChatGPT subscription auth is pointed at the
proxy's Codex backend rail (model_provider = leanctx-chatgpt, openai_base_url = …/backend-api/codex, chatgpt_base_url = …/backend-api for aux calls such as
the codex_apps streamable-HTTP MCP endpoint); API-key Codex keeps the /v1
path. The proxy gained /backend-api/codex/responses (compressed/metered via the
OpenAI Responses path to chatgpt.com) plus credential-preserving passthrough
for non-model /backend-api/* traffic. Header forwarding stays allowlist-based
both ways; a dedicated cookie store persists only Cloudflare anti-bot cookies
(cf_clearance/__cf_bm/cf_chl_*) and drops auth/session cookies; gzip/zstd
request bodies are decoded under a bounded reader (zip-bomb safe) before
compression and re-encoded. Thanks @ousatov-ua.
lean-ctx doctor warns when the MCP server is launched from a directory
without a project root (#547). When an MCP client spawns lean-ctx from an
IDE/agent config dir (.lmstudio, .claude, .codex, .codebuddy) or any
marker-less CWD, every out-of-tree ctx_read fails with "path escapes project
root". The new MCP server CWD doctor check (also surfaced in the structured
health report) explains the cause and the fix (cwd in the client config, or
allow_auto_reroot/extra_roots); .lmstudio is now also treated as a
suspicious persisted root. Thanks @albinekb.
Shadow-mode CLI reads/searches now record Context IR lineage (#566).
Follow-up to #550. The MCP dispatcher records a Context IR provenance entry for
every tool call (server/call_tool.rs), but the shadow-mode hook's single-shot
lean-ctx read/grep subprocess dropped it — so ctx_proof and IR exports
were blind to compressed shadow reads. record_file_read/record_search now
thread the rendered-output excerpt + measured tool duration into a disk-backed
ContextIrV1 load→record→save, mirroring the MCP path (same 200-char
char-boundary excerpt bound; mode/pattern ride the IR pattern slot; the
read's original_tokens and the search's raw matched-line estimate are the IR
input so the stored compression ratio is accurate — no fabricated values). The
two remaining MCP read side effects from #550 — the in-memory loop/correction
detectors and the bounce/adaptive-threshold signals — are now delivered via
connect-only daemon routing: when an MCP daemon is already running, a
shadow-mode lean-ctx read/grep routes the call through it
(/v1/tools/call → call_tool_guarded), so loop detection, correction-loop
auto-degrade, bounce tracking and adaptive thresholds all fire on the daemon's
long-lived state — full MCP parity, for free. The hook child connects only: it
reuses a live daemon but never auto-starts one (a per-call subprocess
auto-starting daemons would proliferate them, the #453 class of bug), falling
back to the enriched standalone path (disk-backed learning sinks + the Context
IR above) when no daemon is reachable or on Windows. This resolves the design
decision #566 was gated on; the connect-only invariant is documented in
daemon_client::try_daemon_tool_call_blocking and regression-guarded by
hook_connect_only_566.
PowerShell-native cmdlets route through lean-ctx (#561). Follow-up to #556:
shadow/harden mode already recognised the Windows powershell shell tool, covering
the Unix-style PS aliases (cat/ls/rg). The command-rewrite layer now also
maps the PowerShell-native cmdlets and their short aliases — Get-Content/gc
→ lean-ctx read (honoring -Path, -TotalCount/-Head/-First and
-Tail/-Last), Select-String/sls → lean-ctx grep (-Pattern, -Path),
and Get-ChildItem/gci → lean-ctx ls (-Path). Parameter names are matched
case-insensitively; anything with an unrecognised flag, a pipeline, multiple
operands, or an out-of-project path passes through untouched (same conservative
contract as the Unix rewrites), so determinism and redaction guarantees are
inherited. The PowerShell cmdlets are detected only in the rewrite path and are
deliberately kept out of the POSIX shell-alias surface.
Addon security hardening — trust, policy, signing, sandbox, audit (#863).
Because an addon is executable trust (a stdio addon runs code on your machine;
an http addon receives your context; its output enters the model), the
ecosystem ships with defense-in-depth across three tiers:
- Trust tier + risk review. A registry-controlled addon.verified flag
  splits the catalog into verified (maintainer-audited) and community, shown
  in addon list/info and the install preview. core::addons::trust::assess
  statically reviews the [mcp] wiring (remote endpoint, non-HTTPS, inline
  shell, fetch-and-exec, unpinned upstream, secret-bearing env) at info/warn/
  danger severity. The same logic backs a registry CI validator
  (registry::validate_entries, run by cargo test): unique slugs, required
  provenance for installable entries, no shell/fetch/non-HTTPS/unpinned wiring,
  and zero findings for verified entries.
- Install policy floor — [addons]. A global-only config block (never
  merged from a project-local file): policy (open/verified_only/
  allowlist/locked), allowlist, require_signature, sandbox,
  block_risky. policy::gate enforces it in install before any gateway
  mutation. Fully permissive by default; distribute via MDM or pin through the
  signed org-policy floor.
- Registry signing. A user-override registry can shadow trusted names; with
  require_signature = true it is honoured only if a sidecar
  addon_registry.json.sig carries a valid Ed25519 signature by a trusted org
  key (same anchor as policy org trust).
- Opt-in OS sandbox. addons.sandbox = auto|strict wraps spawned stdio
  servers in sandbox-exec (macOS) / bwrap (Linux) at the single spawn point
  — outbound-network isolation in auto, read-only fs + refuse-if-no-launcher
  in strict. Off by default.
- Runtime redaction + audit. Downstream tool output is run through the
  shell-layer secret redaction and audit-tagged as untrusted before it reaches
  the model (runtime::scrub_output).
  New small, unit-tested modules core::addons::{trust,policy,signing,sandbox, runtime}; binding registry-review checklist in CONTRIBUTING.md.

Changed

Leaderboard — no top-50 cap, real pagination, everyone findable. The
community leaderboard previously truncated to the top 50 accounts, so most
contributors never appeared and the headline community energy could silently
drop when the cut-off shifted. GET /api/leaderboard now paginates
(?page, ?per_page, default 50 / max 200) and supports case-insensitive
name search (?q=), while two new fields — total_tokens_saved and
total_cost_avoided_usd — report the uncapped community totals across all
opted-in accounts, independent of the displayed page or any filter. The
server-rendered /leaderboard page and the website /metrics page gained
matching search + pagination controls; the landing-page hero energy stat and
the in-app cockpit now read the uncapped totals so headline numbers stay
stable. Global ranks are preserved across pages. Pagination, ranking, totals
and search are pure, unit-tested functions (paginate, all_ranked_cards).
(gitlab #868–#871)

Fixed

ctx_impact dropped C# same-namespace blast radius after the first reindex
(#398). A C# class used within its own namespace (no using, DI-injected)
reappeared as a leaf node after the first background reindex. The private
ctx_impact builder wrote precise type_ref edges into the PropertyGraph, but
every ProjectIndex::save() mirrors graph_index over the graph via
clear_code_graph() — and graph_index emitted no type-usage edges, so the
reindex silently wiped the blast radius (a dual-writer bug). A new
core::type_ref_edges module is now the single source of truth for C#/Java
consumer→definer file resolution (namespace-aware, failsafe-capped), shared by
both the durable graph_index mirror and the ctx_impact builder; graph_index
now emits these precise edges instead of the old coarse alphabetical
namespace-chain heuristic, so a reindex reproduces the blast radius instead of
dropping it. GRAPH_ENGINE_VERSION is bumped (2→3) so stale graphs self-heal on
the next query, and the regression tests now run through the index mirror — the
exact gap every prior #398 fix missed. (The grep hook also now redirects only
output_mode=content, passing files_with_matches/count through untouched,
since the path-swap returned wrong results for those.) (gitlab #915–#918)
ctx_read left an empty [] metadata field on incompressible files (#509).
The entropy (and density) read modes append a [techniques…] tag listing
which compression techniques fired. On a file where none did (high-entropy, no
duplicate blocks) the technique list is empty, so the header rendered a bare
H̄=4.2 [] — the same empty-trailing-field waste fixed for
ctx_semantic_search's (rrf: X, ) in #511. A techniques_tag helper now omits
the bracket segment entirely when the list is empty (and keeps the leading space
- [a, b] form otherwise), so the header is clean on both paths. (#509-A output
  audit; output stays a deterministic function of content/mode per #498.)
Pi AGENTS.md advertised renamed tools that no longer exist (#548). The Pi
installer writes a curated static templates/PI_AGENTS.md, and its tool-mapping
table still listed ctx_grep/ctx_find/ctx_ls — tools renamed long ago to
ctx_search/ctx_glob/ctx_tree. Pi agents that followed the table issued
unknown-tool calls. The template (and the matching pi.rs setup hint) now use the
canonical names, and a new parity test (tests/rules_template_tool_names.rs) ties
every shipped agent template to the live MCP registry: any ctx_* reference
that is not a registered tool fails the build, so a future rename can never drift
silently again. (First slice of the #548 agent-rules unification — marker/dedup
consolidation, content-aware freshness, and rules.toml↔sync semantics follow.)
Rule injection skipped content/compression changes when the version was unchanged (#548).
The injector's freshness check was version-only: it compared the on-disk
 against RULES_VERSION and skipped the rewrite when they
matched. So a change that alters the rendered body without bumping the version —
toggling shadow_mode, switching compression_level, or editing a canonical
section between releases — left every agent's rules block stale until the next
version bump. RulesFile::block_matches_render now compares the on-disk block
byte-for-byte (whitespace-insensitive) against a fresh render for the active
parameters, and the skip path requires both is_current() and that content
match. Re-running sync/inject after a compression-level change now regenerates
the block as expected; an unchanged config stays idempotent. (Second slice of the
#548 agent-rules unification, after the Pi-template parity guard.)
rules diff/sync ↔ .lean-ctx/rules.toml semantics, and a rules diff
false-positive (#548). Two coupled fixes for the rules-governance commands:
- sync/diff do not consume rules.toml — now documented and decoupled.
  rules sync/diff regenerate from the canonical rules_canonical source of
  truth (preserving user text around the markers) and never read rules.toml,
  which is the input for rules lint plus a user-editable inventory from rules init. This is now stated in the rules help, the init next-steps, and the
  RulesConfig/sync docs. detect_drift no longer loads RulesConfig at all,
  so rules diff works without first running rules init (it previously
  failed with "No rules config found") — the dead _config parameter is gone and
  the command is infallible.
- rules diff reported phantom drift after every sync. Drift picked the
  shared-vs-dedicated expected block from a content heuristic ("up_to_date and no
  'existing user rules'"), which misread freshly synced shared files with no
  user text (Copilot CLI, Codex CLI, Gemini/OpenCode in shared mode) as the
  dedicated layout and flagged them as DRIFTED on every run. Drift now compares
  each target against the canonical block for its real RulesFormat via the
  new rules_inject::expected_blocks_by_target, keeping sync and diff in
  agreement. Covered by new tests: detect_drift_without_rules_toml_does_not_ require_init and sync_then_diff_reports_no_drift. (Third slice of the #548
  agent-rules unification.)
Compression block had two disagreeing marker models, so cross-channel dedup
never fired (#548). rules_canonical::render embedded the output-style
compression prompt inline inside the  block with no
delimiters, but the coverage/dedup readers (rules_channel, rules dedup)
detect the payload by a separate  …  block. Since the writer never emitted those markers,
cursor_compression_covered/client_autoloads_compression were always false on
freshly written rule files — so the MCP per-session instructions kept repeating
the compression block even for Cursor/Codex that already load it from their rule
file (double billing), and rules dedup could not thin a render-produced shared
AGENTS.md. The two models are now one: the COMPRESSION_BLOCK_* markers live in
rules_canonical (single marker source, re-exported from rules_channel), and
render wraps the compression prompt in them for the persistent carriers
(Dedicated/Shared — every injected rule file). The ephemeral Bare MCP
channel stays unmarked by design (its inclusion is governed by carrier coverage,
so a per-session marker would be noise). Content-aware freshness (second slice)
re-propagates the new block on the next sync without a version bump. Covered by
new tests asserting carriers wrap / Bare does not / Off emits nothing, and an
end-to-end check that a render-produced Cursor block is now recognised as
compression coverage. (Fourth slice of the #548 agent-rules unification — closes
the "one canonical carrier/marker model" acceptance criterion.)
Shadow-mode hook reads dropped ~75% of the MCP read side effects (#550). When
shadow/harden mode intercepts a native view/grep call it spawns lean-ctx read
as a single-shot subprocess. That CLI path recorded only a fraction of what the MCP
ctx_read pipeline does, and — crucially — never flushed its buffered telemetry
before the process exited, so lean-ctx heatmap stayed empty and lean-ctx gain
reported nothing for compressed reads. Three fixes:
- One flush set, no drift. A new tool_lifecycle::flush_all() is the single
  source of truth for the buffered-telemetry flush (stats, heatmap, path-mode
  memory, auto-mode resolver, edit-quality, mode predictor, feedback, threshold
  learning, LiTM calibration). The daemon shutdown, the parent watchdog and every
  CLI tool arm (read/grep/ls/find/deps/diff/-c/-t) now call it — the
  hand-rolled per-arm copies had drifted (the read arm flushed only stats), which
  is exactly how the gap went unnoticed.
- CLI read learning parity. record_file_read/record_search now run the same
  disk-backed learning sinks the MCP background thread does — mode-predictor training,
  the per-language compression feedback outcome, and the per-call anomaly metric — so
  auto-mode selection, the feedback loop and dashboard signals improve from
  shadow-mode reads too (not just direct MCP calls).
- Mode predictor actually persists now. ModePredictor stored its history in a
  struct-keyed HashMap<FileSignature, _>, which serde_json cannot serialize
  ("key must be a string") — so mode_stats.json was never written and the
  predictor relearned from zero every process. The history now serializes as an entry
  list (round-trip tested). The in-memory-only loop/correction detectors and the
  bounce/adaptive signals that need routing through ctx_read::handle are tracked as
  a follow-up (they cannot be honored from a single-shot subprocess without
  cross-process state).
Windows PowerShell profile path hardcoded to ~\Documents — broke under OneDrive
redirection (#558). proxy enable and the shell-hook install resolved the
PowerShell profile by hardcoding home\Documents\PowerShell\…. Windows OneDrive
folder backup (on by default on most installs) redirects Documents to e.g.
…\OneDrive\Documents\…, so lean-ctx wrote to a file PowerShell never reads — the
active $PROFILE was never updated and the proxy received no traffic in new
terminals. A new resolve_powershell_profile_path asks PowerShell itself for
$PROFILE.CurrentUserCurrentHost (authoritative under any folder redirection,
preferring pwsh then Windows PowerShell, UTF-8 output) and falls back to the
documented default only when no PowerShell host can be launched. Non-Windows hosts
keep the static ~/.config/powershell path and never spawn a process (#356).
Copilot CLI view (read) and rg (search) tool calls passed through uncompressed (#562).
handle_redirect dispatched on the tool name but only matched Read/read/
read_file and Grep/grep/search/ripgrep, so two documented GitHub Copilot
CLI tool names — view (its read tool) and rg (its search alias) — slipped
through without compression in shadow/harden mode. The dispatch is now a tested
classify_redirect helper that includes view (→ read) and rg (→ grep); the
Claude/Cursor/CodeBuddy matchers are unchanged because those hosts never emit
those names and Copilot CLI fires the hook for every tool call.
Copilot/VS Code Claude models ignored lean-ctx — no .github/copilot-instructions.md (#555).
lean-ctx init --agent copilot installed the MCP server plus a deliberately
weak AGENTS.md pointer but never wrote .github/copilot-instructions.md, the
repo-level file VS Code Copilot Chat auto-applies to every request. Claude-
family models (Sonnet/Opus) therefore ignored the tool mapping while GPT-5.x
followed it ~95% of the time. init now writes the strong dedicated ruleset
into .github/copilot-instructions.md as an idempotent 
block (user content is preserved, never clobbered) and pins
github.copilot.chat.codeGeneration.useInstructionFiles: true in the project
.vscode/settings.json as a safety net (an explicit user value is honoured);
uninstall removes the block.
Shadow mode ignored glob and Windows powershell tool calls (#556).
Shadow/harden mode silently passed two documented Copilot CLI tools straight
through: the glob tool ("find files matching patterns") had no arm in the
redirect hook, and the powershell shell tool (paired with bash on Windows)
was not recognised as a shell, so command rewrites never fired there.
handle_redirect now intercepts Glob/glob — warming the shared ctx_glob
core via a new lean-ctx glob subcommand and recording the intercept in
shadow.log, then letting the native path-list result through — is_shell_tool
(now shared by both hook entry points) covers PowerShell/powershell/pwsh,
and the Claude/Cursor/CodeBuddy redirect matchers include Glob so the hook
fires for it. Copilot CLI already dispatches every tool, so its glob/
powershell calls are covered automatically.
Codex proxy never compressed — ChatGPT login bypasses it; the API-key config
was a no-op (#554). lean-ctx proxy enable reported success for Codex yet
Requests/Compressed/Tokens saved stayed at 0, for two reasons. (1) A Codex
ChatGPT login (the default) authenticates via OAuth directly against
chatgpt.com/backend-api, so a custom openai_base_url is ignored and the
proxy never sees the traffic — the Claude Pro/Max situation, but with no
warning. lean-ctx now detects a ChatGPT login (~/.codex/auth.json
auth_mode = "chatgpt", overridable by an explicit OPENAI_API_KEY) and
prints an honest skip notice pointing at the MCP tools instead of writing dead
config. (2) In API-key mode lean-ctx wrote [env] OPENAI_BASE_URL into
~/.codex/config.toml, which Codex does not read; it now writes the documented
top-level openai_base_url key (openai/codex#12031), migrates the dead legacy
entry, and preserves any custom remote endpoint. Uninstall/cleanup/preview
handle both forms.
lean-ctx index build-semantic cold-starts the embedding model again
(#545). On a machine without the model cached, the build dead-ended with
"embedding model not downloaded — auto-download … failed" even though no
download was ever attempted: the build path checked is_available() (a pure
file-existence check) and bailed before the download could run — a regression
from the #519 ORT-teardown guard. build_or_update now downloads the model
first via a new EmbeddingEngine::ensure_downloaded() (pure network/file IO,
no ORT init) and only loads the ONNX Runtime once the files are present, so the
cold bootstrap works again and the #519 teardown safety is preserved. The
passive search path is unchanged.
Copilot CLI hooks silently no-opped — wrong payload field names and missing
modifiedArgs (#551). Copilot CLI sends camelCase toolName + toolArgs
(a JSON-encoded string), but the rewrite/redirect/observe handlers only read
snake_case tool_name/tool_input/command, so every Copilot tool call
passed through unchanged; even once parsed, the preToolUse output omitted
Copilot's modifiedArgs field, so rewrites/redirects would never have taken
effect. A new hook_handlers::payload resolves the tool name, args (a
tool_input object, a toolArgs object, or a toolArgs JSON-string) and
command across all handlers, observe gains a Copilot postToolUse branch so
its telemetry is recorded instead of dropped, and the hook now emits Copilot's
documented permissionDecision + modifiedArgs contract alongside Claude's
hookSpecificOutput.updatedInput and Cursor's updated_input in a single
response. Snake-case (Claude/Cursor) stays regression-tested. Thanks for the
detailed report.
CLAUDE.md/CODEBUDDY.md pointer block duplicated on every setup/doctor --fix (#549). The block-detection constants pointed at the wrong marker:
*_MD_BLOCK_START/END referenced the canonical rules marker
() while the installer writes the AGENTS pointer block
(), so existing.contains(START) was always false — the
doctor reported the block as missing and every run appended a fresh copy,
accumulating duplicates. The constants now point at AGENTS_BLOCK_START/END
(one fix for both the doctor false-negative and the duplication), a new
remove_all_blocks() collapses any already-accumulated duplicates back to a
single canonical block in the installers and strip_*_md_block, and the doctor
fixtures seed the real pointer marker.

Upgrade

lean-ctx update                 # recommended (auto-downloads + refreshes shell hooks)
cargo install lean-ctx          # or
npm update -g lean-ctx-bin      # or
brew upgrade lean-ctx

Note: After upgrading via cargo/npm/brew, run lean-ctx setup to refresh shell aliases. lean-ctx update does this automatically.

Full Changelog: v3.8.12...v3.8.13