vshulcz/deja-vu v0.20.1 on GitHub

Changed

The relevance tier reads and folds the messages that matched instead of every message of every candidate. The ranking already knew which records they were — the per-message signals are keyed by their offsets — and the exact tier has read records at offsets since it was written. On a 2,716-session store one relevance query was folding 120.7 MB across 140,355 messages, of which 10.3% held any query term: measured interleaved, 1,740 ms to 480 ms and peak RSS 377 MB to 158 MB. Quality is unchanged on every benchmark — LoCoMo 69.7% hit@1 / 0.767 MRR, LongMemEval 84.8% / 0.894, day0bench over 19,195 sessions 13/60 hit@1 and 27/60 hit@5 with p50 80 ms to 61 ms, and the recall and prompt benches identical. (#3491)
A search hit carries the passages that matched rather than its session's whole transcript: the matched messages, each with the answer after it, at most twenty per hit, plus messages_total and messages_capped. The size of an answer used to be the size of the reader's longest transcript — on a 2,716-session store deja search --json returned 136 MB over 50 hits and 140,841 messages, and encoding it was a second of the 2.7 it took and half a gigabyte of resident memory. Measured interleaved on that store: 136.18 MB to 2.60 MB, 2,660 ms to 1,790 ms, peak RSS 803 MB to 361 MB, with the recall, context and prompt benches unmoved. deja blame --json takes the same bound. (#3620)

Added

A store can be excluded, not just a project: a line prefixed harness: in ~/.config/deja/exclude, or DEJA_EXCLUDE_HARNESSES. deja then neither walks it nor asks for the tool that would read it, and every screen that mentions stores says so — deja doctor on both its surfaces, deja sources, and the empty screens, which used to blame a machine no agent had run on for a store that is on disk and deliberately unread. Previously the only way to stop needs-sqlite3 advice for a harness the reader does not use was to install the package. (#3499)
deja bench read: what it costs to read a database-backed store, and what one long escape-heavy value does to it. Every other benchmark runs against an already-indexed corpus, which is how a reader that took 2,287s on a 6.16 MB value stayed invisible while all of them held flat. (#3552)
A store that is slow to read says which one it is and that it is still moving, every thirty seconds, and its read time lands on its line. A pass over a 520 MB store gave thirteen minutes of one static line and no way to tell a slow read from a stuck one; the slowest store on a 3.4 GB corpus reads in 10s, so nothing says anything on an ordinary run. (#3555)
deja bench ingest: what an index update costs, per class of change — unchanged, an appended turn, a new transcript, a renamed one, a rewritten one — with whether the pass replaced the records already on file as the gate. (#3507, #3546)

Changed

CI gates what an update costs, per class of change. deja bench ingest has reported it since #3507 and nothing in the build looked, which is the gap that let a path re-tokenise the whole store on every new session for two months and twenty releases (#3500). The gate is the shape rather than the clock: the five classes have to be there and only a rewritten transcript may make the pass replace the records already on file. Verified by putting #3500's shape back — the check fails on it. A wall-clock margin was tried and dropped: locally it was five-fold, on the runner new transcript came in 1% above the rewrite. (#3505)
The nightly migration matrix checks older config shapes as well as older stores: each release writes its own wiring, and the build under test has to read it as wiring rather than as nothing, and has to call it dead once what its entries name is gone. (#3505)
deja doctor --json carries the auto-recall rows, not only the MCP ones: a script could see a missing sqlite3 and not a hook running a binary that is gone. (#3540)
An upgrade says why it is re-reading every source. Every other reason for a full pass names itself; a version difference printed the line a first install prints while doing the longest piece of work deja does on a large store. (#3500)
Index format version 41 applies the new redaction to text already stored. Redaction runs at ingest, so every earlier fix to it reached only the next conversation; a store re-reads its sources once. (#3535)
Index format version 39: dsh names its logs session.v3.jsonl now, and a store that already holds the older names re-reads its sources once so the new ones join it without deja index --rebuild. (#3508)
A new transcript is appended to the index instead of rewriting it. Every new conversation is a new file, and the path that refused an unseen one cost 4.76s against 0.28s on a 171 MB index, growing with the store rather than with the file. (#3503)
A search quotes the hits it serves rather than every session that matched, so its cost follows what was asked for: on a store where every session matches, 806 ms and 127.8 MB against 745 ms and 88.9 MB. (#3544)
The freshness walk no longer runs to be discarded: deja index walked every store a second time whenever it had already reported what it did, 52 ms on a 2.0 GB store. (#3501)

Fixed

A file sitting where the index directory belongs is left alone. The swap parked it as <dir>.old and then deleted it, so pointing DEJA_INDEX_DIR at a database or an archive by mistake removed the file, said nothing and exited 0. A build refuses and names the path now, and doctor says which state it is in — index.state gains path-is-a-file — because "run deja warmup" is advice that cannot be followed there. (#3610)
deja says why a first build could not run. The screen a new install sees reported open /…/index.db.lock: permission denied — an internal lock file and an errno — while deja index in the same state names the directory to fix and the variable that moves it. (#3613)
deja sources no longer reads the two stores whose rows it writes by hand. An excluded opencode kept being opened and kept printing the sqlite3 error the exclusion exists to silence, while doctor said excluded for the same store — the row the loop prints now covers aider and opencode too. (#3611)
deja show no longer prints a session from a store the redaction floor withholds. Every ranked surface rebuilds before answering from a store written under redaction rules this build has moved past; show loads by identity, and so did the MCP tools that share that loader — which is the surface that shows the most at once. The loader refuses and names the state, and show re-reads the sources first, the way the ranked paths do. (#3617)
search --json, last --json, show --json and blame --json strip the characters that make displayed order disagree with stored order. The printed surfaces have stripped U+202E and the invisible tag block since #1090, and the encoder escapes control bytes — which is why the half it does not escape went unnoticed on the one surface a dashboard reads. (#3616)
A secret written api-key "…", api key "…" or x-api-key: "…" is masked. The gate in front of the quoted-secret rule tested three spellings of the name while the rule's own pattern accepted a fourth, so which spelling a tool printed decided whether the value was stored in the clear; the colon form fell between that rule and the key-value one, whose value class starts at sixteen characters. Index format version 46 masks what is already stored. (#3614)
The point-of-action hook reads what a command settled instead of searching for it. Finding it at the moment of the action meant ranking candidates by the command's words and then loading whole sessions to ask which of them had actually run it — 133 ms an action against 21 ms, on the surface that fires on every action, and never warm because almost every action runs a command its session has not run before. The build now stores each session's conclusion and, per command, which session ran it last, so the answer is two map lookups: measured over 42 actions on a real store, decisions 4 to 16 and the p90 call 174 ms to 41 ms. Index format version 45. (#3001, #3605)
The Zed extension asks the host whether it can run deja before downloading its own copy. The check it had asks the filesystem for four absolute paths, which an extension cannot reach from inside the wasm sandbox — measured on the same target it is built for, every candidate comes back NotFound while a run with the filesystem preopened finds both — so a user who already had deja was handed 11 MB of it anyway. Running deja --version through the host answers the only question that matters, by the same mechanism Zed uses to launch the server. (#3392)
Every read against a SQLite store carries a wall-clock budget, ten minutes by default. One sqlite3 child ran 13m54s with 0.75s of CPU in deja itself and nothing in the tree set a deadline, so deja index looked hung rather than slow; a store that runs out is now an ordinary read error, named by deja doctor, and the rest of the index still builds. For scale, the largest store here is a 3.2 GB opencode database that answers in 11.7s. DEJA_STORE_TIMEOUT overrides it, and a zero turns it off. (#3555)
The pre-tool hook says the line behind a fact it has already given this session, instead of going quiet. It picked one line from four producers ordered by certainty, and when that line turned out to be a repeat the call ended there — so one absent program, named once, silenced the surface for every later command that mentioned it: three lines in ninety-one actions on a real store. The retry now walks every producer, paying lookups alone past the first two: 7.5 ms against the 111 ms the same walk costs when it searches. (#3603)
deja doctor --json carries the Claude Code and codex hook rows. Both print in the text report and neither reached auto_recall, because both predate the table the other nineteen harnesses share — so a script watching its own install saw nothing about the harness most people run, including the state an upgrade leaves where every hook exits 127. Each row is decided once and read by both surfaces. (#3608)
deja doctor --json says when the store is not what this build writes. It called a store ok while its recall was off, so a script watching index health saw nothing wrong — the same miss #2292 closed for damage. index.state gains rereading and index.format names which of the four states it is in. (#3600)
deja forget calls a remember note what it is on the run that drops it, not only on --dry-run. The sentence that tells a written note from a promoted one was wired into the dry-run branch alone, so checking first and then doing it gave two different answers, and the wrong one was on the path that changes something. (#3599)
The session-start block no longer quotes a store the redaction floor is withholding. It reads the friction sidecar rather than the records, so it went around the gate that stops text written before a redaction fix from being shown, and put a wall from the old store into the model's context on the first session after an upgrade — in the same message that said recall was not ready yet. Every other surface runs a rebuild first; the hook cannot, which is the reason the floor exists. (#3598)
deja doctor no longer says the index cannot be read and then reads it. The row was driven by the content version and worded for the on-disk layout, so the common upgrade — a role filed better, a title read from a different field, which outnumber the redaction bumps three to one — was reported as a store this build could not open while --deep re-parsed its sessions three lines below. Four states now, four sentences. (#3597)
A secret handed to --passphrase, --secret, --token or --api-key is masked. The flag rule covered --password alone, so gpg --passphrase … and every CLI that takes an account credential on its command line went through in the clear. The same change stops the rule eating a word of prose: run with --password from the keychain lost "from" and counted it as a secret found, and with a space rather than an = a value that reads as an ordinary word is now left alone. Index format version 44 masks what is already stored. (#3596)
deja search says when a hit is stamped later than this machine's clock. It ranks on recency among other things, so the sessions whose date cannot place them are the ones it places first — and it was the one date-ordered surface that printed Jan 1 2099 beside an answer and left the reader to work out what it meant. (#3595)
deja stats --role reaches every role deja help documents. Three of six came back empty on a store holding those records: tool because stats compared the stored name tool-output against the documented alias, and command, files and edit because the index withholds work records from a caller that did not ask for them and stats asked for everything, then narrowed afterwards. It is also why deja index reported 18 messages where deja stats reported 12. (#3592)
deja stats names the filter that emptied it instead of calling the index unbuilt. --since 30d over a store whose sessions are all older answered "nothing indexed yet — run deja index", which is advice for a state deja is not in; --harness and --project did the same. The usage line now also names the four filters stats has always taken. (#3591)
The grok linearity test takes the best of three timing pairs. A ratio of two short timings measures the runner as surely as an absolute bound does: the same parse is a steady 4.0x here and read 8.6x once on the windows leg, failing a pull request that touched neither grok nor parsing. (#3590)
An ordinary Russian word is no longer read as a key word: включены, исключение, переключены and выключен all contain ключ, and the pattern that allows words between the key and the colon reached across the sentence — a markdown link came back as https:[redacted:credential]. (#3589)
A password assigned with = is masked at the length people actually choose, including the DB_PASS= spelling: a JDBC URL, a query string, --from-literal=password= and a dotenv line all kept theirs in the clear under sixteen characters. A colon keeps its older reading. (#3588)
A secret whose value is not ASCII is redacted. Every key-value pattern ended in [A-Za-z0-9/+=._-]{16,}, so пароль: БазаПароль2026 and password: 非常に長いパスワード2026 were stored in the clear whatever the key word — the key words were widened to those languages and the value class was not. Index format version 43 masks what is already stored. (#3587)
A search over a store deja cannot open says so instead of advising the index that just ran. The sentence for a permission wall sat inside the branch for a machine with no history at all, and a store whose files are visible counts as history — so it was unreachable in the case it was written for. (#3585)
deja doctor says what an ignore rule actually hides, and names one that matches nothing. A rule is matched against the project name and the transcript's path, so the natural thing to write — the directory's absolute path — was inert while the row reported it as in force. (#3584)
deja install writes the CLI skill only when it wired something. A target nobody has heard of refused, exited 1, and created ~/.agents/skills/deja-search/SKILL.md on the way. (#3583)
A mistyped command exits non-zero. deja unforget x printed the command that does exist and exited 0, so a script that ran it and checked the code was told the work had been done; a search that simply found nothing still exits 0. (#3567)
deja doctor calls an antigravity store missing when its root is not on disk. The env override was handed back as given, so a variable pointing at a directory that is gone read as a store that is there. (#3568)
blame tells an agent that a file has no history instead of answering []. It is the tool called before an edit, and an empty array reads as a tool that failed; the note names the file and how many sessions were searched. (#3570)
A password handed to a program as a long flag is redacted at the length people actually choose: mysql --password=MyRootPass2026 and app --password=Pass2026Short reached deja show and deja share in the clear, because the key-value floor of sixteen characters applied to a flag that says what its value is. Index format version 42 masks the ones already on disk. (#3572)
A search that lands while the first index build is running says one thing rather than two that contradict each other: there is no index to answer from yet, so it no longer claims to be serving one. (#3574)
deja install refuses a TOML config that is already broken instead of splicing its block in: the JSON targets have always refused one, and an entry in a file the harness cannot load turns a missing bracket into deja's error message. (#3576)
The last slice of deja show says it is the end of the session instead of offering a next slice that is past it. (#3578)
fix no longer tells an agent that its error is not an error. The pair-mining heuristic was used on the caller and refused ten of twenty error lines a tool actually prints — connection refused, permission denied, Exit code 137 among them — while the CLI took all twenty; the advice about pasting the output now rides along with the honest answer instead of replacing it. (#3580)
A sync waits for the peer list's lock instead of writing over it after two seconds. On a box slow enough for sixteen writers not to drain in that time, two machines were dropped from the list — the lost update the lock exists to prevent. A lock left by a dead process is still taken over after thirty seconds. (#3558)
A full rebuild keeps the sessions whose transcripts the client deleted. Claude Code cleans up after 30 days and the incremental pass held them, but a rebuild — what an index-version bump, a changed exclude list and a damaged index all run — wrote the store from the sources alone: 6 of 8 sessions gone on a measured store. deja forget still drops one for good. (#3529)
The first question after an upgrade is answered from the index that is there while the sources are re-read behind it, the way a stale index is already handled. A content-version bump made it wait for the whole pass — 13m54s on a 520 MB store, where the agent that asked gave up after a minute. A layout this build cannot read, and text written before the redaction that masks it, still rebuild first. (#3552)
The opencode reader tests the compaction flag by type instead of reading the value behind it: current opencode keeps an object of file diffs under the same key, which came to 247 MB of query output against 132 MB for the same 78,690 rows on a 3.4 GB store. (#3556)
deja index no longer stalls on a store holding long tool output. Rows from a SQLite-backed harness are built by json_object rather than by the sqlite3 shell's -json mode, which is quadratic in the characters it escapes: 4 MB of quote-heavy text took it 412s against 0.04s, and a 520 MB opencode store took over ten minutes where it now takes three seconds. (#3553)
A renamed transcript is not indexed again. Twenty renames of one 4 KB log left twenty-one copies in records.bin and twenty rows pointing at paths that were gone; search answered once, so nothing on any screen said the store was twenty times its content. (#3546)
deja log says when a compaction capture stored nothing and why. The journal had recorded the reason since the capture was written; the screen printed the same line for a packet that was kept and one that never happened. (#3531)
A bad numeric MCP argument names the number rather than the argument object: {"limit":"five"} answered "arguments must be an object", which is the half of the call that was right. (#3533)
doctor's clock row reads as a sentence — it borrowed the pronoun deja last uses, which carries its own verb. (#3527)
A password handed to a program as an argument is redacted: sshpass -p, curl -u user:pass, docker login -p, redis-cli -a, a .netrc line, a Cookie: header, and a Russian key word with words between it and its colon. Eight of twenty-four planted shapes were stored in the clear and reached deja show, deja recall and deja sync export. (#3535)
DeepSeek Harness: the v3 session logs are read by discovery and by the incremental index, where matching only one of the two left a new log unsearchable until the next rebuild. (#3508)
DeepSeek Harness: the auto-recall plugin asks about the session's workspace rather than the directory dsh was launched from, and the generated plugins load under a home directory whose package.json declares CommonJS. (#3509)
VS Code Copilot Chat: an edited file's URI is percent-decoded, and the older snapshot shape no longer fails to parse — a path with a space landed in the files record encoded, and a v1 working set dropped the record entirely. (#3498)
deja doctor names a hook binary that is gone under every row state, not only a healthy one, and a bare deja says it once a day: after a package upgrade every entry points at the old path and the hooks exit 127. (#3510, #3511)
Zed: the extension cannot reach an installed deja from inside the wasm sandbox, so the install instructions say to name it with the binary setting instead of promising it is found. (#3513)

Changelog

Features

feat(bench): deja bench read measures the stage no benchmark covered (#3561)
feat(bench): what an index update costs, per class of change (#3507)
feat(index): a harness's compaction summary has a role of its own (#3522)
feat: a slow store says which one it is and that it is still reading (#3559)
feat: an upgrade says why it is re-reading every source (#3539)
feat: doctor --json carries the auto-recall rows (#3541)

Fixes

fix(antigravity): a file:// path is decoded, not sliced (#3523)
fix(copilot-chat): decode the edited-file URIs and read the older snapshot shape (#3498) (@Sora-bluesky)
fix(deepseek): read the session.v3.jsonl logs dsh writes now (#3508) (@huszarrichard)
fix(doctor): a row that is not wired still says the binary it names is gone (#3510)
fix(dsh): recall the session's workspace, and load under a CommonJS home (#3509) (@huszarrichard)
fix(hook): a command does not own a note about something else (#3519)
fix(hook): a repeated fact yields to the line behind it (#3604)
fix(opencode): a part the harness ignores is not the person's words (#3518)
fix(peers): wait for the list's lock instead of writing over it (#3560)
fix(zed): the extension cannot see an installed deja, and now says so (#3513)
fix: a bad numeric MCP argument names the number, not the object (#3534)
fix: a bare deja says when the hooks run a binary that is gone (#3511)
fix: a date with no time keeps its date (#3538)
fix: a full rebuild keeps sessions whose transcript the client deleted (#3530)
fix: a readable index answers while an upgrade re-reads the sources (#3562)
fix: a renamed transcript is not indexed again (#3547)
fix: a sqlite read has a deadline, the Zed extension asks the host, and a store can be excluded (#3606)
fix: deja log says when a compaction capture stored nothing (#3532)
fix: doctor's clock row no longer reads "leads with that one is" (#3528)
fix: forget refuses a selector that named nothing (#3619)
fix: redact a password passed as a command argument (#3536)
fix: sources honours the store exclusion, and brief says why a build could not run (#3612)
fix: sqlite readers build rows with json_object, not sqlite3 -json (#3554)
fix: the 0.20.1 sweep — eight fixes and two test gates in one pass (#3582)
fix: the 0.20.1 sweep, round four — a leak the redaction floor was supposed to stop, and five more (#3602)
fix: the 0.20.1 sweep, round three — three secret leaks, two stats screens, one flaky gate (#3594)
fix: three from the redaction sweep — api-key spellings, --json, and show below the floor (#3615)
fix: three more from the 0.20.1 sweep — install, doctor, and a wall deja could not name (#3586)

Performance

perf(hook): what a command settled is a lookup, not a search (#3607)
perf(index): a new transcript is appended instead of rewriting the store (#3503)
perf(index): the freshness walk no longer runs to be thrown away (#3501)
perf: a detached warmup takes half the cores and yields to the foreground (#3525)
perf: quote the hits a search serves, not every session that matched (#3545)
perf: read opencode's compaction flag by type, not by the object behind it (#3557)
perf: the read side — a hit carries what matched, and the tier reads only that (#3621)

Documentation

docs: changelog for today's fixes (#3543)
docs: cut 0.20.1 (#3618)
docs: the store exclusion, in the README and on the privacy page (#3622)

Other

chore(index): version 39 so the dsh v3 logs reach stores already indexed (#3514)
chore: pin the manifests to 0.20.0 (#3497)
ci: a red main says so in the tracker (#3517)
ci: ingest cost gate; doctor --json gains the claude and codex hook rows; a file at the index path is no longer deleted (#3609)
ci: only a new push cancels the run before it (#3506)
ci: the nightly asks what an upgrade costs a machine that already ran deja (#3565)
ci: the suite runs on main merged with each open pull request (#3520)
ci: the windows label follows what the branch touches (#3524)
tools: a migration matrix over the released binaries (#3551)
tools: commandgate says which gate keeps the action hook quiet (#3566)

Install

curl -fsSL https://raw.githubusercontent.com/vshulcz/deja-vu/main/install.sh | sh
go install github.com/vshulcz/deja-vu/cmd/deja@v0.20.1
brew install deja-vu