[2.9.0] - 2026-11-17
QMD adds document processors, metadata filtering, and remote model support.
This release also updates llama.cpp and hardens concurrent indexing and MCP
shutdown.
Fixed
-
Serialize trust-record mutations and atomically publish the trust file, so
concurrent approvals and revocations do not overwrite each other. -
Share a crash-recoverable SQLite sidecar lock across CLI, SDK and direct
embedding runs, preventing concurrent partial-vector cleanup from deleting
another run's completed vectors. -
Hold the index's embed lock in
clearAllEmbeddings, so a clear cannot drop
the vector table under a running embed or resurrect vectors it just deleted. -
Keep SDK configuration sources separate per store. Coordinate LLM sessions,
idle unloading and disposal so in-flight operations retain their resources. -
Wait for admitted SDK operations before closing their store, and reject
reentrant closes instead of closing SQLite beneath active work. -
Bound MCP shutdown to a five-second grace period by default. On timeout,
cancel requests and report failure without disposing resources still in use. -
Stream MCP event responses instead of buffering them indefinitely, and cancel
requests when their clients disconnect. -
Serialize overlapping scans on one connection and make processor publication
and embedding retry states explicit. -
Reconcile document changes made during CJK FTS shadow rebuilding before
publishing, so concurrent writes remain searchable. -
Serialize YAML configuration mutations and publish them atomically, retaining
concurrent edits and the permissions and targets of existing config files.
Synchronize SQLite against the current locked config snapshot so an older
opener cannot remove another process's newly added collection, and publish
SDK collection and context changes before mirroring them, so a refused config
write leaves no database-only edit or staleconfig_hash. A rename records
the old→new pair before publishing, so an interrupted index move is finished
by repeating the command and is never inferred from a name that merely exists;
repeating a removal finishes it the same way. While that marker is live, a
second rename or any change to either name is refused with
PendingRenameErrorinstead of destroying the only evidence, and an
unreadable or malformed marker fails closed withCorruptPendingRenameError
rather than being read as "no rename". -
Coordinate MCP daemon PID-file claims and stale recovery between current
versions, preventing concurrent starters from displacing a live daemon. -
Avoid leaking Nix libc to host subprocesses, and use the build target instead
of runningunameto select the Darwin launcher mitigation. -
Expose the matching C++ runtime in the Linux Nix development shell so Bun can
load the packaged llama.cpp backend. -
Correct the Python and Go grammar repair hints to use their pinned versions.
-
Keep memory-regression coverage effective when Bun's SQLite does not track
allocations forhard_heap_limit. -
Security: MCP stdio and HTTP startup now enforce project-local config trust
before selecting models, ignoring all unapproved model URIs and options,
including remote embedding credential environment variable names. -
qmd doctorselects vector sample identities before loading document bodies,
avoiding excessive SQLite memory use on large indexes with duplicate paths.
#978 (thanks @naveenspark) -
Vector diagnostics match passages by their saved character position, avoiding
false mismatches when earlier chunks change the sequence numbering.
#978 (thanks @naveenspark)
Added
-
Check bounded TLA+ models for lock, trust, indexing, processor, SDK and
server lifecycles in CI, including intentional counterexample controls. -
Per-collection processors turn source files into stable-key Markdown index
documents through explicit SDK callbacks or trusted argv commands. Bounded
whole-source validation and SQL savepoints retain the last complete result on
processing failures; source fingerprints skip unchanged work and preserve
unchanged output vectors. Empty coverage and processor-mode markers prevent
silent raw-input fallback after configuration removal. Commands are not an OS
sandbox; seedocs/PROCESSORS.mdfor trust, versioning and crash boundaries. -
Optional TypeSafe System One reranking through
models.rerank: typesafe:<model>
and copied per-storererankOptions(revision and credential variable name).
Independent per-document scores use bounded two-worker HTTP, caller/session
cancellation and revision-specific cache identities, without a local reranker.
Public SDK search accepts a signal; duplicate-file chunks cache by result index.
Status/doctor do not probe the remote service; pull skips its download.
Local models and cache keys are unchanged. Adapted from #1014
(thanks @caseymanos). -
Explicit OpenAI-compatible remote embeddings through
models.embedAPI-base
URIs (http[s]://host/v1#model-id), with bounded requests/responses,
caller/session cancellation and opt-in credential environment variable names.
SDK and CLI indexing need no local embedding model or tokenizer. Remote space
identities pin the endpoint, model, dimensions, revision and input format;
vector search enforces remote space boundaries in scope and reads matches
from one post-request SQLite snapshot. Remote queries require exact model/
fingerprint matches; local queries refuse remote rows without changing existing
local-to-local or legacy-fingerprint behavior. Local GGUF defaults are unchanged.
Adapted from #761 (thanks @epruseal). -
Added Oxlint lint fence.
-
Document metadata and metadata filtering. Markdown documents can opt into typed metadata through a namespaced frontmatter block (
qmd.metadatawith strings, numbers, booleans, or flat homogeneous arrays), and every search surface — CLIsearch/vsearch/queryvia--filter <json>, the SDK'sfilteroption onsearch()/searchLex()/searchVector(), the MCPquerytool, and HTTPPOST /queryand/search— accepts one shared recursive filter AST discriminated byoperator:and/or/notlogical groups,eq/ne/gt/gte/lt/ltecomparisons,in/nin/allmembership,contains/prefix/suffixtext matching,typefor the stored type of a key's values, andexistspresence, with an optionalcaseInsensitiveflag on any condition whose value is a string (ASCII folding). Every returned result satisfies the filter (applied before RRF fusion and reranking); like collection filtering, highly selective filters remain best-effort for top-K completeness. Frontmatter stays ordinary searchable content — no chunking, embedding, snippet, or line-number changes — and documents withoutqmd.metadatabehave exactly as before. JSON/SDK/MCP/HTTP results now include each document's indexed metadata, andqmd statusreports how many documents still need metadata extraction (a normalqmd updatebackfills existing indexes). -
Metadata discovery. Filtering is only useful when the caller knows what to filter on, so every surface now reports the metadata keys, types, and value counts already in the index — read from the existing metadata tables, with no re-indexing. The CLI adds
qmd collection metadata [name...]with--filter <json>to count only documents matching a filter (same AST as search) and--match <json>to select which metadata entries are reported (the same AST evaluated per entry, where a condition'sfieldis the entry'skeyorvalue, so one grammar covers a single key, a family of keys, reverse lookup of a value, typed thresholds, and anyand/or/notcomposition), plus--key-limit/--key-offset/--all-keysand--value-limit/--value-offset/--all-valuesto page both windows, and--sort count|valueand--min-count <n>to order and trim values. Every windowed list reports its exact remainder, counts are documents rather than values, numbers report min/median/max, keys whose documents disagree on type are split per type with their own counts, and with--filterevery coverage count is measured against the documents the filter admits.qmd collection listnames each collection's top keys,qmd collection showdetails them with a value preview, andqmd statussummarizes coverage. The SDK addslistMetadata(options)(the same options, spelledkeyLimit,valueLimit, and so on, withMetadataOptionErrorfor a value outside its domain andMetadataBindingBudgetErrorfor a filter and match that together bind too many SQL parameters), the MCP server adds ametadatatool and lists each collection's most covered keys instatus, and the HTTP server addsPOST /metadata.MetadataFilterandMetadataMatchare bothMetadataPredicate<Condition>, one recursive grammar over the conditions a document or a metadata entry admits.
Fixed
-
Filtered vector search binds candidate IDs as one JSON list, so a parser-valid metadata filter cannot exhaust Node's SQL variable limit during document lookup. Applies to both exact scans and the capped global fallback.
-
Collection- and metadata-scoped BM25 search (
qmd search -c, the lex leg of
qmd query, MCP, SDKsearchLex) no longer returns false-empty or
incomplete results when stronger matches outside the scope fill the old
limit * 10candidate window (#922). The scope is now applied to the full
FTS5 match set, materialized once, sosearch -c <collection>is exact for
common terms; unscoped search keeps its early-terminating plan. A search over
several collections, such as the default ones, runs one keyword query instead
of one per collection, which made keyword search about 12x faster over 13
collections (thanks @brettdavies). Builds on the approach in #918 (thanks
@fxstein). #953 (thanks @Mr-Beasley) -
qmd updateno longer deactivates every document in a collection whose root
folder is missing, for example an unmounted drive or an offline network share.
The folder globbed to an empty list, which read as "every file was deleted",
so search went empty and aqmd cleanupbefore the drive came back deleted
the rows for good. The collection is now reported as not found and its index
is left unchanged. Permission and I/O errors still fail with their original
cause (#989). #990 (thanks @ParkerRex) -
Embedding generation and legacy fingerprint adoption now tokenize documents
with the store-selected embedding model instead of the global default. This
keeps chunk boundaries aligned with the model that creates and verifies the
stored vectors without initializing an unrelated provider. -
qmd doctorno longer stalls or fills the temp directory on large indexes
when checking legacy (empty-fingerprint) embeddings. The adoption sample
query joinedcontentand grouped by the document body, so SQLite
materialized the full body once per legacy chunk and per active path before
LIMIT 1discarded it, the same pattern as the doctor vector-sample check
(#978). It now picks the sample row through indexes and loads only that
row's body; the sampled chunk is unchanged (#994). #995 (thanks @mjaverto)
Changed
-
Update
node-llama-cpp3.20.0 → 3.22.3 (llama.cpp v0.9.0), MCP server
2.0.0 → 2.4.0,web-tree-sitter0.26.12 → 0.27.2, and other compatible
runtime and development dependencies. Refresh Bun, pnpm and Nix dependency
metadata; Vitest stays on 3 and TypeScript on 5 for compatibility. -
The MCP server caches its instructions per store for up to 60 seconds instead
of rebuilding them, with a full index-status scan, for every HTTP request.
Concurrent requests share one build and a failed build is not cached; index
changes from other processes show up in the instructions within a minute.
#815 (thanks @fxstein) -
Avoid a redundant runtime process on packaged CLI calls where the launcher is
already running under its selected Node or Bun runtime. #923 (thanks @ilepn) -
qmd updateno longer reads files whose modification time and size match
the last pass (tracked in a newfile_sync_statetable), so re-indexing an
unchanged collection takes seconds. A cached entry counts only while its
document is still active at that path with that content, so a collection
removed and added back is re-indexed, while a renamed one keeps its entries;
files with missing or outdated metadata are still re-extracted.qmd update
andqmd collection addskip files over 10 MB withFILE_TOO_LARGE, and
qmd updatedeactivates a previously indexed file that becomes empty or is
over 10 MB, including one indexed by an earlier release. #962 (thanks
@rikvanriel) -
Search results carry at most the first 262,144 characters of each document
body (qmd search --full, MCP results, keyword and vector hits), which
bounds memory on indexes with very large documents. A match past that point
is still found, but its snippet, and the passage the reranker scores, come
from the start of the document. #962 (thanks @rikvanriel) -
qmd updateandqmd collection addwrite a collection scan in short
transactions instead of committing every row on its own, so a first index
of 20,000 files takes about 5 s instead of about 3 minutes. #1021 (thanks
@brettdavies) -
qmd queryandqmd vsearchnow drop repeated query expansions before
searching. The expansion model can repeat a line, and the cache kept every
copy: one reported query ran 23 expansions where 9 were distinct, and each
copy ran its own search. Cached expansions are deduplicated when read, so
existing caches need no rebuild. Repeated lex lines used to count more than
once in the rank fusion, so result order can shift slightly. (#921)
#1000 (thanks @ParkerRex) -
Structured searches over several collections (
qmd querywith
lex:/vec:/hyde:lines, the MCPquerytool, SDKqueries) run one
search per line over the whole collection list, a keyword search for a
lex:line and a vector search for avec:orhyde:line, and fuse one
ranked list per search. Rankings no longer depend on the order the
collections are named. #946 (thanks @shalom-t), #1009 (thanks @xidus90)
Changed
- The vector index is partitioned by collection. Each embedded chunk is
stored once per collection that holds it, under an integer collection id,
so a collection-scopedqmd queryorqmd vsearchscans only that
collection's vectors instead of pre-filtering the whole index, and a small
collection is never crowded out of the results by a large one. The first
command after upgrading converts the index in place and prints its
progress; the conversion resumes from where it stopped if interrupted, two
commands started at once share it, and a VACUUM at the end reclaims the
space of the old table.qmd collection renameno longer touches vectors,
qmd updateremoves, as it finishes, the vectors of content that no
indexed document holds any more (content that comes back later is embedded
again), andqmd updateandqmd embedcopy an already-embedded document
into a collection that gains it instead of embedding it again. A
metadata-filtered vector search returns the nearest documents the filter
admits however many chunks it admits, and a vector search fills its limit
even when one long document holds all of the nearest chunks. Downgrading
to an older qmd afterwards needsqmd embed -f, and aqmd mcpserver
started before the upgrade must be restarted. #983 (thanks @brettdavies)
Changed
qmd cleanuprepacks the vector table when fewer than 90% of its chunk
reads hold live rows. vec0 fills the first free slot of its newest chunk
and reclaims a chunk only once it is empty, so re-embeds and orphan removal
leave holes in older chunks that every brute-force scan still reads; a
794k-vector index at 36% occupancy scanned twice as slowly as a packed copy
of the same rows. The repack moves the live rows of each mostly-empty chunk
to the tail, one chunk per short transaction, so it never holds the write
lock for long and an interrupted run leaves a consistent table. The cleanup
output andqmd cleanup --dry-runreport the chunk counts.
#937 (thanks @brettdavies)