Agents learn on connecting how to search well, sift index survives readers that crash or hang and keeps a
collection whose folder came back empty, and keyword search no longer falls back to FTS5 for a collection without
documents.
Changed
- The instructions
sift servegives agents when they connect start with how to search well: which tool fits what
the agent has (a question, an exact string such as a version or error code, an idea, a document), and how to research
in rounds: several phrasings in one call, a compact survey, the best few read in oneget, thensimilar,links
and new terms, until a round finds nothing new. The collections come last: a client may keep only the start of a
server's instructions (Claude Code is reported to cut them at 2,048 characters), and a long list of collections
then loses its own end, not the tools. Each search tool's description says what its results carry and how to read
on, for agents that find a tool by tool search and read its description alone. - The MCP tools
statusandlinksreturn their result as data too (structured content), as the other tools do,
beside the same text as before: a client that hands its model the data hands it compact, less than half the size
forstatus. - Where stderr is no terminal, in a file, a pipe, a service's log or an agent's shell, the progress line is a line
a minute and one where it ended, as in the logs. It was drawn again there too, one more line an update and a
second: seven for five files, thousands for a night of embedding. In a terminal nothing changes;sift worker's
count of what it served shows in a terminal only.
Fixed
- A worker that shows no certificate is refused as one showing another than its key names: "has another
certificate than its key", before anything is sent. It ended in a TypeError. - A worker whose name resolves only to addresses this Python can't connect to, IPv6 ones where Python was built
without IPv6, says so: it ended in a TypeError. - A collection's index that doesn't say which collection it is, as only damage leaves one, says so, and what to do,
when its keyword index is built or brought up to date: it ended in a TypeError. sift indexkeeps what is indexed of a collection whose folder holds none of its files any more, and says so,
with how to let it go (sift rm NAME). A drive not mounted leaves its folder there, empty, as does a folder being
filled anew: every document went, and the vectors embedded for them.- A file whose reader ends its worker process, as a converter's native code can, or comes to no end, is left out and
said, and the other files are read:sift indexended in a traceback, or never, and every later run met the same
file again. A file gets ten minutes, and a second more for every 100 KB. Ctrl+C ends the worker processes at once,
not once they have read every file started. - A sift whose stderr no one reads any more, a pipe to a program that ended, goes on with its work.
sift serve
stopped at its next message, five minutes later that the models had left the GPU, and a batch too large for the
GPU ended in the error of saying so. - A keyword search of every collection works between
sift addand the new collection's firstsift index: it
ended in "The weights and list don't have the same length", as didquery, on the command line and over MCP. - A text file of more than a billion bytes, the most a document's text may be, is left out unread, and said, until
it changes: it was read whole, cut into passages, then refused by the index, and so again on every run. - A server keeps what it has read of the keyword postings' long lists within 256 MB, as it keeps their blocks and
the phrases: it kept those of 1,024 words per collection however long, 12 MB for a word in every passage of 50
million. linksreads a note's links and the notes they name as they are at one moment: asift indexbeside the server
that removed a linking note in between ended the call in "KeyError".sift indexpasses over what is no file under a name it reads, a named pipe or a folder linked asx.md: a pipe
held it for good, a folder was unreadable on every run.- The page looks for a document's file off the server's event loop: on a share that doesn't answer, the server heard
no request for as long as the system waited. reindexsays a file whose reader failsunreadable, with why, and reads the call's other files, assift index
does: the call failed whole.- A server keeps its last searches' results through a search of a phrase it hasn't counted yet: each one let every
result kept go, and wasn't kept itself, so an agent asking phrases got little from memory. - A collection without documents sends the searches of the others to their keyword postings, not to FTS5, seconds
over millions of passages; andreindexbuilds the postings of a small collection that has none (under 2,000
passages), as one an agent fills throughreindexalone, which had none until asift index.