MemPalace/mempalace v3.7.0

v3.7.0 — agent logstream, integrity spine, and pluggable embeddings

on GitHub

v3.7.0 — agent logstream, integrity spine, and pluggable embeddings

The headline of this release is agent logstream coordination: an append-only, local-first event layer so agents on different machines can delegate work, wait for replies, exchange patches, and acknowledge handoffs through the MemPalace hub — without a human relaying messages. Everything else in 3.7.0 hardens long-lived palaces under multi-session load (single-writer ownership, safer repair/re-mine, MCP lifecycle) and expands embed/search options (OpenAI-compatible embeddings, search date windows) without changing the local-first default.

Features

• Agent logstream coordination (RFC 003). Append-only event layer for multi-agent work: durable task packets, wait/ack handoffs, patch and file artifacts, and live tailing over the MCP HTTP hub (GET /logstream/stream SSE). MCP tools include mempalace_event_append / list / wait / ack, artifact put/get, and patch submit; CLI mempalace logstream mirrors the core verbs. Events and artifacts stay local, verbatim, and separate from the drawer palace (logstream.sqlite3). Shared-brain / multi-machine agent fleets no longer need a human to relay status between hosts. (#2162, phases 1–5)

• Logstream multi-master sync foundation (RFC 004 step 0). Estate-level logstream replication hooks so coordinated agents can share the coordination layer across palace replicas — the storage step for a replicated shared brain. (#2162, logsync)

• OpenAI-compatible embeddings. Opt-in embedding_model: "openai-compat" talks to any /v1/embeddings endpoint (LM Studio, llama.cpp, vLLM, Ollama's OpenAI shim, or a self-hosted server) over stdlib urllib — no new dependency. Config and env vars set URL, model, and key; the model id is part of the embedder name so a model change forces a reindex. Default MiniLM/ONNX path is unchanged. (#1671, #1559)

• Search date window. mempalace_search and mempalace search accept since / before ([since, before) on filed_at), shared with list_drawers via mempalace.date_window. Undated drawers are excluded while a bound is active; the vector candidate pool widens under a filter and reports truncation when the pool is full. (#2000, #463)

• Hermes memory provider (core). In-package Hermes MemoryProvider files live turns through file_conversation_exchange() so metadata matches convo mining, with a background worker so the agent loop never blocks. Install/backfill/docs remain a stacked follow-up. (#1915, #2215)

• RFC 002 source adapters on mine. mempalace mine <source> --source <adapter> resolves registered adapters, holds the palace writer lease for the full ingest, and keeps dry-runs inert. Legacy --mode paths are unchanged. (#2068, #2062)

• Hook write-routing through the daemon. Background hook saves and mines can honor the shared write-routing policy so multi-session setups serialize mutations through one local owner. (#2030, #1963)

Performance

• EmbeddingGemma groups documents by size before sub-batching, cutting padded-token work on long sweeps without changing vector meaning. Applies only to embedding_model: embeddinggemma. (#2104)

• HNSW capacity probes are cached and invalidated by palace file signature, so repeated status/taxonomy paths no longer re-scan native segments every call. (#2051, #1471)

• chunk_text line numbering is O(N), fixing multi-second hangs on large sources. (#2054, #2055)

Bug Fixes

• Mining works again on the default Chroma backend. Explicit-embedding writes no longer hand Chroma np.float32 scalars that normalize_embeddings rejects. (#2187)

• ChatGPT data exports are parsed as conversations, not stored as raw JSON arrays. (#2160)

• Local backends enforce process-lifetime single-writer ownership. File-backed palaces require one writer owner for the process lifetime; read-only MCP may coexist; remote backends remain multi-process. (#2079, #2045)

• MCP refuses writes when the served library drifts (mempalace, and chromadb when that backend is active) after an upgrade without restart. Opt out with MEMPALACE_MCP_ALLOW_STALE_LIBRARY. (#2081, #899)

• Chroma HNSW write defaults match chromadb (batch_size=100 / sync_threshold=1000), retiring the old 2/2 bloat guard. (#2107, #2106)

• Repair and recovery are safer under contention. Mine-lock before archive, preserve temp collections on failed swap, fail loud on truncated pagination; dry-run is a true preview in both legacy and from-sqlite modes; backups skip sockets/pipes/device nodes so a live palace socket no longer aborts repair/migrate. (#2109, #2086, #2087, #2133, #2144, #2207, #2212)

• rebuild_index holds the palace writer lease for the full snapshot→rebuild/swap cycle. (#2195)

• Orphaned per-source mine locks are reaped (age + nonblocking flock), throttled to once per 15 minutes; palace-level locks are untouched. (#2200)

• HNSW divergence is preflighted before remaining count() crash sites across mine, dedup, migrate, repair, and palace helpers. (#2093)

• Re-mine and conversation ingest no longer lose or duplicate drawers. Content-hash dedup, sweeper purge scope, round-trippable drawer_id on search; Claude Code subagents/ skipped by default (--include-subagents to opt in). (#2050, #2125, #2090, #1330, #1217)

• MCP and daemon lifecycle harden multi-agent use. Read-only refuses config/checkpoint-ack host mutations; stdio exits on EOF; daemon defers lock-refused jobs; Windows stdout capture falls back or fails closed if the protocol stream cannot be restored. (#2126, #2072, #2029, #2211, #2210)

• Encoding and Windows locale hardening. Pin UTF-8 on config/dialect opens; mojibake repair no longer destroys clean Portuguese/Vietnamese/Turkish prose; repair-encoding CLI reconfigures stdio; non-ASCII CLI symbols replaced for GBK consoles. (#2098, #2208, #2194, #1104, #1034, #2193)

• Small correctness fixes. Entity-candidate ReDoS guard; reject pathological chunk_overlap; worktree cwd maps to the project wing; markdown emphasis is not emotion; service entrypoints restore MEMPALACE_PALACE_PATH; systemd Restart=always with idle watchdog docs; valid docker-compose.yml environment key. (#2127, #2056, #2206, #2199, #2192, #2203, #2204, #2188)

Documentation

• Agent logstream concept page, coordination protocol, shared-brain fleet guide, and RFC 003/004. (#2162)
• Operator write-routing / single-writer recovery notes. (#2079)
• Remote-server idle watchdog and read-only semantics. (#2126, #2204)
• README Docker section leads with the published image and real mount/permission pitfalls. (#2196)
• MX3 public-shim example and CONTRIBUTING Discussions cleanup. (#597, #555)

Internal

• Docker publish is gated on a real smoke script (Compose parse, mine, MCP handshake). (#2189)
• Embedding empty-batch / plain-sequence regression tests; HNSW defaults assertions. (#2191, #2159)

A big thank-you to everyone who contributed to this release.

Install: pip install -U mempalace==3.7.0

Docs: Agent logstream · RFC 003