github kenn-io/msgvault v0.22.0

5 hours ago

This release builds msgvault with Go 1.27.3. Its crypto/x509 rejects the compromised intermediate certificates listed in GO-2026-6809 for every TLS connection msgvault makes, even when the operating system's trust store has not been updated yet. Upgrade from 0.21.0 and earlier.

Known issue: documents commands that use Mistral still fail because Mistral retired the pinned mistral-ocr-4-0 model (#1436).

Before upgrading

  • Cron schedules now use go-cron. Schedules with a step as large as its range, such as */60 * * * *, and an empty zone name, such as CRON_TZ= 0 3 * * *, are rejected. Replace them first, for example with 0 * * * * and CRON_TZ=UTC 0 3 * * *.
  • A standalone | between full-text searches now means OR. Quote it as "a | b" to search for the literal text. Saved Views created before this release keep the literal meaning.
  • Stored people provider keys move into tokens/provider-credentials.json the first time msgvault uses them. An older release cannot read that file, so before downgrading run msgvault person provider remove <name> for each profile with a stored key.
  • A stored person-enrichment suppression key now overrides suppression_key_env. Make the two agree before upgrading.
  • A server bound to a non-loopback address without an API key now generates tokens/server-api-key and reuses it, so the Web UI asks for a login on 127.0.0.1 too.
  • msgvault now sends anonymous usage telemetry. Turn it off with [telemetry] enabled = false, MSGVAULT_TELEMETRY_ENABLED=0 or DO_NOT_TRACK=1.

New features

  • Archive joined Matrix and Element rooms from any homeserver with msgvault add-matrix --homeserver <url> --user-id <id> and sync-matrix, on demand or on the [matrix] schedule. Edits show their newest text, and redacted edits and reactions are removed. Only plaintext rooms are supported.
  • Archive Plaud cloud recordings, full transcripts with speaker labels, and every note tab as searchable meetings with add-plaud and sync-plaud. --limit, --after, --full and --probe bound, filter, repair or dry-run a sync. Audio stays at Plaud.
  • Archive Twenty call recordings, including those from its Granola, Fathom, Fireflies and Teams integrations, as searchable meetings with add-twenty and sync-twenty.
  • Archive Omi conversations with summaries, action items and speaker transcripts from hosted or self-hosted Omi. Sources share resumable sync and per-key request pacing.
  • Archive Chatwoot shared inboxes, files, source transcripts and linked call meetings.
  • Archive Twilio calls as searchable meetings, with their recordings saved locally up to 1 GiB each by default (max_media_mb).
  • Keep Muesli M4A and WAV recordings as meeting attachments with recordings = true, including from a remote recorder's Mac. Sync Muesli to a remote daemon with the owner credentials in [remote], and keep it current with sync-muesli --watch or Muesli's completion hook.
  • Give Notion meeting attendees verified email addresses, guests included, by adding a workspace-users integration token as users_token on a [[notion_meetings]] entry, then rerunning sync-notion-meetings.
  • Archive Slack with a token limited to public channels, including channels you haven't joined; search:read is no longer required. Turn private channels on or off with [slack].private_channels or sync-slack --private-channels.
  • Search spoken words in recording transcripts with msgvault media search, MCP search_media and GET /api/v1/media/search. Everything search shows transcript matches for plain-word full-text queries, and the Web reader shows each recording's transcript under the expanded message.
  • Route Loom and Cap links and native Teams recording pointers to Docbank for transcripts, with separate reference consent. Self-hosted Cap requires an exact configured origin.
  • Mirror stored attachments to a Docbank folder, including files retained after source deletion. Status reports progress and failures, and message and attachment details show delivery receipts. Requires HTTP API schema 3.23.0.
  • Extract document text on your own hardware with a self-hosted Docling Serve instance: set provider = "docling" and endpoint under [attachments.documents].
  • Match any of up to eight pipe-separated alternatives in full-text message search, for example invoice | receipt. Remote clients need a daemon with API schema 3.19.0 or newer.
  • Find mail by the address that received it with received:work@example.org, and add sent mail and calendar events with account:work@example.org. The TUI account picker, the Web UI's Explore account filter, MCP and vector and hybrid search accept the same choices.
  • Create, update, delete and move calendar events, respond to invitations and check availability from the CLI, HTTP API and MCP. Write consent and source permissions are opt-in, and guest notifications default to none.
  • Read and edit Gmail labels, IMAP keywords and Microsoft Graph categories on an archived message with msgvault message-tags, HTTP or MCP. Microsoft Graph sync now archives categories, so the first sync of an existing Microsoft account walks every folder once.
  • Quote an exact passage from a message, transcript or file in a Kata issue from the Web UI, msgvault kata, HTTP or MCP (--allow-kata-writes). msgvault kata issues finds issues that already cite a message or file, and msgvault kata context reads back each cited passage.
  • Prepare and manage drafts through MCP with the caller's own daemon permissions, including restricted agent grants. msgvault never sends.
  • List the email addresses, phone numbers and chat IDs your archive holds for a person with msgvault person identities or GET /api/v1/people/{id}/identities, to pick an address for draft-compose --to.
  • Find people by name and read the messaging routes your archive has verified for a person through MCP find_contact_candidates and get_person_messaging_routes, or the matching /api/v1/people/ routes.
  • Let MCP clients check source status and scheduler work, request syncs, change Slack conversation selection and edit display names. Source writes need --allow-source-writes, display-name edits need --allow-profile-writes, and HTTP also needs --http-allow-writes.
  • Sync Microsoft 365 and Outlook.com contacts over CardDAV with msgvault add-carddav --microsoft <email>.
  • Look up a person by a subscribed CardDAV card's UID with msgvault person get --vcard-uid <uid>.
  • Find a person's attachment when you misremember its name by one letter with msgvault person files <person-id> --filename <name> --fuzzy-filename.
  • Link many people at once from Relationships: hover to show checkboxes, Shift-click for ranges or select all, then choose Same person…. Directory and Saved views add bulk deletion.
  • Click a day on a relationship's activity heatmap to show that day's messages in the timeline.
  • Merge a replaced or duplicate account into one live source with msgvault merge-account --from <account> --into <account> --dry-run, then again with --yes. update-account adds --alias, --history-only and --accept-reanchor.
  • Grant read-only remote clients their own reader credentials with response limits and optional collection writes.
  • Issue agent tokens for scoped archive reads through the remote CLI and MCP with msgvault agent-token issue, optionally with --expires. Requires [server] agent_access = true.
  • Keep Google OAuth client secrets and tokens in a password manager or macOS Keychain with [oauth] client_secrets_command and the commands under [oauth.tokens]. Existing plaintext credentials are not migrated.
  • Configure containers and supervised installs without startup scripts: MSGVAULT_HOME, MSGVAULT_BIND_ADDR, MSGVAULT_API_PORT and MSGVAULT_REMOTE_URL override config, iface:NAME binds to an interface's address, key files mount through api_key_file, MSGVAULT_REMOTE_API_KEY_FILE and mcp --http-token-file, and msgvault credentials set <id> --from-file stores provider keys.
  • Use msgvault as a Go library through go.kenn.io/msgvault/pkg/archive: set up and open SQLite or PostgreSQL archives, run the Slack and Discord importers, purge channels and sources, scope reads per request, and mount the HTTP API in your own server. PostgreSQL-only programs build without CGO.
  • Check embedding coverage, running and queued jobs, throughput with an ETA, and per-batch timings with msgvault embeddings status, GET /api/v1/embeddings/status or MCP get_embeddings_status.
  • Compare a new binary's expected archive version with the stored one using schema-version, then run migrate before replacing the executable.
  • Anonymous usage telemetry: a daemon ping at startup and once each later UTC day, Web UI opens, screen views at most once per day, session length in four buckets, and counts of searches, empty searches, message opens, manual syncs, displayed errors and failing sources. Events carry the version, commit, OS, architecture and a random install ID, never query text, IDs or archive content. CI runs and non-release builds never send, and the CLI and TUI wait at most 50 ms for pending reports at exit.

Improvements

  • Rerunning import-whatsapp on an Apple ChatStorage.sqlite reads only chats that changed and writes only new and changed messages, and picks up senders that LID.sqlite resolves later. --full compares every message, and --after and --before let a first import fetch recent messages first.
  • Repeat import-emlx runs skip unchanged Apple Mail files and write less to disk while still noticing late attachments. --full-reconcile revisits every file, and --max-message-bytes raises the 128 MiB merge limit.
  • SQLite metadata searches use substring indexes. Initialization backfills them once, so large archives need extra startup time and disk space.
  • Changing owned addresses rechecks only matching messages. SQLite upgrades may take several minutes to build a one-time From-address index.
  • Search help explains the fts, vector and hybrid modes, and an empty keyword search suggests --mode hybrid when embeddings can answer it. Search adds --snippet, --sort date for semantic pages and a filename: operator.
  • show aliases show-message, which gains --body-only and --strip-quoted, and show-thread reads a conversation oldest first with quoted history removed.
  • Root help groups commands by task and starts with the commands to try first. Unknown subcommands under help-only groups exit 1 with suggestions.
  • list-accounts and embeddings list keep working while a sync, import or embedding build runs, showing pending for counts that aren't ready.
  • A bad value in a setting only the daemon reads, such as [integrations.kata] endpoint or MSGVAULT_API_PORT, no longer stops other commands. They print a warning and still reach the daemon.
  • Cron schedules accept 7 as Sunday and ranges that wrap around, such as 22-2. A run in an hour skipped for daylight saving starts after the jump, and a schedule in a repeated hour runs once.
  • MCP message searches, search_in_message and list_messages page past offset 1000. search_by_domains reports paging and caps pages at 500 rows.
  • MCP metadata searches run fewer queries, and a disconnected HTTP MCP client cancels its search on the daemon. GET /api/v1/search/fast omits aggregate stats unless you set view_type.
  • Message listings filtered by domain, and relationship cache builds for large group chats, do less database work and use less temporary disk.
  • Reply drafts from IMAP and Gmail send from the confirmed alias that received the original message.
  • Remote edits update source-owned values on published CardDAV imports while keeping unrelated local values. Edits that would overwrite another owner create a conflict.
  • Granola and Circleback save meetings through the shared meeting path, with trimmed names and deduplicated attendees. Run sync-granola --full or sync-circleback --full to rewrite existing meetings.
  • Deduplication prefers copies with native message IDs, thread evidence and an RFC822 Message-ID when earlier rules tie, and identity discover --provider includes the signed-in Gmail profile.
  • The Google and Microsoft contacts sign-in commands in Settings come in both POSIX shell and Command Prompt forms.
  • Vector search, daemon status and Explore coverage say to restart when settings changed after the daemon started.
  • On Windows, private folders such as token, export and spool folders are created owner-only from the start.
  • Release binaries include gRPC-Go 1.84.0, and container images install Debian's patched PCRE2 package.

Bug fixes

  • On SQLite archives, the CardDAV account upgrade added in 0.21.0 no longer stops msgvault sync at startup with "CardDAV account upgrade left dangling references" when unrelated rows have broken references.
  • On Linux, saving or editing config.toml works on filesystems that don't record file birth time, such as ext4 with 128-byte inodes. Before, every save, settings change and CardDAV account test failed with a config conflict.
  • add-o365 no longer saves a token for a different Microsoft account when sign-in returns no email. When an IMAP sign-in name differs from the mailbox, pass it with --sign-in.
  • When an IMAP server marks one mailbox as both Sent and Drafts, msgvault treats it as Drafts, so unfinished drafts are no longer counted as sent mail.
  • Messages that declare charset=utf-8 keep their accented characters in the body and search index. Messages archived earlier keep their old text.
  • Slack syncs interrupted by the scheduler finish their pass and reach stale channels, so the last successful sync advances.
  • On PostgreSQL, channel purge, account removal and exclusive maintenance no longer deadlock with another source's sync.
  • Sync now and POST /api/v1/sync/{id} on a running generic source queue a follow-up run and report started, pending or coalesced. Before, they claimed a sync started when none did, or failed with 503.
  • Contact sync keeps working on CardDAV servers that require Depth: 0 for sync reports.
  • Choosing Use msgvault version for a CardDAV conflict no longer duplicates an email you added in msgvault.
  • A CardDAV update or unpublish attempted while the Google sign-in is missing goes out after sign-in instead of becoming a conflict.
  • Renaming a published person updates the name on the CardDAV card. Run msgvault sync-carddav to fix existing publications.
  • Adding Calendar to a Gmail token recognizes Google's short and expanded email and profile scope names.
  • Embedding builds finish when some messages contain only invisible text, and embedding passes no longer count successful messages as failed.
  • msgvault backup create no longer fails with no active backup freeze with that token when a cache reader delays SQLite checkpointing.
  • msgvault backup and backup restore read pack blobs larger than 512 MiB on 64-bit systems.
  • import-emlx restores attachments in Apple Mail messages that mix LF and CRLF line endings, and msgvault deduplicate keeps the restored copy over one with placeholder attachments.
  • import-whatsapp counts and warns about messages it could not add to the search index.
  • Beeper syncs stop re-requesting files a bridge reports as missing, scheduled Beeper imports that yield mid-group stop repeating work, and Beeper voice notes from older archives reach Docbank for transcripts.
  • Saved Views in the Web UI keep their identity filter when the stored source ID or direction uses a looser form.
  • MCP semantic_search_messages results include deleted_at again, and Claude Desktop accepts the MCP server again when query_sql is enabled.
  • MCP CardDAV approval prompts summarize inline photos, logos, sounds and keys instead of showing base64.
  • repair-message --audit stops flagging messages with inline binary attachments after a successful repair.
  • Single-message export-eml refuses ambiguous provider IDs, and export-messages reports missing --start and --end before contacting the daemon.
  • New clients find a keyed daemon that is slow to start instead of failing with msgvault daemon is already running for this data directory.
  • After a client disconnects during a search that hits a stale vector index, the daemon keeps advising msgvault daemon restart instead of a full rebuild.
  • Renaming or linking a person over HTTP refreshes the people cache of the archive the daemon serves when msgvault serve was started with --home or --config.
  • On Windows, a keyed daemon's first start no longer fails with provider credential DACL permits inherited access, and saving provider keys, CardDAV connections or settings no longer fails intermittently.

Acknowledgements

Thanks to everyone who contributed to this release:

  • @eserie for Matrix room archiving, faster repeat WhatsApp imports from Apple devices, and accurate Sync now results.
  • @exactmike for starting Microsoft 365 and Outlook.com contacts sync.
  • @freivokh for fixing config saves on Linux filesystems without file birth time.
  • @fucx for restoring Apple Mail attachments in messages with mixed line endings.
  • @jr-k for OR searches with | and linking many people at once in Relationships.
  • @mariusvniekerk for cron schedules that handle daylight saving, and patched toolchains for builds.
  • @rodboev for searching recording transcripts, Microsoft 365 contacts sync, quoting archive passages in Kata issues, anonymous telemetry, and Windows startup fixes.
  • @salmonumbrella for Plaud, Twenty, Omi, Chatwoot and Twilio archiving, calendar event control, message tag editing, account merging, and new MCP tools.
  • @wesm for using msgvault as a library in other Go programs, Slack archives with public-channel-only tokens, the clickable relationship heatmap, and backup and Slack sync fixes.
  • @X1pheR for patching PCRE2 in container images.

Don't miss a new msgvault release

NewReleases is sending notifications on new releases.