This release brings two-factor authentication into the open-source edition, adds an OAuth-based ingestion source that adds support for Outlook.com and other accounts that require OAuth, adds more robust ingestion and indexing, and closes a list of ingestion and indexing issues.
Highlights
Two-factor authentication is now available in the open-source edition
TOTP two-factor authentication was previously only available in the Enterprise advanced-security module. Its core now ships in the backend and works in every deployment, including open-source installations.
Users enrol from their account settings by scanning a QR code and confirming with a code from their authenticator, and receive eight single-use backup codes. Signing in as an enrolled user issues a short-lived pending session that is exchanged for full access once a valid code is supplied.
The Enterprise Edition retains the organisation-wide policy layer on top: the feature switch, mandatory enforcement for all users, and the grace window for enrolment.
OAuth Mailbox: a single-mailbox source that supports Outlook.com
Microsoft has retired basic authentication for personal accounts, which means Outlook.com, Hotmail and Live.com mailboxes can no longer be added as Generic IMAP sources at all. The new OAuth Mailbox provider archives a single mailbox using an OAuth sign-in instead of a stored password.
- Two sign-in methods. Either redirect to the provider in the browser, or display a short code to enter on another device — the second option works for instances that are not reachable from the public internet.
- Built-in Microsoft presets for personal accounts (Outlook.com, Hotmail, Live.com) and for a single mailbox inside a Microsoft 365 organization. Both read mail over Microsoft Graph, because Microsoft refuses IMAP sessions on personal mailboxes unpredictably while the same account answers every Graph request.
- A Custom preset for any other server that accepts IMAP XOAUTH2, with Gmail documented as a worked example.
- Tokens are stored encrypted and refreshed automatically during syncing. A source whose authorization expires or is revoked reports that plainly and offers a Re-authorize action rather than failing silently.
Use OAuth Mailbox to archive one person's mailbox. To archive an entire organization, the existing Microsoft 365 and Google Workspace providers remain the right choice.
Access control and roles
- A visual role policy editor. Roles can now be built from action and subject pickers with condition rows that prepopulate from real data — ingestion sources by name, mailboxes by address, and the signed-in user. The visual editor is an Enterprise Edition feature, and open-source builds explain this on the tab and keep the JSON editor fully usable.
- Permission checks now apply to the individual record. Routes acting on a single ingestion source, archived email or stored file previously checked only whether a role could act on that type of resource. A role scoped to one ingestion source could reach any other, and a mailbox-scoped role could download any stored file whose path it could guess. These routes now load the record and check against it.
- The users endpoint no longer returns sensitive columns. It previously included the password hash, encrypted two-factor secret and backup codes to any role with read access on users, which the shipped Read-only role grants. It now returns public fields only.
Search indexing
- Orphaned search entries can be cleaned up. When an email or an ingestion source was deleted but the search index was not updated, the index kept entries for emails that no longer existed and search returned results that could not be opened. A clean up feature in the Index page can remove orphaned index documents.
Performance
Indexing and ingestion both processed one item at a time while spending most of that time waiting on storage, the search engine or the mail provider. The indexing worker now processes several jobs at once and builds the next batch of documents while the previous one is being confirmed, and a mailbox archives several emails concurrently. Deduplication guarantees are unchanged: emails sharing a Message-ID are still processed one after another, and emails are marked as indexed only after the search engine confirms the write.
Two settings control this — INDEXING_WORKER_CONCURRENCY (default 4) and INGESTION_EMAIL_CONCURRENCY (default 3). Both are validated on startup and rejected with a warning if set to something unusable.
Bug fixes
- A fresh Docker installation could not start (#347, #448). The example environment file set a Redis username that the bundled Valkey server does not have, so every queue and worker was refused on connection. The variable is now commented out and the Redis password has a default. Existing installations are unaffected.
- The setup page was unreachable on a new installation (#421). When the API could not be reached, the interface sent users to the sign-in page instead. It now reports that the API is unavailable. The status endpoint also used to create the first administrator from environment variables; that account is now created only on the setup page.
- Unsent drafts are no longer archived from live mailboxes (#447, #287). Auto-saved drafts could leave twenty or more copies of a single message. On many IMAP servers, a draft archived first caused the finished message to be discarded as a duplicate. Drafts are now skipped. Set
ARCHIVE_DRAFTS=trueto keep archiving them. - Overlapping sync cycles archived the same email twice. Sources covering one mailbox reported different email counts while all of them reported success. Two cycles could run over the same source at once, and that is now prevented. Existing copies are left in place.
- Sources could stop syncing while still showing "syncing". A worker that stopped at the wrong moment left a source that nothing picked up again. On one instance this went unnoticed for four months. Such sources are now released and retried on the next cycle.
- Microsoft 365 re-downloaded mail that was already archived (#428). Microsoft changes a message identifier when the message moves between folders, so the stored identifiers no longer matched. Requests now ask for identifiers that survive folder moves. Existing sources keep their sync position and no resynchronisation is needed.
- Microsoft 365 tenants with guest accounts failed every sync cycle (#351). Guests invited to Teams or SharePoint have no mailbox, but were listed as mailboxes to archive, and one failure marked the whole cycle failed. Guests are now excluded. An account with no mailbox is recorded as skipped and named in the status message.
- One provider error could cost an entire mailbox (#441, #443). Google Workspace stopped the whole mailbox after a single Gmail error. Microsoft 365 dropped the message instead and recorded nothing. Both now retry, skip what cannot be fetched, and recover from an expired sync marker.
- Emails with very long Message-IDs were never archived (#440). A malformed header running to kilobytes exceeded a database limit, so the email failed on every attempt. These keys are now shortened. The stored
.emlkeeps the header exactly as it arrived. - One malformed PDF could stop all indexing. A scanned document used memory faster than it could be parsed and terminated the indexing worker, taking every job in progress with it. Parsing now stops properly on timeout and a memory guard abandons a parse that grows too far.
- Indexing stopped without reporting anything (#227, #284, #285). The worker could run out of memory or stop taking jobs while the process stayed alive, and reindexing kept reporting success. Documents are now built in smaller batches, a worker that stops taking jobs is restarted, and a short search engine outage no longer removes emails from search.
Upgrade notes
- New environment variables, all optional with working defaults:
ARCHIVE_DRAFTS,INDEXING_WORKER_CONCURRENCY,INGESTION_EMAIL_CONCURRENCY,MEILI_INDEXING_CHUNK,INDEXING_MAX_TEXT_BYTES,INDEXING_WORKER_MAX_OLD_SPACE_MB, andPDF_PARSE_HEAP_BUDGET_RATIO. See the installation guide for the full list. - Existing duplicate emails are not removed. The overlapping-sync fix stops new duplicates from being created but deliberately leaves existing copies in place, since removing archived mail is not something an upgrade should do on its own.
ADMIN_EMAILandADMIN_PASSWORDare no longer used. Very early versions created the administrator account from these variables, and a compatibility shim kept honouring them. That shim has been removed, and the first administrator is created only on the setup page. If your.envstill defines them they are ignored and can be deleted. Existing installations are unaffected, because the account is stored in the database.
Contributors
- @weishest / wayneshn — two-factor authentication, OAuth Mailbox ingestion, role policy editor and permission scoping, index maintenance, ingestion and indexing reliability, and performance work.
- @EddMCambs — Meilisearch upgrade guide (#444), and contributions to the indexing and performance work.
Thanks to everyone who reported the issues that shaped this release.
Full Changelog: v0.5.2...v0.6.0