github LanternOps/breeze v0.122.1
on GitHub

2 hours ago

Breeze RMM v0.122.1: remote desktop clipboard v2, AI suggested fixes that research and remember, administrator-approved diagnostic file access, Partner API ticket read and write, customer portal read APIs for hardware and performance data, restores that check snapshot integrity, and a large cut in database work per agent heartbeat.

v0.122.0 was never released. Its tag and container images exist, but a database migration in it fails on installations that have security policy links. Upgrade from v0.121.x straight to v0.122.1, and don't deploy 0.122.0. If you already started v0.122.0 and the API is restarting with cannot ALTER TABLE "config_policy_feature_links" because it has pending trigger events, move to 0.122.1: the failed migration rolled back, and 0.122.1 applies the corrected file (#8396).

⚠️ Removed in this release (announced in v0.121): /api/v1/ai/provider now returns 404, and requests that still send the retired AI model fields (reviewerModel, allowedModels, agent model) are rejected with 400. The AI model registry (/api/v1/ai/models) replaces them. Details under Self-Hosting / Upgrade Notes (#7867).

⚠️ Restores need a v0.120.0 or later backup helper. Restores are refused for devices whose backup helper cannot check snapshot attestations, and recovery media older than v0.120.0 are refused. Update agents and rebuild recovery media before upgrading the server (#8047, #8302).

⚠️ Partner API tickets:* grants take effect now. Service principals that already hold tickets:read or tickets:write can use the new ticket routes as soon as you upgrade. Review those grants first (#8037, #8235).

⚠️ Some roles lose webhook visibility. Viewing webhooks now needs the new webhooks:read permission. A custom role with organizations:read but not organizations:write, and the built-in Partner Viewer and Partner Security Approver, no longer see webhooks until you add it (#8285).

⚠️ Roll forward only for installers. Installer bootstrap tokens are stored hashed from the first start. Rolling back afterwards breaks every installer issued since the upgrade and the tokens hashed at boot; re-issue installers if you roll back (#7978).

⚠️ Avoid deploying during a database backup window. Three migrations briefly take ACCESS EXCLUSIVE locks on busy tables and give up after 5 s if a long reader holds one; the API then fails to boot and re-applies cleanly on the next restart. Details under Database (#8011).

Security — hardening

This release includes security hardening across:

  • partner service principal key ownership, scope delegation and MCP access;
  • permission checks for webhooks, AI for Office templates, tunnels, DNS filtering policies, CIS baselines and IOC auto-quarantine, fleet findings, audit baselines, site-restricted users and partner-scope audit data;
  • second-factor confirmation for charging a client, changing payment settings, approving audit baselines and restoring unattested backups;
  • backup restore integrity, device purge and org erasure handling of backup storage;
  • command delivery checks over WebSocket and queued org merges;
  • installer bootstrap token storage and lifetime;
  • the agent's own data directory on Windows file tools;
  • diagnostic file access, which never grants credential material;
  • endpoint URLs shown in alerts, tickets, monitor results and logs, and outbound request handling;
  • customer portal responses for draft invoices;
  • dependency updates.

Before upgrading: have each partner service principal's owner rotate any of its keys that another admin issued, and review the scopes granted before the upgrade beyond tickets:*. Scopes granted before this release are not re-checked against the granting admin's own permissions.

Self-hosters are encouraged to upgrade. (#7907, #7908, #7916, #7920, #7954, #7956, #7957, #7958, #7959, #7978, #7979, #8008, #8010, #8012, #8039, #8042, #8045, #8047, #8060, #8088, #8097, #8108, #8118, #8253, #8255, #8267, #8279, #8280, #8285, #8286, #8302, #8327, #8328, #8332, #8358, #8363)

Summary

  • Remote desktop clipboard v2 (#8262, #8266, #8263). Large text copies in both directions, and PNG/JPEG images too, except that images copied on a Windows device do not yet reach the viewer (see Known gaps, #8393). A status chip shows what policy allows in each direction and offers copy and send actions. A paste that can't be delivered is cancelled instead of pasting stale content, and connecting no longer overwrites the technician's clipboard. Each session writes one audit entry summarising its clipboard activity (never the content). Needs this release's agent and viewer.
  • Remote input fixes (#8244, #8251, #7848). Keys and mouse buttons are released when the viewer loses focus or the session drops, so nothing stays stuck down on the remote. German and French keyboards type the right letters on Windows and Linux remotes.
  • AI suggested fixes research and remember (#7939, #7946). The suggestions panel on an alert groups Proven, AI suggestions and Similar fixes. A research agent proposes fixes from the catalog, built-in actions or manual steps, and a new Fix memory page lists what worked. Triage runs check fix memory first.
  • Diagnostic file access (#7406, #8327). The AI assistant can ask to list and read files in named folders on one device, read-only and time-boxed, after an administrator approves. Credential material is never grantable, and file content is redacted for secrets before the assistant sees it.
  • Partner API: tickets, device status and MCP (#8037, #8235, #8303, #7586, #8090). Service principals can read a ticket change feed and create, update, assign and comment on tickets. Ticket webhooks gain ticket.updated and ticket.assigned. An opt-in device-status:read scope adds a live device-status feed, and service principal keys can authenticate MCP at partner scope.
  • Customer portal visibility APIs (#7797, #8119, #8175, #8357, #8101). Per-organization toggles open read-only portal APIs for hardware health, hardware inventory and performance metrics, all off by default; a software inventory read API sits behind its own toggle. The portal has no pages for this data yet, so customers don't see it in the portal UI in this release.
  • Restores check snapshot integrity (#8047, #8302). Restores proceed automatically for verified snapshots. Restoring an unattested snapshot needs a technician's second-factor confirmation, and restore wizards show a Verified / Not verified badge.
  • Topology overview (#7892, #7845, #7882, #7883, #7846, #7843, #7847). Cards zoom out to summary panels, members are grouped by role, VPN and overlay networks are classified by interface type, and stale or decommissioned devices no longer read as online.
  • Who's connected (#7461). The device page shows a banner naming colleagues with a live Desktop, Terminal or VNC session on that device.
  • Reliability baseline markers (#8319, #8323). Mark a device reimaged, remediated or hardware-replaced, and its reliability score restarts from that point with a before/after comparison.
  • Lighter heartbeats (#8128, #8222, #8264, #8055, #8054, #8282). A steady agent heartbeat now runs 14 database statements in 2 transactions (v0.121: 9 transactions), policy resolution is one read, and requests queue fairly for database connections under load.

Added

  • Diagnostic file access for the AI assistant: requests name the device, paths, list/read, a purpose and a duration (5–1440 minutes); administrators decide in the Approvals inbox or the mobile app; grants are read-only and revocable. Diagnostic reads need this release's agent; older agents return agent_update_required (#7406, #8327).
  • AI suggested fixes: Generate and "Research deeper" on an alert, built-in actions (reboot, restart service, kill process, disk cleanup) runnable from a suggestion, "Draft a script" into the script builder, and the Fix memory page with Retire (#7939). Triage verdict and full alert runs consult fix memory first, behind ml.remediation_suggestions.enabled (#7946).
  • Partner API tickets: GET /api/v1/partner-api/tickets (signed change feed with tombstones), /tickets/ids, /tickets/:id, /tickets/:id/comments for tickets:read (#8037); POST /tickets, PATCH /tickets/:id, POST /tickets/:id/status, /assign, /comments for tickets:write, with X-Idempotency-Key support and external ticket references (#8235, #8304).
  • Ticket webhooks: ticket.updated (with changed field names) and ticket.assigned; existing ticket events carry more ids, including the comment's origin so a mirror can skip its own comments (#8303). Customer email replies now fire ticket.commented (#8344).
  • Partner API device-status feed (GET /api/v1/partner-api/device-status) behind the opt-in device-status:read scope (#7586).
  • Partner service principal keys on MCP at partner scope, with opt-in ai:read, ai:write, ai:execute and ai:execute_admin scopes. MCP_UNATTENDED_TIER3_PRINCIPALS accepts partner_sp:<principal id> (#8090).
  • Remote desktop clipboard v2 in the agent and Breeze Viewer, and one session_clipboard_summary audit entry per session (#8262, #8266, #8263).
  • Who's connected banner on the device page (GET /remote/devices/:deviceId/active-sessions) (#7461).
  • Customer portal: hardware health (#7797), per-device hardware inventory (#8119) and performance metrics (#8175) toggles, each off by default and included in Enable all. They open read-only portal APIs; there are no portal pages for this data yet. Also a Site filter on the Network assets table (#8101).
  • Customer portal software inventory (API only): GET /portal/software/summary and GET /portal/software/devices/:deviceId, behind the new enableSoftwareInventory toggle. It is off by default and not turned on by Enable all; switch it on individually per organization. The portal has no software page yet, so the toggle only opens the read API (#8357).
  • Reliability baseline markers: "Mark work done" on the device reliability panel, provisional scoring for 14 days, before/after comparison and marker history; a completed bare-metal recovery places a reimaged marker automatically (#8319, #8323).
  • Physical placement (room, rack, rack unit, height) for devices and discovered assets, API only (/devices/:id/placement, /discovery/assets/:id/placement) (#8348).
  • Site location pins: POST /api/v1/orgs/sites/:id/location pins a site's coordinate and optional 50–1000 m geofence radius from the field; it requires the new sites:set_location permission and deliberately no MFA. PATCH /orgs/sites/:id also accepts latitude/longitude/geofenceRadiusM (still MFA-gated; both coordinates or neither). Only the site's coordinate is stored, never a technician position. Audited as site.location_set (#8223).
  • Time entries carry an org and site without a ticket: POST /api/v1/time-entries/start and POST /api/v1/time-entries accept optional orgId, siteId and source. A ticket's org always wins; a mismatch is 422 ORG_MISMATCH, a site outside the org is 422 SITE_ORG_MISMATCH. GET /time-entries returns siteId and filters by it (#8223).
  • Location suggestions (partner setting, off by default): Ticketing settings → Time Tracking gains a toggle and a default arrival radius (timeTracking.locationSuggestions). GET /api/v1/time-entries/location-sites returns the flag, the default radius, whether the caller may pin sites, and the caller's sites (capped at 2000 with a truncated flag) for the mobile app's arrival prompt. The mobile prompt itself ships later (#8223).
  • EDR integrations (read path): a provider framework with Bitdefender GravityZone as the first provider. Partner admins connect an API key, map GravityZone companies to organizations, and endpoints are linked to devices by hostname; detections sync in the background. API only under /api/v1/edr (#8220, #8259).
  • Hosted partner sign-in page at /login/<partner-slug>, shown on the Login Branding card (#8070).
  • Pushover for ticket assignments to the assignee's own Pushover key, set on the profile page; an optional partner fallback to the default key is off by default (#8080).
  • Default assignee for new inbound tickets, per partner, on the Inbound email card (#8106).
  • Staff forward routing: with the new partner switch on, a customer email forwarded by your staff into a connected Gmail mailbox is filed under the original sender's organization. Off by default (#7961).
  • Gmail mailboxes can label and archive mail that became a ticket. Off until a label is set; needs the gmail.modify scope on the domain-wide delegation grant (#7950).
  • Delete time entries from the ticket's Time & Billing rail and from the timesheet (#8135).
  • Quote Sign & pay: a customer signing a quote with money due now goes straight to Stripe checkout (#8249).
  • Payment reminders for every unpaid invoice (upcoming, repeat and overdue), configured under Billing → Payments. Off by default and independent of automatic payments (#7866).
  • Automatic payments (enrollment, charging, retries, processing fee, client pages and emails). Ships switched off for every partner behind the per-partner rollout switch (#7822, #7877, #7893, #7987, #7983, #7986).
  • Access reviews: reviewers can't decide their own access, except in a single-admin setup, where the decision is flagged "Self-decided" (#7878). "Notify reviewer" sends the email from the server (#7851).
  • Organization column on the Custom Fields page for partner-level users (#8100).
  • Agent display branding (build time): operators building their own agent can set the display name, description and publisher of the agent and watchdog (#8061).
  • Windows recovery console groundwork: the breeze-backup recovery console runs on WinPE. Building Windows recovery media is not part of this release (#7926).

Improved

  • Heartbeat database work: a steady beat now runs 14 statements in 2 transactions, down from 9 transactions in v0.121. UniFi collector polls make no database calls for devices without collectors, and permanent agent auth rejections are cached for 60 s (#8128, #8222, #8264, #8055, #8054).
  • Database connections under load: requests queue fairly for a pool slot and fail cleanly after a bounded wait, timed-out transactions always roll back, and short event-loop stalls no longer cause false timeouts (#8243, #8282).
  • Restores download files in parallel (up to 8 at a time) while installing in manifest order (#7417).
  • Fewer noisy metric anomaly episodes: three consecutive 5-minute buckets, and RAM growth only on a new 7-day high (#7815).
  • Quote PDFs match the web proposal and use partner branding, with an optional contents list (#7713). Quote parts orders include only lines with a SKU or part number, and the editor and send dialog say so (#8248).
  • Topology: neighbor-cache corroboration places phones and printers and splits conflicting gateways (#7845); summary cards, role sections and a compact toolbar (#7892).
  • Microsoft 365 admin consent signs the administrator in first, shows a confirm-tenant screen, and pins consent to the verified tenant (#7933, #7938). Consent failures name the cause, for example a Conditional Access block (#7915).
  • Ticket-assigned email includes a facts table, the first 1,200 characters of the message and an "Open ticket" link (#7952).
  • Customer email sender name: customer-facing mail is sent as your company name instead of "Company via Breeze" (#6574).
  • AI chat: read-only Windows event log queries against System or Setup run without an approval tap in interactive chat (#7917); the assistant sees longer validation errors (#7836); monitor creation explains valid kinds and conditions (#7840, #8103).
  • macOS agent updates download the installer package from your own server first, then fall back to the GitHub release; the package must still match the signed checksum (#8107).
  • Breeze Assist starts hidden in the tray when the agent starts it; launching it yourself still opens the window (#8213).
  • AI connections: adding an OpenAI-compatible connection reports whether discovery was queued (#8099); a refused Base URL shows on the field (#8271); env-managed offerings lock name and price (#8098).
  • FORCE_HTTPS rejections return a reason code and link to the TLS docs (#8093).
  • Sign-in page partner logo renders larger (#7929).
  • Scoped contact responsibilities (backend): caller verification, account readiness, billing contact resolution and report recipients read from a new contact_roles model, back-filled from existing contact roles. No UI change in this release (#8226).
  • 65 more operator settings (retention windows, agent ingest quotas, rate limits, screenshot and log-forwarding caps, DB_POOL_MAX) are passed from .env into the API container by both Compose files (#7858).

Fixed

  • Time Tracking settings show the stored values. Settings → Ticketing → Time Tracking always displayed both suggestion toggles as off, with default thresholds, and saving the card wrote those defaults back over the stored settings (since v0.109). It now loads and keeps the stored values. If you saved that card on an earlier version, re-check "Suggest time entries from remote sessions" and its thresholds (#8387).
  • AI run_script and other tools that wait on a device no longer report failure when the device takes longer than a minute (#7928). Tool results are matched to the right call in chat and in AI agent runs (#7932, #8214).
  • AI chat shows the provider error when a turn fails before any output (#8273); an aborted chat no longer puts the model on cooldown (#8177); OpenAI-compatible gateways such as Ollama no longer refuse every tool call (#8091).
  • Moved devices pick up their new organization and site on the agent through a signed identity update (#8320).
  • Device control (peripheral policy v2) applies to devices that aren't in any device group (#8322).
  • Agent command results finishing during shutdown are kept and delivered on the next connect (#8379).
  • Windows service monitoring no longer reports protected services such as WinDefend as not_found (#8038).
  • Antivirus and firewall: a failed collection reports Unknown instead of off, without the unprotected penalty or alerts (#8043, #8272). N-able is no longer detected from any process named agent (#8085).
  • macOS: the desktop helper no longer re-raises the screen recording prompt after a restart (#8154); reinstalling right after uninstalling works (#7876); a breeze group without a GID is repaired (#7833); long group member lists are read correctly (#7972).
  • Windows: restore point descriptions keep their first characters (#8069); the Storage Spaces collector no longer flaps on hosts without pools (#8150); the agent's own service restart no longer counts against reliability (#8227); the Migrate Agent Edition script no longer stops itself midway, and automatic edition migration canaries one device per org (#7336).
  • Windows installs with object storage install the watchdog again, and the install one-liner enrolls before installing the service (#7585).
  • Add Device install command uses the API's configured public URL on split deployments (#8124).
  • Linux patches on update rings with a deferral are no longer held forever; the deferral starts from when the device first reported the patch (#7930).
  • Bare-metal recovery: a failed attempt no longer consumes the recovery token (#7411); a snapshot from a changed destination is refused up front (#7410).
  • Backup: a backup is stopped if the device moves org or its config changes mid-send; verifications of a failed snapshot fail immediately (#8044). The policy Backup tab offers "New destination" in its empty state and lists the policy owner's destinations (#8314, #8336).
  • Org erasure completes for an organization that is in use, and partner staff whose home org is erased are kept (#8011).
  • Moving a device to another org no longer times out after 30 s in the dashboard (#8025).
  • Scripts queued by partner technicians for offline devices are delivered when the device comes online (#7964).
  • Scheduled network discovery jobs no longer stick in scheduled (#7945); discovery no longer overwrites an agent-detected device role (#8034).
  • DNS filtering: policy changes sync to the provider again (#8337); Cloudflare Gateway event sync and list edits use the real Cloudflare APIs (#7619); a private-address integration host shows a clear sync error (#8073).
  • Network proxy loads device stylesheets and scripts in Firefox and for crossorigin resources (#8110).
  • Microsoft 365 ticket mailboxes keep polling through transient Graph errors (#8300); converting a quarantined email keeps its body and sender (#8316).
  • Ticket assignment notifications are no longer lost when the worker runs first (#8040).
  • Patch compliance Export downloads the CSV (#8315).
  • Security recommendations can be completed or dismissed by partner users across organizations (#8094).
  • Step-up prompts work when enabling the AI unattended script lane and the partner script-authoring ceiling with ENABLE_2FA on (#8095, #8115).
  • Service principal edits no longer shift the expiry by the browser's time zone (#8383).
  • Audit Trail shows partner-level events to partner users and records partner-level and document writes that were dropped before (#7808, #8009); filtering by user no longer returns 500 (#8374).
  • Webhook delivery Retry no longer returns 404 (#8336).
  • AI tools: manage_tags add and C2C item restore work again (#8298); manage_patches device actions resolve the device's own org (#7839); get_device_details keeps core fields when compacted (#8064); monitor updates keep the stored URL (#7955).
  • Service and process monitor alert titles show the name instead of {{serviceName}} (#8063).
  • Org branding logo saves correctly (#8062); redirects are never cached by browsers (#8048); long dashboard pages are no longer cut off (#8018).
  • Agent change reports with a NUL character no longer fail the whole batch (#8026).
  • Helper screenshots are removed from disk when their records are deleted (#8216, #8116).
  • Guided setup boot helper recreates Caddy reliably after a reboot (#7962).
  • pt-BR no longer translates vendor names (#7728).
  • Smaller fixes from the pre-release sweeps (#8114, #8336, #8374, #7834, #7837, #7838, #7849, #7859, #7860, #7871, #7835, #8105, #8178, #8212, #8027, #8036, #8270).

Known gaps in this release

  • Clipboard images from Windows devices don't reach the viewer (#8393). Text copies work in both directions, including large ones, but an image copied on a Windows device isn't sent to the viewer. The desktop session runs as SYSTEM and can't read the user's copied image formats. Images pasted into a Windows device aren't affected by this. Planned for a follow-up release.
  • Identity sync after a device move needs a deployment signing key (#8320). A key exists when the server registers its own agent binaries (BINARY_SOURCE=local) or has issued an agent rollback directive. On the default BINARY_SOURCE=github, the server has no key, so a moved agent keeps its enrolled organization and site, as before this release, and the API logs a one-time warning.
  • Checked before the release: live Windows and Linux agents for identity sync, clipboard text, Breeze Assist start-hidden, Storage Spaces, diagnostic file access and the AI event-log query, and real Stripe test mode for Charge now, the consent cap, payment-settings step-up, quote Sign & pay, and partner suspension. Not lab-run: macOS agent rows (#8154, Breeze Assist Dock reopen), the viewer's clipboard status chip and image paste (#8266), and held-key release on disconnect (#8251).

Self-Hosting / Upgrade Notes

Upgrade command. Check how your install pins its images: grep '^BREEZE_API_IMAGE_REF=' .env.

  • Digest-pinned (value ends in @sha256:…, every guided-setup.sh install from v0.112.0): fetch the current guided-setup.sh from main, then run bash guided-setup.sh --upgrade 0.122.1. It verifies the signed image inventory, updates BREEZE_VERSION and all four image digests, then pulls and restarts.
  • Tag-form (value ends in :${BREEZE_VERSION}): set BREEZE_VERSION=0.122.1, then run docker compose pull api web portal && docker compose up -d.
  • Do not edit only BREEZE_VERSION on a digest-pinned install. It re-pulls the images you already run.
  • --upgrade does not change your docker-compose.yml. To set any of this release's new optional variables, add their lines to the api service's environment: block (or take the v0.122.1 docker-compose.yml); a value only in .env does not reach the API.
  • See the Upgrade Guide.

Breaking changes and required actions.

  1. Legacy AI model settings removed (#7867). The AI model registry (Settings → AI Providers & Models) is now the only AI model configuration.
    • /api/v1/ai/provider is gone (404). Use /api/v1/ai/models: POST /ai/models/connections, POST /ai/models/connections/:id/key, POST /ai/models/connections/:id/endpoint, DELETE /ai/models/connections/:id, and PUT /ai/models/assignments for per-feature defaults.
    • Requests that still send reviewerModel (script policy), allowedModels (AI for Office org policy) or model (AI agent create/update/preview; send offeringId) are rejected with HTTP 400 naming the replacement. These fields were already ignored; GET responses no longer include them.
    • BREEZE_AI_SCRIPT_REVIEWER_MODEL and WORKSPACE_CONTENT_LLM_MODEL are no longer read; remove them from .env. New partners start every AI feature on the platform default model.
    • Upgrading straight from a release older than v0.121.0: each partner is set up from the platform default; a partner with its own Anthropic key keeps using it, but its legacy per-feature model choices are not carried over.
    • No database change for this in this release; the legacy table and columns are archived and dropped in the next release.
  2. Restores now require a v0.120.0 or later backup helper (#8047, #8302). Restores that write, import or boot a backup (file, SQL Server, Hyper-V, restore as VM, instant boot, bare-metal recovery and rebuild) are refused for devices whose backup helper does not check restores against snapshot attestations, with "Update the Breeze agent on this device, then try again." Recovery media and recovery tools older than v0.120.0 are refused at code exchange and authentication. Verification and test restore are unchanged. Update agents and rebuild recovery media before upgrading the server.
    • Backups without an attestation restore after a two-factor confirmation. Snapshots taken before attestations existed are labelled "Not verified"; restoring one requires a technician's two-factor confirmation (an explicit confirmation when ENABLE_2FA=false; a technician without a second factor must set one up first, and SMS-only accounts need an authenticator app or passkey), recorded in the new backup_restore_authorizations table and the audit log (backup.restore.unattested_override). AI tools and DR plans refuse such snapshots. breeze_backup_restore_integrity_total gains status="override" and status="refused".
  3. tickets:* grants on partner service principals take effect in this release (#8037, #8235). v0.121.x already let admins grant tickets:read and tickets:write to a partner service principal, but the Partner API ticket routes were not live, so those grants did nothing. This release adds the routes (/api/v1/partner-api/tickets*), and any principal that already holds those scopes can use them immediately: tickets:read reads every ticket and every comment, including internal notes, in every organization the principal reaches, and tickets:write creates and updates tickets and posts comments, including customer-visible replies that email the requester. Before upgrading, review which partner service principals hold tickets:* (Settings → Partner Service Principals) and remove the scopes from any that should not have them. Scopes granted before this upgrade are not re-checked against the granting admin's role, so include principals whose scopes were granted by an admin who does not hold them. submitterEmail on ticket writes is not verified.
  4. Partner service principal keys are issued by the principal's owner and end with the owner's sign-in credential (#8332). Only the admin who created a partner service principal can issue or rotate its keys, re-enable it, add a scope, or remove or extend its expiry or source-CIDR restriction; other partner-wide admins get 403 (SERVICE_PRINCIPAL_OWNER_REQUIRED) but can still revoke keys, disable the principal, remove scopes and rename it. Every scope is now a delegation the granter must hold, not only the MCP ai:* scopes: for example tickets:read/tickets:write need tickets.read/tickets.write, contracts:write needs contracts.write and alerts:read needs alerts.read (the full list is under "Who can manage a partner service principal" in the API keys reference).
    • Keys are bound to the owner's credential state like a person's own API keys: the owner's password change or reset, an administrator changing the owner's account status (including disabling it), a change to the owner's MFA factors, or removing the owner from the partner stops the principal's keys on both the Partner API and MCP; signing out does not. Existing keys are bound to their owner's current state on upgrade and keep working.
    • Over MCP, a key that another admin issued on someone else's principal is refused; the owner rotates it to restore access (the Partner API keeps accepting it). Have principal owners rotate such keys before upgrading.
    • Plan for owner off-boarding: an integration whose principal belongs to a departing admin stops when that admin is disabled or removed, so create a replacement principal under another admin first.
    • Organization API keys: only a key's creator can rotate it now (403, API_KEY_CREATOR_REQUIRED, for anyone else, who can still revoke it), and rotating re-binds the key to the creator's current sign-in.
    • Rolling deploys: API containers still on v0.121 cannot issue or rotate service principal keys once the migration has run, until they are replaced.
  5. New webhooks:read permission (#8285, #7956). Viewing webhooks and their delivery history (the Webhooks tab under Integrations, GET /webhooks, GET /webhooks/:id, GET /webhooks/:id/deliveries and the AI assistant's query_webhooks tool) now requires webhooks:read instead of organizations:read. Managing webhooks still requires organizations:write. The built-in Org Admin, Org Technician and Partner Technician roles receive it on upgrade (Partner Admin already holds everything), and so does any custom role that holds organizations:write (or a wildcard grant covering it). Viewer, billing and security-approver roles do not, so Partner Viewer and Partner Security Approver, which could see webhooks through organizations:read, no longer can.
    • A custom role that holds organizations:read but not organizations:write loses access to webhooks and their delivery history until an administrator adds webhooks:read to it under Settings → Roles. The upgrade log lists those roles: the migration writes a webhooks-read-permission: warning with their count and up to 20 names.
    • In the dashboard, the Webhooks tab is hidden for users without webhooks:read, and users with webhooks:read but not organizations:write see webhooks and their delivery history without the create, edit, delete, test, pause/resume or retry controls.
    • API contract change: the delivery list no longer returns delivery payload or responseBody.
  6. Roll forward only for installers (#7978). Installer bootstrap tokens are stored as a keyed hash (ENROLLMENT_KEY_PEPPER), and a background sweep at first start hashes existing tokens, logging [startup] Installer bootstrap tokens hashed: X/Y token(s) (Z changed concurrently, F failed). A rollback strands every installer issued after the upgrade, and the pre-upgrade tokens hashed by the sweep stop working on the old version too: re-issue installers after a rollback. Rotating ENROLLMENT_KEY_PEPPER now invalidates every outstanding installer link and every downloaded installer that has not enrolled yet.

Other behaviour changes.

  • Bootstrap tokens embedded in downloaded installers now default to 7 days and last at most 30 days (#8280). The previous default was 30 days in code and 1 day under the bundled compose files (1440). INSTALLER_BOOTSTRAP_TOKEN_TTL_MINUTES now defaults to 10080 in code and in both compose files. A value above 43200 (30 days) is clamped, with a warning at API startup. Set the variable to keep a different value up to 30 days.
    • The installer download (GET /enrollment-keys/:id/installer/:platform) and POST /enrollment-keys/:id/bootstrap-token now reject an explicit ttlMinutes above 43200 with 400. The child enrollment key embedded in the legacy macOS zip is held to the same 30 days.
    • The Add Device modal's Installer tab offers expiries up to 30 days (it previously offered 90 days and 1 year); a longer partner/org default preselects 30 days there. That picker also sizes links generated from the tab. The CLI tab is unchanged.
    • The installer-link API (POST /enrollment-keys/:id/installer-link) and enrollment keys still accept lifetimes up to a year; each installer downloaded from a link carries a token of at most 30 days.
    • Installers already downloaded keep their original expiry.
  • Patch compliance report files are kept for 30 days (#8284). A generated compliance report CSV can be downloaded for 30 days after it completes; after that a daily job removes the file and the report shows as expired (download returns 410 — generate a new report). Set PATCH_REPORT_RETENTION_DAYS to change the window (minimum 1). Report files are also removed when their organization is erased, and stray report files with no report record are cleaned up from PATCH_REPORT_STORAGE_PATH once they are an hour old (PATCH_REPORT_ORPHAN_MIN_AGE_MS). On the first run after upgrading, existing reports older than the window are expired.
  • Diagnostic file access: credential material is never grantable, and self-approval needs a fresh second factor (#7406, #8327). Browser secrets, credential stores, private keys and stored tokens can never be requested or granted: requests naming them are refused, listings of approved folders leave them out, and the agent refuses them on its own (E_DIAG_CREDENTIAL_MATERIAL); the request_diagnostic_access tool takes no sensitive-class option. File content read under a grant is passed through secret redaction before the assistant sees it, over a window with 16 KiB of context on each side so a read cannot start inside a secret (contentRedacted: true when anything was replaced; one read returns at most 992 KiB). A sole operator approving their own request must sign that approval with a hardware-backed factor (passkey or Breeze Authenticator, L3 or higher; a reused step-up does not count), and a diagnostic_access.self_approval audit record is written before the grant activates; if it cannot be written the approval returns 503 audit_unavailable and can be retried. Diagnostic reads need agents from this release. See Diagnostic File Access.
  • Approving an audit baseline apply request now requires MFA and checks its target devices (#8279). On deployments with MFA enabled (ENABLE_2FA), approving a request returns 403 MFA_REQUIRED unless the approver's session has completed MFA. API keys cannot approve apply requests when MFA is enabled. Rejecting a request does not need MFA. Approving is also refused with 409 ("One or more target devices no longer exist in this organization; ask the requester to submit a new apply request") when any device in the request has since been decommissioned, deleted or moved to another organization. Applying an approved request runs the same check and leaves the approval unconsumed. When two decisions on the same request race, or a decision races the request's expiry, the later one returns 409 ("Apply request was already decided") instead of overwriting the first. Pending requests refused this way can be rejected and resubmitted.
  • Deleting a role that an access review covered now returns 409 ("Role is referenced by access reviews") instead of a server error, so review evidence is kept. Deleting a notification channel keeps the alert delivery history it produced (shown without a channel name), and deleting a maintenance window removes all of its occurrences (#8011).
  • AI for Office prompt templates have their own permissions (#8286). Two new permissions, client_ai_templates:read and client_ai_templates:write, replace organizations:read / organizations:write on the AI for Office template manager. On upgrade, every role that holds organizations:read (built-in and custom) receives client_ai_templates:read, and every role that holds organizations:write receives client_ai_templates:write. Partner Technician, Partner Viewer and Partner Security Approver get read. The built-in Org Admin gets read and write, which is new access for that role. Partner Admin is covered by its wildcard. Org Technician and Org Viewer get neither. Creating, editing and deleting a template now also requires MFA when your MFA policy requires it. The Templates tab and its edit controls are hidden from users without the matching permission.
  • CIS hardening and IOC auto-quarantine run under an approving user (#7979, #8008). Editing CIS baselines or remediations and turning on IOC auto-quarantine need devices:execute and MFA. Existing schedules keep running under the user who created them while that user is active and holds devices:execute; otherwise a schedule pauses with "Needs re-approval", an IOC scan runs detect-only, and a remediation returns to pending approval until someone with devices:execute and MFA saves it again. Approved CIS remediations not dispatched within 7 days return to pending approval.
  • Automatic payments run only for active partners; resetting an invoice link also resets its automatic-payment links (#8328). When a partner is suspended, churned, offboarding, pending or deleted, automatic payments stop for all of its clients, even with the rollout switch on: enroll and update-method links stop working, setup creates no Stripe customer or Checkout session, a setup that finishes saves no payment method and does not turn automatic payments on, due charges and retries are deferred (and end after the usual grace, as when automatic payments are switched off), reconciliation creates or confirms no payment (an unconfirmed one is cancelled), card-expiry emails stop, and cancelled charging notices are no longer re-planned. Stop and skip links keep working for clients. Payments already in progress at Stripe are still recorded. Reactivating the partner resumes charges that were already scheduled or awaiting a retry. Invoices whose charging notice was cancelled while the partner was inactive are not rescheduled on reactivation; collect those through the invoice's pay link. Separately, an invoice's Reset link now also revokes the skip, confirm-payment and update-method links sent for that invoice, and a revoked automatic-payment link no longer offers the invoice's pay link.
  • Charging a client now and changing payment settings ask for a second-factor confirmation; the consent cap is checked before a payment is started (#8358, #8363). Charge now on an invoice, saving the partner Billing settings → Payments tab, saving an organization's own payment settings (its Billing tab, or PUT /orgs/:orgId/billing/payment-settings) when a value actually changes, and sending a client's automatic-payment request to an address other than its billing contact now ask for a fresh authenticator-app code or passkey, bound to that invoice, those settings or that recipient, and only from a signed-in user session (API keys cannot do these). Whether an organization's payment settings change is decided by the server against the stored values: an organization save that changes none of them needs no confirmation and writes nothing, and the organization Billing tab no longer sends them at all when only other fields (such as the tax ID or address) were edited. Deployments running with ENABLE_2FA=false are not asked. Separately, an invoice over the automatic-payment limit (the one the client authorized, or a lower one you set later) is now refused before anything is created at Stripe, instead of a payment being started and then cancelled; the schedule ends and the client is told the planned charge will not happen, as before.
  • Device purge and org erasure keep backup objects in storage (#8215, #7980). Purging a device or erasing an organization no longer lets storage cleanup delete its backup files later; the objects stay, including ones past retention, until an explicit deletion feature ships. Storage is not reclaimed on purge or erasure. Legal holds set on a backup policy or configuration policy now block purge and erasure (409 BACKUP_PROTECTED).
  • Helper screenshot cleanup catches up (#8216). The first retention run after the upgrade deletes expired screenshots and orphaned files that accumulated before. Screenshot storage is local disk, so with several API containers each sweeps only its own files.
  • DNS policy changes reach providers (#8337). Pending DNS filtering policy changes are pushed to providers for the first time after the upgrade. Sending-domain auto-suspension now runs; on self-host it is on only if you set an EMAIL_DOMAINS_AUTOSUSPEND_* variable.
  • Linux patches on deferred rings start auto-approving (#7930) once first-seen plus the deferral period has passed. Windows and macOS patches with no release date still wait for manual approval.
  • Metric anomaly episodes: METRIC_ANOMALY_EPISODE_MIN_BUCKETS defaults to 3 (was 2); the variable still overrides it (#7815).
  • Policy assignment ties: when two configuration policy assignments tie on every ranking key, the earliest-created now wins. A device decided by an exact tie may change once after the upgrade (#8264).
  • Several API instances: some org and partner values (agent update policy, version pins, topology flags, helper settings, policy probe, PAM fallback) are cached per API process for up to 60 s, the remote-management relay setting for up to 180 s. The instance that handled a change applies it immediately (#8128, #8222). A reinstated device or tenant can take up to 60 s to be accepted again (#8054).
  • Database pool waits are bounded (#8243, #8282). A request queued behind a saturated pool used to wait indefinitely; it now fails after DB_POOL_ACQUIRE_TIMEOUT_MS (default 10000, 0 restores the unbounded wait) with DbPoolAcquireTimeoutError, logged as [db-pool-acquire]. Workers may show more job retries under saturation. With DB_POOL_MAX below 3 a warning is logged at startup.
  • Moved devices restart their agent once (#8320) after a random 0–60 s delay, when the new signed identity is applied. Deployments that have never created an agent deployment signing key send no identity update until one exists; agents with BREEZE_ORG_ID / BREEZE_SITE_ID in their environment cannot take the new identity.
  • Helper (Breeze Assist) API returns 403 helper_disabled for a device whose effective Helper setting is off (the agent already stops the Helper in that state) (#7959). Breeze Assist's tray menu loses Exit, and the showTrayIcon policy option is removed; the tray icon always shows (#8213).
  • Breeze Viewer shortcuts move to a prefix chord: Ctrl+Alt+Shift+F/V/R on Windows/Linux and ⌃⌥⌘F/V/R on macOS (fullscreen, paste as keystrokes, release stuck keys), so Ctrl/Cmd+Shift+V and +F reach the remote (#8244).
  • Windows install one-liner on Add Device enrolls before installing the service and no longer ends with service start. With object storage configured, watchdog downloads stream through the API instead of redirecting (#7585).
  • Split deployments: the Add Device command uses PUBLIC_API_URL (else API_URL) from the API; set PUBLIC_API_URL on the API if the dashboard and API are on different hosts (#8124).
  • Cloudflare Gateway DNS Security tokens now also need Account Analytics: Read (#7619).
  • Microsoft 365 Customer Graph executors you run yourself: deploy executor images that serve POST /v1/verify-identity (read and actions) before this API version, or onboarding fails as executor_unavailable. A consent in progress during the upgrade must be restarted. On Azure Container Apps, executors can set BIND_HOST=127.0.0.1 (#7933, #7938, #7682).
  • Email: customer-facing mail uses your company name as the display name when EMAIL_FROM is a bare address; an EMAIL_FROM with its own display name is sent unchanged (#6574). Every transactional email now renders in a sans-serif font (#7987).
  • Webhooks and automations receive ticket.commented for customer email replies (#8344). The assignee notification for a new ticket arrives on the next outbox pass, about 5 s later (#8303, #8040).
  • Remote tunnels: closing one needs devices:execute, remote:access and MFA (#7907). DNS filtering policy writes refuse site- or device-restricted users (#8118). Fleet findings status changes require MFA (#8045). Opening the live event stream requires devices:read (#8012).
  • Enrollment keys with an expiresAt in the past are refused with 400; discovery profiles with a subnet larger than /16 need deepScan (#8374).
  • Quotes: hardware lines without a SKU or part number are left out of the parts order (#8248). Quotes use the partner's Settings → Branding logo and colors, and contract PDFs generated from now on may paginate slightly differently (#7713).
  • Self-hosted docker/Caddyfile.prod copies: a new @webBillingAutopay matcher routes /billing/autopay* to the web app; carry it over if you maintain your own copy (#7822).
  • Building the agent from source now needs Go 1.26.9. The agent module and the release builds move from Go 1.26.6 to 1.26.9, and golang.org/x/net from v0.59.0 to v0.60.0, picking up the upstream net/http and HTTP/2 fixes in those releases. With the default GOTOOLCHAIN=auto, go build fetches 1.26.9 on its own; with GOTOOLCHAIN=local, install Go 1.26.9 first. Pre-built agent binaries need no action (#8305).

Database.

  • 76 migrations (idempotent; applied automatically on boot via autoMigrate unless AUTO_MIGRATE=false). Expect [auto-migrate] Applied 76 migration(s). A 77th new file, migrations/preflight/2026-11-28-100000-ai-model-registry-legacy-drop-preflight.sql, is a read-only check for the next release's legacy AI drop and is never applied by autoMigrate.
  • Mostly additive DDL: new tables for diagnostic access grants, backup restore authorizations and erasure fences, partner API idempotency keys, EDR providers, device ownership history, reliability baselines, physical placements, device workloads, contract hour periods, ticket approval settings and requests, contact roles and fix memory; new columns on sites, time_entries, tickets, portal_branding, audit_logs, installer_bootstrap_tokens, partner_service_principal_keys and others; new permissions (webhooks:read, client_ai_templates:read/:write, sites:set_location, tickets:record_approval) granted to roles as described above; and RLS policy updates.
  • Brief exclusive locks on busy tables — avoid a backup window (#8011). The 2026-12-13-110000/110100/110200 migrations redefine foreign keys on child tables of users, devices, alerts, tickets, roles and others. Each swap briefly takes an ACCESS EXCLUSIVE lock on the child and the parent table, and waits at most 5 s for it (lock_timeout). If a long-running reader holds one of those tables (a large report, a pg_dump/backup window), the migration stops, the API fails to boot with a lock-timeout error, and it re-applies cleanly on the next restart. Avoid deploying this release during a database backup window. No data is rewritten; the new constraints are validated afterwards without blocking writes. The time_entries, tickets/ticket_parts and sites column migrations (2026-12-17-100300, 2026-12-20-200300, 2026-12-20-220000) use the same 5 s lock timeout and validate without blocking writes.
  • audit_logs index and validation scan at boot (#7808). 2026-12-05-110100-audit-logs-partner-scope-index.sql runs outside a transaction: it builds audit_logs_partner_scope_idx with CREATE INDEX CONCURRENTLY, then runs VALIDATE CONSTRAINT under SHARE UPDATE EXCLUSIVE. Audit writes continue, but on a large audit_logs table both steps scan it and can add minutes to the first boot. If the build is interrupted, the next start refuses with an "INVALID index" message: run DROP INDEX CONCURRENTLY audit_logs_partner_scope_idx; and restart the API. Partner-level events recorded before the upgrade stay hidden (no backfill).
  • Other concurrent index builds: on remote_sessions/tunnel_sessions, tickets (partner feed), time_entries and sites. Each waits for long-running transactions, does not block writes, and refuses the next start with an "INVALID index" message naming the index if interrupted; drop it with DROP INDEX CONCURRENTLY <name>; and restart.
  • Network-monitor alerts backfill (#7954). 2026-12-07-110000-network-monitor-alert-endpoint-display.sql scans alerts and rewrites message, context.target and context.error on network-monitor alerts so endpoints show as scheme + host. On a large alerts table this adds time to the first boot. Each rewritten alert is re-delivered once on the Partner API alerts feed.
  • Other backfills: one ownership row per existing device, in batches of 5,000 followed by a validation scan of devices (#8219); report run ownership copied from each report, holding SHARE ROW EXCLUSIVE on report_runs and reports until it commits, so report generation waits briefly on installs with a large report history (#8255); contact_roles back-filled from existing contacts (#8226); existing partner service principal keys bound to their owner's current state (#8332); grandfathered CIS and IOC approvals, with counts logged (#7979).
  • backup_configs.encryption_key is dropped (#7977). The column has no writer; the migration refuses, and the API does not boot, if any row still holds a value (refusing to drop backup_configs.encryption_key: N row(s) still hold a non-null value). Review and clear those rows, then restart.
  • New enum values: expired report status, awaiting_approval billing status, hour_block contract line type and the research AI agent kind.

Environment variables.

  • Required: none. No new required environment variables; the config validator gains no new boot-refused variable, and neither Compose file adds a new :? guard.

  • New, optional:

    • PATCH_REPORT_RETENTION_DAYS (default 30, minimum 1) and PATCH_REPORT_ORPHAN_MIN_AGE_MS (default 1 hour, never below 10 minutes) (#8284).
    • DB_POOL_ACQUIRE_TIMEOUT_MS (default 10000; 0 disables) and DB_TIMER_LAG_GRACE_MS (default 2000; 0 disables) (#8243, #8282).
    • PARTNER_API_TICKET_WRITE_RATE_LIMIT_PER_HOUR (default 1200), PARTNER_API_TICKET_WRITE_PARTNER_RATE_LIMIT_PER_HOUR (default 6000), PARTNER_API_IDEMPOTENCY_RETENTION_ENABLED (default true) and PARTNER_API_IDEMPOTENCY_RETENTION_DAYS (default 7) (#8235).

    All are mapped in the root docker-compose.yml. The four PARTNER_API_* variables are not yet in deploy/docker-compose.prod.yml; if you run that file and need them, add them to the api service's environment: block.

  • Changed default: INSTALLER_BOOTSTRAP_TOKEN_TTL_MINUTES is now 10080 (7 days), capped at 43200 (above).

  • Changed meaning: MCP_UNATTENDED_TIER3_PRINCIPALS also accepts partner_sp:<partner service principal id> (#8090). METRIC_ANOMALY_EPISODE_MIN_BUCKETS default is now 3.

  • No longer read: BREEZE_AI_SCRIPT_REVIEWER_MODEL, WORKSPACE_CONTENT_LLM_MODEL (#7867).

  • 65 more variables passed to the API by both Compose files (#7858): retention windows, agent ingest quotas and rate limits, screenshot and log-forwarding caps, DB_POOL_MAX, the offboarding drain window, the public download key TTL and login lockout. Every one is passed as ${VAR:-}, so unset keeps the built-in default, but a value already in your .env that Compose used to ignore now takes effect. Check .env for stale entries before upgrading.

Agents. Agents, Breeze Assist and Breeze Viewer update to 0.122.1 through your normal update channel. Agent-side changes: clipboard v2 and the per-session clipboard summary (#8262, #8263); key and button release on disconnect (#8251); signed org/site identity sync after a device move (#8320); diagnostic file access commands (#7406, #8327); command results kept across shutdown, with a new outbox/inflight/ directory (#8379); macOS update package from your own server first (#8107); macOS install, uninstall, group and screen-capture fixes (#7876, #7833, #7972, #8154); Windows service monitoring, antivirus/firewall unknown state, Storage Spaces, System Restore and agent-data-directory path checks (#8038, #8043, #8150, #8069, #8108, #8010); N-able detection (#8085); peripheral policy for ungrouped devices (#8322); interface types for topology (#7846); parallel restore downloads (#7417); WinPE recovery console groundwork (#7926); build-time display branding (#8061). Breeze Assist starts hidden in the tray (#8213). Breeze Viewer: clipboard v2, input and focus fixes, QWERTZ/AZERTY letters and the new shortcut chord (#8266, #8244, #7848).

Feature flags.

  • Automatic payments ship switched off for every partner (per-partner rollout switch). Payment reminders are off until configured under Billing → Payments.
  • Customer portal toggles enableHardwareHealth, enableHardwareInventory, enablePerformanceMetrics and enableSoftwareInventory default off for every organization. Enable all turns on the first three, not enableSoftwareInventory, which only opens the read API.
  • Location suggestions (timeTracking.locationSuggestions), staff forward routing, the default inbound assignee, Gmail labelling and the Pushover fallback are all off by default.
  • Customer approval of ticket work (settings API only) defaults to enabled = false; workload inventory enumeration is off by default; EDR has no UI yet.
  • AI suggested fixes fix-memory consultation in triage is behind ml.remediation_suggestions.enabled.

What's Changed

New Contributors

Full Changelog: v0.121.0...v0.122.1

Check out latest releases or
releases around LanternOps/breeze v0.122.1

Don't miss a new breeze release

NewReleases is sending notifications on new releases.

Get notifications