github LanternOps/breeze v0.101.0
on GitHub

latest releases: v0.124.0, v0.123.0, v0.122.1...
3 months ago

Breeze RMM v0.101.0 — agent-fleet reliability: healthy agents stop getting killed, and prod stops deadlocking its own DB pool.

This is a fix-and-hardening release. Two of the fixes address live production incidents from 2026-07-24: a fresh v0.100.0 install whose watchdog killed the healthy agent and left it permanently stopped, and a US web-UI hang caused by the API self-deadlocking its Postgres pool. No breaking changes. No migrations. No new environment variables.

Fixed

  • Watchdog no longer restart-loops and strands healthy agents — a fresh v0.100.0 install could enroll and heartbeat normally, then be killed five times in nine minutes and left permanently STOPPED in FAILOVER. Operator symptom: "it installed, but the service is not running after." Six independent defects combined to cause it, each fixed here: staleness now uses the freshest of the on-disk agent.state or the IPC state_sync channel (an AV/EDR-blocked state file is a blocked writer, not a dead agent); IPC liveness counts any received envelope, not only pongs, so a busy agent isn't read as silent; graceful service recovery gets a 5s SCM acknowledge grace instead of failing in 0–1ms with a stop already in flight; and exhaustion can no longer park the service stopped in FAILOVER. agent.state is also recreated when it goes missing instead of failing forever. (#2769)
  • Agent command polling no longer double-holds pooled DB connections — GET /agents/:id/commands opened a request-long org-scoped transaction and a second system-scoped one, holding two pool slots per poll. Under agent load that filled every slot with outer transactions waiting on an inner connection that could never be acquired, reaped every 60s and immediately re-polled — a self-sustaining spiral that hung the web UI. The route now self-manages its DB context; the command result route is unchanged. (#2765)
  • Breeze Assist and the watchdog updater resolve the server URL at call time — both took a by-value copy of cfg.ServerURL at startup, so after a backup-server promotion they kept downloading binaries from the dead primary while heartbeating the promoted backup. The URL is now a provider function, making the stale-copy mistake unrepresentable at the call site. (#2478)
  • User-context helpers ship their logs again — helpers running as the logged-in user (Windows --role user, the macOS desktop helper, the Linux user helper) always fell back to a default config because config.Load() failed opening the root-only secrets.yaml, and the empty AgentID/ServerURL failed the log-shipper gate. Their diagnostics never reached the API on any platform. They now load agent.yaml directly. (#2483)
  • Scheduled Windows session helpers can no longer be duplicated — when a helper kill failed, ownership was cleared but never restored, so the next reconcile spawned a second live helper for the same session and role. Ownership is now retained on kill failure with a self-clearing 5-minute deadline cap. (#2530)
  • Security-posture reports no longer show one protected device as full coverage — the "Security products in use" inventory OR-merged per-device antivirus evidence, so Defender on 200 devices with real-time protection on just one rendered as green with a 200-device count, contradicting the same page's honest 1% coverage metric. It now reads 200 devices, 1 with real-time protection on. (#2517)
  • breeze-ext CLI entrypoint and validate — the entrypoint guard compared a symlinked argv[1] against the module realpath, so a pnpm-installed breeze-ext parsed nothing and exited 0 — a signing/packing step that "passed" without packing or signing anything. Both sides now resolve to realpath. breeze-ext validate is also implemented (it was a stub that threw) on the same machinery pack uses, plus entry-path existence checks that pack does not perform. (#2657)

Security

  • Portal session cookies derive Secure from the real transport, not NODE_ENV — the client portal carried a private copy of the pre-#1618 cookie logic and stamped Secure from NODE_ENV === 'production' alone. A production deployment the browser reaches over plain HTTP (failed ACME, an http:// URL, a TLS-stripping proxy) issued portal cookies the browser silently discards — login appeared to succeed but the session never persisted, with no diagnostic anywhere. The portal now uses the same X-Forwarded-Proto → request-URL → NODE_ENV decision as the admin app, gated on the existing TRUST_PROXY_HEADERS / TRUSTED_PROXY_CIDRS posture, with set/clear symmetry and transport-mismatch warnings. (#2611)
  • PID-reuse TOCTOU closed in desktop-helper termination — the helper manager verified a process and then killed it across two separate bare-PID OpenProcess calls. If Windows recycled that PID in between, the check passed on the old process and the terminate landed on an unrelated one. Windows now opens the PID once and verifies the image path and terminates on the same handle, which pins the exact kernel process object. (#2531)

Improved

  • Helper Role is typed as ipc.HelperRole instead of a bare string, removing a class of mismatched-role bugs at compile time. (#2532)
  • The compiled breeze-backup binary is no longer tracked in git. (#2594)

Self-Hosting / Upgrade Notes

Upgrade command — bump BREEZE_VERSION to 0.101.0 in /opt/breeze/.env, then: 

docker compose pull api web portal && docker compose up -d binaries-init api web portal

Include portal — it is a separate container and does not follow api/web. If you build from source, run pnpm install first.

Database — nothing required. This release contains zero migrations. Upgrading is a container swap; there is no schema change, backfill, or table rewrite, and no boot-time migration delay.

No new required environment variables. Nothing added here is read by the config validator, so no deployment will refuse to boot for a missing variable.

Behavior changes

  • Portal cookies now honor your proxy posture. If you terminate TLS at a reverse proxy, make sure TRUST_PROXY_HEADERS / TRUSTED_PROXY_CIDRS are configured so the API recognizes X-Forwarded-Proto: https — otherwise portal cookies will correctly but undesirably be issued without Secure. If the portal is reached over plain HTTP, cookies are now issued without Secure so login actually works, and a throttled [portal] warning is logged. The optional PORTAL_COOKIE_FORCE_SECURE and PORTAL_COOKIE_SAME_SITE overrides are unchanged and still force Secure when set.
  • The watchdog fixes are agent-side. Deploying the server registers the new agent, watchdog, helper, and user-helper binaries, but devices do not pick them up until the new version is promoted. If you run with AGENT_AUTO_PROMOTE=false, promote deliberately. If you saw v0.100.0 installs that enrolled and then showed the service not running, this is the release that fixes it — the fix only lands once the fleet is promoted.
  • Consider your DB_POOL_MAX headroom. #2765 removes the deadlock mechanism itself, so no config change is required. But if you run a large agent fleet against a small pool, DB_POOL_MAX should have real headroom under your Postgres max_connections — our own hosted region runs 35 against a server limit of 50.

No breaking changes.

Full Changelog: v0.100.0...v0.101.0

What's Changed

Full Changelog: v0.100.0...v0.101.0

Check out latest releases or
releases around LanternOps/breeze v0.101.0

Don't miss a new breeze release

NewReleases is sending notifications on new releases.

Get notifications