Agent Watchdog Service
New lightweight breeze-watchdog binary that independently monitors the primary breeze-agent, recovers it from crashes and hangs, and provides a failover API connection when the agent is down.
New
- breeze-watchdog binary — separate ~5-8MB Go binary, ships alongside the agent
- Layered health checks — process liveness (5s), IPC ping/pong (30s), heartbeat staleness (2min)
- State machine — CONNECTING → MONITORING → RECOVERING/STANDBY → FAILOVER
- Transparent failover — shares agent identity, polls API with
X-Breeze-Role: watchdog - Cross-platform — launchd (macOS), systemd (Linux), Windows SCM with
Restart=always - Graceful shutdown detection — IPC
shutdown_intent+agent.statefile prevents false recovery - Failover commands — restart_agent, update_agent, update_watchdog, collect_diagnostics, start_agent
- Cross-updating — agent updates watchdog normally; watchdog self-updates in failover
- Tamper protection interfaces defined for v2 (integrity_check, integrity_result, tamper_alert)
API Changes
- Watchdog heartbeat handling (updates
watchdog_status,watchdog_last_seen,watchdog_version) - Command routing via
target_rolecolumn ondevice_commands GET /devices/:id/watchdog-logsendpoint- DB migration:
watchdog_statusenum, watchdog columns on devices
Build
make build-watchdog/build-watchdog-{linux,darwin,windows}make dev-push-watchdog/make dev-push-both- Install scripts updated to include watchdog binary
- macOS
.pkgand Windows MSI ready for watchdog inclusion
Config
New watchdog: section in agent.yaml with tunable intervals, recovery limits, and journal settings.
Full PR: #339
What's Changed
- feat: JIT VNC fallback for older macOS at login screen by @ToddHebebrand in #338
- feat: add agent watchdog service by @ToddHebebrand in #339
Full Changelog: v0.50.9...v0.60.0