github LanternOps/breeze v0.60.0
v0.60.0 — Agent Watchdog Service

latest releases: v0.127.0-rc.1, v0.126.0, v0.126.0-rc.1...
7 months ago

Agent Watchdog Service

New lightweight breeze-watchdog binary that independently monitors the primary breeze-agent, recovers it from crashes and hangs, and provides a failover API connection when the agent is down.

New

  • breeze-watchdog binary — separate ~5-8MB Go binary, ships alongside the agent
  • Layered health checks — process liveness (5s), IPC ping/pong (30s), heartbeat staleness (2min)
  • State machine — CONNECTING → MONITORING → RECOVERING/STANDBY → FAILOVER
  • Transparent failover — shares agent identity, polls API with X-Breeze-Role: watchdog
  • Cross-platform — launchd (macOS), systemd (Linux), Windows SCM with Restart=always
  • Graceful shutdown detection — IPC shutdown_intent + agent.state file prevents false recovery
  • Failover commands — restart_agent, update_agent, update_watchdog, collect_diagnostics, start_agent
  • Cross-updating — agent updates watchdog normally; watchdog self-updates in failover
  • Tamper protection interfaces defined for v2 (integrity_check, integrity_result, tamper_alert)

API Changes

  • Watchdog heartbeat handling (updates watchdog_status, watchdog_last_seen, watchdog_version)
  • Command routing via target_role column on device_commands
  • GET /devices/:id/watchdog-logs endpoint
  • DB migration: watchdog_status enum, watchdog columns on devices

Build

  • make build-watchdog / build-watchdog-{linux,darwin,windows}
  • make dev-push-watchdog / make dev-push-both
  • Install scripts updated to include watchdog binary
  • macOS .pkg and Windows MSI ready for watchdog inclusion

Config

New watchdog: section in agent.yaml with tunable intervals, recovery limits, and journal settings.

Full PR: #339

What's Changed

Full Changelog: v0.50.9...v0.60.0

Don't miss a new breeze release

NewReleases is sending notifications on new releases.