github kOlapsis/maintenant v1.6.0
Agent keeps collecting & Containers labels read

5 hours ago

An agent now keeps collecting through an outage and replays afterwards. A server restart, a network cut or an HA failover leaves a backlog instead of a hole in the history.

This release also reads the labels your containers already carry: Traefik and Caddy routes become endpoint and certificate monitors, and OCI image labels show up on the container.

Outage spool

Before, a broken stream stopped the agent's collector until the next reconnect, so everything measured during the outage was lost. The collector now runs for the whole life of the process, and events queue in a local SQLite database (<data-dir>/spool.db) until the server acknowledges them. On reconnect, the agent first sends a fresh inventory and topology, then replays from the last acknowledged event. An agent that restarts while the server is down keeps measuring too.

What is replayed: container lifecycle events, container resource samples and endpoint probe results. What is not: container inventory, Swarm and Kubernetes topology, host samples and certificate scans. Those describe current state, and a stale replay would overwrite a live value.

Replay is history, not alerting

Each event carries the time it was observed. The server records history at that time, so graphs, uptime and the container timeline are filled in where the outage was. A replayed event never opens an alert or sends a notification: nobody is paged for an incident that is already over.

A replayed probe also no longer touches an endpoint's status, its consecutive counters or its status-change event. Receiving the same probe twice, which the replay does by design, was inflating the counter that decides when an alert fires.

Visibility

Agents report their queue depth, whether they are catching up, and how many events were dropped for lack of room. The Agents page shows it.

Limits

Setting Default
MAINTENANT_AGENT_SPOOL_MAX_MEMORY_BYTES / --agentSpoolMaxMemoryBytes 16 MB Buffer held in memory before spilling to disk
MAINTENANT_AGENT_SPOOL_MAX_DISK_BYTES / --agentSpoolMaxDiskBytes 128 MB Spool ceiling, oldest events dropped first
MAINTENANT_AGENT_SPOOL_MAX_AGE_SECONDS / --agentSpoolMaxAgeSeconds 24 h Age past which a queued event is neither kept nor replayed

Setting both budgets to 0 disables the spool. A clean stop flushes it, and a revoked agent deletes it. See Multi-host: Outage Spool.

Monitors from reverse proxy labels

A container behind Traefik or Caddy already says where it answers from. Set MAINTENANT_PROXY_LABELS=true (or --proxyLabels) and maintenant turns those labels into HTTP endpoints, and the hostnames of the HTTPS ones into certificate monitors. Nothing to declare twice.

From Traefik it reads the Host(...) hostnames of the HTTP routers, plus a single Path or PathPrefix when the rule has one, and picks https when the router has a tls label or a secure entrypoint. From Caddy docker-proxy it reads the site addresses in caddy and caddy_<n>, and turns TLS verification off for the sites served with tls internal. traefik.enable=false is respected.

Discovered endpoints behave like the ones you declare yourself: they accept 2xx,3xx, since a proxied site often answers with a redirect, and they follow the global maintenant.endpoint.* settings such as interval or failure-threshold. Your own maintenant.endpoint.* target always wins over discovery, and a single container opts out with maintenant.proxy-labels=false.

The setting is off by default and is read by the server and by each agent separately, so a host can be enrolled without its routes becoming monitors. Docker only: Swarm services and Kubernetes ingresses are out. See Docker labels: reverse proxy labels.

Image metadata

The OCI labels an image carries (org.opencontainers.image.version, description, source, url, with the older org.label-schema.* as a fallback) are now stored with the container and shown in its detail panel, source and documentation as links. Nothing to configure: most public images already ship them.

Also fixed: the retention job's incremental vacuum freed a single page per run instead of a full slice, so a SQLite database gave disk space back far more slowly than intended.

Upgrading

Upgrade the server before the agents. A 1.5.0 server does not understand the replay flag: it would evaluate alerts on replayed events and record them at the time they arrive rather than the time they were observed.

One server migration, applied at startup: four columns on the containers table for the image metadata. The agent creates its spool database on first start.

Don't miss a new maintenant release

NewReleases is sending notifications on new releases.