mr-karan/calert v2.4.0

on GitHub

What's new

v2.4.0 adds opt-in group threading, deterministic thread keys for replicated deployments, better Helm secret handling, and separate Ubuntu and scratch container images. It also changes webhook delivery so Alertmanager can retry when Google Chat fails instead of receiving an early success response.

Group threading and multiple replicas

Set threading_mode = "group" to keep every alert in an Alertmanager group in one Google Chat thread. Group notifications use an aggregated CardsV2 message with firing and resolved counts, capped alert sections, and a notice when Alertmanager truncates the payload.

Per-alert mode remains the default. Its thread keys now come from the room, fingerprint, and alert start time instead of process-local UUIDs. Replicas and restarted instances therefore agree on the same thread without leader election.

Redis is optional in group mode. It shares deduplication and resolved-status history between replicas. State is recorded only after Google Chat accepts the message, so a failed delivery cannot suppress an Alertmanager retry. Concurrent replicas may still produce a duplicate rather than risk dropping an alert.

See Group threading and Redis shared state for setup details.

Container images

The default image remains a regular Ubuntu image with a shell and basic debugging tools:

docker pull ghcr.io/mr-karan/calert:v2.4.0

A smaller scratch variant is available separately:

docker pull ghcr.io/mr-karan/calert:v2.4.0-scratch

Both variants are published for amd64 and arm64. Moving to scratch is optional; latest continues to point to the Ubuntu image, while latest-scratch points to scratch.

Helm chart

The chart can now:

• generate a Kubernetes Secret for config.toml
• mount an existing Secret with a configurable key
• load environment variables through standard envFrom.secretRef and envFrom.configMapRef entries
• mount custom message templates correctly in every Secret mode
• configure group threading, deduplication, message limits, and Redis

Example:

config:
  asSecret: true

providers:
  production:
    endpoint: https://chat.googleapis.com/v1/spaces/example/messages
    threading_mode: group
    dedup_window: 2m
    max_alerts_per_message: 10
    redis:
      address: redis:6379
      db: 0
      key_prefix: calert

To keep the complete config outside Helm values, set config.asSecret: true with config.existingSecret.name. To load provider endpoints from environment variables instead, use envFrom with a Secret containing keys such as CALERT_PROVIDERS__PRODUCTION__ENDPOINT.

Delivery and security fixes

• /dispatch now waits for provider delivery and returns HTTP 502 on failure, allowing Alertmanager to retry.
• Request deadlines propagate through retries and Redis operations and finish before the HTTP server write timeout.
• Google Chat webhook keys, tokens, transport errors, and response bodies are no longer written to logs.
• CardsV2 templates JSON-escape labels, annotations, URLs, and other dynamic fields.
• Oversized text messages split without empty chunks or broken UTF-8.
• Group deduplication includes label, annotation, status, and truncation changes.
• Redis state keys are bounded and isolated by room.

Tooling and tests

• Go and the release toolchain now use Go 1.26.5.
• Go modules, GitHub Actions, GoReleaser, golangci-lint, Prometheus, and Alertmanager were updated.
• The release config now uses GoReleaser's current dockers_v2 format.
• Unit tests, race tests, Helm render tests, lint, and startup smoke tests cover the new paths.

Upgrade notes

Group threading is opt-in, so existing configurations continue to use per-alert messages.

/dispatch no longer acknowledges the webhook before Google Chat delivery finishes. Alertmanager will retry a 502 response. If your provider retries need more time, increase app.server_timeout; calert keeps its dispatch deadline below that timeout so it can still return a response.

Thread keys are now deterministic. An alert that was already firing before the upgrade may resolve into a new thread after the process restarts because older releases used an in-memory random UUID. New alert lifecycles remain stable across restarts and replicas.

In group mode, thread_ttl controls deduplication and resolved-status state retention. It does not rotate Google Chat threads. A recurring Alertmanager GroupKey continues in the same thread.

Thanks

Thanks to @jvlxz for the group-threading implementation, @gregseb for the scratch image and Helm Secret work, and @marco-m for documenting the health endpoint.

Full changelog: v2.3.0...v2.4.0