github roxy-wi/IncidentRelay v2.5.0
v2.5.0 — Incident Collaboration and Change Intelligence Preview

2 hours ago

IncidentRelay 2.5 brings incident response into a dedicated workspace. Responders can see who is coordinating the incident, which services are affected, what the team has tried, and how the incident was resolved. Activity from linked AlertGroups appears alongside the operational discussion, with each entry attributed to its source.

Incident collaboration

  • Assign an incident commander, request responders and manage stakeholders from the Incident. Inspect affected services, runbooks, dashboards and dependencies while investigating.
  • Record operational comments, root cause and a resolution summary. Technical comments stay on their original AlertGroups. If two responders edit the same Incident, the workspace preserves the unsaved text so the second responder can reconcile it with the latest version.
  • Incident notifications track delivery per event, channel and recipient. The scheduler retries failed deliveries and recovers interrupted attempts. Delivery is at least once, so a provider can receive a duplicate if a worker stops after the provider accepts a message.
  • A manual resolve that sees an already resolved AlertGroup while a reopen is in progress leaves the new occurrence active. This fixes a race on PostgreSQL.

Change Intelligence preview

For teams evaluating deployment correlation, the Changes inbox brings together deployments and other changes reported by CI/CD. On services that opt in, IncidentRelay suggests changes that might explain an alert episode. Responders can confirm or reject suggestions, record a mitigation, or link a change directly to an Incident. Reviewed relations are visible in the alert and Incident workspaces, and authorized service changes appear in service timelines.

Change Event intake is disabled by default and is intended for internal testing in this version. Automatic correlation also requires an explicit opt-in per service. Configuration and limits

Google Cloud Monitoring

Route Google Cloud Monitoring alerts directly to an IncidentRelay team. Create a Google Cloud Monitoring route and point a Webhook notification channel at /api/integrations/google-cloud-monitoring. Use HTTP Basic with username incidentrelay and the route intake token as the password. Enable closure notifications on the Google alerting policy so IncidentRelay can resolve the alert when the incident closes.

Repeat notifications update the same alert. Once a close arrives, a delayed open for the same route, project and Google incident ID is ignored, even if the close arrived first.

Grafana dashboard for IncidentRelay

Import tools/grafana/incidentrelay-overview.json in Grafana and select the Prometheus data source that scrapes IncidentRelay’s /metrics endpoint. The IncidentRelay · Operations dashboard has 22 panels covering database health, HTTP traffic and latency, incoming alerts, worker heartbeats, notification queues, Event Orchestration and Change Intelligence.

Filter by scrape job or web target to investigate a single replica. When viewing replicas together, request rates are summed and queues shared through the database are counted once.

Other interface changes

Users can choose a personal color in their profile to make duty shifts and participants easier to recognize. The Silences list now shows rule details directly in the table.

Full changelog since v2.4.0

Don't miss a new IncidentRelay release

NewReleases is sending notifications on new releases.