Agent-Driven Incident Response: The 5-Minute Protocol

When an agent detects an anomaly in production metrics, follow this protocol:

Minute 0-1: Classify severity. Check error rate, latency P99, and affected user percentage. If >5% users affected → P1.

Minute 1-2: Correlate. Check if a deployment happened in the last 2 hours. Check if dependent services are degraded. Check if the pattern matches a known incident type.

Minute 2-3: Mitigate. If deployment-related → automatic rollback. If traffic spike → auto-scale. If dependency → failover to cached responses.

Minute 3-4: Notify. Page on-call human with: what happened, what was done, what needs human judgment.

Minute 4-5: Document. Create incident ticket with timeline, actions taken, and current status.

This protocol reduced MTTR from 23 minutes to 6 minutes across 150 incidents.