← ClaudeAtlas

alerting-rules-tunerlisted

Cut alert noise and make every page mean something — rewrite alerting rules to fire on user-felt symptoms (error rate, latency SLO burn, failed requests) instead of causes (high CPU, full disk), with duration windows and severity routing so only urgent, actionable conditions reach a human. Use when on-call is fatigued by low-value pages, when real incidents get missed in the noise, or when alerts fire on causes rather than impact.
imtiazrayhan/agentscamp-library · ★ 1 · AI & Automation · score 75
Install: claude install-skill imtiazrayhan/agentscamp-library
On-call exhaustion is rarely an "alert quantity" problem you fix by muting things — it's an *altitude* problem. Pages fire on causes (a node at 95% CPU, a disk at 80%, a saturated thread pool) that may or may not hurt anyone, instead of on symptoms the user actually feels. This skill audits every rule against one question — *does this fire only when a human must act now?* — then rewrites the survivors to alert on symptoms with duration windows and severity routing, and demotes the rest to dashboards or tickets. ## When to use this skill - On-call is fatigued: frequent pages that resolve themselves or need no action, night pages for non-urgent conditions. - Real incidents get missed because they're buried under low-value noise, or everyone has muted the channel. - Alerts fire on causes (CPU, memory, disk, queue depth, pod restarts) rather than user impact. - One incident generates a storm of 50 correlated pages instead of one. - You have alerts with no owner and no runbook — nobody knows what to do when they fire. - Standing up alerting for a new service and want to start symptom-first instead of bolting on host metrics. ## Instructions 1. **Inventory the rules and classify each as symptom or cause.** Grep the alerting config (`*.yml`/`*.yaml` Prometheus rules, Datadog monitor exports, Grafana alert JSON, Alertmanager routes) for every rule that pages a human. For each, label it: **symptom** (something the user experiences — request errors, latency, failed checkouts, SLO bu