naradalisted

# Narada — Messenger of the Gods (Alerts, Notifications & Webhooks) Narada carries only messages worth hearing — every alert actionable, every webhook delivered exactly once in effect. ## Alerting rules - Alert on symptoms, not causes: "checkout p99 > 2s" or "error rate > 1%", not "CPU at 80%". - Every alert must be actionable. If the response is "ignore it", delete the alert or fix the threshold. - Severity tiers — route accordingly: - **Page** (wake someone): user-facing outage, data loss risk, SLO burn rate critical. - **Ticket** (fix this week): degradation with headroom, elevated retries, disk trending full. - **Log/dashboard only**: everything else. No notification. - Alerts fire on sustained conditions (e.g. 5 min windows), not single data points. - Every alert links to a runbook or dashboard. An alert with no next step is noise. ## Avoiding fatigue - Dedupe: group identical alerts into one incident, don't send 200 copies. - Rate-limit and batch notifications; use escalation instead of repetition. - Review alert volume monthly: anything that fired > 5 times without action gets deleted or re-thresholded. ## Webhooks — producing - Sign every payload (HMAC-SHA256 header) so consumers can verify origin. - Retry failed deliveries with exponential backoff + jitter (e.g. 1m, 5m, 30m, 2h, give up at 24h). - Include an event ID and timestamp in every payload; document the schema and version it. - Failed-forever deliveries land in a dead-letter queue with alerting,