how-to-biz-monitoringlisted

Monitor business metrics so that the alerts are worth reading - build a causal seasonal baseline on the log scale, calibrate the null before trusting any threshold, control the false discovery rate across every series watched on the same day, and score the whole thing against changes planted at known times so recall is measured rather than claimed. Then attribute a real move exactly, splitting a ratio metric into mix and rate so a healthy business does not get escalated. Use when asked to "set up alerting on our metrics", "why did revenue drop", "our dashboard alerts too much", "nobody reads the alerts", "detect anomalies in our KPIs", "is this dip real", "build a weekly metrics digest", or when someone proposes a percent-change threshold on a noisy series. Ships five standalone modules - baseline, detect, attribute, bench, digest - a runnable baseline script, and an injection bench that scores any detector you write.
phoebefu6/phoebe-data-skills · ★ 1 · DevOps & Infrastructure · score 72

Install: claude install-skill phoebefu6/phoebe-data-skills

# how-to-biz-monitoring Data Analytics layer skill (layer 2 of the phoebe-data-skills 4-layer roadmap). Most metric alerting is built backwards. Somebody picks a threshold, ships it, and then spends a year tuning it against complaints. Nobody ever measures how many real changes it missed, because the misses are by definition absent from the alert log, and so the only available evidence is the alerts that fired - the one population that cannot tell you anything about recall. This skill builds the monitor in the opposite order: establish what normal looks like, calibrate the null, control multiplicity, and only then choose a threshold - with a bench that scores the result against changes you planted yourself, so every number on the page is measured. Showcase walkthrough (Everrest marketplace case, real executed charts): `docs/how-to-biz-monitoring/` in https://github.com/phoebefu6/phoebe-data-skills ## Where this sits in the lineage `raw dump -> lake -> warehouse -> marts` **-> the thing that watches the marts**. Reads the stage-3 marts. It sits beside `how-to-metric-to-scorecard`, which reports a metric on a schedule, and `how-to-data-quality`, which gates the data on the way in. The boundary matters and is the first rule below. ## The seven rules 1. **Anomaly detection is not data quality.** They fail differently. A duplicate load that inflates every day of a month by a quarter is invisible to a threshold on the metric, because the baseline simply learns it: in