experiment-suite
SolidUse when the user has a research question and needs a complete experiment package — design document, runnable code, results (measured or simulated with honest provenance), publication-grade figures, structured report. Single-stage, no Python runtime.
Install
Quality Score: 83/100
Skill Content
Details
- Author
- ai4s-research
- Repository
- ai4s-research/ai4s-skills
- Created
- 2 months ago
- Last Updated
- 1 months ago
- Language
- Python
- License
- MIT
Similar Skills
Semantically similar based on skill content — not just same category
experiment-designer
Turn an assumption or a growth bet into an experiment brief with a mechanism-stated hypothesis, one primary metric, guardrails with floors, an exposure design, a sample size reasoned from the minimum detectable effect, and stop rules written before launch. Use when a growth plan bet needs a test, when an assumption register row is low confidence and high impact, when a price change should be tried on a slice first, or when someone says "let us just A/B it". Takes the assumption, the metric candidate with its baseline, the eligible traffic, and the guardrail candidates; returns the test card, the filled brief, the sizing record with its calculator inputs, and the pre-committed decision rule.
experiment-design
A discipline for designing experiments (A/B tests, multivariate, holdouts) so the results actually answer the question you asked. Hypothesis writing, sample size, duration, segment analysis, running discipline, matching a result to a pre-committed decision rule, and the common failure modes that produce confidently wrong shipping decisions. Use this skill whenever the user is planning a test that has not run yet: framing a hypothesis, sizing the sample, setting duration, choosing guardrails, or deciding whether something is worth testing at all. Triggers on design an experiment, experiment plan, A/B test, split test, multivariate test, holdout, experiment hypothesis, sample size, minimum detectable effect, MDE, test duration, guardrail metric, no peeking, pre-committed decision rule, is this worth testing. Use `experimentation-analytics` instead when the test has already run and the question is how to read the result panel.
experiment-rigor
Structure an experiment and its write-up so the Methods reconstruct without the conversation and the uncertainty is declared, not hidden — a typed record.yaml across a probe / measurement / decision tier ladder, every load-bearing rule a script that exits non-zero rather than a line of prose. Use when you pre-register an A/B, freeze a plan before running it, write up an experiment or a comparison, add error bars or a confidence interval to a rate, reconcile declared cells against the runs that happened, separate confirmatory from exploratory outcomes, name the threats to validity, design the 2x2, or ask whether a skill, tier, model, or strategy actually helps and how you would show it rigorously. Covers the small-n refusal (no CLT below 30 — Wilson, Clopper-Pearson, or a within-experiment Beta-Binomial), the design-arithmetic reconciliation (declared cells == disposition == denominators), the plan freeze and its drift gate, the rate that needs both a numerator and a denominator, probe self-labeling, and recor