ab-test-plan

Featured

Design a statistically rigorous A/B or multivariate test plan — If/Then/Because hypothesis, control and variant specs, required sample size per variant (absolute vs relative MDE via sample-size-calculator.py), test duration, guardrail metrics, stopping rules, and go/no-go decision criteria. Triggers on "/digital-marketing-pro:ab-test-plan", "set up an A/B test", "how long should my test run", "calculate sample size for an experiment", "is this test result significant". Reads the brand profile and past campaign-tracker results to avoid re-testing validated hypotheses; finished tests are evaluated with significance-tester.py by the cro-specialist agent.

AI & Automation 809 stars 134 forks Updated 4 days ago MIT

Install

View on GitHub

Quality Score: 95/100

Stars 20%
97
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# /digital-marketing-pro:ab-test-plan ## Purpose Dedicated A/B test planning with a structured hypothesis framework, statistical sample size calculation, variant design, and monitoring plan. Produces a complete experiment specification with statistical rigor and clear decision criteria. ## Input Required The user must provide (or will be prompted for): - **Element to test**: The specific page, component, or experience being tested (landing page headline, CTA button, pricing page layout, email subject line, checkout flow, form design, etc.) - **Current conversion rate**: Baseline conversion rate for the metric being tested (or best estimate) - **Desired minimum detectable effect (MDE)**: The smallest improvement worth detecting. **MDE is ABSOLUTE by default** — expressed in the same units as the baseline (baseline 5.0% and you want to catch a +1.0 percentage-point lift, i.e. 5.0% → 6.0% ⇒ `--mde 0.01 --mde-type absolute`). To express it as a **relative** lift instead (a 10% relative improvement on a 5% baseline = 5.5% ⇒ `--mde 0.10 --mde-type relative`), pass `--mde-type relative`. This distinction is the single most common sample-size error: the same "10%" read as absolute vs. relative changes the required sample size by roughly two orders of magnitude (~200×) at a 5% baseline. Always confirm which the user means. - **Daily traffic or impressions**: Average daily visitors or impressions to the test page or element - **Significance level**: Desired confidence level, defau...

Details

Author
indranilbanerjee
Repository
indranilbanerjee/digital-marketing-pro
Created
7 months ago
Last Updated
4 days ago
Language
Python
License
MIT

Integrates with

Bundled in these plugins

Similar Skills

Semantically similar based on skill content — not just same category

Testing & QA Listed

ab-test-plan

Design a statistically rigorous A/B or multivariate test plan — If/Then/Because hypothesis, control and variant specs, required sample size per variant (absolute vs relative MDE via sample-size-calculator.py), test duration, guardrail metrics, stopping rules, and go/no-go decision criteria. Triggers on "/digital-marketing-pro:ab-test-plan", "set up an A/B test", "how long should my test run", "calculate sample size for an experiment", "is this test result significant". Reads the brand profile and past campaign-tracker results to avoid re-testing validated hypotheses; finished tests are evaluated with significance-tester.py by the cro-specialist agent.

0 Updated 3 weeks ago
teachskillofskills-ai
AI & Automation Featured

ab-test-planner

Design statistically rigorous A/B tests for product features, UI changes, onboarding flows, and pricing experiments. Use when asked to set up an experiment, design an A/B test, calculate sample size, or interpret test results. Produces a complete test plan with hypothesis, variant definitions, sample size, duration estimate, guardrail metrics, and a results interpretation guide.

1,356 Updated yesterday
mohitagw15856
AI & Automation Featured

ab-test-setup

When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "conversion experiment," "statistical significance," or "test this." For tracking implementation, see analytics-tracking.

25,834 Updated 1 weeks ago
alirezarezvani