ad-test-designer

Featured

Use when the user asks to "design an A/B test", "set up a creative/landing test", "run an incrementality test", or "is this result statistically and practically material?"; produces a hypothesis, variant matrix, sample-size/duration/power plan, and a documented effect/uncertainty read from own exported results. It applies only a precommitted owner-approved action rule; the statistical helper never chooses a business action. Not for producing variants — use ad-creative-builder; not for reading back one shipped change — use paid-measurement-loop. 广告AB测试设计/实验设计/显著性判定/增效测试

Web & Frontend 2,766 stars 358 forks Updated today Apache-2.0

Install

View on GitHub

Quality Score: 98/100

Stars 20%
100
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
80
License 10%
100
Description 5%
100

Skill Content

# Ad Test Designer Designs paid-ad creative/landing A/B/n and incrementality tests and reads them out: hypothesis, variant matrix, sample-size/duration/power plan, effect size, uncertainty, practical-effect status, and guardrail state. This skill owns **experiment design + statistical interpretation**. It may apply an owner-approved, precommitted action rule, but it never treats a p-value or helper output as an automatic business decision. It does not produce variants (`ad-creative-builder`), read back one already-shipped change (`paid-measurement-loop`), or do cross-channel reporting (`performance-analyzer`). ## Quick Start ```text Design an A/B test for two landing-page hero variants. Baseline CVR is 3%, I want to detect a 15% lift. Goal is DR. ``` ```text I have 4 RSA creative variants to test on a prospecting set. Build the variant matrix, sample size, and run duration. ``` ```text Here's my finished test results CSV (variant, sessions, conversions). Is the winner significant — promote or kill? ``` ## Skill Contract - **Expected output**: a test design (hypothesis, variant matrix, primary/secondary/guardrail metrics, sample-size + duration + power plan) **and/or** a read-out (effect estimate, interval, statistical flag, practical-effect flag, guardrails, and either an owner-governed recommendation or `decision: UNDECIDED`). - **Reads**: what the user wants to test, the ROAS profile (`direct-response|prospecting|incremental-profit`), baseline CVR/CTR and traffic volum...

Details

Author
aaron-he-zhu
Repository
aaron-he-zhu/aaron-marketing-skills
Created
8 months ago
Last Updated
today
Language
Python
License
Apache-2.0

Bundled in these plugins

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Listed

ad-test-designer

Use when the user asks to 'design an A/B test', 'set up a creative/landing test', 'run an incrementality test', or 'is this result statistically and practically material?'; produces a hypothesis, variant matrix, sample-size/duration/power plan, and a documented effect/uncertainty read from own exported results.

1 Updated 1 months ago
imMamdouhaboammar
Web & Frontend Listed

experiment-designer

Turn an assumption or a growth bet into an experiment brief with a mechanism-stated hypothesis, one primary metric, guardrails with floors, an exposure design, a sample size reasoned from the minimum detectable effect, and stop rules written before launch. Use when a growth plan bet needs a test, when an assumption register row is low confidence and high impact, when a price change should be tried on a slice first, or when someone says "let us just A/B it". Takes the assumption, the metric candidate with its baseline, the eligible traffic, and the guardrail candidates; returns the test card, the filled brief, the sizing record with its calculator inputs, and the pre-committed decision rule.

0 Updated today
RizwanZafaris
AI & Automation Listed

ab-test-analyzer

Designs and analyzes A/B tests end-to-end — frames a sharp hypothesis, computes required sample size and test duration, runs significance tests (two-proportion z-test, Welch's t-test, chi-square), reports confidence intervals and lift, and flags common pitfalls like peeking, multiple comparisons, and Simpson's paradox. Use this skill when the user mentions A/B testing, split testing, experiment design, conversion-rate experiments, statistical significance, p-values, sample size or power calculations, "did this experiment win", minimum detectable effect (MDE), or asks to interpret experiment results honestly. Includes a runnable stats script (scripts/abtest.py).

5 Updated 1 months ago
JayRHa