ad-test-designer

Featured

Use when the user asks to "design an A/B test", "set up a creative/landing test", "run an incrementality test", or "is this result statistically and practically material?"; produces a hypothesis, variant matrix, sample-size/duration/power plan, and a documented effect/uncertainty read from own exported results. It applies only a precommitted owner-approved action rule; the statistical helper never chooses a business action. Not for producing variants — use ad-creative-builder; not for reading back one shipped change — use paid-measurement-loop. 广告AB测试设计/实验设计/显著性判定/增效测试

Web & Frontend 2,465 stars 335 forks Updated today Apache-2.0

Install

View on GitHub

Quality Score: 99/100

Stars 20%
100
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Ad Test Designer Designs paid-ad creative/landing A/B/n and incrementality tests and reads them out: hypothesis, variant matrix, sample-size/duration/power plan, effect size, uncertainty, practical-effect status, and guardrail state. This skill owns **experiment design + statistical interpretation**. It may apply an owner-approved, precommitted action rule, but it never treats a p-value or helper output as an automatic business decision. It does not produce variants (`ad-creative-builder`), read back one already-shipped change (`paid-measurement-loop`), or do cross-channel reporting (`performance-analyzer`). ## Quick Start ```text Design an A/B test for two landing-page hero variants. Baseline CVR is 3%, I want to detect a 15% lift. Goal is DR. ``` ```text I have 4 RSA creative variants to test on a prospecting set. Build the variant matrix, sample size, and run duration. ``` ```text Here's my finished test results CSV (variant, sessions, conversions). Is the winner significant — promote or kill? ``` ## Skill Contract - **Expected output**: a test design (hypothesis, variant matrix, primary/secondary/guardrail metrics, sample-size + duration + power plan) **and/or** a read-out (effect estimate, interval, statistical flag, practical-effect flag, guardrails, and either an owner-governed recommendation or `decision: UNDECIDED`). - **Reads**: what the user wants to test, the ROAS profile (`direct-response|prospecting|incremental-profit`), baseline CVR/CTR and traffic volum...

Details

Author
aaron-he-zhu
Repository
aaron-he-zhu/aaron-marketing-skills
Created
7 months ago
Last Updated
today
Language
Python
License
Apache-2.0

Bundled in these plugins

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Solid

experiment

Design, run, and evaluate a controlled A/B test (split test) on your ad accounts with evidence discipline — one variable, a falsifiable hypothesis, a fixed window, and a per-variant outcome verdict. Use when the user asks to run an A/B test, split test, or experiment, to 'test whether X beats Y', to validate a creative-refresh or learning hunch properly, or requests A/Bテスト / スプリットテスト設計 / 実験を回したい / どちらが勝ったか評価して / 仮説を検証したい. Forces a designed experiment instead of an ad-hoc change, records the baseline in action_log, and forbids peeking-based decisions before the window closes.

16 Updated today
logly
Web & Frontend Listed

a-b-test-designer

You are a conversion optimization expert. When given a conversion problem, design statistically valid A/B tests with clear hypotheses, variants, and success metrics. ## Process 1. Identify the conversion problem and current metrics 2. Formulate a clear, testable hypothesis 3. Design control and variant(s) 4. Define success metrics and statistical significance 5. Estimate sample size and test duration ## Output Format ## A/B Test Design ### Problem \[Current conversion rate and goal\] ### Hypothesis 'If we \[change\], then \[metric\] will improve because \[reasoning\].' ### Variants - Control (A): Current design - Variant (B): \[Specific change description\] ### Success Metrics - Primary: \[Main metric to track\] - Secondary: \[Supporting metrics\] - Guardrail: \[Metrics that shouldn't decrease\] ### Statistical Plan - Confidence level: 95% - Minimum detectable effect: X% - Estimated...

0 Updated 1 weeks ago
prvthmpcypher
AI & Automation Featured

ab-test-plan

Design A/B and multivariate tests. Use when: sample size calculation, testing hypothesis, CRO experimentation.

636 Updated 1 weeks ago
indranilbanerjee