send-experiment-designer

Featured

Use when the user asks to "design an email A/B test", "set up a multivariate subject/CTA test", "run a send-time test", "build a hold-out group", or "is this email result statistically and practically material?"; produces a falsifiable hypothesis, one-variable-per-cell matrix, sample-size/MDE/duration/power plan, and an effect/uncertainty read from own ESP data. Applies only a precommitted owner-approved action rule; the helper never chooses a business action. Not for EQS/vetoes or writing the email. 邮件AB测试设计/多变量测试/发送时间测试/留出组/显著性判定

Web & Frontend 2,766 stars 358 forks Updated today Apache-2.0

Install

View on GitHub

Quality Score: 98/100

Stars 20%
100
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
80
License 10%
100
Description 5%
100

Skill Content

# Send Experiment Designer Designs email experiments across four modes and reads them out: a falsifiable hypothesis, a variant matrix that isolates **one** variable per cell, a sample-size / minimum-detectable-effect / run-duration / power plan, and a documented effect/uncertainty read. It may apply an owner-approved precommitted action rule, but statistical output alone never chooses a business action. **Mode set (pick one):** | Mode | Isolated variable | Primary metric | |------|-------------------|----------------| | `a-b` | one change — subject *or* preheader *or* CTA *or* creative | open (subject) / click / CTOR (CTA/creative) | | `multivariate` | 2+ factors crossed (e.g. subject × CTA), one variable per cell | the goal metric, powered per cell | | `send-time` | deploy hour/day; subject, segment, creative held constant | same-window engagement (open/click) | | `hold-out` | send vs no-send (randomized control receives nothing / current default) | conversion or revenue-per-recipient (incremental lift) | Default the mode from the request when it is unambiguous (e.g. "test two subject lines" → `a-b`, "best hour to send" → `send-time`, "measure incremental revenue" → `hold-out`); state the picked mode back and proceed. **Scope guard:** this skill owns email **experiment design + the significance read** only. It scores the SEND **E (Engagement)** lever as a test signal — it does **not** compute the profile-weighted **EQS** or run the `S1/S2/N1/D1` vetoes ([email-quality-a...

Details

Author
aaron-he-zhu
Repository
aaron-he-zhu/aaron-marketing-skills
Created
8 months ago
Last Updated
today
Language
Python
License
Apache-2.0

Bundled in these plugins

Similar Skills

Semantically similar based on skill content — not just same category

Web & Frontend Featured

ad-test-designer

Use when the user asks to "design an A/B test", "set up a creative/landing test", "run an incrementality test", or "is this result statistically and practically material?"; produces a hypothesis, variant matrix, sample-size/duration/power plan, and a documented effect/uncertainty read from own exported results. It applies only a precommitted owner-approved action rule; the statistical helper never chooses a business action. Not for producing variants — use ad-creative-builder; not for reading back one shipped change — use paid-measurement-loop. 广告AB测试设计/实验设计/显著性判定/增效测试

2,766 Updated today
aaron-he-zhu
Web & Frontend Listed

experiment-designer

Turn an assumption or a growth bet into an experiment brief with a mechanism-stated hypothesis, one primary metric, guardrails with floors, an exposure design, a sample size reasoned from the minimum detectable effect, and stop rules written before launch. Use when a growth plan bet needs a test, when an assumption register row is low confidence and high impact, when a price change should be tried on a slice first, or when someone says "let us just A/B it". Takes the assumption, the metric candidate with its baseline, the eligible traffic, and the guardrail candidates; returns the test card, the filled brief, the sizing record with its calculator inputs, and the pre-committed decision rule.

0 Updated today
RizwanZafaris
AI & Automation Featured

experiment-design

A discipline for designing experiments (A/B tests, multivariate, holdouts) so the results actually answer the question you asked. Hypothesis writing, sample size, duration, segment analysis, running discipline, matching a result to a pre-committed decision rule, and the common failure modes that produce confidently wrong shipping decisions. Use this skill whenever the user is planning a test that has not run yet: framing a hypothesis, sizing the sample, setting duration, choosing guardrails, or deciding whether something is worth testing at all. Triggers on design an experiment, experiment plan, A/B test, split test, multivariate test, holdout, experiment hypothesis, sample size, minimum detectable effect, MDE, test duration, guardrail metric, no peeking, pre-committed decision rule, is this worth testing. Use `experimentation-analytics` instead when the test has already run and the question is how to read the result panel.

843 Updated 3 days ago
rampstackco