ab-test-analysis

## A/B Test Analysis Evaluate A/B test results with statistical rigor and translate findings into clear product decisions. ### Context You are analyzing A/B test results for **$ARGUMENTS**. If the user provides data files (CSV, Excel, or analytics exports), read and analyze them directly. Generate Python scripts for statistical calculations when needed. ### Instructions 1. **Understand the experiment**: - What was the hypothesis? - What was changed (the variant)? - What is the primary metric? Any guardrail metrics? - How long did the test run? - What is the traffic split? 2. **Validate the test setup**: - **Sample size**: Is the sample large enough for the expected effect size? - Use the formula: n = (Z²α/2 × 2 × p × (1-p)) / MDE² - Flag if the test is underpowered (<80% power) - **Duration**: Did the test run for at least 1-2 full business cycles? - **Randomization**: Any evidence of sample ratio mismatch (SRM)? - **Novelty/primacy effects**: Was there enough time to wash out initial behavior changes? 3. **Calculate statistical significance**: - **Conversion rate** for control and variant - **Relative lift**: (variant - control) / control × 100 - **p-value**: Using a two-tailed z-test or chi-squared test - **Confidence interval**: 95% CI for the difference - **Statistical significance**: Is p < 0.05? - **Practical significance**: Is the lift meaningful for the business? If the user provides raw data, generate...