← ClaudeAtlas

agenthog-experimentlisted

Run an A/B test end-to-end with AgentHog as the source of truth — create a feature flag, wire the variant read into the app, verify exposure is flowing, open an experiment window with the right metric and horizon, read results honestly, ship the winner, and retire the flag. Use when setting up an A/B or A/B/n test, ramping a rollout percentage, checking experiment results, or cleaning up finished flags.
AnniesAI/agenthog-claude · ★ 0 · AI & Automation · score 75
Install: claude install-skill AnniesAI/agenthog-claude
# Run an experiment with AgentHog An experiment is an analysis window on a multivariate feature flag. The flag delivers variants (deterministic per-user hashing in the SDK — no assignment service); the experiment compares treatments against a control on a metric and reports each arm's chance to beat it. Everything runs through the `ah` CLI; every mutation needs a write-scope token. Follow the phases in order. Each ends with a check — do not proceed past a failed check. ## 0. Preconditions - The AgentHog SDK is integrated and verified in the target app (events arrive in `ah events`). If not: stop and run the **agenthog-integrate** skill first. - `ah` is authenticated with **write** scope: `ah whoami` shows the project and `scope: write`. If read-only, ask the user to run `ah login --write`. - You know the **decision metric**: one event name that already exists (check `ah events top`) or a goal (`ah goals list`). If the metric event doesn't exist yet, instrument it and verify it arrives before doing anything else. ## 1. Create the flag ``` ah flags create checkout_cta --variants control:50,b:50 --desc "new checkout CTA copy" ``` - Key: lowercase `[a-z0-9_-]`, ≤64 chars. Variant names the same. One arm should be named `control` unless the user says otherwise. - A/B/C: `--variants control:34,b:33,c:33`. Percentage rollout of a non-experiment feature: `ah flags create new_nav` (boolean) then `ah flags rollout new_nav 10`. - Start experiments at full traffic (`--