ab-test-setup
When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," or "hypothesis." For tracking implementation, see analytics-tracking.
81
86%
Does it follow best practices?
Impact
81%
1.50xAverage score across 2 eval scenarios
Passed
No findings from the security scan
Evaluation results
63%
14%Checkout Flow Experiment: Feasibility Assessment and Design
Multi-variant A/B test feasibility and design assessment
1.5x multiplier applied
8%
100%
Bonferroni correction mentioned
25%
87%
Duration estimate calculated
100%
100%
4–8 week limit flagged
20%
70%
Multiple alternatives presented
90%
50%
Reduce variants as option
100%
0%
Sequential testing tool named
0%
20%
Single-variable violation flagged
13%
13%
Segment sizing to smallest sub-population
93%
93%
Variant detectability assessed
80%
80%
100%
40%Pricing Page Experiment Plan
A/B test plan for pricing page billing toggle
Hypothesis: Because clause
0%
100%
Hypothesis: We believe clause
0%
100%
Hypothesis: Will cause clause
0%
100%
Hypothesis: For audience clause
0%
100%
Hypothesis: We'll know clause
50%
100%
Guardrail metrics
100%
100%
Sample size ~18k/variant
0%
100%
Minimum 7-day duration
100%
100%
Segment analysis plan
100%
100%
50/50 traffic split
100%
100%
Checklist: hypothesis documented
66%
100%
Checklist: primary metric defined
66%
100%
Checklist: sample size calculated
100%
100%
Checklist: variants implemented
100%
100%
Checklist: tracking verified
100%
100%
Checklist: QA completed
100%
100%
Success criteria section
100%
100%
- Repository
- TheCraigHewitt/seomachine
- Commit
e818d5e
- Evaluated
- Agent
- Claude Code
- Model
- Claude Sonnet 4.6
Table of Contents
Is this your skill?
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.