ab-test-setup

When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," or "hypothesis." For tracking implementation, see analytics-tracking.

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

78%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, actionable instruction-only skill that moves the user through the full A/B test lifecycle with concrete templates, tables, and checklists, and cleanly offloads detail to two reference files. It would benefit from de-duplicating the peeking guidance and tightening the conceptual asides.

Suggestions

Remove the duplicated peeking discussion: keep it once (either 'The Peeking Problem' or the DON'T bullet) and cross-reference instead of restating.

Tighten 'Statistical Significance' to drop the explanatory gloss ('Means <5% chance result is random, Not a guarantee—just a threshold') which Claude already knows.

Add an explicit go/no-go gate after the pre-launch checklist so the workflow has an unmistakable validation checkpoint before launch.

Dimension	Reasoning	Score
Conciseness	Largely efficient with tables, checklists, and bullets, but has minor trimmable repetition (peeking restated under both 'Running the Test' and 'The Peeking Problem') and light over-explanation of a known concept (95% confidence).	4 / 5
Actionability	Concrete, usable artifacts (fill-in hypothesis template, sample-size quick-reference table, pre-launch/analysis checklists, test-type and traffic-allocation tables), with a few higher-level 'best practices' bullets keeping it just below fully executable.	4 / 5
Workflow Clarity	Clear phased sequence from initial assessment through documentation with pre-launch and analysis checklists acting as checkpoints; not a 5 because there is no explicit go/no-go gate and the analysis checklist lacks a feedback loop.	4 / 5
Progressive Disclosure	Overview body points clearly to two real one-level-deep references (references/sample-size-guide.md and references/test-templates.md), both well-signaled with 'See [references/...]' phrasing and appropriately splitting detail (sample-size tables, templates) from the overview.	5 / 5
	Total	17 / 20 Passed

Description

95%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description that crisply states capabilities and gives a rich, natural-language trigger list with explicit boundary guidance to a sibling skill. Minor improvement would be enumerating a few more concrete deliverables, but it already hits the top anchors for completeness and triggers.

Dimension	Reasoning	Score
Specificity	Names the domain and several concrete actions ('plan, design, or implement an A/B test or experiment'), with minor gaps in coverage rather than a comprehensive action list.	4 / 5
Completeness	Explicitly answers both 'what' (plan/design/implement A/B tests) and 'when' (concrete 'Also use when the user mentions...' trigger list), matching the anchor for clearly answering both with concrete triggers.	5 / 5
Trigger Term Quality	Comprehensive coverage of natural trigger phrases including synonyms ('A/B test,' 'split test,' 'experiment,' 'test this change,' 'variant copy,' 'multivariate test,' 'hypothesis') that users would actually say.	5 / 5
Distinctiveness Conflict Risk	Clear experimentation niche with distinct triggers, and explicit boundary delegation ('For tracking implementation, see analytics-tracking') minimizes conflict with adjacent skills.	5 / 5
	Total	19 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: freekmurze/dotfiles
Path: config/claude/skills/ab-test-setup/SKILL.md
Commit: a7899fe

Reviewed: 3 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.