ab-test-setup

When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," or "hypothesis." For tracking implementation, see analytics-tracking.

Quality

62%

Does it follow best practices?

Impact

—

No eval scenarios have been run

Securityby

Passed

No known issues

Optimize this skill with Tessl

npx tessl skill review --optimize ./config/claude/skills/ab-test-setup/SKILL.md

Quality

Content

62%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

This is a comprehensive A/B testing guide with strong workflow structure and useful reference tables, but it leans verbose for a skill file—explaining concepts Claude already knows (statistical significance, why peeking is bad) and lacking concrete implementation code. The progressive disclosure structure is partially implemented with references to external files, but those files don't exist in the bundle, and the main file retains too much detail that could be offloaded.

Suggestions

Remove explanations of concepts Claude already knows (statistical significance definitions, why single-variable testing matters, what the peeking problem is) to improve conciseness.

Add concrete, executable code examples for at least one testing tool (e.g., PostHog feature flag setup, LaunchDarkly SDK snippet) to improve actionability.

Move detailed reference tables (sample size, common mistakes, metrics examples) into bundle reference files and keep only summaries in the main SKILL.md to improve progressive disclosure.

Provide the referenced bundle files (references/sample-size-guide.md, references/test-templates.md) or remove the broken references.

Dimension	Reasoning	Score
Conciseness	The skill is reasonably well-organized but includes some unnecessary explanations Claude already knows (e.g., explaining what statistical significance means, explaining the peeking problem in detail, defining test types). The tables are efficient, but sections like 'Core Principles' explain basic experimentation concepts that Claude would already understand.	2 / 3
Actionability	The skill provides structured frameworks (hypothesis template, checklists, tables) which are useful, but lacks executable code or concrete implementation commands. The implementation section mentions tools but gives no actual code snippets for setting up tests in any of them. The guidance is more conceptual/procedural than copy-paste ready.	2 / 3
Workflow Clarity	The skill presents a clear end-to-end workflow from initial assessment through hypothesis formation, design, implementation, running, and analysis. It includes explicit checklists (pre-launch, analysis), DO/DON'T lists during execution, and a clear decision framework for interpreting results. The sequential flow is well-defined with validation checkpoints.	3 / 3
Progressive Disclosure	The skill references two external files (references/sample-size-guide.md and references/test-templates.md) and related skills, which is good structure. However, no bundle files are provided, meaning these references are broken. Additionally, the main file is quite long (~200+ lines) and some content (like the full sample size table, common mistakes, detailed analysis guidance) could be split into reference files to keep the overview leaner.	2 / 3
	Total	9 / 12 Passed

Description

62%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description excels at trigger term coverage and distinctiveness, with strong 'when to use' guidance and helpful cross-referencing to related skills. However, it is weak on specificity — it fails to describe what the skill actually does in concrete terms (e.g., generates hypothesis documents, creates test plans, writes variant copy, calculates sample sizes). The 'what' portion needs significant improvement.

Suggestions

Add specific concrete actions the skill performs, e.g., 'Generates hypothesis documents, creates experiment plans with success metrics, writes variant copy, calculates required sample sizes, and designs test frameworks.'

Restructure to lead with concrete capabilities before the trigger terms, following the pattern: '[What it does]. Use when [triggers].'

Dimension	Reasoning	Score
Specificity	The description mentions 'plan, design, or implement an A/B test or experiment' but these are very high-level actions without concrete specifics. It doesn't list what the skill actually does — no specific outputs, deliverables, or concrete capabilities are named.	1 / 3
Completeness	The 'when' is very well covered with explicit trigger terms and a 'Use when' equivalent clause. However, the 'what' is weak — it says 'plan, design, or implement' but doesn't describe what concrete outputs or actions the skill performs. The cross-reference to analytics-tracking is a nice touch for disambiguation.	2 / 3
Trigger Term Quality	Excellent coverage of natural trigger terms: 'A/B test,' 'split test,' 'experiment,' 'test this change,' 'variant copy,' 'multivariate test,' 'hypothesis.' These are terms users would naturally use when needing this skill.	3 / 3
Distinctiveness Conflict Risk	The description carves out a clear niche around A/B testing and experimentation, and explicitly disambiguates from analytics-tracking for tracking implementation. The trigger terms are specific enough to avoid conflicts with other skills.	3 / 3
	Total	9 / 12 Passed

Validation

90%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 10 / 11 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	10 / 11 Passed

Repository: freekmurze/dotfiles
Commit: ef5ffa0

Reviewed: 3 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.