CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

ab-test-setup

When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "conversion experiment," "statistical significance," or "test this." For tracking implementation, see analytics-tracking.

64

Quality

81%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

78%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured skill body: lean tabular guidance, a real and correctly documented bundled calculator, explicit launch/analysis checkpoints, and exemplary progressive disclosure into two reference files and a script. The only notable gaps are textbook statistics restatements and the absence of a concrete significance-testing command or error-recovery loop in the analysis phase.

DimensionReasoningScore

Conciseness

The body is largely lean: tables, checklists, short bullets, and a copy-paste hypothesis template, with every section earning its place. Minor over-explanation of concepts Claude already knows keeps it below anchor 5 — e.g., '95% confidence = p-value < 0.05 / Means <5% chance result is random' and the 'Peeking Problem' paragraph restate textbook statistics.

4 / 5

Actionability

Concrete, verified executable guidance: the bundled calculator is invoked with exact flags ('python3 scripts/sample_size_calculator.py --baseline 0.05 --mde 0.20 --json', '--daily-traffic') that match the script's actual argparse, plus a weak-vs-strong hypothesis example and specific metric tables. It falls short of anchor 5 only because the analysis section says 'Check confidence intervals' without a command or formula for actually computing significance.

4 / 5

Workflow Clarity

The lifecycle is clearly sequenced (Initial Assessment → Hypothesis → Design → Sample Size → Pre-Launch Checklist → Running → Analysis → Documentation) with explicit checkpoints: the pre-launch checklist, 'Paste sample_size_per_variation... before any test is approved to run', the DO/DON'T list, and the 6-step analysis checklist. It misses anchor 5 because there is no error-recovery loop (e.g., what to do when QA or tracking verification fails) and the overall sequence is implied by section order rather than stated as one master workflow.

4 / 5

Progressive Disclosure

The body acts as a clean overview with well-signaled, one-level-deep references that all exist in the bundle: 'See [references/sample-size-guide.md](references/sample-size-guide.md)' for detailed tables, 'See [references/test-templates.md](references/test-templates.md)' for templates, and the bulk calculation logic delegated to scripts/sample_size_calculator.py. Content is appropriately split — only the small quick-reference table and usage examples stay inline — matching the anchor-5 example structure.

5 / 5

Total

17

/

20

Passed

Description

73%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong trigger-focused description with excellent natural-language keyword coverage and explicit 'use when' guidance. Its weaknesses are an implicit capability statement (what the skill produces is never named) and overly broad triggers like 'test this' that risk conflicts with code-testing skills.

Suggestions

Lead with a concrete capability sentence naming the deliverables, e.g., 'Designs statistically valid A/B tests: hypothesis framing, sample-size calculation, variant design, and results analysis. Use when...' before the trigger list.

Qualify ambiguous triggers such as 'test this' and 'experiment' with the marketing/conversion context (e.g., '"test this change" when discussing page copy, CTAs, or conversion rates') to reduce conflict with code-testing skills.

Add one or two outcome nouns users search for (e.g., 'sample size', 'test duration', 'experiment brief') to the trigger list to sharpen both specificity and discoverability.

DimensionReasoningScore

Specificity

The description names the domain ('A/B test or experiment') and a small set of actions ('plan, design, or implement'), but these are near-synonymous verbs for one activity and no concrete outputs (test plans, sample-size calculation, variant design, results analysis) are stated. It matches anchor 3 (domain plus 1-2 concrete actions, not comprehensive) better than anchor 4, which expects several distinct specific actions.

3 / 5

Completeness

The 'when' is explicit and rich (two 'use when' clauses with concrete trigger phrases), and a 'what' is present but folded inside the when-clause ('plan, design, or implement an A/B test') rather than stated as a standalone capability list. It answers both what and when but sits below the anchor-5 example, which pairs a full capability sentence with an equally explicit trigger sentence; the delegation note ('For tracking implementation, see analytics-tracking') is a plus.

4 / 5

Trigger Term Quality

Comprehensive natural-language coverage including synonyms: '"A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "conversion experiment," "statistical significance," or "test this." These are phrases a user would naturally say when needing this skill, matching the anchor-5 example's breadth (e.g., 'PDF files, PDFs, forms, document extraction, .pdf').

5 / 5

Distinctiveness Conflict Risk

The A/B-testing vocabulary is distinctive, but generic triggers '"test this," "test this change," "experiment," and "hypothesis" would also naturally occur in code-testing and general engineering contexts, creating overlap risk with test-writing or experimentation-adjacent skills. This matches anchor 3 (somewhat specific but could still overlap) rather than anchor 4's 'minor overlap risk'.

3 / 5

Total

15

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

alirezarezvani/claude-skills

Path

marketing-skill/skills/ab-test-setup/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.