CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

ab-test-setup

When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "conversion experiment," "statistical significance," or "test this." For tracking implementation, see analytics-tracking.

62

Quality

78%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./.gemini/skills/ab-test-setup/SKILL.md

The canonical home for this skill is ab-test-setup in alirezarezvani/claude-skills

SKILL.md
Quality
Evals
Security

Quality

Content

63%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is a well-organized, largely actionable A/B-testing playbook with strong lifecycle sequencing, checklists, and a concrete hypothesis framework. Its two real weaknesses are the missing bundle files — the calculator script both reference links point to files that are not present — and a few spots that re-teach statistics Claude already knows.

Suggestions

Ship the referenced bundle files (scripts/sample_size_calculator.py, references/sample-size-guide.md, references/test-templates.md) or remove the 'bundled tool' framing and dead links — as written, the 'don't eyeball it, use this calculator' instruction cannot be executed.

Trim textbook-statistics asides Claude already knows ('95% confidence = p-value < 0.05', 'Means <5% chance result is random') and fold 'The Peeking Problem' paragraph into the existing DON'T list to tighten conciseness.

Add an explicit error-recovery loop for the pre-launch phase (e.g., 'if tracking verification or variant QA fails, fix and re-verify before launching') to complete the workflow's validation checkpoints.

DimensionReasoningScore

Conciseness

The body is dense and checklist/table-driven with almost no padding — hypothesis template, weak/strong example, test-type table, checklists, interpretation tables. Only minor instances of over-explaining what Claude already knows: '95% confidence = p-value < 0.05 / Means <5% chance result is random' and 'The Peeking Problem' paragraph restating the DON'T list. This fits anchor 4 (efficient, minor instances that could be trimmed) rather than anchor 3, since the textbook-statistics asides are a small fraction of 300 otherwise-earning-their-place lines.

4 / 5

Actionability

Guidance is largely concrete — copy-paste calculator commands ('python3 scripts/sample_size_calculator.py --baseline 0.05 --mde 0.20 --json'), a fill-in hypothesis template, explicit DO/DON'T lists, and a 6-item analysis checklist — but the centerpiece executable instruction ('Use this skill's own calculator — don't eyeball it') targets a script that does not exist in the bundle, so running it verbatim fails. This sits between anchor 3 (concrete but incomplete, missing key details) and anchor 4; the broken primary tool tips it to 3 rather than 4.

3 / 5

Workflow Clarity

The sections sequence a full lifecycle (assess → hypothesis → design → sample size → implement → run → analyze → document) with explicit checkpoints in the Pre-Launch Checklist, the DO/DON'T during-test rules, and the ordered Analysis Checklist ('Reach sample size?', 'Statistically significant?', 'Guardrail concerns?'). Anchor 5 requires explicit error-recovery feedback loops, which are absent (e.g., no 'if tracking verification fails, fix before launch' loop), so anchor 4 (clear sequence, most checkpoints, minor validation gaps) is the best fit.

4 / 5

Progressive Disclosure

Structure is good on paper: quick-reference tables inline, detailed material delegated one level deep and clearly signaled ('For detailed sample size tables and duration calculations: See references/sample-size-guide.md', 'For templates: See references/test-templates.md'). However, the referenced files — references/sample-size-guide.md, references/test-templates.md, and scripts/sample_size_calculator.py — do not exist in the bundle, so the promised navigation dead-ends. Well-signaled but non-functional references place this at anchor 3 rather than anchor 4.

3 / 5

Total

14

/

20

Passed

Description

86%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description with excellent, exhaustive trigger-term coverage and an explicit what/when structure that follows best-practice patterns. Its only weakness is that the capability statement relies on generic verbs ('plan, design, or implement') rather than naming the skill's specific deliverables.

DimensionReasoningScore

Specificity

The description names the domain and the actions 'plan, design, or implement an A/B test or experiment', but these are generic verbs — it never enumerates the skill's concrete capabilities (hypothesis framing, sample-size calculation, variant design, results analysis). This matches anchor 3 (domain plus 1-2 actions, not comprehensive) better than anchor 4, which expects several specific listed actions; it also exceeds anchor 2 since the domain is named precisely.

3 / 5

Completeness

Both what and when are explicit: the what ('plan, design, or implement an A/B test or experiment') and the when ('When the user wants to...' plus 'Also use when the user mentions...' followed by concrete trigger phrases). It parallels the anchor-5 example structure exactly; anchor 4 would require the when to be less explicit, which it is not.

5 / 5

Trigger Term Quality

Comprehensive natural-term coverage with synonyms: "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "conversion experiment," "statistical significance," and "test this" — phrases a user would naturally say. This matches anchor 5 (comprehensive coverage including synonyms) and clearly exceeds anchor 4 (a few terms missing).

5 / 5

Distinctiveness Conflict Risk

The A/B-testing niche is clear with distinct triggers, but broad phrases like "test this," "experiment," and "hypothesis" could overlap with code-testing or general analytics skills (minor overlap risk), matching anchor 4. The explicit boundary handoff ('For tracking implementation, see analytics-tracking') reduces conflict, keeping it above anchor 3 but below anchor 5's minimal-conflict bar.

4 / 5

Total

17

/

20

Passed

Validation

87%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

relative_links

Relative link issues: 2 missing

Warning

referenced_paths_exist

Referenced path issues: 7 missing

Warning

Total

14

/

16

Passed

Repository

alirezarezvani/claude-skills

Path

.gemini/skills/ab-test-setup/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.