Name: ab-test-setup
Rating: 64 (1 reviews)
Author: alirezarezvani

ab-test-setup

When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "conversion experiment," "statistical significance," or "test this." For tracking implementation, see analytics-tracking.

Quality

77%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./.gemini/skills/ab-test-setup/SKILL.md

The canonical home for this skill is ab-test-setup in alirezarezvani/claude-skills

Quality

Content

63%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is well-structured and largely actionable with concrete commands, templates, and checklists, but it loses points for restating statistics basics an expert already knows and, more seriously, for referencing three bundle files (a script and two reference docs) that are not actually present.

Suggestions

Ship the missing bundle files the body depends on — scripts/sample_size_calculator.py, references/sample-size-guide.md, and references/test-templates.md — or remove the references and inline only what is essential.

Cut or move the beginner-explanation sections (Statistical Significance, The Peeking Problem, Common Mistakes) into a reference doc; assume Claude already knows these concepts.

Move the large sample-size quick-reference table into references/sample-size-guide.md and keep only the calculator command and a one-line pointer in SKILL.md.

Dimension	Reasoning	Score
Conciseness	The body is mostly efficient (tables, checklists, command blocks), but several sections restate A/B-testing basics Claude already knows: "95% confidence = p-value < 0.05," "The Peeking Problem," and the Common Mistakes list.	3 / 5
Actionability	Concrete, copy-paste-ready bash commands for sample_size_calculator.py plus a hypothesis template, tables of real numbers, and checklists — but the referenced calculator script does not exist in the bundle, so the commands cannot actually execute.	4 / 5
Workflow Clarity	A clear sequenced flow (Assessment → Hypothesis → Sample Size → Metrics → Variants → Implementation → Running → Analysis → Documentation) with pre-launch and analysis checklists serving as validation gates, though no explicit validate→fix→retry feedback loop is stated.	4 / 5
Progressive Disclosure	Internal section structure and markdown-linked references are clearly signaled, but the referenced files (scripts/sample_size_calculator.py, references/sample-size-guide.md, references/test-templates.md) do not exist in the bundle, and substantial reference material (sample-size table, test-type tables) is inlined rather than split out.	3 / 5
	Total	14 / 20 Passed

Description

91%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is strong: it covers what the skill does, when to invoke it, and a broad set of natural trigger terms, plus a cross-reference that reduces overlap with analytics-tracking. The only soft spot is that a few generic triggers ("test this") could conflict with non-experimentation skills.

Suggestions

Tighten broad triggers like "test this" to an experimentation context (e.g., "test this change on a page") to reduce false matches against code-testing skills.

Make the "what" actions slightly more concrete (e.g., "calculate sample size, define metrics, document the experiment brief") to lift specificity toward the 5-anchor.

Dimension	Reasoning	Score
Specificity	"plan, design, or implement an A/B test or experiment" names the domain and lists several concrete actions across the experimentation lifecycle, though the verbs are slightly more abstract than the rubric's 5-anchor examples.	4 / 5
Completeness	Explicitly answers both what ("plan, design, or implement an A/B test") and when ("Also use when the user mentions...") with concrete trigger phrases.	5 / 5
Trigger Term Quality	Comprehensive natural terms including synonyms ("A/B test," "split test," "multivariate test") and phrases users actually say ("test this change," "test this," "statistical significance").	5 / 5
Distinctiveness Conflict Risk	A clear A/B-testing niche with a boundary clarification ("For tracking implementation, see analytics-tracking"), though broad triggers like "test this" / "experiment" carry minor overlap risk with general code-testing skills.	4 / 5
	Total	18 / 20 Passed

Validation

87%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

Criteria	Description	Result
relative_links	Relative link issues: 2 missing	Warning
referenced_paths_exist	Referenced path issues: 7 missing	Warning

	Total	14 / 16 Passed

Repository: alirezarezvani/claude-skills
Path: .gemini/skills/ab-test-setup/SKILL.md
Commit: 19392f7

Reviewed: ~1 hour ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.