ab-test-setup

Use when designing an A/B or split test: define the hypothesis, control and variants, estimate sample size, verify tracking, and predeclare metrics and stopping rules.

Quality

81%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

70%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A rigorous, well-sequenced experimentation guide with strong validation gates and an executable sample-size example. Its main weaknesses are redundancy across admonition sections and a lack of file-level progressive disclosure for a skill of this length.

Suggestions

Consolidate the 'Key Principles (Non-Negotiable)' recap and the repeated MUST/DO NOT language into the existing gates to remove redundancy and trim length.

Move the full sample-size calculation and/or worked example into a separate references file (e.g. references/sample-size.md) linked from the main body, since the skill exceeds the 50-line simple-skill threshold.

Add a concrete, executable check or code snippet for sample-ratio mismatch (SRM) verification rather than leaving it as a qualitative instruction.

Dimension	Reasoning	Score
Conciseness	Mostly efficient and free of conceptual fluff, but redundant sections (e.g. 'Key Principles (Non-Negotiable)' rehashing earlier gates) and repeated MUST/DO NOT emphasis could be tightened.	3 / 5
Actionability	Provides a copy-paste-ready Python sample with expected output plus concrete tracking checks and refusal conditions, though some verification steps (SRM, guardrail dashboards) remain qualitative rather than executable.	4 / 5
Workflow Clarity	Clear numbered sequence with explicit validation gates (Hypothesis Lock, Tracking Verification before Gate 8, Execution Readiness Hard Stop), feedback loops ('stop and resolve'), and checklists throughout.	5 / 5
Progressive Disclosure	Internal sections are well-headed and navigable, but the ~290-line skill is monolithic with no external reference files and inlines the sample-size and worked examples that could be split out.	3 / 5
	Total	15 / 20 Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, well-scoped description that explicitly pairs a 'Use when' trigger with a comprehensive list of concrete actions, using natural domain terms and a synonym pair. Keyword coverage could be marginally broader but is otherwise excellent.

Dimension	Reasoning	Score
Specificity	Lists five concrete actions (define hypothesis/control/variants, estimate sample size, verify tracking, predeclare metrics and stopping rules), giving comprehensive coverage of the setup workflow.	5 / 5
Completeness	Explicitly answers both 'when' ('Use when designing an A/B or split test') and 'what' (the colon-separated action list) with concrete trigger phrases.	5 / 5
Trigger Term Quality	Strong natural coverage with the synonym pair 'A/B or split test' plus 'hypothesis' and 'sample size', but common synonyms like 'experiment', 'conversion rate', and 'statistical power' are absent.	4 / 5
Distinctiveness Conflict Risk	Targets a clear, narrow niche (A/B/split test setup) with distinct triggers and minimal overlap risk with other skills.	5 / 5
	Total	19 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: sickn33/agentic-awesome-skills
Path: skills/ab-test-setup/SKILL.md
Commit: b1aebac

Reviewed: ~18 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.