CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

ab-test-setup

Plan, design, or implement an A/B test or experiment. Includes methodology, statistical significance, and metric selection.

72

1.27x
Quality

75%

Does it follow best practices?

Impact

97%

1.27x

2 of 3 eval scenarios. Add 1 more for a full score.

SecuritybySnyk

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./plugins/AI-Agents-Safe-Coding-Skills/skills/ab-test-setup/SKILL.md
SKILL.md
Quality
Evals
Security

Quality

Content

71%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, largely actionable experimentation guide with excellent progressive disclosure — the overview stays lean while detailed tables and templates live in two real, clearly linked reference files. The main drag is conciseness: it explains basics Claude already knows (p-values, the peeking problem) and the Common Mistakes section repeats earlier principles, and the Implementation section could name concrete setup steps for its tools.

Suggestions

Cut concept explanations Claude already knows — the p-value definition under "Statistical Significance" and the "The Peeking Problem" paragraph can be reduced to one-line rules ("pre-commit to sample size; do not stop early").

Deduplicate "Common Mistakes" against "Core Principles" and "During the Test" (stopping early, changing mid-test, and no-hypothesis each appear twice), or fold them into those sections.

Make the Implementation section actionable: add concrete setup steps or a minimal code/config example for one tool (e.g., a PostHog feature-flag or Optimizely experiment snippet) instead of only naming tools.

DimensionReasoningScore

Conciseness

Tables and checklists are mostly tight, but the body explains concepts Claude already knows ("95% confidence = p-value < 0.05. Means <5% chance result is random", the "The Peeking Problem" rationale paragraph, "Otherwise you don't know what worked"), and the "Common Mistakes" section largely duplicates "Core Principles" and "During the Test". Matches anchor 3; not 4 because the over-explanation is more than minor.

3 / 5

Actionability

Concrete instruction-only guidance: a fill-in hypothesis template with weak/strong examples, a sample-size quick-reference table with real numbers, named calculators, per-category variant tables, and checklists. Matches anchor 4; not 5 because the Implementation section names tools (PostHog, Optimizely, LaunchDarkly) without concrete setup steps or commands, and not 3 because the guidance goes well beyond high-level hints.

4 / 5

Workflow Clarity

The full lifecycle is clearly sequenced (Initial Assessment -> Hypothesis -> Design -> Sample Size -> Pre-Launch Checklist -> During the Test -> Analyzing Results -> Documentation) with an explicit pre-launch checklist and a conditional 6-step analysis checklist that catches errors ("Reach sample size? If not, result is preliminary"). Matches anchor 4; not 5 because the end-to-end sequence is implicit in section order rather than stated, and during-test monitoring offers no recovery path for technical issues.

4 / 5

Progressive Disclosure

The SKILL.md is a well-organized ~260-line overview that keeps a quick sample-size table inline and pushes detail to clearly signaled one-level-deep references ("For detailed sample size tables and duration calculations: See references/sample-size-guide.md"; "For templates: See references/test-templates.md"), both of which exist and deliver the promised content. Matches anchor 5: clear overview, well-signaled references, appropriate split, easy navigation.

5 / 5

Total

16

/

20

Passed

Description

66%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A clear, domain-specific description that names concrete actions and sub-topics, but it omits any 'Use when...' trigger guidance and misses common synonyms like 'split test' and 'multivariate'. Adding an explicit when-to-use clause with natural trigger phrases would lift completeness and trigger quality.

Suggestions

Append an explicit trigger clause, e.g. "Use when the user mentions A/B tests, split tests, multivariate tests, experiments, hypothesis testing, statistical significance, or which version performs better."

Include common synonyms and phrasings ("split test", "multivariate test", "hypothesis") in the description so it matches the natural language users actually say.

Make the action list slightly more concrete (e.g., "size samples, define primary/guardrail metrics") instead of generic verbs like "plan, design, or implement".

DimensionReasoningScore

Specificity

"Plan, design, or implement an A/B test or experiment" plus "Includes methodology, statistical significance, and metric selection" lists several actions and sub-topics, matching anchor 4. Not 5 because the verbs (plan, design, implement) are generic rather than concrete; not 3 because it goes beyond 1-2 actions with enumerated coverage areas.

4 / 5

Completeness

The 'what' is clear (plan/design/implement A/B tests with methodology, statistical significance, metric selection) but there is no 'Use when...' clause or equivalent explicit trigger guidance, which caps completeness at 3 per the judging guidelines. Not 4 because 'when' is entirely absent rather than merely vague.

3 / 5

Trigger Term Quality

Includes natural terms users say ("A/B test", "experiment", "statistical significance") but omits common synonyms like "split test", "multivariate test", and "hypothesis test", matching anchor 4 rather than the comprehensive synonym coverage of anchor 5.

4 / 5

Distinctiveness Conflict Risk

The experimentation/statistical-significance niche is distinct from most skills, matching anchor 4 with only minor overlap risk from generic software 'testing' skills. Not 5 because without explicit trigger phrases, bare 'test'/'experiment' wording could pull in adjacent analytics or QA skills.

4 / 5

Total

15

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

administrakt0r/AI-Agents-Safe-Coding-Skills

Path

plugins/AI-Agents-Safe-Coding-Skills/skills/ab-test-setup/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.