CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

testland/experiment-results-interpreter

Use for any question about reading an A/B test or experiment result, including quick ones: is it significant, should we ship variant B, why a week-one win faded. Computes significance per metric with multiple-comparison correction, reads confidence intervals, and checks novelty, interactions, Simpson's paradox, guardrails and stop-early rules before a ship decision.

Quality

90%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

81%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A strong, well-sequenced interpretation workflow with concrete decision rules, hard stops, and a fully worked example; actionability and workflow clarity are exemplary. The main weaknesses are token cost from concept re-explanations and citation padding, plus two bundle files that are orphaned from the main skill file.

Suggestions

Cut or compress the definitional explanations of concepts Claude already knows (Simpson's paradox definition, what a 95% CI means, novelty/primacy definitions in Step 3) into one-line operational cues, saving ~30 lines.

Replace the nine repeated inline citation parentheticals and three ISBN mentions with a single attribution block in the References section.

Link references/significance-anti-patterns.md and references/significance-output-example.md from SKILL.md (e.g., from Step 1/2 or the References section) so they are discoverable.

DimensionReasoningScore

Conciseness

The body re-explains concepts Claude already knows — "Simpson's paradox is when 'a trend appears in several groups of data but disappears or reverses when the groups are combined'", "A 95% confidence interval gives the plausible range of the true effect", and the novelty/primacy definitions — and pads with repeated citation parentheticals ("(Nielsen Norman Group, [sources](references/sources.md))" appears 9 times; the Kohavi ISBN three times). Mostly efficient operational tables and checklists, so this fits anchor 3 rather than anchor 2's padded prose, but it could clearly be tightened.

3 / 5

Actionability

Concrete decision rules throughout: the CI-position table ("Entirely above zero and above MPSE | Strong positive - candidate for ship"), specific thresholds ("Minimum run time: two full weeks", "a negative segment that represents > 20% of users"), an ordered guardrail procedure with explicit stop conditions, and a fully worked numeric example ("lift = +1.2pp (95% CI: +0.3pp, +2.1pp)") covering the common case. For an instruction-only skill this is copy-paste-ready guidance; the statistical computation details are properly delegated to references/significance-check.md rather than missing.

5 / 5

Workflow Clarity

Six steps in explicit order with an enforced gate — "Each step has a hard stop: if a step blocks, do not proceed to the next" — plus a numbered guardrail check with "(STOP - no-ship)" checkpoints and recovery paths ("extend experiment", "accepted risk + rationale"), and a worked example that walks the same sequence. Matches anchor 5's explicit validation steps, feedback loops, and checklists; this is an analysis/decision skill, so the destructive/batch cap does not apply.

5 / 5

Progressive Disclosure

Good structure with clearly signaled one-level-deep links (references/significance-check.md, references/guardrails.md, references/peeking.md, references/thresholds-and-corrections.md, references/sources.md) and a References section. However, two real bundle files — references/significance-anti-patterns.md and references/significance-output-example.md — are never referenced from SKILL.md, a discoverability gap, and the ~300-line body inlines some methodology detail (Step 3 novelty detection, Step 6 anti-pattern table) that could live in references. Fits anchor 4's 'minor organization gaps', not 5.

4 / 5

Total

17

/

20

Passed

Description

96%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description explicitly answers both what and when, with concrete capability lists and natural question-form trigger phrases that are exemplary. Its only weakness is the 'any question' breadth, which creates minor overlap risk with closely related experiment-validity skills.

DimensionReasoningScore

Specificity

"Computes significance per metric with multiple-comparison correction", "reads confidence intervals", and "checks novelty, interactions, Simpson's paradox, guardrails and stop-early rules" list multiple concrete actions covering the entire interpretation workflow. Matches anchor 5's comprehensive coverage; not 4, since the action list spans every step of the skill with no coverage gaps.

5 / 5

Completeness

Explicit 'when' — "Use for any question about reading an A/B test or experiment result, including quick ones" — paired with an explicit 'what' listing the concrete capabilities and the ship-decision framing. Clearly matches anchor 5 (both what and when with concrete trigger phrases); not 4, since neither half is implicit or underspecified.

5 / 5

Trigger Term Quality

The description embeds literal natural user questions — "is it significant, should we ship variant B, why a week-one win faded" — alongside domain terms "A/B test" and "experiment result", matching anchor 5's comprehensive natural-term coverage including question-form synonyms. Not 4: the question-form triggers go beyond anchor 4's 'good keyword coverage with a few missing', covering exactly what a user would say when they need this skill.

5 / 5

Distinctiveness Conflict Risk

The post-hoc interpretation/ship-decision niche is well carved out, but "Use for any question about reading an A/B test or experiment result, including quick ones" is aggressive breadth that overlaps closely related skills — e.g., the companion `ab-test-validity-checklist` covering experiment harness validity, or general statistics skills. Fits anchor 4 ('mostly distinct; minor overlap risk with closely related skills'); not 5 because of that breadth, not 3 because the interpretive niche is clearly distinguishable.

4 / 5

Total

19

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

On this page