CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

testland/experiment-results-interpreter

Use when an A/B test result is read for a ship decision, especially off a dashboard, a small sample or many metrics. Computes significance per metric with multiple-comparison correction, reads confidence intervals, checks novelty, interactions, Simpson's paradox, guardrails and stop-early rules.

Quality

85%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

73%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured interpretive reference: the six-step workflow with hard stops, decision tables, and a worked example gives it excellent workflow clarity and strong actionability. The main costs are token efficiency — redundant citations, duplicate reference listings, and explanations of concepts Claude already knows — plus a couple of orphaned bundle files that weakens navigation.

Suggestions

Cut known-concept explanations (the Wikipedia definition of Simpson's paradox, the generic novelty/primacy primer) and keep only the A/B-specific detection signals, trimming the body toward its tables and decision rules.

Cite the Kohavi source once (in the References section) instead of repeating the full title and ISBN three times in the body, and remove the duplicate reference enumeration in Step 6's parenthetical.

Link significance-anti-patterns.md and significance-output-example.md from the body (e.g., in the Anti-patterns or Example sections) so all bundle files are discoverable from SKILL.md.

DimensionReasoningScore

Conciseness

The body is mostly efficient tables and decision rules, but includes unnecessary material: a Wikipedia definition of Simpson's paradox and explanations of novelty/primacy that Claude already knows, plus the Kohavi citation with ISBN repeated three times and the reference set enumerated twice (in Step 6 prose and again in the References section). It sits at 'mostly efficient but includes some unnecessary explanation or could be tightened', above the 2 anchor's pervasive padding but below the 4 anchor's minor trims.

3 / 5

Actionability

For an instruction-only skill the guidance is largely concrete: per-step decision tables with pass criteria, specific thresholds ("two full weeks", "> 20% of users"), a segment-detection procedure, and a fully worked example with real numbers ending in a ship decision. It falls short of 5 because some steps remain directive rather than executable (e.g., "List all experiments running in the same user population" with no query or method), and the actual computation is delegated to a reference file.

4 / 5

Workflow Clarity

Six explicitly ordered steps with a declared hard-stop rule ("Each step has a hard stop: if a step blocks, do not proceed to the next"), per-step pass/block criteria, a numbered guardrail check with STOP conditions, explicit remedies when a check fails (isolate and re-run, factorial model, block ship; accepted-risk-or-extend for guardrail alerts), and a worked example running the full sequence. This matches the 5 anchor's explicit validation steps, feedback loops, and checklists; the operation is interpretive, not destructive/batch, so no cap applies.

5 / 5

Progressive Disclosure

SKILL.md is a clear overview with well-signaled, one-level-deep markdown links, and every referenced file exists with substantive content. It falls short of 5 due to minor organization gaps: two bundle files (significance-anti-patterns.md, significance-output-example.md) are never referenced or discoverable from the body, and the Step 6 reference paragraph is a dense parenthetical that duplicates the References section.

4 / 5

Total

16

/

20

Passed

Description

88%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: concrete action list, explicit 'Use when' triggers tied to realistic situations (dashboard-only verdict, small sample, many metrics), and a clearly delineated ship-decision niche. Its only weaknesses are a few missing natural synonyms and some adjacency to the companion experiment-validity skill.

DimensionReasoningScore

Specificity

The description lists multiple concrete actions — "Computes significance per metric with multiple-comparison correction, reads confidence intervals, checks novelty, interactions, Simpson's paradox, guardrails and stop-early rules" — with comprehensive coverage of the skill's capabilities. It matches the anchor for multiple specific concrete actions rather than the 4 anchor, which would require minor gaps in coverage.

5 / 5

Completeness

Both what and when are explicit: an "Use when an A/B test result is read for a ship decision..." trigger clause with concrete conditions (dashboard, small sample, many metrics), followed by a full enumeration of what the skill does. This matches the 5 anchor (clear, explicit what AND when with concrete trigger phrases), not the 4 anchor where the 'when' could be more specific.

5 / 5

Trigger Term Quality

Good natural keyword coverage: "A/B test result", "ship decision", "dashboard", "small sample", "many metrics", "confidence intervals", "guardrails" are phrases a user would naturally say. A few common variations are missing (e.g., "experiment results", "significance", "p-value", "lift"), which places it just below the comprehensive-synonyms anchor of 5.

4 / 5

Distinctiveness Conflict Risk

The niche is clear (interpreting a finished experiment's result for a ship/no-ship call), and triggers like "Simpson's paradox" and "guardrails" are distinctive. However, the body names a companion skill `ab-test-validity-checklist` covering adjacent A/B-testing territory, so there is minor overlap risk with a closely related skill — the 4 anchor — rather than the minimal-conflict 5 anchor.

4 / 5

Total

18

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

On this page