CtrlK
BlogDocsLog inGet started
Tessl Logo

testland/experiment-results-interpreter

Interprets the results of a valid online controlled experiment, one whose harness, SRM, and telemetry have already been confirmed. Covers the distinction between practical and statistical significance, reading confidence intervals instead of binary p-values, novelty and primacy week-over-week decay that causes post-ship reversion, interaction effects from concurrent experiments, Simpson's paradox in segmented results, and the ordered guardrail-check sequence required before a ship decision - with the deep methodology in references/: the peeking problem and its corrections (fixed-horizon, alpha-spending, always-valid mSPRT) in references/peeking.md, and guardrail-metric methodology (taxonomy, OEC relationship, pre-commitment, thresholds) in references/guardrails.md. Use when a data scientist or PM is ready to draw conclusions from an experiment, when designing a stop-early policy, or when declaring an experiment's guardrail set. Distinct from ab-test-validity-checklist (harness setup and SRM detection).

72

Quality

90%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

85%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is a well-structured reference with a clear ordered workflow, explicit validation checkpoints, and appropriate progressive disclosure into real reference files. It is mostly lean and actionable, with minor over-explanation in inline citations and structured-text rather than executable code blocks.

Suggestions

Tighten the inline verbatim source citations to paraphrases or move them entirely to references/sources.md to reduce token cost in the body.

Replace the structured-text blocks in Steps 5 and 6 with concrete, copy-pasteable commands (e.g., explicit SQL/Python for per-segment lift and CI computation) where feasible.

DimensionReasoningScore

Conciseness

The body is dense and table-driven and assumes Claude's knowledge of CIs, p-values, and experimental design, but inline verbatim citations and some narrative prose around each step could be trimmed to be fully lean.

4 / 5

Actionability

Provides concrete decision tables, specific thresholds ('> 20% of users', 'two full weeks', '14+ day window'), and a fully worked example, but the Simpson's and guardrail 'code' blocks are structured numbered text rather than executable code, leaving minor gaps.

4 / 5

Workflow Clarity

A six-step ordered workflow with explicit hard stops ('if a step blocks, do not proceed'), block/alert thresholds, and a validate-then-proceed feedback loop in the guardrail step matches the anchor for clear sequencing with explicit validation and error-recovery checkpoints.

5 / 5

Progressive Disclosure

The body is an overview that pushes deep methodology to clearly signaled, one-level-deep reference files (sources.md, peeking.md, guardrails.md, thresholds-and-corrections.md), all of which exist, giving easy navigation.

5 / 5

Total

18

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific, complete, and distinctive, with an explicit 'Use when' trigger clause and a clear carve-out from the companion validity-checklist skill. Trigger-term coverage is strong but misses the 'A/B test' synonym in the description body itself.

DimensionReasoningScore

Specificity

Lists multiple concrete actions — 'distinction between practical and statistical significance', 'reading confidence intervals instead of binary p-values', 'novelty and primacy week-over-week decay', 'interaction effects', 'Simpson's paradox', 'ordered guardrail-check sequence' — giving comprehensive coverage of what the skill does.

5 / 5

Completeness

Explicitly answers 'what' (interprets results across the listed sub-topics) and 'when' via a concrete 'Use when…' clause with three distinct triggers, matching the anchor for a clear what-and-when with concrete trigger phrases.

5 / 5

Trigger Term Quality

Includes natural phrases a user would say — 'ready to draw conclusions from an experiment', 'designing a stop-early policy', 'declaring an experiment's guardrail set' — but the common synonym 'A/B test' appears only in metadata keywords, not the description body, leaving a few natural terms missing.

4 / 5

Distinctiveness Conflict Risk

Carves out a clear niche and explicitly states 'Distinct from ab-test-validity-checklist (harness setup and SRM detection)', minimizing conflict risk with the closely related sibling skill.

5 / 5

Total

19

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

metadata_version

'metadata.version' is missing

Warning

Total

15

/

16

Passed

Reviewed

Table of Contents