CtrlK
BlogDocsLog inGet started
Tessl Logo

testland/experiment-results-interpreter

Interprets the results of a valid online controlled experiment, one whose harness, SRM, and telemetry have already been confirmed. Covers the distinction between practical and statistical significance, reading confidence intervals instead of binary p-values, novelty and primacy week-over-week decay that causes post-ship reversion, interaction effects from concurrent experiments, Simpson's paradox in segmented results, and the ordered guardrail-check sequence required before a ship decision. Use when a data scientist or PM is ready to draw conclusions from an experiment whose telemetry and randomisation have already passed the ab-test-validity-checklist. Distinct from ab-test-validity-checklist (harness setup and SRM detection) and from interaction-effect overlap auditing during experiment design.

74

Quality

93%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

85%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured reference skill: ordered, gated workflow with concrete numeric thresholds and a verified single-level reference layer. Its only weakness is mild redundancy between the step prose, the Anti-patterns table, and the worked Example.

Suggestions

Tighten conciseness by cross-referencing the Anti-patterns table from the relevant steps instead of restating each failure mode twice, or trim the Anti-patterns table to entries not already covered in Steps 1-6.

Consider collapsing the Example's per-step walkthrough into a single compact decision table, since it mirrors the six-step structure already presented.

The Limitations and References sections partially overlap the Overview's framing of what the skill does not do; merge the harness-validity exclusion so it is stated once.

DimensionReasoningScore

Conciseness

The body is dense and table-driven and assumes expert competence (MPSE, CUPED, SUTVA, Kendall's tau are used without over-explaining), but the Anti-patterns table and the worked Example restate concepts already covered in Steps 1-6, so it could be tightened.

2 / 3

Actionability

Concrete numeric decision rules throughout: 'Do not ship on a positive aggregate with a negative segment that represents > 20% of users', 'Minimum run time: two full weeks', plus CI-position tables and alert/block guardrail thresholds make the guidance copy-paste ready for an interpretation skill.

3 / 3

Workflow Clarity

Six interpretation steps are explicitly ordered with hard stops ('if a step blocks, do not proceed to the next'), the guardrail check has STOP gates, and feedback loops (extend runtime, isolate and re-run) handle error recovery.

3 / 3

Progressive Disclosure

Body points to a real one-level-deep reference (references/sources.md, verified present) via clearly signaled links and is organized into well-separated sections, with companion catalogs named rather than inlined.

3 / 3

Total

11

/

12

Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific, trigger-rich, complete, and explicitly disambiguated from sibling skills, all in third-person voice. It is a model 'what + when + distinct-from' description.

DimensionReasoningScore

Specificity

Lists multiple concrete interpretation actions: 'distinction between practical and statistical significance', 'reading confidence intervals instead of binary p-values', 'novelty and primacy week-over-week decay', 'interaction effects from concurrent experiments', 'Simpson's paradox in segmented results', and 'ordered guardrail-check sequence'.

3 / 3

Completeness

Explicitly answers both what ('Interprets the results of a valid online controlled experiment...') and when ('Use when a data scientist or PM is ready to draw conclusions from an experiment whose telemetry and randomisation have already passed the ab-test-validity-checklist').

3 / 3

Trigger Term Quality

Good coverage of natural terms a data scientist or PM would say: 'online controlled experiment', 'statistical significance', 'confidence intervals', 'p-values', 'novelty', 'primacy', 'interaction effects', 'Simpson's paradox', 'guardrails', and 'ship decision'.

3 / 3

Distinctiveness Conflict Risk

Explicit disambiguation clause ('Distinct from ab-test-validity-checklist (harness setup and SRM detection) and from interaction-effect overlap auditing during experiment design') carves out a clear niche unlikely to trigger the wrong skill.

3 / 3

Total

12

/

12

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

metadata_version

'metadata.version' is missing

Warning

Total

15

/

16

Passed

Reviewed

Table of Contents