CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

trustworthy-experiments

Use when asked to "run an A/B test", "design an experiment", "check statistical significance", "trust our results", "avoid false positives", or "experiment guardrails". Helps design, run, and interpret controlled experiments correctly. Based on Ronny Kohavi's framework from "Trustworthy Online Controlled Experiments".

Quality

63%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./.agents/skills/trustworthy-experiments/SKILL.md
SKILL.md
Quality
Evals
Security

Quality

Content

30%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body reads as a motivational overview of the Kohavi framework rather than an operational skill: it is padded with boilerplate sections, provides no executable procedure for any of its claimed capabilities, and lacks both a workflow and reference material to back its scope. The when/when-not lists and the failure statistics are its only genuinely useful content.

Suggestions

Replace the "Domain Context" and "Input Requirements" boilerplate with a concrete workflow: design (hypothesis, OEC, power calculation) → run (SRM check, runtime) → validate (significance, Twyman's Law screen) → decide.

Add executable specifics for the claimed capabilities: a sample-size formula or rule of thumb, a chi-square SRM check, and numeric thresholds for the false-positive-risk adjustment.

Move deeper material (metric/OEC guidance, diagnosis playbook, speedup techniques) into a references/ file linked from the body instead of leaving the claims unsupported.

DimensionReasoningScore

Conciseness

The Kohavi statistics and the when/when-not lists earn their tokens, but the "Domain Context" section ("a proven product management framework... combines best practices from industry leaders") and the generic "Input Requirements" section are boilerplate padding that could be cut entirely.

3 / 5

Actionability

The body names capabilities ("Determine sample size and runtime for statistical power", "Diagnose why results look suspicious") without giving any procedure, formula, threshold, or example for executing them — it describes rather than instructs, offering only high-level hints like "Need tens of thousands minimum".

2 / 5

Workflow Clarity

Despite claiming to help design, run, and validate experiments, the body contains no sequenced steps whatsoever — there is no design procedure, no validation method for the "Validate experiment results before making ship/no-ship decisions" step, and no error-recovery guidance.

1 / 5

Progressive Disclosure

Section headers are clear and the body is short, but the skill claims seven distinct capabilities (sample sizing, OEC metrics, SRM diagnosis, speedup, culture) whose supporting detail lives nowhere — the only external pointer is a book citation rather than actual reference files.

3 / 5

Total

9

/

20

Passed

Description

78%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description with an explicit, natural-language trigger list and a clear third-person capability statement. Its main weakness is that the stated actions (design, run, interpret) are generic verbs rather than concrete operations, and a few common synonym triggers are absent.

DimensionReasoningScore

Specificity

Names the domain ("controlled experiments") and the actions "design, run, and interpret", but these are generic verbs rather than concrete operations, matching the anchor for domain-plus-generic-actions rather than the several-specific-actions level.

3 / 5

Completeness

Explicitly answers both parts: an explicit trigger clause ("Use when asked to..." with six concrete quoted phrases) and a clear third-person statement of what it does ("Helps design, run, and interpret controlled experiments correctly"). Not the level below because the 'when' is fully explicit with concrete triggers, not merely present.

5 / 5

Trigger Term Quality

Natural phrases like "run an A/B test", "check statistical significance", and "avoid false positives" are exactly what users would say, but common synonyms such as "split test", "hypothesis test", or "p-value" are missing, so coverage is good rather than comprehensive.

4 / 5

Distinctiveness Conflict Risk

Triggers like "experiment guardrails", "avoid false positives", and the Kohavi framework citation carve out a fairly distinct experimentation niche, with only minor overlap risk against a generic statistics or data-analysis skill.

4 / 5

Total

16

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

saski/arnesto

Path

.agents/skills/trustworthy-experiments/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.