experimental-design

Best practices for designing reproducible ML experiments. Use when planning ablations, baselines, or controlled experiments.

Quality

79%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./researchclaw/skills/builtin/experiment/experimental-design/SKILL.md

Quality

Content

80%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is an efficient, actionable checklist of experimental-design practices that avoids over-explanation and is well-organized for a short single-purpose skill. Its main weakness is workflow clarity: it presents parallel practices rather than a sequenced process with validation checkpoints.

Suggestions

Reframe the checklist as a light sequenced workflow (e.g., plan baselines -> set seeds -> design ablations -> control variables -> report metrics) so the ordering reflects the actual experimental process.

Add an explicit verification/reporting checkpoint, such as 'Before reporting, confirm each ablation differs from baseline and std is computed across >=3 seeds.'

Include a minimal results-reporting template (table columns: method, mean +/- std, seeds, wall-clock, memory) to make the reporting guidance copy-paste ready.

Dimension	Reasoning	Score
Conciseness	The body is a lean numbered checklist with no padding and no re-explanation of concepts Claude already knows (e.g., it never defines 'baseline' or 'ablation'); every line earns its place, matching the score-5 'lean and efficient' anchor.	5 / 5
Actionability	Each item is a concrete, specific directive ('minimum 3, ideally 5 seeds', 'Report mean +/- std', 'change only ONE thing per comparison'), but as an instruction-only skill it lacks output examples or a reporting template, leaving minor gaps versus the fully-copy-paste score-5 anchor.	4 / 5
Workflow Clarity	The numbered items read as parallel best practices rather than a sequenced process, and there are no validation/verification checkpoints; this matches the score-3 anchor of listed steps with implicit sequence and missing checkpoints.	3 / 5
Progressive Disclosure	The skill is under 50 lines, needs no external references (references/, scripts/, assets/ are absent), and is organized under a single clear section header with a well-structured list, satisfying the score-5 simple-skill exception for progressive disclosure.	5 / 5
	Total	17 / 20 Passed

Description

78%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is concise, uses third person, and cleanly pairs a clear 'what' with an explicit 'Use when' trigger clause. It is strong on completeness and trigger quality, with only minor gaps in action specificity and distinctiveness.

Dimension	Reasoning	Score
Specificity	Names the domain ('reproducible ML experiments') and a few actions ('planning ablations, baselines, or controlled experiments'), but the actions are abstract 'designing/planning' verbs rather than a comprehensive concrete capability list, matching the score-3 anchor rather than the multi-action score-4 anchor.	3 / 5
Completeness	Explicitly answers both 'what' ('Best practices for designing reproducible ML experiments') and 'when' with a concrete 'Use when planning ablations, baselines, or controlled experiments' trigger clause, matching the score-5 anchor pattern.	5 / 5
Trigger Term Quality	Includes natural terms an ML practitioner would say — 'ablations', 'baselines', 'controlled experiments', 'reproducible ML experiments' — giving good coverage, though a few common synonyms are missing so it falls short of the comprehensive score-5 anchor.	4 / 5
Distinctiveness Conflict Risk	The experimental-design niche ('ablations, baselines, controlled experiments, reproducible ML') is mostly distinct with clear triggers, but carries minor overlap risk with general ML-training or reproducibility skills, placing it at score 4 rather than the minimal-conflict score 5.	4 / 5
	Total	16 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: aiming-lab/AutoResearchClaw
Path: researchclaw/skills/builtin/experiment/experimental-design/SKILL.md
Commit: be4ba47

Reviewed: 5 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.