CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

experiment-guide

Choose experiments and use their results when investigating program improvements with the portfolio research tools.

Quality

74%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./agentpilot/src/agentpilot/policy/proposer/skills/experiment-guide/SKILL.md
SKILL.md
Quality
Evals
Security

Quality

Content

78%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A tight, token-efficient orchestration reference with concrete command definitions and an understandable experiment-assess-revise workflow. Its main weakness is actionability: core protocol vocabulary (T track, B comparison, whole-task A runs) is used but never defined, so a reader outside the surrounding system cannot fully execute the guidance.

Suggestions

Briefly define the protocol vocabulary on first use (e.g., what the 'T track', 'B comparison', and 'whole-task A runs' are), or point to where those definitions live.

Restructure the process paragraphs into a short numbered sequence (submit → wait → check result → revise/narrow → note/drop) so the workflow checkpoints are impossible to miss.

Add one concrete example invocation of `experiment` and `note` showing the expected fields (mechanism, comparison, track) in practice.

DimensionReasoningScore

Conciseness

The ~15-line body is lean with zero padding and no explanations of concepts Claude already knows — every sentence carries operational content ("Reads and experiments may be interleaved; independent experiments can run together"). It clearly matches the 'lean and efficient; every token earns its place' anchor rather than 4, which reserves room for over-explanation that is absent here.

5 / 5

Actionability

Concrete commands and parameters are present (`experiment`, `build`, `from_experiment`, `track="new"`, "send a nonempty assistant note without tool calls"), but key protocol terms are used without definition — "Use the related T track", "the full B comparison", "request whole-task A runs" — leaving missing key details for execution. This fits 'some concrete guidance but incomplete' better than 4, where gaps would be minor.

3 / 5

Workflow Clarity

A clear sequence exists (run experiments → wait via a nonempty note → check results → revise/narrow → note or drop) with explicit checkpoints ("check whether the tested behavior actually occurred and whether the comparison distinguishes the explanations") and a revise-on-counterevidence loop. It falls short of 5 because the workflow is scattered across prose rather than presented as explicit steps or a checklist.

4 / 5

Progressive Disclosure

The skill is under 50 lines, needs no external references (none exist in the bundle), and is well organized as a heading, a terse command definition list, and two short process paragraphs. Per the simple-skill guideline this earns 5; there is no inlined content that belongs in separate files.

5 / 5

Total

17

/

20

Passed

Description

58%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description states both what the skill does and when to use it in a single concise sentence, avoiding fluff and staying in third person. It is held back by abstract action verbs, thin natural-trigger coverage, and jargon ("portfolio research tools") that limits distinctiveness for users who don't already know the tooling.

Suggestions

Replace the abstract actions with concrete capabilities, e.g., "Run experiments to test predicted mechanisms, inspect their results, and commit supported changes to the build comparison."

Expand trigger coverage with natural phrases users would say, such as "Use when testing a hypothesized improvement, checking experiment results, or deciding whether to keep or drop a change."

Clarify the domain jargon or add synonyms (experiments, trials, A/B comparisons) so the description is distinguishable from generic evaluation skills.

DimensionReasoningScore

Specificity

The description names the domain ("investigating program improvements with the portfolio research tools") and 1-2 actions ("Choose experiments", "use their results"), but the actions are abstract and coverage is not comprehensive. It matches 'names domain and 1-2 concrete actions' rather than 4, which requires several specific actions.

3 / 5

Completeness

Both parts are explicitly stated: what ("Choose experiments and use their results") and when ("when investigating program improvements with the portfolio research tools"). The 'when' is embedded in a single sentence and could be more explicit with concrete trigger phrases, matching anchor 4 rather than 5, and it is clearly stated rather than merely implied, ruling out 3.

4 / 5

Trigger Term Quality

Some relevant keywords exist ("experiments", "results", "program improvements"), but "portfolio research tools" is internal jargon and common natural variations (test, try changes, benchmark, evaluate) are absent. This fits 'some relevant keywords but missing common variations or synonyms' better than 4's 'good keyword coverage'.

3 / 5

Distinctiveness Conflict Risk

"portfolio research tools" carves out a niche, but the action phrasing "choose experiments and use their results" is generic and could overlap with other experimentation or evaluation skills. It sits at 'somewhat specific but could still overlap with similar skills' rather than 4's 'mostly distinct; minor overlap risk'.

3 / 5

Total

13

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

HKUDS/OpenSpace

Path

agentpilot/src/agentpilot/policy/proposer/skills/experiment-guide/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.