CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

ml-benchmark-evaluation

Rigorous methodology for evaluating ML models on established benchmarks. Covers proper train/val/test splits, baseline verification from original papers, exact metric formula discrepancies, data-leak detection checklist, multi-seed robustness, and honest reporting templates. Use when claiming to beat published baselines, writing methods papers, or auditing existing results.

61

Quality

77%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./backend/cli/skills/ml-training/ml-benchmark-evaluation/SKILL.md
SKILL.md
Quality
Evals
Security

Quality

Content

56%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The core methodology is genuinely strong — concrete, near-executable code for splits, leak detection, metric formulas, and multi-seed runs, organized into a clear numbered workflow with validation checkpoints. Weaknesses center on the injected "Preserve the execution evidence" prose (off-topic and non-actionable), a fabricated paper-parsing tool, and a monolithic single-file structure with no progressive disclosure for domain-specific annexes.

Suggestions

Replace the fabricated `npm i -g @llamaindex/liteparse` workflow with a real, executable way to fetch baseline numbers (e.g., arxiv API via curl or Semantic Scholar API) and define or stub `train(seed)`/`evaluate()` so the code is copy-paste runnable.

Remove or rewrite the 'Preserve the execution evidence' subsection — its evaluator-storage, run-manifest, and fake-provider content is unrelated to ML benchmarking; replace it with ML-relevant evidence preservation (fixed data splits, checkpoint hashes, config dumps).

Move the PDE-specific physics-informed validation table (Section 7) into a one-level-deep reference file (e.g., references/physics-validation.md) linked from the body, keeping SKILL.md as a tight overview.

DimensionReasoningScore

Conciseness

The body is largely lean and code-forward with no basic-concept padding, but the "Preserve the execution evidence" subsection under Section 5 is dense prose about evaluator seed ledgers, run manifests, fake-provider checks, and virtual-environment readiness — material largely unrelated to ML benchmark evaluation that reads as an off-topic insertion and should be trimmed or rewritten for this domain. This matches the anchor 'mostly efficient but includes some unnecessary explanation or could be tightened', not the level above where only minor instances would exist.

3 / 5

Actionability

Sections 1, 3, 4, and 5 supply near-executable code (splits, six leak-detection asserts, an nRMSE implementation, a seed loop), but Section 2's verification workflow depends on a non-existent tool (`npm i -g @llamaindex/liteparse`) plus a placeholder arxiv URL, helper functions like `train(seed)` and `evaluate()` are never defined, and the evidence subsection gives no concrete steps. This sits between 'some concrete guidance but incomplete; pseudocode... missing key details' and 'mostly executable guidance with minor gaps' — closer to the lower one given the fabricated dependency.

3 / 5

Workflow Clarity

Numbered sections 1-7 form a coherent sequence (split → verify baselines → leak checks → metric → seeds → report → physics checks) with real validation checkpoints ("Run these 6 checks before reporting any result", assert-based checks, "Report final epoch OR val-selected"). It falls short of the top anchor because there is no explicit feedback loop stating what to do when a leak check or verification fails (fix, re-check, re-run), and the sequence is implied by section order rather than stated.

4 / 5

Progressive Disclosure

There are no bundle files at all — no references/, scripts/, or assets/ — so everything (including the PDE-specific physics validation table and the off-topic evidence prose) is inlined in a ~215-line monolithic SKILL.md. Section headers are well-organized, but per the anchor, content that should be separate is inline (e.g., PDE/scientific-ML validation and the evidence-prose could live in one-level-deep reference files), which fits 'some structure but could be better organized' rather than 'most content is appropriately placed'.

3 / 5

Total

13

/

20

Passed

Description

88%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: it names the domain, lists six concrete capabilities comprehensively, and includes an explicit third-person "Use when..." trigger clause. The only shortcoming is trigger-term breadth — a few natural synonyms users might say (reproducibility, SOTA/leaderboard, benchmark names) are missing.

DimensionReasoningScore

Specificity

The description enumerates six concrete capabilities — "proper train/val/test splits, baseline verification from original papers, exact metric formula discrepancies, data-leak detection checklist, multi-seed robustness, and honest reporting templates" — giving comprehensive, non-abstract coverage of the domain. This matches the top anchor ('lists multiple specific concrete actions; comprehensive coverage') and not the level below, which allows minor gaps in coverage.

5 / 5

Completeness

Both questions are explicitly answered: the "what" via "Covers proper train/val/test splits... honest reporting templates" and the "when" via the explicit trigger clause "Use when claiming to beat published baselines, writing methods papers, or auditing existing results". This matches the anchor for clear and explicit what-and-when with concrete triggers, not the level below where the "when" is merely present but less specific.

5 / 5

Trigger Term Quality

Natural phrases users would say include "evaluating ML models", "beat published baselines", "writing methods papers", and "auditing existing results", giving good coverage. It falls short of the top anchor because common variations a user might say are absent — e.g., "reproducibility", "SOTA", "leaderboard", "generalization", or specific benchmark names.

4 / 5

Distinctiveness Conflict Risk

The niche (rigorous ML benchmark evaluation protocol) is well-delineated by distinct triggers like "claiming to beat published baselines" and "auditing existing results". It does not reach the top anchor because it could minorly overlap with adjacent skills on general experiment tracking, reproducibility tooling, or ML training, which share terms like "train/val/test splits".

4 / 5

Total

18

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

15

/

16

Passed

Repository

synthetic-sciences/openscience

Path

backend/cli/skills/ml-training/ml-benchmark-evaluation/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.