CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

evaluate-cluster

Evaluates an ML GPU cluster for a cloud trial or acceptance test: environment dump, isolated newest PyTorch, matmul FLOPS (MAMF/MSMF) on every GPU while the others compute, intra-node all-reduce bandwidth and per-call latency of every collective, the same inter-node on every node you were given (omit those sections if there is only one node), fio on local disk and shared FS, dated markdown report. Use when the user asks to evaluate a cluster, kick the tires on trial nodes, run cluster acceptance, or measure GPU/network/storage. Canonical copy: https://github.com/stas00/ml-engineering/blob/master/skills/evaluate-cluster/SKILL.md

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

SKILL.md
Quality
Evals
Security

Quality

Content

77%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

An exceptionally actionable runbook: executable commands for every phase, explicit gates, validation checkpoints, and error-recovery loops that most skills lack. The costs are size and redundancy — many rules restated across sections, and the report template, final checklist, and training-wheels notes are inlined rather than split into reference files, making the single file very expensive to load.

Suggestions

Deduplicate rules stated 3-4 times across sections (1-node omission rules, 'never % of NIC spec', NVLS ~30%/~25% figures): state each rule once in its governing section and cross-reference it elsewhere instead of restating it — the conciseness dimension's main penalty.

Move the ~200-line report template, the final-pass checklist table, and the 'Training wheels' past-job notes into a references/ file (e.g. references/report-template.md, references/final-checklist.md) linked from a short overview section, so the SKILL.md body is a navigable runbook rather than a single ~100 KB inline document.

Move the 'Canonical copy' GitHub-URL paragraph out of the frontmatter description into the body — description tokens are trigger text, and the URL is not a trigger.

DimensionReasoningScore

Conciseness

Nearly every line is domain-specific operational knowledge Claude does not have (NVLS gains, is_host/rdzv failure modes, fio --unlink=1 scratch math, EFA/rdma link quirks) — no filler explaining known concepts — but the same rules are repeated 3–4 times across sections (1-node omission in ~6 places, 'never % of NIC spec' in 4, NVLS ~30%/~25% figures in 4) and the ~740-line body could be meaningfully tightened. This fits the 3 anchor ('mostly efficient… could be tightened') better than 2 ('noticeably verbose… padded sections'), since the density is deliberate rule placement, not padding — though the redundancy is real and the file is very large.

3 / 5

Actionability

Fully executable throughout: copy-paste curl fetch commands, venv install on the newest torch with pip index discovery, 'dcgmi diag -r 2 | tee dcgm-diag-r2.txt', a complete pdsh + torch.distributed.run command with remote-side escaping, './fio-scan "$NVME_PATH"', and exact report table templates with column names and alignment. Specific commands cover the common cases — the 5 anchor.

5 / 5

Workflow Clarity

The 0–10 numbered loop is explicitly sequenced with hard gates and validation checkpoints: the preflight gate ('Missing → message the user with the exact list, then stop'), SSH/TCP reachability pass/fail criteria, three-pass rerun confirmation before calling a GPU under-performing, grepping logs to verify the isolated torch actually ran, and a 24-row final-pass checklist run 'line by line, before you tell the user it is ready'. Batch/destructive scratch operations (165 GiB fio files) are covered by move-not-rm cleanup with 'du -sh' verification, so the workflow-clarity cap for unvalidated batch ops does not apply.

5 / 5

Progressive Disclosure

No references/, scripts/, or assets/ bundle exists; everything — the ~200-line report template, the final-pass checklist table, and the 'Training wheels' past-job notes — is inlined in a single ~100 KB file where the anchor expects such material in separate referenced files. Structure within the file is strong (clear headed sections, clearly signaled GitHub links to external scripts and tables), which rules out 2 ('minimal structure') and matches 3 ('some structure; content that should be separate is inline'), but not 4 since no content at all is offloaded to a navigable bundle.

3 / 5

Total

16

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: third person, specific capability enumeration across compute/network/storage, and an explicit multi-phrase 'Use when' trigger clause. The only weakness is a few missing natural synonyms ('benchmark'), and the trailing 'Canonical copy:' URL paragraph inside the description field adds a small amount of non-trigger noise.

DimensionReasoningScore

Specificity

The description enumerates concrete capabilities: 'environment dump', 'isolated newest PyTorch', 'matmul FLOPS (MAMF/MSMF) on every GPU while the others compute', 'intra-node all-reduce bandwidth and per-call latency of every collective', 'fio on local disk and shared FS', 'dated markdown report' — comprehensive coverage with no gaps, matching the 5 anchor ('multiple specific concrete actions; comprehensive coverage'). It is not 4 because the action list covers compute, network, storage, and reporting without missing a phase.

5 / 5

Completeness

Both what and when are explicit: a detailed capability list followed by 'Use when the user asks to evaluate a cluster, kick the tires on trial nodes, run cluster acceptance, or measure GPU/network/storage' — an exact match to the 5 anchor (clear what AND when with concrete trigger phrases). Voice is third person ('Evaluates…'), so no penalty applies.

5 / 5

Trigger Term Quality

Natural user phrasings are present: 'evaluate a cluster', 'kick the tires on trial nodes', 'run cluster acceptance', 'measure GPU/network/storage'. A few natural synonyms are missing — e.g. 'benchmark the GPUs/cluster' and 'acceptance test' phrasing users would say — so it fits the 4 anchor ('good keyword coverage; a few natural terms missing') rather than the 5 anchor, and is clearly above 3 (coverage is not partial).

4 / 5

Distinctiveness Conflict Risk

A clear niche (ML GPU cluster cloud-trial/acceptance evaluation) with distinct triggers ('kick the tires on trial nodes', 'cluster acceptance') that no generic document- or code-processing skill would claim — minimal conflict risk, matching the 5 anchor. Not 4: there is no meaningful overlap with a closely related skill.

5 / 5

Total

19

/

20

Passed

Validation

87%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

skill_md_line_count

SKILL.md is long (739 lines); consider splitting into references/ and linking

Warning

relative_links

Relative link issues: 6 missing, 4 deeper-than-1-level

Warning

Total

14

/

16

Passed

Repository

stas00/ml-engineering

Path

skills/evaluate-cluster/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.