CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

aris-training-check

Periodically check WandB metrics during training to catch problems early (NaN, loss divergence, idle GPUs). Avoids wasting GPU hours on broken runs. Use when training is running and you want automated health checks.

Quality

83%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

78%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body delivers a clear, well-sequenced workflow with decision tables, executable WandB/SSH snippets, and strong error-recovery loops (fallbacks, interval resets, evidence-saving rules before destructive stops). The main gaps are an undefined divergence threshold (">N steps"), no concrete kill mechanism for the stop action, and a sketchy CronCreate example. Minor redundancy between the intro, the When to Use section, and the watchdog integration table keeps it from full token efficiency.

Suggestions

Replace the undefined ">N steps" divergence threshold with a concrete value (e.g., "loss increasing over the last 10+ logged checkpoints") so the stopping decision is unambiguous.

Specify the actual stop mechanism — the command or procedure used to kill the training session — and turn the CronCreate example into a literal invocation with the prompt and interval arguments.

Trim the motivational opener and condense the watchdog integration table, since both repeat the description and the When to Use boundary note.

DimensionReasoningScore

Conciseness

The body is mostly lean — compact decision tables, runnable snippets, and no explanation of concepts Claude already knows — but the motivational opener ("Do not wait until training finishes to discover it was a waste of GPU time") and the "Integration with Watchdog" table partially repeat the description and the When to Use boundary note. This matches anchor 4 (efficient with minor over-explanation that could be trimmed), not 5, since a few tokens don't earn their place.

4 / 5

Actionability

Provides executable guidance — `wandb.Api()` history snippet, `ssh server "tail -100 ..."` fallback, a fill-in Codex MCP prompt template — matching anchor 4 (mostly executable with minor gaps). It falls short of 5 because "increasing for >N steps" leaves N undefined, the CronCreate block is a prose sketch rather than an actual invocation, and "Kill the training session" gives no concrete command.

4 / 5

Workflow Clarity

A clear 4-step sequence (read metrics → judgment table → escalate only when unsure → act) with genuine feedback loops (SSH fallback, retry next interval when unreachable, interval reset after anomaly, save evidence before stopping). Not 5 because the destructive checkpoint is underspecified: the divergence threshold is an undefined ">N steps" and the kill step has no concrete mechanism, which are real validation gaps.

4 / 5

Progressive Disclosure

No bundle files exist (references/, scripts/, assets/ are absent) and the ~117-line body is self-contained under well-organized headers with nothing that clearly belongs in a separate file. The only external reference ([watchdog.py](../aris-infra/tools/watchdog.py)) is clearly signaled and one level deep, matching anchor 5 (clear overview, easy navigation, appropriately split content).

5 / 5

Total

17

/

20

Passed

Description

83%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description clearly and explicitly states both what the skill does and when to use it, with concrete problem signals (NaN, loss divergence) and a specific trigger clause. Trigger coverage is good but misses natural synonyms like "Weights & Biases" or "loss spike", and the "idle GPUs" claim overstates the skill's scope, which the body assigns to the watchdog tool.

Suggestions

Add natural synonyms such as "Weights & Biases", "loss spike", or "monitor my training run" to broaden trigger-term coverage.

Align claimed capabilities with the skill's actual scope: drop or reword "idle GPUs" (the body assigns process health to watchdog.py) to avoid overlap with the sibling tool.

DimensionReasoningScore

Specificity

"Periodically check WandB metrics… (NaN, loss divergence, idle GPUs)" names the domain and several concrete detection signals, plus a value statement ("Avoids wasting GPU hours on broken runs") — matching anchor 4 (several specific actions, minor gaps). Not 5: it omits the response actions (stop/escalate) and claims "idle GPUs", which the body assigns to watchdog.py; not 3, since coverage goes well beyond 1-2 actions.

4 / 5

Completeness

It explicitly answers both questions: what ("Periodically check WandB metrics… catch problems early (NaN, loss divergence, idle GPUs)") and when ("Use when training is running and you want automated health checks") with concrete trigger phrasing. This matches anchor 5; anchor 4 ("when could be more explicit") does not fit because the when-clause is concrete and specific.

5 / 5

Trigger Term Quality

Natural phrases a user would say are present: "training is running", "WandB", "NaN, loss divergence", "health checks" — good coverage per anchor 4. Not 5: common synonyms and variations like "Weights & Biases", "loss spike", or "experiment monitoring" are missing; not 3, since the existing keywords are relevant and naturally phrased rather than partially covered.

4 / 5

Distinctiveness Conflict Risk

"WandB metrics during training" carves a clear niche with distinct triggers, but the "idle GPUs" claim overlaps the sibling watchdog.py tool (the body explicitly assigns process health/GPU activity to watchdog), creating minor conflict risk with a closely related tool — anchor 4. Not 5 due to that overlap; not 3, since the training-quality framing is otherwise distinct.

4 / 5

Total

17

/

20

Passed

Validation

81%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 13 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

allowed_tools_field

'allowed-tools' contains unusual tool name(s)

Warning

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

relative_links

Relative link issues: 2 suspicious

Warning

Total

13

/

16

Passed

Repository

OpenLAIR/dr-claw

Path

skills/aris-training-check/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.