CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

nemo-mbridge-resiliency

Resiliency features in Megatron Bridge including fault tolerance, straggler detection, in-process restart, preemption, and re-run state machine.

Quality

85%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

90%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

An excellent, dense reference document: copy-paste-ready enablement code for every feature, non-obvious defaults and semantics captured in tight tables, a pitfalls section that encodes real failure modes (ft_launcher vs torchrun, NCCL watchdog vs hard_timeout), and a verification section with executable checks and expected outputs. The only structural observation is that it is fully self-contained — some per-feature parameter tables could be offloaded to reference files if the skill grows.

DimensionReasoningScore

Conciseness

The body is a lean reference: every section is copy-paste code, parameter tables with defaults Claude cannot know, or targeted one-line facts ('Exit codes: 16 = resume to disambiguate, 17 = failed validation'). It never explains concepts Claude already knows (no 'what is fault tolerance' or NCCL tutorials). Not score 4: there are no over-explanatory passages left to trim — the only prose ('Section-based timeout monitoring covers...') conveys non-obvious behavior.

5 / 5

Actionability

Fully executable throughout: complete Python config snippets for every feature, the exact ft_launcher command with all flags, required env vars ('export TORCH_NCCL_RETHROW_CUDA_ERRORS=0'), and runnable verification commands with expected log lines ('Look for [FaultTolerance] / [RankMonitorServer] log lines'). Not score 4: no key details are missing; even failure semantics and exit codes are given.

5 / 5

Workflow Clarity

The structure (Enablement → Code Anchors → Pitfalls → Verification) gives each feature a clear enable path plus an explicit validation section with concrete commands and expected outputs — most checkpoints are present. Not score 5: the document is a per-feature reference rather than a sequenced workflow, and there are no explicit feedback loops (e.g., 'if the log line is missing, check X'); pitfalls partially but not systematically cover error recovery. Not score 3: validation steps are explicit and per-feature, not merely implicit.

4 / 5

Progressive Disclosure

No bundle files exist (references/, scripts/, assets/ are absent), so this is scored on structure alone: sections are well-organized with clear headers, and external pointers (@docs/training/resiliency.md, per-feature Code Anchors into repo paths) are one level deep and clearly signaled. Not score 5: the body is self-contained rather than an overview deferring detail to reference files — full parameter tables for six sub-features (~300 lines) are inlined, some of which (e.g., the 14-row InProcessRestartConfig table) could live in reference files; not score 3: organization is good and all pointers are clear.

4 / 5

Total

18

/

20

Passed

Description

75%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A specific, well-scoped description that clearly states what the skill covers with an accurate feature enumeration, but it omits the 'when to use' trigger guidance from the description itself (parking it in a separate when_to_use field) and lacks troubleshooting-oriented synonyms like 'training hangs'. These gaps cap completeness and trigger coverage despite otherwise excellent specificity and distinctiveness.

Suggestions

Fold the when_to_use triggers into the description field, e.g. append: 'Use when enabling resiliency features or investigating training hangs, straggler detection failures, or broken restarts' — the rubric caps completeness at 3 without an explicit 'Use when' clause in the description.

Add natural troubleshooting synonyms users would actually say ('training hangs', 'hang detection', 'automatic restart', 'crash recovery') to improve trigger-term coverage beyond the current feature names.

Consider mentioning the 'nvidia-resiliency-ext' package name (currently only in when_to_use) since users debugging imports would naturally search for it.

DimensionReasoningScore

Specificity

The description names the domain ('Resiliency features in Megatron Bridge') and enumerates all five concrete capability areas the skill covers: 'fault tolerance, straggler detection, in-process restart, preemption, and re-run state machine' — matching the body's full coverage. Not score 4: there are no minor gaps; the enumerated list is comprehensive relative to the skill's scope.

5 / 5

Completeness

It has a clear 'what' (five named resiliency features), but no 'Use when...' clause or equivalent trigger guidance inside the description itself — the when-guidance lives only in a separate when_to_use field. Per the rubric guideline, a missing explicit trigger clause caps completeness at 3. Not score 4: the 'when' is not merely implicit in this text, it is entirely absent from the description field.

3 / 5

Trigger Term Quality

Includes strong natural domain terms users would say ('fault tolerance', 'straggler detection', 'preemption', 'in-process restart'), but misses common synonyms and phrasings users actually use when troubleshooting, such as 'training hangs', 'hang detection', 'crash recovery', or 'automatic restart' — several of which appear only in the separate when_to_use field, not the description. Not score 5: anchor 5 requires comprehensive coverage including synonyms/extensions, which is absent; not score 3: the terms present go well beyond 'some relevant keywords'.

4 / 5

Distinctiveness Conflict Risk

'Resiliency features in Megatron Bridge' establishes a clear niche with distinct, framework-specific trigger terms (Megatron Bridge, in-process restart, re-run state machine) that are unlikely to fire for unrelated skills. Not score 4: the trigger surface is both specific and non-overlapping with generic training or checkpointing skills.

5 / 5

Total

17

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

referenced_paths_exist

Referenced path issues: 1 missing, 1 deeper-than-1-level

Warning

Total

15

/

16

Passed

Repository

NVIDIA/skills

Path

skills/nemo-mbridge-resiliency/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.