CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

training-llms-megatron

Trains large language models (2B-462B parameters) using NVIDIA Megatron-Core with advanced parallelism strategies. Use when training models >1B parameters, need maximum GPU efficiency (47% MFU on H100), or require tensor/pipeline/sequence/context/expert parallelism. Production-ready framework used for Nemotron, LLaMA, DeepSeek.

65

Quality

82%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

SKILL.md
Quality
Evals
Security

Quality

Content

75%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable body: copy-able checklists, complete executable launch scripts, concrete troubleshooting fixes, and a clean pointer structure to four real reference files. The main gaps are a missing scripts/examples bundle referenced in quick start, absent validation checkpoints in the MoE and optimization workflows, and partial duplication between inline scripts and training-recipes.md.

Suggestions

Bundle or remove references to examples/run_simple_mcore_train_loop.py and ./examples/llama/train_llama3_8b_fp8.sh — the quick start currently points at files that don't ship with the skill.

Add explicit validation/feedback checkpoints inside Workflows 2 and 3 (e.g., 'after launch, verify MFU >40%; if lower, apply fixes in Common issues') rather than keeping error recovery only in the separate issues section.

Move the full inline launch scripts to references/training-recipes.md, keeping one representative script in the body, to remove the duplication between SKILL.md and training-recipes.md.

DimensionReasoningScore

Conciseness

The body is dominated by commands, flag lists, tables, and checklists rather than concept explanations, but a few sentences could be trimmed (e.g., 'Expert parallelism distributes different experts across GPUs, reducing memory while maintaining capacity' and the memory-savings arithmetic block). Matches anchor 4 (efficient, minor instances of over-explanation) rather than anchor 5.

4 / 5

Actionability

Nearly all guidance is copy-paste-ready: full torchrun invocations with complete flag sets for 70B and Mixtral, an OOM/divergence fix set with exact flags. Minor gaps keep it at anchor 4: 'torchrun ... --micro-batch-size $MBS' uses a literal ellipsis placeholder, and the quick start references 'examples/run_simple_mcore_train_loop.py' and './examples/llama/train_llama3_8b_fp8.sh', which are not present in the bundle.

4 / 5

Workflow Clarity

Each of the three workflows ships a copy-able checklist with numbered, well-sequenced steps, and Workflow 1 includes a monitoring checkpoint ('MFU: Target >40% on H100', 'Loss: Should decrease steadily'). Anchor 4 rather than 5 because Workflows 2 and 3 lack validation checkpoints, and validate/fix/retry feedback loops live only in the separate Common issues section rather than inside the workflows.

4 / 5

Progressive Disclosure

Four one-level-deep references (parallelism-guide, benchmarks, production-examples, training-recipes) are clearly signaled with descriptive pointers in the Advanced topics section, and all four exist with substantial content. Anchor 4 rather than 5 because the body inlines complete ~50-line launch scripts that overlap training-recipes.md, which contains full LLaMA 70B/8B and Mixtral configurations.

4 / 5

Total

16

/

20

Passed

Description

83%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description with explicit what/when structure, concrete parameter ranges, and a full enumeration of parallelism modes. Its main weaknesses are a single-verb action list and a somewhat broad >1B-parameter trigger that could overlap with alternative training-framework skills.

DimensionReasoningScore

Specificity

Concrete specifics are present: 'Trains large language models (2B-462B parameters) using NVIDIA Megatron-Core with advanced parallelism strategies' and an explicit list of 'tensor/pipeline/sequence/context/expert parallelism'. Falls short of anchor 5 because action coverage centers on a single verb (train) with minor gaps (no fine-tuning, checkpointing, or data-prep actions); above anchor 3 because it names far more than 1-2 concrete capabilities with concrete numbers.

4 / 5

Completeness

Explicitly answers both what ('Trains large language models (2B-462B parameters) using NVIDIA Megatron-Core with advanced parallelism strategies') and when with concrete trigger phrases ('Use when training models >1B parameters, need maximum GPU efficiency (47% MFU on H100), or require tensor/pipeline/sequence/context/expert parallelism'). This matches the anchor-5 example structure exactly.

5 / 5

Trigger Term Quality

Includes natural user-facing phrases such as 'training models >1B parameters', 'maximum GPU efficiency (47% MFU on H100)', and model names 'Nemotron, LLaMA, DeepSeek'. A few common natural terms are missing ('pretrain', 'distributed training', 'fine-tune', 'GPU cluster'), which matches anchor 4 rather than the comprehensive synonym coverage of anchor 5.

4 / 5

Distinctiveness Conflict Risk

Clear niche (NVIDIA Megatron-Core, large-scale production training) with distinct framework-specific triggers. Not anchor 5 because the broad 'training models >1B parameters' trigger overlaps with closely related skills like DeepSpeed or FSDP — alternatives the skill itself acknowledges.

4 / 5

Total

17

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

15

/

16

Passed

Repository

davila7/claude-code-templates

Path

cli-tool/components/skills/ai-research/distributed-training-megatron-core/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.