distributed-llm-pretraining-torchtitan

Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

High

Do not use without reviewing

Quality

Content

72%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is highly actionable and well-structured with verified progressive-disclosure references, but loses points on minor marketing verbosity and missing explicit validation checkpoints in its training workflows.

Suggestions

Remove marketing claims ('65%+ speedups', 'PyTorch's official platform') from the Quick start to tighten conciseness.

Add explicit validation/verification steps to each workflow (e.g., confirm GPU health, verify checkpoint integrity before resuming, validate config before launch) to lift workflow_clarity.

Make the 'Monitor and checkpoint' step concrete with a validation command rather than just opening TensorBoard.

Dimension	Reasoning	Score
Conciseness	The body is mostly efficient with executable TOML and commands, but includes marketing padding ('achieving 65%+ speedups over baselines', 'PyTorch's official platform') and could be tightened.	2 / 3
Actionability	Provides fully executable, copy-paste-ready guidance: concrete torchrun/sbatch commands, complete TOML configs, and exact CLI flags across all four workflows.	3 / 3
Workflow Clarity	Workflows use clear step checklists, but lack explicit validation/verification checkpoints for batch distributed-training operations, which caps clarity at 2.	2 / 3
Progressive Disclosure	Clear overview body with well-signaled, one-level-deep references (fsdp.md, float8.md, checkpoint.md, custom-models.md) that all exist as real files with substantive content.	3 / 3
	Total	10 / 12 Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is concise, third-person, and concrete, with explicit what/when guidance and natural trigger terms covering named models and GPU scales. It is a strong, low-conflict skill description.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete capabilities ('PyTorch-native distributed LLM pretraining', '4D parallelism (FSDP2, TP, PP, CP)', 'Float8, torch.compile, and distributed checkpointing') rather than vague language.	3 / 3
Completeness	Explicitly answers both what it does and when to use it via the 'Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale...' clause.	3 / 3
Trigger Term Quality	Covers natural user terms ('pretraining', 'Llama 3.1', 'DeepSeek V3', 'custom models', '8 to 512+ GPUs', 'Float8') that a user would actually say when needing this skill.	3 / 3
Distinctiveness Conflict Risk	A clearly defined niche (torchtitan-based distributed pretraining with 4D parallelism) with distinct triggers unlikely to collide with other skills.	3 / 3
	Total	12 / 12 Passed

Validation

87%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning
referenced_paths_exist	Referenced path issues: 2 missing	Warning

	Total	14 / 16 Passed

Repository: synthetic-sciences/openscience
Path: backend/cli/skills/ml-training/torchtitan/SKILL.md
Commit: 52845c3

Reviewed: about 9 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.