moe-training

Train Mixture of Experts (MoE) models using DeepSpeed or HuggingFace. Use when training large-scale models with limited compute (5× cost reduction vs dense models), implementing sparse architectures like Mixtral 8x7B or DeepSeek-V3, or scaling model capacity without proportional compute increase. Covers MoE architectures, routing mechanisms, load balancing, expert parallelism, and inference optimization.

Quality

81%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Quality

Content

62%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The skill is rich with executable code and well-structured references, but leans verbose on conceptual explanations and lacks an explicit sequenced workflow with validation checkpoints for large-scale training. Progressive disclosure and actionability are its strongest dimensions.

Suggestions

Add an explicit ordered workflow (prepare data -> configure MoE -> launch training -> validate loss/imbalance -> tune capacity/LR) with concrete validation commands, since large-scale batch training warrants feedback loops and the current cap caps workflow_clarity at 3.

Trim conceptual padding (the ASCII routing diagram and repeated top-k/capacity explanations) and move detailed tuning heuristics to references/training.md to improve conciseness.

Make the inference optimization example fully runnable — replace the illustrative model.load_expert() call with a concrete expert-loading pattern or link to references/inference.md.

Dimension	Reasoning	Score
Conciseness	Mostly efficient with executable code, but includes padded conceptual scaffolding (ASCII routing diagram, repeated explanations of top-k/capacity, general commentary Claude already knows about MoE) that could be trimmed.	3 / 5
Actionability	Provides copy-paste-ready Python (MoELayer, load-balancing loss, Mixtral block) and concrete DeepSpeed bash configs with specific flags; minor gaps such as the inference example's load_expert is illustrative rather than runnable.	4 / 5
Workflow Clarity	Sections are thematic (Architecture, Routing, Training Config) rather than a sequenced train->validate->tune workflow, and there are no validation checkpoints or feedback loops for these batch/destructive-scale training runs despite the rubric cap requiring them.	3 / 5
Progressive Disclosure	SKILL.md is an organized overview with a clear 'See Also' section pointing to three real one-level-deep reference files (architectures.md, training.md, inference.md); bulk detail is appropriately offloaded, though some advanced material could move entirely to references.	4 / 5
	Total	14 / 20 Passed

Description

100%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, specific description with explicit what/when guidance, concrete model anchors, and natural trigger phrases. It clearly distinguishes MoE training from general ML training skills.

Dimension	Reasoning	Score
Specificity	Lists many concrete capabilities — architectures, routing mechanisms, load balancing, expert parallelism, inference optimization — plus concrete model examples (Mixtral 8x7B, DeepSeek-V3), giving comprehensive coverage of the skill's scope.	5 / 5
Completeness	Explicitly answers both what ('Train Mixture of Experts models using DeepSpeed or HuggingFace' + coverage list) and when ('Use when training large-scale models with limited compute...') with concrete trigger phrases.	5 / 5
Trigger Term Quality	Rich natural triggers users would actually say — 'training large-scale models with limited compute', 'scaling model capacity', plus synonyms (MoE, Mixture of Experts, sparse architectures) and model names, with the explicit 'Use when' clause.	5 / 5
Distinctiveness Conflict Risk	Highly specific MoE niche with distinct triggers (sparse architectures, expert parallelism, Mixtral/DeepSeek-V3) unlikely to collide with general training skills.	5 / 5
	Total	20 / 20 Passed

Validation

81%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 13 / 16 Passed

Validation for skill structure

Criteria	Description	Result
skill_md_line_count	SKILL.md is long (528 lines); consider splitting into references/ and linking	Warning
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning
relative_links	Relative link issues: 1 missing	Warning

	Total	13 / 16 Passed

Repository: synthetic-sciences/openscience
Path: backend/cli/skills/ml-training/moe-training/SKILL.md
Commit: fcd2e11

Reviewed: 1 day ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.