knowledge-distillation

Compress large language models using knowledge distillation from teacher to student models. Use when deploying smaller models with retained performance, transferring GPT-4 capabilities to open-source models, or reducing inference costs. Covers temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.

Quality

72%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./backend/cli/skills/ml-training/knowledge-distillation/SKILL.md

Quality

Content

57%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is actionable with solid executable code, but it is verbose for its context budget, duplicates the MiniLLM material already in references/minillm.md, and omits validation/feedback loops for an expensive training workflow. Tightening explanations and routing detail to the existing reference would lift the weakest dimensions.

Suggestions

Replace the inline MiniLLM/reverse-KLD section with a short summary that links to references/minillm.md, removing the duplicated explanation and code to cut ~80 lines.

Trim concept primers Claude already knows (softmax-at-temperature worked example, forward-vs-reverse KL intuition) down to one-line rules.

Add validation checkpoints to the training workflows: monitor/plot loss and eval perplexity, gate save on a held-out metric, and include a quick post-train sanity check (e.g., student loads and generates) before declaring success.

Dimension	Reasoning	Score
Conciseness	The ~450-line body mostly delivers content efficiently but explains concepts Claude already knows (e.g., softmax behavior at low vs high temperature, forward-vs-reverse KL intuition) and repeats MiniLLM material that also appears in references/minillm.md, fitting the 'mostly efficient but includes some unnecessary explanation' anchor.	3 / 5
Actionability	Provides multiple executable, copy-paste-ready code blocks (distillation_loss, reverse_kl_loss, DistillationTrainer, multi_teacher_distillation) with only minor gaps such as Strategy 2 calling undefined distill()/fine_tune() stubs, matching the 'mostly executable guidance' anchor.	4 / 5
Workflow Clarity	Training loops are sequenced conceptually, but this is an expensive batch/training operation with no validation checkpoints, loss-curve monitoring, or post-training evaluation gate, and the guideline caps batch operations without validation at 3, matching the 'steps listed but validation gaps' anchor.	3 / 5
Progressive Disclosure	Sections are well-organized and a reference file (references/minillm.md) exists, but the body never links to it and instead inlines the full MiniLLM reverse-KL treatment, duplicating content that should be offloaded, fitting the 'some structure but content that should be separate is inline' anchor.	3 / 5
	Total	13 / 20 Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is strong: it clearly states what the skill does and when to use it with concrete, natural trigger phrases and a distinctive niche. Voice is appropriately third person throughout. Minor gains are possible by adding a couple of common synonyms (e.g., 'model compression', '.safetensors').

Dimension	Reasoning	Score
Specificity	Lists several concrete capabilities ("Compress large language models using knowledge distillation from teacher to student models", "Covers temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies"), with only minor coverage gaps, matching the 'lists several specific actions' anchor.	4 / 5
Completeness	Explicitly answers both 'what' (compress LLMs via teacher-to-student knowledge distillation) and 'when' via a concrete "Use when deploying smaller models... transferring GPT-4 capabilities... or reducing inference costs" clause, matching the top anchor.	5 / 5
Trigger Term Quality	Includes natural trigger phrases users would actually say ("deploying smaller models with retained performance", "transferring GPT-4 capabilities to open-source models", "reducing inference costs") with good coverage, though a few synonyms are missing, fitting the 'good keyword coverage' anchor.	4 / 5
Distinctiveness Conflict Risk	Occupies a clear niche (knowledge distillation / model compression) with distinct, specific triggers and minimal overlap with other skills, matching the 'clear niche with distinct triggers' anchor.	5 / 5
	Total	18 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: synthetic-sciences/openscience
Path: backend/cli/skills/ml-training/knowledge-distillation/SKILL.md
Commit: fcd2e11

Reviewed: 2 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.