knowledge-distillation

Compress large language models using knowledge distillation from teacher to student models. Use when deploying smaller models with retained performance, transferring GPT-4 capabilities to open-source models, or reducing inference costs. Covers temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.

Quality

82%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

65%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable with extensive executable code, but it is padded with redundant re-explanations of the same concepts and lacks validation checkpoints in its training workflows. It also fails to leverage its existing minillm.md reference, keeping organization at an intermediate level.

Suggestions

Deduplicate the forward/reverse KLD and temperature-scaling explanations: keep one authoritative treatment in Core Concepts and reference it from Quick Start and Best Practices rather than re-explaining each time.

Add explicit validation/feedback checkpoints to the training workflows (e.g., evaluate student-vs-teacher quality and only proceed or deploy when a threshold passes), since long-running batch training is a context the rubric expects validation in.

Replace the inline MiniLLM section with a concise summary plus a link to references/minillm.md (e.g., 'See [minillm.md](references/minillm.md) for the full reverse-KLD writeup') so the reference file is actually navigated and the body stays lean.

Dimension	Reasoning	Score
Conciseness	At 458 lines the body restates concepts Claude already knows multiple times — forward vs reverse KLD is explained in Quick Start, Core Concepts, and Best Practices, and the reverse-KL loss is implemented twice with overlapping code; it is not level 1 because most content is concrete code rather than prose fluff, but redundancy keeps it below the lean level-3 bar.	2 / 3
Actionability	Provides fully executable, copy-paste-ready code throughout (distillation_loss, reverse_kl_loss, a complete DistillationTrainer, multi-teacher distillation) with specific parameters and a runnable train_distilled_model entry point, matching the level-3 anchor.	3 / 3
Workflow Clarity	Multi-step strategies (two-stage distillation, multi-teacher) are listed but lack validation checkpoints or feedback loops for the long-running batch training operations; per the rubric, missing validation in batch operations caps workflow clarity at 2 rather than 3.	2 / 3
Progressive Disclosure	A references/minillm.md bundle (334 lines) exists but the body never links to it and instead duplicates MiniLLM content inline, and there are no markdown navigation links at all; structure exists but references are not signaled, keeping it at level 2 rather than the well-navigated level-3 bar.	2 / 3
	Total	9 / 12 Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific, trigger-rich, and fully answers both what the skill does and when to use it, with a distinct niche that minimizes conflict risk. It is a strong, concise description.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions ('Compress large language models using knowledge distillation', 'transferring GPT-4 capabilities to open-source models', 'reducing inference costs') plus enumerated techniques (temperature scaling, reverse KLD, logit distillation), matching the level-3 anchor; it exceeds level 2 which expects only some actions.	3 / 3
Completeness	Explicitly states both what ('Compress large language models using knowledge distillation from teacher to student models') and when ('Use when deploying smaller models with retained performance, transferring GPT-4 capabilities... or reducing inference costs'), matching the level-3 'what AND when' anchor.	3 / 3
Trigger Term Quality	Natural user phrasings are well covered ('compress large language models', 'deploying smaller models with retained performance', 'reducing inference costs', 'transferring GPT-4 capabilities to open-source models'); it is above level 2 because common variations are present rather than missing.	3 / 3
Distinctiveness Conflict Risk	Occupies a clear niche (knowledge distillation / model compression) with distinct triggers (GPT-4 to open-source, reverse KLD, MiniLLM) that are unlikely to fire for unrelated skills, matching the level-3 anchor.	3 / 3
	Total	12 / 12 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: Orchestra-Research/AI-Research-SKILLs
Path: 19-emerging-techniques/knowledge-distillation/SKILL.md
Commit: 773a529

Reviewed: about 12 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.