implementing-llms-litgpt

Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA. Single-file implementations, no abstraction layers.

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

72%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The skill is highly actionable and well-structured with clean one-level references, but loses points on conciseness due to redundant command blocks and on workflow clarity due to missing validation/verification steps for destructive and large-scale batch operations.

Suggestions

Add explicit verification checkpoints to destructive/batch workflows — e.g. after fine-tuning 'verify out/finetune/final/ loads and generates', after GGUF conversion 'test the .gguf in llama.cpp before deploying', after pretraining 'confirm checkpoint shard integrity before SLURM scale-out'.

De-duplicate the finetune_lora command: show the full flagged version once in Workflow 2 and reference it (or link to a reference file) from Workflow 1 and 3 instead of re-pasting.

Trim padding such as the full two-record Alpaca JSON example (one record suffices) and commentary like 'Most memory-efficient option' to improve token efficiency.

Dimension	Reasoning	Score
Conciseness	The body is dense and mostly efficient, but carries redundancy (the finetune_lora command appears in three workflows) and light padding ('Most memory-efficient option', the full Alpaca JSON block) that could be tightened, so it sits at 'mostly efficient' rather than fully lean.	2 / 3
Actionability	Every workflow provides fully executable, copy-paste-ready bash/python with specific flags, model names, and hyperparameters, matching the score-3 anchor.	3 / 3
Workflow Clarity	Workflows are sequenced with checklists, but batch/destructive operations (pretraining on 8 nodes, quantization, GGUF conversion, deployment) lack explicit validation/verification checkpoints and error-recovery loops, which the rubric caps at 2.	2 / 3
Progressive Disclosure	The body is a clear overview with four well-signaled, one-level-deep references (supported-models, training-recipes, distributed-training, custom-models), all verified to exist in references/ and each linked once in the 'Advanced topics' section.	3 / 3
	Total	10 / 12 Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific, trigger-rich, and explicitly pairs capabilities with a 'Use when' invocation clause, hitting all four anchors at the top of the scale. Minor grammatical roughness ('Use when need') does not reduce any dimension's score.

Dimension	Reasoning	Score
Specificity	Quotes multiple concrete actions ('Implements and trains LLMs', 'production fine-tuning with LoRA/QLoRA', 'Single-file implementations') and enumerates 20+ architectures, matching the score-3 anchor of listing multiple specific concrete actions.	3 / 3
Completeness	Explicitly states both what it does and when to use it via the 'Use when need clean model implementations, educational understanding of architectures, or production fine-tuning' clause, matching the score-3 anchor.	3 / 3
Trigger Term Quality	Uses natural domain keywords a user would say ('LLMs', 'fine-tuning', 'LoRA/QLoRA', 'Llama, Gemma, Phi, Qwen, Mistral', 'clean model implementations'), giving good coverage rather than jargon.	3 / 3
Distinctiveness Conflict Risk	The LitGPT-specific framing ('Lightning AI's LitGPT', 'Single-file implementations, no abstraction layers') carves a clear niche unlikely to trigger for competing training tools.	3 / 3
	Total	12 / 12 Passed

Validation

87%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning
referenced_paths_exist	Referenced path issues: 2 missing	Warning

	Total	14 / 16 Passed

Repository: synthetic-sciences/openscience
Path: backend/cli/skills/ml-training/litgpt/SKILL.md
Commit: 52845c3

Reviewed: about 13 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.