implementing-llms-litgpt

Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA. Single-file implementations, no abstraction layers.

Quality

82%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

72%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

Highly actionable with copy-paste-ready code and well-structured progressive disclosure to real reference files. Weakest spots are repeated LoRA command blocks that hurt conciseness and the absence of validation feedback loops in batch training workflows.

Suggestions

Consolidate the repeated litgpt finetune_lora command blocks: show the full parameter set once and reference it, rather than reprinting overlapping invocations across Workflow 1, Workflow 2, and Common issues.

Add explicit validation checkpoints to the training workflows (e.g., 'verify loss is decreasing before step N', 'if loss diverges, reduce learning rate and restart'), creating a validate→fix→retry loop for these batch operations.

Replace the 'Common issues' OOM guidance that re-states LoRA with a brief pointer back to the LoRA workflow, keeping only the genuinely new advice (gradient checkpointing, micro-batch tuning).

Dimension	Reasoning	Score
Conciseness	Mostly efficient with executable code and minimal concept re-explanation, but the litgpt finetune_lora command is repeated verbatim across sections with overlapping parameters and the OOM section re-states LoRA guidance already shown, so it could be tightened.	3 / 5
Actionability	Fully executable, copy-paste-ready commands and complete code span every workflow (load/generate, finetune, LoRA, pretrain, quantize, deploy API), covering the common cases comprehensively.	5 / 5
Workflow Clarity	Each workflow has a clearly sequenced numbered checklist, but training and fine-tuning are batch operations with no explicit validate→fix→retry feedback loops, which caps workflow clarity at 3 per the rubric guideline.	3 / 5
Progressive Disclosure	The body is an overview with four one-level-deep references (supported-models, training-recipes, distributed-training, custom-models), all clearly signaled in the Advanced topics section and verified to exist, with content appropriately split between inline and reference files.	5 / 5
	Total	16 / 20 Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, well-scoped description that clearly states both what it does and when to use it, with concrete trigger phrases and a distinct LitGPT niche. Keyword coverage is good but could add a few more natural synonyms to reach full marks.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions — 'Implements and trains LLMs', 'production fine-tuning with LoRA/QLoRA', 'Single-file implementations' — giving comprehensive coverage of the skill's capabilities, matching the score-5 anchor.	5 / 5
Completeness	Explicitly answers 'what' ('Implements and trains LLMs...') and 'when' with a concrete 'Use when need clean model implementations, educational understanding..., or production fine-tuning...' trigger clause.	5 / 5
Trigger Term Quality	Strong natural keyword coverage (LLMs, LitGPT, Lightning AI, LoRA/QLoRA, named model families) but a few common synonyms are missing, placing it just below the comprehensive score-5 anchor.	4 / 5
Distinctiveness Conflict Risk	Scoped specifically to Lightning AI's LitGPT with single-file, no-abstraction implementations — a clear niche with distinct triggers and minimal overlap with broader training skills.	5 / 5
	Total	19 / 20 Passed

Validation

81%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 13 / 16 Passed

Validation for skill structure

Criteria	Description	Result
metadata_version	'metadata.version' is missing	Warning
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning
referenced_paths_exist	Referenced path issues: 2 missing	Warning

	Total	13 / 16 Passed

Repository: synthetic-sciences/openscience
Path: backend/cli/skills/ml-training/litgpt/SKILL.md
Commit: 543b17f

Reviewed: 5 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.