CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

implementing-llms-litgpt

Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA. Single-file implementations, no abstraction layers.

61

Quality

77%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./backend/cli/skills/ml-training/litgpt/SKILL.md
SKILL.md
Quality
Evals
Security

Quality

Content

65%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable skill body with exemplary progressive disclosure — four complete workflows plus real, clearly signaled one-level-deep references. The main weaknesses are redundancy (the LoRA fine-tuning command is repeated across two workflows with overlapping explanations) and the complete absence of validation checkpoints in workflows where mistakes are costly (dataset validity before training, model testing before deployment).

Suggestions

Add validation checkpoints to each workflow: verify the JSON dataset loads and has the expected fields before launching `litgpt finetune`, confirm the checkpoint exists in `out/finetune/` after training, and run a test generation on the merged/quantized model before the deploy step.

Fix the non-executable code blocks: move the `# comment` explanations off the `\`-continuation lines in the Workflow 2 LoRA command (bash treats them as arguments and the command fails), and replace the literal `... \` placeholder in the OOM fix with the full command.

Deduplicate the LoRA content — Workflow 1's Step 3 and Workflow 2's Steps 2–3 run nearly the same `litgpt finetune_lora` invocation three times; keep one canonical command and one LoRA parameter guide (or move the flag-by-flag guide to references/training-recipes.md).

DimensionReasoningScore

Conciseness

The body is command-dense and explains nothing Claude already knows, but it can clearly be tightened: the `litgpt finetune_lora` command with the same `--data.json_path`/`--lora_r 16` arguments appears three times across Workflow 1 and Workflow 2, the LoRA flag walkthrough repeats the Workflow 1 invocation, and the "Training too slow" section carries a Python block containing only comments ("# Already enabled by default... # No configuration needed"). Fits the 3 anchor ("mostly efficient... could be tightened") rather than 4, where over-explanation would be only minor.

3 / 5

Actionability

Most guidance is copy-paste executable: install, `litgpt download`, both finetune commands with full flags, `merge_lora`, pretrain single/multi-GPU, and a working FastAPI deploy snippet. Not 5 because some blocks break on execution — the Workflow 2 LoRA block puts `# comments` after `\` line continuations (which bash treats as arguments, invalidating the whole command), the OOM fix uses a literal `... \` placeholder, and `scripts/prepare_dataset.py` / `scripts/convert_lit_checkpoint.py` are invoked but no scripts/ bundle exists. Still well above 3, where guidance would be pseudocode or missing key details.

4 / 5

Workflow Clarity

All four workflows (fine-tune, LoRA, pretrain, deploy) have explicit copyable step checklists and per-step commands — clear sequencing. However, no workflow contains any validation or verification checkpoint: nothing checks the dataset is valid before a multi-hour training run, verifies the checkpoint after training, or tests the merged/quantized model before deployment. This matches the 3 anchor ("steps listed but validation gaps; checkpoints missing") rather than 4 ("most checkpoints present").

3 / 5

Progressive Disclosure

Structure matches the top anchor: a Quick start, four core workflows in the main file, and an "Advanced topics" section that cleanly offloads depth to four well-signaled, one-level-deep links — [references/supported-models.md], [references/training-recipes.md], [references/distributed-training.md], [references/custom-models.md] — all of which exist in the bundle as real files. Not 4: there are no organization gaps — nothing that belongs in a reference file is inlined (model zoo, recipes, FSDP, and custom-architecture detail all live in references), and navigation is a single hop.

5 / 5

Total

15

/

20

Passed

Description

83%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: third-person, concrete about capabilities, with an explicit multi-trigger "Use when..." clause naming the tool, model families, and techniques. Only a minor grammatical slip ("Use when need" instead of "Use when you need") slightly mars the trigger phrasing, and coverage omits pretraining/deployment that the skill body actually provides.

DimensionReasoningScore

Specificity

Quotes several concrete actions — "Implements and trains LLMs", "production fine-tuning with LoRA/QLoRA", "20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral)" — which is several specific actions with only minor gaps (pretraining, quantization, and deployment covered by the skill body are not mentioned). Not 5 because coverage is not comprehensive across the skill's actual capabilities; not 3 because far more than 1–2 concrete actions are named.

4 / 5

Completeness

Explicitly answers both: what — "Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures"; when — "Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA" — three concrete trigger phrases. Matches the 5 anchor ("Use when..." clause with concrete triggers); not 4 because the 'when' is fully explicit rather than needing more specificity.

5 / 5

Trigger Term Quality

Good natural keyword coverage: "LLMs", "fine-tuning", "LoRA", "QLoRA", and model names users would actually say ("Llama, Gemma, Phi, Qwen, Mistral"). Not 5 because common phrasings users would use such as "pretrain", "train from scratch", or library-comparison terms are missing; clearly above 3 since both the tool name and multiple natural synonyms are present.

4 / 5

Distinctiveness Conflict Risk

A clear niche (LitGPT specifically, with named model families and "Single-file implementations, no abstraction layers") distinguishes it from generic LLM-training skills. Not 5 because generic requests like "fine-tune a LLM with LoRA" could plausibly match several competing skills (TRL, Axolotl-oriented ones); not 3 because the tool-specific and model-specific triggers make overlap unlikely except with closely related skills.

4 / 5

Total

17

/

20

Passed

Validation

81%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 13 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

metadata_version

'metadata.version' is missing

Warning

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

referenced_paths_exist

Referenced path issues: 2 missing

Warning

Total

13

/

16

Passed

Repository

synthetic-sciences/openscience

Path

backend/cli/skills/ml-training/litgpt/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.