nanogpt

Educational GPT implementation in ~300 lines. Reproduces GPT-2 (124M) on OpenWebText. Clean, hackable code for learning transformers. By Andrej Karpathy. Perfect for understanding GPT architecture from scratch. Train on Shakespeare (CPU) or OpenWebText (multi-GPU).

Quality

75%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./backend/cli/skills/ml-training/nanogpt/SKILL.md

Quality

Content

80%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is highly actionable and well-structured with real, cleanly organized reference files, but its multi-step training/data-prep workflows lack validation checkpoints, capping workflow clarity. Adding verify-and-retry steps would lift the weakest dimension.

Suggestions

Insert explicit validation steps into each workflow (e.g., after prepare.py: 'Verify train.bin/val.bin exist and are non-empty'; after loading GPT-2: 'Confirm checkpoint loaded'; during training: 'Check val loss decreases before eval_interval') to satisfy the batch-operation feedback-loop requirement.

Add an error-recovery feedback loop for the long training runs (e.g., 'If loss spikes or NaNs, reduce learning_rate and resume from last checkpoint').

Trim the 'Use alternatives instead' list and the duplicated Quick start vs Workflow 1 commands to tighten conciseness toward 5.

Dimension	Reasoning	Score
Conciseness	The body is largely lean executable guidance with little concept-explanation padding, but sections like 'Use alternatives instead' (HuggingFace/Megatron/LitGPT) restate what a competent model already knows and the Quick start duplicates Workflow 1's three commands, so minor trimming would reach a 5.	4 / 5
Actionability	It provides fully executable, copy-paste-ready commands and configs (pip install, train.py invocations, torchrun DDP, full config files, custom-dataset prepare.py, issue-fix snippets) covering all common cases from Shakespeare to GPT-2 reproduction, fine-tuning, and custom data.	5 / 5
Workflow Clarity	Workflows are clearly numbered (Step 1/2/3) across all four paths, but this is a batch/training skill with no validation checkpoints — no 'verify loss decreases', no check that train.bin/val.bin exist after data prep, no confirmation that GPT-2 weights loaded — so per the rubric's batch-operation cap workflow clarity cannot exceed 3.	3 / 5
Progressive Disclosure	SKILL.md serves as a clear overview with well-signaled, one-level-deep markdown links to real files (references/architecture.md, training.md, data.md, all verified present), with detail appropriately split into those references and easy navigation.	5 / 5
	Total	17 / 20 Passed

Description

70%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific and distinctive with strong natural trigger terms, but it lacks an explicit 'Use when...' clause, so its completeness is capped. Adding a concrete trigger-guidance sentence would lift the weakest dimension.

Suggestions

Add an explicit 'Use when...' clause (e.g., 'Use when learning GPT/transformer architecture, reproducing GPT-2, or teaching nanoGPT') to satisfy the completeness 'when' requirement and raise the cap.

Include the skill's own name 'nanoGPT' and common learner synonyms like 'language model' or 'from-scratch transformer' to round out trigger term coverage.

Trim evaluative fluff ('Clean, hackable code', 'Perfect for understanding') in favor of one more concrete capability to push specificity toward 5.

Dimension	Reasoning	Score
Specificity	Names the domain plus several concrete actions ('Reproduces GPT-2 (124M)', 'Train on Shakespeare (CPU) or OpenWebText (multi-GPU)', 'understanding GPT architecture from scratch'), though phrases like 'Clean, hackable code' and 'Perfect for understanding' are evaluative rather than additional concrete capabilities, keeping it just below a 5.	4 / 5
Completeness	It clearly answers 'what' (educational GPT implementation reproducing GPT-2 124M, training on Shakespeare/OpenWebText) but has no explicit 'Use when...' trigger clause — 'when' is only weakly implied by 'for learning transformers' and 'Perfect for understanding', which per the rubric caps completeness at 3.	3 / 5
Trigger Term Quality	Includes natural terms a user would say ('GPT', 'GPT-2', 'transformers', 'Shakespeare', 'OpenWebText', 'Andrej Karpathy', 'GPU'), but is missing some synonyms a learner might use such as 'nanoGPT', 'language model', or explicit file/task terms, so it stops short of comprehensive coverage.	4 / 5
Distinctiveness Conflict Risk	It carves a clear, well-scoped niche (educational GPT-2 124M reproduction tied to OpenWebText/Shakespeare and Andrej Karpathy) with distinct triggers and minimal overlap risk against other ML skills.	5 / 5
	Total	16 / 20 Passed

Validation

87%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

Criteria	Description	Result
metadata_version	'metadata.version' is missing	Warning
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	14 / 16 Passed

Repository: synthetic-sciences/openscience
Path: backend/cli/skills/ml-training/nanogpt/SKILL.md
Commit: 543b17f

Reviewed: 2 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.