CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

nanogpt

Educational GPT implementation in ~300 lines. Reproduces GPT-2 (124M) on OpenWebText. Clean, hackable code for learning transformers. By Andrej Karpathy. Perfect for understanding GPT architecture from scratch. Train on Shakespeare (CPU) or OpenWebText (multi-GPU).

59

Quality

74%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./backend/cli/skills/ml-training/nanogpt/SKILL.md
SKILL.md
Quality
Evals
Security

Quality

Content

68%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable skill body with real executable commands and configs for all four workflows and a useful troubleshooting section. Its main weaknesses are the verbatim duplication between Quick start and Workflow 1, the complete absence of validation checkpoints in multi-step (and multi-day) training pipelines, and heavy inlining of detail despite having well-organized reference files.

Suggestions

Add validation checkpoints to each workflow, e.g. after data preparation verify train.bin/val.bin exist, and after training check the final val loss or sample output before declaring success — especially before the 4-day 8-GPU GPT-2 run.

Remove the verbatim duplication between Quick start and Workflow 1 by having Quick start link to the workflow section, and trim filler lines like 'Educational: Best for understanding transformers from scratch'.

Move the full config dumps and the complete custom-dataset prepare script into references/ (e.g. references/data.md) and keep only the key parameter excerpts inline, tightening the body to a true overview.

DimensionReasoningScore

Conciseness

The body is mostly efficient (terse commented configs, command blocks), but the Quick start section duplicates Workflow 1's exact three commands verbatim, and filler like 'Educational: Best for understanding transformers from scratch' and a repeated '~300 lines' claim pad it out. Noticeable tightening opportunities exist, which fits the 'mostly efficient but could be tightened' anchor better than the 'minor instances' anchor above it.

3 / 5

Actionability

Every workflow is copy-paste executable: install command, 'python data/shakespeare_char/prepare.py', 'torchrun --standalone --nproc_per_node=8 train.py config/train_gpt2.py', complete config files with values, and a full runnable custom-dataset prepare script. Troubleshooting entries give exact parameter changes (batch_size = 1, compile = True, dtype = 'bfloat16').

5 / 5

Workflow Clarity

Workflows are clearly numbered (Step 1/2/3) with expected runtimes, but there are no explicit validation checkpoints anywhere — no check that train.bin/val.bin were created before training, no val-loss check before the 4-day 8-GPU run, no verification step before sampling. This matches 'sequence present but checkpoints missing or implicit' rather than the level-4 anchor, which requires most checkpoints to be present.

3 / 5

Progressive Disclosure

The Advanced topics section cleanly signals three one-level-deep references ('See [references/architecture.md](references/architecture.md) for GPT block structure...'), and all three files exist in the bundle. However, the body inlines substantial detail that could live in those references (full config dumps, the complete custom-dataset script, hardware requirement tables) and duplicates Quick start content, so it falls just short of the 'content appropriately split' level-5 anchor.

4 / 5

Total

15

/

20

Passed

Description

70%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A specific, distinctive description with concrete capabilities and good natural keywords, held back mainly by the absence of any explicit 'Use when...' trigger guidance and by filler phrases. Adding a trigger clause and natural synonyms like 'language model' or 'fine-tune' would lift it substantially.

Suggestions

Add an explicit trigger clause, e.g. 'Use when the user wants to train or understand a small GPT/language model from scratch, reproduce GPT-2, or experiment with transformer architecture.'

Include natural synonyms users would say, such as 'language model', 'fine-tune', and 'from scratch', to broaden trigger-term coverage.

Trim filler like 'Perfect for understanding GPT architecture from scratch' and mention concrete additional capabilities (fine-tuning from OpenAI GPT-2 checkpoints, sampling/generating text) instead.

DimensionReasoningScore

Specificity

Several concrete capabilities are stated — 'Educational GPT implementation in ~300 lines', 'Reproduces GPT-2 (124M) on OpenWebText', 'Train on Shakespeare (CPU) or OpenWebText (multi-GPU)' — but coverage has gaps (fine-tuning and text sampling are absent) and filler like 'Perfect for understanding GPT architecture from scratch' pads it, so it sits between the 'several specific actions, minor gaps' and 'comprehensive' anchors.

4 / 5

Completeness

The 'what' is clear (educational GPT implementation that reproduces GPT-2 and trains on Shakespeare/OpenWebText), but there is no explicit 'Use when...' clause or equivalent trigger guidance — 'Perfect for understanding GPT architecture from scratch' only weakly implies when to use it. Per the judging guidelines, a missing 'Use when...' clause caps completeness at 3.

3 / 5

Trigger Term Quality

Good natural keyword coverage: 'GPT', 'GPT-2', 'transformers', 'train', 'learning', 'OpenWebText', 'Shakespeare' — phrases a user would plausibly say. A few natural terms are missing, such as 'language model', 'fine-tune', and 'from scratch' as a trigger phrase, keeping it below the comprehensive level-5 anchor.

4 / 5

Distinctiveness Conflict Risk

The description carves out a clear niche — a ~300-line educational GPT that reproduces GPT-2 (124M) on OpenWebText — with distinct triggers (nanoGPT, GPT-2, Shakespeare training) that are unlikely to fire for production frameworks like HuggingFace or Megatron-LM, matching the 'clear niche with distinct triggers; minimal conflict risk' anchor.

5 / 5

Total

16

/

20

Passed

Validation

87%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

metadata_version

'metadata.version' is missing

Warning

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

14

/

16

Passed

Repository

synthetic-sciences/openscience

Path

backend/cli/skills/ml-training/nanogpt/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.