nlp-pretraining

Best practices for language model pretraining and fine-tuning. Use when generating or reviewing NLP training code.

Quality

81%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

87%Weight 40%Scale 1-3

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is an exceptionally lean, actionable reference of hyperparameter ranges and evaluation metrics with clean organization, but it reads as a static recipe rather than a sequenced workflow with validation checkpoints.

Suggestions

Add a brief ordered training workflow (prepare data -> configure optimizer/schedule -> train -> evaluate) with a validation checkpoint (e.g. monitor eval loss, early-stop on degradation) to lift workflow clarity.

Include one minimal executable snippet (e.g. a HuggingFace Trainer / PEFT LoRA config) so users can copy a working starting point.

Dimension	Reasoning	Score
Conciseness	The body is lean and assumes Claude's competence—only parameter ranges with no concept explanation or padding—matching anchor 3 ('every token earns its place').	3 / 3
Actionability	Provides concrete, copy-applicable numeric guidance ('AdamW optimizer, lr=2e-5 to 5e-5', 'LoRA: r=8-64, alpha=16-128, apply to q/v projections'); for an instruction-only skill the guidance is actionable without code, meeting anchor 3.	3 / 3
Workflow Clarity	Content is organized into clear sections but presents a parameter reference rather than a sequenced workflow, with no explicit step ordering or validation checkpoints, capping at anchor 2.	2 / 3
Progressive Disclosure	Under 50 lines with no external references and three well-organized sections (fine-tuning recipe, parameter-efficient methods, evaluation), qualifying for anchor 3 under the simple-skills note.	3 / 3
	Total	11 / 12 Passed

Description

75%Weight 40%Scale 1-3

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is concise and well-formed with an explicit 'Use when' trigger and a clear niche, but it under-specifies concrete capabilities and omits common keyword variations like 'LLM', 'transformer', and 'fine-tune'.

Suggestions

Add concrete capabilities to the description, e.g. 'configure optimizers, learning-rate schedules, and LoRA/adapter training'.

Broaden trigger terms with natural variations users say, such as 'LLM', 'transformer', 'BERT', 'GPT', and 'fine-tune a model'.

Dimension	Reasoning	Score
Specificity	Names the domain ('language model pretraining and fine-tuning') and two actions ('generating or reviewing NLP training code'), but does not enumerate multiple concrete capabilities; falls at anchor 2 rather than 3's 'lists multiple specific concrete actions'.	2 / 3
Completeness	Explicitly answers both what ('Best practices for language model pretraining and fine-tuning') and when via an explicit 'Use when generating or reviewing NLP training code' trigger clause, meeting anchor 3.	3 / 3
Trigger Term Quality	Includes relevant terms ('language model', 'pretraining', 'fine-tuning', 'NLP training code') but omits common variations users would say such as 'LLM', 'transformer', 'fine-tune', 'BERT/GPT', matching anchor 2.	2 / 3
Distinctiveness Conflict Risk	Targets a clearly defined niche (NLM pretraining/fine-tuning) with a distinct explicit trigger ('NLP training code') unlikely to broadly conflict with other skills, matching anchor 3.	3 / 3
	Total	10 / 12 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: aiming-lab/AutoResearchClaw
Path: researchclaw/skills/builtin/domain/nlp-pretraining/SKILL.md
Commit: e2e23c9

Reviewed: 6 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.