nlp-pretraining

Best practices for language model pretraining and fine-tuning. Use when generating or reviewing NLP training code.

Quality

71%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./researchclaw/skills/builtin/domain/nlp-pretraining/SKILL.md

Quality

Content

80%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is a concise, well-organized reference recipe with highly specific numeric guidance, but it provides configuration values rather than executable code and lacks an explicit workflow with validation checkpoints. It works well as lookup material but would benefit from a runnable example and a brief end-to-end training sequence.

Suggestions

Add one short runnable code snippet (e.g. a HuggingFace Trainer setup) to lift actionability toward anchor 5 and bridge the config-to-code gap.

Add a brief end-to-end fine-tuning workflow with a validation/checkpoint step (train -> evaluate on held-out set -> adjust lr/epochs) so workflow_clarity can score above 3.

Include a one-line 'verify' cue (e.g. confirm loss decreases and run eval after each epoch) to add a feedback loop for the training process.

Dimension	Reasoning	Score
Conciseness	The body is lean bullet-list reference material with concrete numbers and no padding or explanation of concepts Claude already knows, matching anchor 5 ('every token earns its place').	5 / 5
Actionability	It provides concrete, directly-usable configuration values (lr=2e-5 to 5e-5, LoRA r=8-64, alpha=16-128) but no executable code blocks or commands, placing it between anchor 3 (pseudocode) and anchor 4 (executable guidance); as actionable instruction-style guidance it earns anchor 4.	4 / 5
Workflow Clarity	Content is organized into three labeled sections but presents reference configuration rather than a sequenced process, with no validation checkpoints; matching anchor 3 ('sequence present but checkpoints missing') it lacks an explicit workflow despite clear structure.	3 / 5
Progressive Disclosure	With no bundle files present and a sub-50-line body organized into three clear section headers, the simple-skill exception applies: well-organized sections alone warrant anchor 5.	5 / 5
	Total	17 / 20 Passed

Description

62%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description clearly states what the skill covers and gives an explicit 'Use when' trigger, but the trigger terms are somewhat narrow and the listed actions are not comprehensive. It is a solid, concise description with minor room for richer trigger phrasing.

Suggestions

Broaden the 'Use when' clause to enumerate natural trigger phrases users would say, e.g. 'Use when training, fine-tuning, or evaluating language models (BERT, GPT, LLMs), or when the user asks to set up an NLP training run.'

Add a couple more concrete actions to the 'what' portion (e.g. 'configure optimizers, schedules, and PEFT methods') to lift specificity toward anchor 4-5.

Include common synonyms or model family names (transformer, LLM) directly in the description rather than only in metadata trigger-keywords.

Dimension	Reasoning	Score
Specificity	Names the domain ('language model pretraining and fine-tuning') and a couple of concrete actions ('generating or reviewing NLP training code'), matching anchor 3; it lists fewer specific actions than anchor 4's 'several specific actions'.	3 / 5
Completeness	It states both the 'what' ('Best practices for language model pretraining and fine-tuning') and an explicit 'when' ('Use when generating or reviewing NLP training code'), but the trigger is narrower than anchor 5's comprehensive enumeration, so it sits at anchor 4.	4 / 5
Trigger Term Quality	It contains relevant keywords ('NLP training code', 'generating or reviewing') but misses the natural phrases users say (e.g. 'train a model', 'fine-tune BERT') and any file extensions, fitting anchor 3.	3 / 5
Distinctiveness Conflict Risk	The pretraining/fine-tuning niche with NLP-training-code triggers is mostly distinct with only minor overlap risk against a general ML-training skill, matching anchor 4 rather than the fully-niche anchor 5.	4 / 5
	Total	14 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: aiming-lab/AutoResearchClaw
Path: researchclaw/skills/builtin/domain/nlp-pretraining/SKILL.md
Commit: be4ba47

Reviewed: 5 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.