CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

nemo-rl-auto-research

Autonomous NeMo-RL research agent workflow for directed hypothesis testing and open-ended discovery. Guides agents through the full experiment lifecycle: understanding recipes and environments, wiring RL or NeMo-gym runs, launching reproducible baselines and iterations, analyzing results, preserving human oversight, and using git plus TSV logs as the research ledger. Do NOT use for: bug fixes, code review, documentation, refactoring, dependency updates, or single-file changes.

Quality

84%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

85%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable campaign playbook with real validation checkpoints, feedback loops, and a clean one-level-deep reference split verified against the actual bundle. Its only weaknesses are repeated stop-rule exposition across sections and a hand-waved metric-extraction step.

Suggestions

Consolidate the stop-rule guidance repeated in Workflow steps 4-5, Loop steps 5/11, the examples, and the Stop section into one authoritative location with a single cross-reference.

Add a one-line concrete metric-extraction example for a typical recipe log (e.g. a grep/awk one-liner for the authoritative metric) so step 8 is copy-paste ready.

Consider moving the three worked stop-rule examples (lines 72-75) into a reference file alongside exploration-ideas.md to tighten the main body.

DimensionReasoningScore

Conciseness

The body is dense and prescriptive with no concept explanations and full assumption of competence, but stop-condition guidance is restated across Workflow steps 4-5, Loop steps 5 and 11, the stop-rule examples, and the Stop section, which could be consolidated. This matches the 4 anchor (efficient, minor instances that could be trimmed) rather than 3, since the padding is repetition of genuinely critical rules, not unnecessary explanation.

4 / 5

Actionability

Concrete, executable guidance throughout: a runnable launch snippet with a unique log dir, explicit timeout-enforcement rules ('prefer a recipe-level timeout... otherwise wrap the command with an external timeout... honor the tighter limit'), a 14-field TSV record spec, concrete branch patterns, and worked stop-rule translations ('10h total, 1h each'). The gap that keeps it from 5 is step 8 — 'Extract the primary metric with a command appropriate for the actual log format' — which is direction rather than a concrete extraction example.

4 / 5

Workflow Clarity

A clear Workflow → Branching → Loop sequence with explicit validation checkpoints at every stage: pre-launch stop-condition check (step 5), a feedback loop on empty metric extraction ('inspect the last log lines and the recipe's logging path', step 8), explicit post-run re-check with concrete status strings ('stop condition met: 24/24 attempted', step 11), keep/discard/crash outcome marking (step 12), and mandatory user confirmation before destructive or batch operations. This is a batch/destructive-adjacent skill whose validation is present, so the cap does not apply; it matches the 5 anchor (explicit validation, feedback loops, and a de facto checklist).

5 / 5

Progressive Disclosure

The core decision loop stays inline while detailed git patterns (references/git-workflow.md, 111 lines), hypothesis generation (references/exploration-ideas.md, 131 lines), and the TSV schema (references/experiment-log-template.md) are split into three one-level-deep references, each listed with its purpose in the References section and one inline pointer at the Branching section. All three files exist and match their stated purposes. Matches the 5 anchor; not a 4 since navigation and signaling are unambiguous.

5 / 5

Total

18

/

20

Passed

Description

75%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, specific, third-person description with excellent boundary guidance via the Do-NOT-use list, weakened only by having no affirmative 'Use when...' trigger phrase inside the description itself. Affirmative triggers currently rely on the separate when_to_use field.

Suggestions

Add an affirmative trigger clause to the description itself, e.g. 'Use for long-running NeMo-RL/NeMo-gym research campaigns, running experiments, or improving recipe metrics.'

Fold one or two of the most natural when_to_use phrases ('run experiments', 'find a better recipe', 'improve accuracy') into the description so trigger terms survive if when_to_use is not surfaced.

Keep the Do NOT use list; it is the description's strongest distinctiveness asset.

DimensionReasoningScore

Specificity

The description lists six concrete actions spanning the full lifecycle — 'understanding recipes and environments, wiring RL or NeMo-gym runs, launching reproducible baselines and iterations, analyzing results, preserving human oversight, and using git plus TSV logs as the research ledger' — in third-person voice, matching the comprehensive-coverage anchor. It is not a 4 because there are no meaningful gaps in the action inventory.

5 / 5

Completeness

The 'what' is comprehensive, but the only trigger guidance inside the description is negative ('Do NOT use for: bug fixes, code review, documentation, refactoring, dependency updates, or single-file changes') with no affirmative 'Use when...' clause. Per the rubric guideline, missing equivalent explicit affirmative trigger guidance caps this at 3; it is not a 4 because the 'when' is exclusion-only rather than present-but-could-be-more-specific.

3 / 5

Trigger Term Quality

Good keyword coverage ('research', 'hypothesis testing', 'experiment', 'NeMo-RL', 'NeMo-gym'), but natural user phrasings like 'run experiments' or 'improve accuracy' live in the separate when_to_use field rather than the description itself, and 'recipes' is project jargon. Falls between the 3 anchor (some keywords, missing variations) and the 5 anchor (synonyms and extensions in the description), so 4.

4 / 5

Distinctiveness Conflict Risk

A narrow NeMo-RL/NeMo-gym research niche plus an explicit 'Do NOT use for' exclusion list gives distinct triggers and minimal conflict risk with adjacent skills. The 4 anchor ('minor overlap risk') fits less well since the exclusion list actively disambiguates.

5 / 5

Total

17

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

NVIDIA/skills

Path

skills/nemo-rl-auto-research/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.