CtrlK
BlogDocsLog inGet started
Tessl Logo

nemo-curator

GPU-accelerated data curation for LLM training. Supports text/image/video/audio. Features fuzzy deduplication (16× faster), quality filtering (30+ heuristics), semantic deduplication, PII redaction, NSFW detection. Scales across GPUs with RAPIDS. Use for preparing high-quality training datasets, cleaning web data, or deduplicating large corpora.

67

Quality

82%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

65%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A code-rich, well-sectioned skill with concrete executable examples and clear staged pipelines, weakened by repeated benchmark/marketing content and an absence of validation checkpoints for batch operations. Progressive disclosure is only partially realized with two reference files.

Suggestions

Consolidate the duplicated performance benchmarks (16× fuzzy dedup, exact dedup) into a single section and trim the cost-comparison/marketing detail to improve token efficiency.

Add validation checkpoints to the curation pipeline (e.g. inspect dataset size before/after filtering, verify dedup recall on a sample) so batch operations have explicit feedback loops.

Move the detailed multi-modal (image/video/audio), classifier, and benchmark content into separate reference files and link them from the overview to better split inline content.

DimensionReasoningScore

Conciseness

Mostly code-driven and actionable, but the '16× faster / 120h→7.5h' benchmark recurs at multiple points (When-to-use, GPU table, Performance benchmarks, references) and the Quick-start pipeline duplicates Stage 1/2 code; cost-comparison and star-count details pad the body.

2 / 3

Actionability

The body is dominated by concrete, executable code with real import paths and specific parameters (e.g. WordCountFilter(min_words=50, max_words=100000), FuzzyDuplicates(num_hashes=260, num_buckets=20)), going well beyond pseudocode.

3 / 3

Workflow Clarity

Stage 1→4 sequencing and the Common Crawl pipeline list are clear, but no validation checkpoints or error-recovery loops exist for these batch curation operations, capping the score at 2 per the batch-operation guideline.

2 / 3

Progressive Disclosure

Two real, well-signaled one-level references exist (filtering.md, deduplication.md, both verified present), but the ~370-line body keeps full multi-modal, classifier, benchmark, and cost content inline that could be split into additional reference files.

2 / 3

Total

9

/

12

Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, third-person description that names concrete capabilities, provides explicit 'Use for' trigger guidance, and occupies a distinct niche. It is concise yet comprehensive without fluff.

DimensionReasoningScore

Specificity

Lists multiple concrete actions — 'fuzzy deduplication (16× faster)', 'quality filtering (30+ heuristics)', 'semantic deduplication', 'PII redaction', 'NSFW detection' — rather than vague language.

3 / 3

Completeness

Explicitly answers what it does (the feature list) and when to use it via the explicit 'Use for ...' trigger clause, satisfying both halves.

3 / 3

Trigger Term Quality

The 'Use for preparing high-quality training datasets, cleaning web data, or deduplicating large corpora' clause covers natural terms a user would say, alongside domain terms like 'LLM training' and 'deduplication'.

3 / 3

Distinctiveness Conflict Risk

The GPU-accelerated LLM-training-data curation niche with RAPIDS and named alternatives (datatrove, dolma, Ray Data) makes it clearly distinguishable and unlikely to trigger for the wrong skill.

3 / 3

Total

12

/

12

Passed

Validation

81%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation13 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

metadata_version

'metadata.version' is missing

Warning

metadata_field

'metadata' should map string keys to string values

Warning

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

13

/

16

Passed

Repository
NousResearch/hermes-agent
Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.