CtrlK
BlogDocsLog inGet started
Tessl Logo

sentencepiece

Language-independent tokenizer treating text as raw Unicode. Supports BPE and Unigram algorithms. Fast (50k sentences/sec), lightweight (6MB memory), deterministic vocabulary. Used by T5, ALBERT, XLNet, mBART. Train on raw text without pre-tokenization. Use when you need multilingual support, CJK languages, or reproducible tokenization.

70

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Medium

Suggest reviewing before use

SKILL.md
Quality
Evals
Security

Quality

Content

72%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable with executable code and well-structured progressive disclosure into real reference files. It is slightly verbose due to repeated performance figures and lacks explicit validation checkpoints in its training workflow, keeping conciseness and workflow_clarity at the middle anchor.

Suggestions

State the performance figures (50k sentences/sec, ~6MB) once and remove the duplicate mentions in the 'When to use' and 'Performance benchmarks' sections to tighten conciseness.

Add a validation checkpoint after training — e.g., load the resulting 'm.model' and assert the vocabulary size or that encoding round-trips — so the training workflow has an explicit verify step.

Avoid repeating the 'Use SentencePiece when' / 'Use alternatives instead' guidance that already appears in the description; keep the body focused on how-to content.

DimensionReasoningScore

Conciseness

Mostly efficient with executable snippets and tables, but restates the frontmatter's 'When to use' list and repeats the performance figures ('50,000 sentences/sec', '~6MB memory') three times across the body, which is unnecessary padding.

2 / 3

Actionability

Provides fully executable, copy-paste-ready guidance throughout — 'pip install sentencepiece', 'spm_train --input=data.txt ...', and complete Python API snippets for training, encoding, and decoding.

3 / 3

Workflow Clarity

Training and encode/decode flows are sequenced with command-line and API variants, but there are no explicit validation or verification checkpoints (e.g., confirming a model trained successfully or vocabulary size is correct before use).

2 / 3

Progressive Disclosure

Clear overview body with two well-signaled, one-level-deep references — '[Training Guide](references/training.md)' and '[Algorithms](references/algorithms.md)' — both verified to exist as real files; content is appropriately split.

3 / 3

Total

10

/

12

Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is strong across the board: it states concrete capabilities, includes natural trigger terms, supplies an explicit 'Use when' clause, and carves out a distinct niche. No substantive weaknesses; the only minor note is that performance figures ('50k sentences/sec', '6MB') lean slightly marketing-flavored but remain factual.

DimensionReasoningScore

Specificity

Lists multiple concrete capabilities and properties — 'Supports BPE and Unigram algorithms', 'Fast (50k sentences/sec), lightweight (6MB memory), deterministic vocabulary' — matching the highest anchor for naming multiple specific concrete actions.

3 / 3

Completeness

Clearly answers both what ('Language-independent tokenizer treating text as raw Unicode') and when with an explicit 'Use when you need multilingual support, CJK languages, or reproducible tokenization' clause.

3 / 3

Trigger Term Quality

Good coverage of natural terms users would say — 'SentencePiece', 'multilingual support', 'CJK languages', 'reproducible tokenization', 'BPE', 'Unigram' — rather than abstract jargon.

3 / 3

Distinctiveness Conflict Risk

Occupies a clear niche (SentencePiece language-independent tokenization) with distinct triggers ('multilingual', 'CJK', 'reproducible'), unlikely to fire for the wrong skill.

3 / 3

Total

12

/

12

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

15

/

16

Passed

Repository
OpenLAIR/dr-claw
Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.