CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

llm-evaluation

Master comprehensive evaluation strategies for LLM applications, from automated metrics to human evaluation and A/B testing.

Quality

53%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./skills/llm-evaluation/SKILL.md
SKILL.md
Quality
Evals
Security

Quality

Content

32%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is a verbose catalog that re-teaches concepts Claude already knows and inlines hundreds of lines of implementation code while pointing to eight bundle files that do not exist. Its most actionable examples are undermined by a fictional llm_eval library, a removed OpenAI API, and a stub function, and it never lays out a sequenced evaluation workflow with validation checkpoints.

Suggestions

Create the advertised bundle files (references/metrics.md, assets/evaluation-framework.py, scripts/evaluate-model.py, etc.) or remove the Resources section, and fix the inconsistent "resources/implementation-playbook.md" path — every reference is currently a dead link.

Cut concept glosses Claude already knows (BLEU/ROUGE/accuracy definitions) and the boilerplate intro/"Do not use when" filler; move the long per-metric and A/B-testing implementations into the reference files, keeping SKILL.md as a lean overview.

Replace the fictional llm_eval Quick Start and the removed openai.ChatCompletion.create calls with runnable code against real libraries, and add a sequenced evaluation workflow (define cases → pick metrics → baseline → compare with significance test → CI regression gate) with explicit validation steps.

DimensionReasoningScore

Conciseness

The body re-explains concepts Claude already knows ("BLEU: N-gram overlap", "Accuracy: Percentage correct", "Precision/Recall/F1: Class-specific performance"), repeats the frontmatter description verbatim as the intro, carries boilerplate filler ("Do not use this skill when - The task is unrelated to llm evaluation"), and inlines ~480 lines of implementation code. This matches anchor 2 (noticeably verbose, several padded sections) rather than 3, where padding would be incidental.

2 / 5

Actionability

Real concrete code exists (nltk BLEU, rouge_score, bert_score, scipy ABTest), but the flagship Quick Start imports a nonexistent "llm_eval" module, the LLM-judge examples call the removed "openai.ChatCompletion.create" API and use json without importing it, "calculate_factuality" is a pass stub, and the kappa interpretation dict uses boolean keys that collapse duplicates. Concrete guidance is present but key details are broken or missing — anchor 3, not anchor 4's "minor gaps".

3 / 5

Workflow Clarity

There is no sequenced evaluation workflow (define test cases → choose metrics → establish baseline → compare → integrate into CI); sections are a topical catalog. The Instructions section offers only generic filler ("Apply relevant best practices and validate outcomes") with no validation checkpoints anywhere. A rough, poorly-defined sequence with validation absent fits anchor 2.

2 / 5

Progressive Disclosure

Every cited bundle file is dangling: references/metrics.md, references/human-evaluation.md, references/benchmarking.md, references/a-b-testing.md, references/regression-testing.md, assets/evaluation-framework.py, assets/benchmark-dataset.jsonl, scripts/evaluate-model.py, and resources/implementation-playbook.md (whose "resources/" prefix is also inconsistent) do not exist — no references/, scripts/, assets/, or resources/ directory is present in the bundle. Meanwhile metric implementations that clearly belong in those files are inlined in SKILL.md. This matches anchor 2: content that belongs in separate files is inlined and the reference structure is broken.

2 / 5

Total

9

/

20

Passed

Description

53%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description clearly names its domain and enumerates the right high-level areas, but relies on the vague verb "Master", omits concrete actions, and lacks any "Use when" trigger guidance. It would benefit from concrete capability verbs and an explicit trigger clause.

Suggestions

Replace the vague verb "Master" with concrete actions, e.g. "Compute automated metrics (BLEU, ROUGE, BERTScore), run LLM-as-judge and human evaluation workflows, and A/B test model or prompt variants with statistical significance testing."

Add an explicit trigger clause, e.g. "Use when evaluating LLM outputs, benchmarking models, comparing prompts, or detecting performance regressions before deployment."

Include natural trigger synonyms users actually say — "benchmark", "score outputs", "prompt comparison", "regression testing" — to improve trigger term coverage.

DimensionReasoningScore

Specificity

The description names the domain ("LLM applications") and three evaluation approach categories ("automated metrics", "human evaluation", "A/B testing"), but the only action verb is the generic "Master" — no concrete operations like computing BLEU/ROUGE or running significance tests. It matches anchor 3 (domain plus 1-2 named areas, not comprehensive concrete actions) rather than anchor 4, which requires several specific actions.

3 / 5

Completeness

The "what" is reasonably clear (evaluation strategies spanning automated metrics, human evaluation, and A/B testing), but the description contains no "Use when..." clause or equivalent trigger guidance, which caps completeness at 3 per the judging guidelines. It cannot score 4 without an explicit when-clause.

3 / 5

Trigger Term Quality

"LLM applications", "evaluation", "metrics", and "A/B testing" are phrases users would plausibly say, but common variations are absent — "benchmark", "test prompts", "regression", "LLM-as-judge", "score model outputs". Some relevant keywords with missing synonyms fits anchor 3, below anchor 4's good keyword coverage.

3 / 5

Distinctiveness Conflict Risk

The "LLM applications" scope plus the three named evaluation paradigms carve a fairly distinct niche with only minor overlap risk against generic testing or benchmarking skills. Anti-drift check: it is more specific than anchor 3's "could still overlap with similar skills" but lacks the crisp trigger phrases of anchor 5.

4 / 5

Total

13

/

20

Passed

Validation

87%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

referenced_paths_exist

Referenced path issues: 8 missing

Warning

Total

14

/

16

Passed

Repository

administrakt0r/AI-Agents-Safe-Coding-Skills

Path

skills/llm-evaluation/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.