Name: llm-evaluation
Rating: 38.4 (1 reviews)
Author: administrakt0r

llm-evaluation

Master comprehensive evaluation strategies for LLM applications, from automated metrics to human evaluation and A/B testing.

Quality

36%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./plugins/AI-Agents-Safe-Coding-Skills/skills/llm-evaluation/SKILL.md

The canonical home for this skill is llm-evaluation in administrakt0r/AI-Agents-Safe-Coding-Skills

Quality

Content

36%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

This skill reads more like a comprehensive tutorial or textbook chapter on LLM evaluation than a concise, actionable skill for Claude. It over-explains well-known concepts, uses fictional libraries in key examples, and lacks a clear workflow for actually conducting an evaluation. The content would benefit enormously from being trimmed to essential guidance with real code moved to bundle files.

Suggestions

Drastically reduce inline content: move all code implementations to the referenced bundle files (e.g., assets/evaluation-framework.py) and keep SKILL.md as a concise overview with only the most essential examples.

Remove explanations of basic concepts Claude already knows (what BLEU, ROUGE, accuracy, precision/recall are) and replace metric definition lists with a reference to references/metrics.md.

Add a clear end-to-end workflow with numbered steps for conducting an evaluation (e.g., 1. Define eval criteria → 2. Build test set → 3. Run metrics → 4. Validate results → 5. Compare to baseline), including validation checkpoints.

Replace fictional library imports (llm_eval, gpt-5) with real, executable code using actual libraries, or clearly mark them as illustrative patterns rather than copy-paste-ready code.

Dimension	Reasoning	Score
Conciseness	The skill is severely bloated at ~400+ lines. It explains concepts Claude already knows (what BLEU, ROUGE, accuracy, precision are), provides extensive code examples for well-known libraries (nltk, rouge_score, bert_score, scipy), and includes class definitions that are more tutorial-like than actionable skill guidance. The metric definition lists (e.g., 'Accuracy: Percentage correct') add no value for Claude.	2 / 5
Actionability	The code examples are concrete and mostly executable, but many rely on fictional libraries (e.g., `from llm_eval import EvaluationSuite, Metric`) or outdated APIs (`openai.ChatCompletion.create` with `gpt-5`). The Quick Start uses a non-existent library. Real, executable code is mixed with pseudocode-like fictional frameworks, reducing practical actionability.	3 / 5
Workflow Clarity	There is no clear workflow or sequenced process for conducting an evaluation end-to-end. The content is organized as a reference catalog of techniques rather than a step-by-step guide. The Instructions section is extremely vague ('Apply relevant best practices and validate outcomes'). There are no validation checkpoints or feedback loops for the evaluation process itself.	2 / 5
Progressive Disclosure	The skill references multiple external files (references/metrics.md, assets/evaluation-framework.py, etc.) which is good structure, but no bundle files are provided, so these references are unverifiable. Meanwhile, the SKILL.md itself inlines hundreds of lines of code that should be in those referenced files, defeating the purpose of the progressive disclosure structure it sets up.	3 / 5
	Total	10 / 20 Passed

Description

36%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description reads more like a course title than a skill description—it uses aspirational language ('Master comprehensive evaluation strategies') rather than describing concrete actions the skill performs. It lacks a 'Use when...' clause entirely, and the capabilities listed are high-level categories rather than specific actions. The use of imperative/second-person-adjacent voice ('Master...') is also problematic.

Suggestions

Add an explicit 'Use when...' clause with trigger phrases like 'evaluate prompts', 'benchmark LLM output', 'measure model quality', 'run evals', or 'compare model performance'.

Replace the vague 'Master comprehensive evaluation strategies' with concrete actions in third person, e.g., 'Designs evaluation rubrics, implements automated scoring metrics, sets up human evaluation workflows, and configures A/B tests for LLM applications.'

Include natural synonyms and related terms users might say, such as 'evals', 'benchmarks', 'scoring', 'prompt testing', 'model comparison', or 'quality assessment'.

Dimension	Reasoning	Score
Specificity	Names the domain (LLM evaluation) and lists broad categories (automated metrics, human evaluation, A/B testing), but these are high-level categories rather than concrete actions. No specific verbs describing what the skill actually does—'Master comprehensive evaluation strategies' is vague and aspirational rather than action-oriented.	2 / 5
Completeness	Has a vague 'what' (evaluation strategies for LLM applications) but completely lacks a 'when' clause. There is no 'Use when...' or equivalent trigger guidance, which per the rubric should cap completeness at 3, and the 'what' itself is too vague to merit even that.	2 / 5
Trigger Term Quality	Includes some relevant keywords like 'LLM', 'automated metrics', 'human evaluation', 'A/B testing', but misses natural user phrases like 'evaluate my model', 'benchmark', 'evals', 'scoring', 'test prompts', 'measure quality', or 'evaluation framework'. The terms present are somewhat generic.	3 / 5
Distinctiveness Conflict Risk	The LLM evaluation domain is somewhat specific, but 'evaluation strategies' is broad enough to overlap with skills related to testing, benchmarking, prompt engineering, or general ML evaluation. The mention of A/B testing could also conflict with product experimentation skills.	3 / 5
	Total	10 / 20 Passed

Validation

90%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 10 / 11 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	10 / 11 Passed

Repository: administrakt0r/AI-Agents-Safe-Coding-Skills
Path: plugins/AI-Agents-Safe-Coding-Skills/skills/llm-evaluation/SKILL.md
Commit: 57c135b

Reviewed: 4 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.