CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

llm-evaluation

Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.

54

Quality

68%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./skills/llm-evaluation/SKILL.md
SKILL.md
Quality
Evals
Security

Quality

Content

42%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is well-organized and rich with mostly-real code, but it is a long monolith that pads token budget with glossaries of standard ML metrics Claude already knows, generic filler sections, and several non-executable or buggy code examples (fictional llm_eval package, deprecated openai API, pass stub, kappa dict bug). It reads as a reference cookbook rather than a skill that sequences an evaluation workflow with checkpoints.

Suggestions

Move the metric implementation code (BLEU/ROUGE/BERTScore, ABTest, RegressionDetector, BenchmarkRunner) into reference files linked from a lean SKILL.md overview, and delete the glossaries of standard metrics (BLEU, ROUGE, precision/recall, confusion matrix) and the generic 'Instructions' / 'Do not use this skill when' filler — Claude already knows these concepts.

Make the code executable: replace the fictional `llm_eval` Quick Start with a working example using real libraries, update `openai.ChatCompletion.create` to the current `openai` client API, implement or remove `calculate_factuality`, and fix the kappa interpretation dict (the boolean keys collapse, so `interpretation[True]` mislabels almost every kappa value).

Add an explicit end-to-end evaluation workflow with sequence and validation checkpoints (e.g., define goals → select metrics → build test set → run baseline → compare → check for regression → iterate), so the guidance sequences a process instead of only cataloging techniques.

DimensionReasoningScore

Conciseness

Several sections are unnecessary explanations of concepts Claude already knows: glossaries of standard metrics ("BLEU: N-gram overlap", "Accuracy: Percentage correct", "Precision/Recall/F1", "Confusion Matrix"), a glossary of human-evaluation dimensions, and generic filler like "The task is unrelated to llm evaluation" and "Clarify goals, constraints, and required inputs. Apply relevant best practices and validate outcomes." This matches anchor 2 ('noticeably verbose; several unnecessary explanations or padded sections') better than anchor 3's 'some unnecessary explanation', given roughly a third of the body is padding.

2 / 5

Actionability

Much of the code is concrete (nltk BLEU, rouge_score, bert_score, sklearn kappa, scipy ABTest), but the anchor-4 'minor gaps' bar is exceeded: the Quick Start imports a fictional "llm_eval" package (EvaluationSuite, Metric), "openai.ChatCompletion.create" with "gpt-5" is a deprecated API that throws on modern openai, "calculate_factuality" is a `pass` stub, and the kappa "interpretation[True]" dict collapses duplicate boolean keys and mislabels results. This fits anchor 3 ('some concrete guidance but incomplete; missing key details') more than anchor 2, since most snippets are real and complete.

3 / 5

Workflow Clarity

There is a rough topic-level sequence (metric types → implementations → LLM-as-judge → human eval → A/B testing → regression → benchmarking) but no explicit end-to-end workflow for conducting an evaluation, and validation checkpoints are absent beyond generic best-practice bullets; this sits between anchor 2 ('rough sequence, poorly defined, validation absent') and anchor 3 ('sequence present, checkpoints missing'), landing at 3 because the section ordering does imply a usable progression.

3 / 5

Progressive Disclosure

The body has good section structure (## and ### headers, logical grouping), but it is a ~465-line monolith with zero bundle files — the metric implementations, LLM-judge prompts, and framework classes (~200+ lines of reference material) clearly belong in separate reference files. This matches anchor 3 ('some structure but content that should be separate is inline') rather than anchor 2, because the structure is well-organized and navigable, not minimal.

3 / 5

Total

11

/

20

Passed

Description

78%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A solid description with an explicit and well-phrased 'Use when...' trigger clause covering both what and when. The main weakness is the generic 'Implement comprehensive evaluation strategies' framing, which names method categories rather than concrete actions, plus a mild buzzword ('comprehensive').

DimensionReasoningScore

Specificity

"Implement comprehensive evaluation strategies" is a generic verb plus the buzzword "comprehensive", and the specific items that follow ("automated metrics, human feedback, and benchmarking") are capability categories rather than concrete actions — matching anchor 3's 'names domain and actions but not comprehensive' better than anchor 4, which expects several concrete action verbs like 'extracts, fills, converts'.

3 / 5

Completeness

It explicitly answers both questions: what ("Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking") and when ("Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks") with three concrete trigger phrases, matching anchor 5's exemplar structure; anchor 4 applies only when the 'when' clause is less explicit than here.

5 / 5

Trigger Term Quality

"testing LLM performance", "measuring AI application quality", and "establishing evaluation frameworks" are natural phrases users would say, giving good keyword coverage; a few common variations are missing (e.g., "benchmark my model", "compare models", "evals", "A/B test"), so it falls just short of anchor 5's comprehensive synonym coverage.

4 / 5

Distinctiveness Conflict Risk

LLM evaluation is a clear niche with distinct triggers ("testing LLM performance", "evaluation frameworks") that is unlikely to fire for unrelated skills; minor overlap risk remains with general software-testing or benchmarking skills, so anchor 4 ('mostly distinct; minor overlap risk') fits better than anchor 5.

4 / 5

Total

16

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

rmyndharis/antigravity-skills

Path

skills/llm-evaluation/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.