CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

testland/llm-eval-anti-patterns

Use when an LLM eval suite already gates a release, model swap or prompt change and nobody has audited how it was built. Checks eight methodology errors (too few cases, exact match on open text, no baseline, unpinned models, no adversarial cases and more) and how to validate an LLM judge.

Quality

85%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

75%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A high-quality, deeply actionable review catalog: every check names exact harness keys and assertion types, fixes are specific changes rather than categories, and the worked example demonstrates the full output contract. The main cost is length — front-matter sections overlap each other and several explanatory passages could be compressed without losing substance.

Suggestions

Collapse the "What this owns, and what it does not" table and the follow-up paragraph into the intro or Limitations section — they make the same scoping point three times.

Compress the secondary "Why..." subsections ("Related risk when adding coverage", "Why the comparison itself needs care", "Secondary concern") to one sentence each, or fold them into the Fix lines they modify.

Give "The judge is a model under test" a number or move it after anti-pattern 8 so the 1–8 check sequence reads unbroken, and consider moving the worked example to a reference file to shorten the always-loaded body.

DimensionReasoningScore

Conciseness

The body is expert-level with no basic-concept padding, but at roughly 430 lines it could be meaningfully tightened: the "What this owns, and what it does not" table restates both the intro and the Limitations section, and multi-paragraph "Why it invalidates" passages (e.g. anti-pattern 5's "Why the comparison itself needs care") run long where one sentence would carry the point. This fits anchor 3 ('could be tightened') better than anchor 4's 'minor instances of over-explanation'.

3 / 5

Actionability

Guidance is fully concrete and executable: exact assertion names per harness (`equals`, `contains`, `llm-rubric`, `similar` with a `threshold`), copy-ready YAML snippets, the three-level Promptfoo grader override syntax (`promptfoo eval --grader <provider>`, `defaultTest.options.provider`, assertion-level `provider`), a measured procedure for setting similarity thresholds, and a complete worked example that produces the exact specified output table. Per the rubric's code-vs-instruction note, an instruction-only skill with this level of specificity matches anchor 5.

5 / 5

Workflow Clarity

The review sequence is clear from structure — map the five slots (numbered 1–5), apply eight Detect/Why/Fix checks, then report in the mandated format with checkpoints like "Every row cites a file and line, or names the file and marks the finding absent". It stops short of anchor 5 because the overall procedure order is implied by layout rather than stated as an ordered list, and the unnumbered "The judge is a model under test" section interrupts the 1–8 check numbering.

4 / 5

Progressive Disclosure

Both referenced files exist, are one level deep, and are well signaled with their contents described (citations.md for full academic citations, harness-mapping.md for per-harness slot vocabulary), with the core catalog correctly kept inline. Minor gaps keep it at anchor 4 rather than 5: the unnumbered judge-validation section is awkwardly interleaved between anti-patterns 4 and 5, and the body length pushes the limit of what belongs in SKILL.md versus a reference file.

4 / 5

Total

16

/

20

Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: it pairs an explicit, multi-condition 'Use when' trigger with an enumerated list of the specific methodology errors it checks, and occupies a well-delineated niche. Keyword coverage is good but misses a few natural synonyms (benchmark, regression, LLM-as-judge) that would push trigger quality to the top anchor.

DimensionReasoningScore

Specificity

The description enumerates concrete check targets ("too few cases, exact match on open text, no baseline, unpinned models, no adversarial cases and more" plus "how to validate an LLM judge"), listing several specific items as anchor 4 requires. It is not anchor 5 because the items are error categories rather than concrete actions and "and more" concedes incompleteness.

4 / 5

Completeness

It explicitly answers both questions: the what is "Checks eight methodology errors (... ) and how to validate an LLM judge" and the when is the concrete trigger clause "Use when an LLM eval suite already gates a release, model swap or prompt change and nobody has audited how it was built". This matches the anchor-5 example's structure of enumerated capabilities plus explicit trigger conditions, clearly above anchor 4 where the 'when' is only generally stated.

5 / 5

Trigger Term Quality

Natural phrases the target user would say appear throughout ("LLM eval suite", "gates a release", "model swap", "prompt change", "baseline", "adversarial cases"). It falls short of anchor 5 because common synonyms like "benchmark", "regression", "LLM-as-judge", or "CI gate" are missing.

4 / 5

Distinctiveness Conflict Risk

Auditing the methodology of an already-existing eval suite is a clear niche with distinct triggers ("nobody has audited how it was built", "gates a release"), creating minimal overlap risk with adjacent skills like benchmark-running or test-writing. It clearly fits anchor 5 rather than anchor 4's 'minor overlap risk with closely related skills'.

5 / 5

Total

18

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

On this page