CtrlK
BlogDocsLog inGet started
Tessl Logo

validate-evaluator

Calibrate an LLM judge against human labels using data splits, TPR/TNR, and bias correction. Use after writing a judge prompt (write-judge-prompt) when you need to verify alignment before trusting its outputs. Do NOT use for code-based evaluators (those are deterministic; test with standard unit tests).

72

Quality

88%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, actionable calibration procedure with executable code, explicit validation checkpoints, and useful anti-patterns. The only gap is that a long single file inlines content (the bootstrap CI code) that could be split into a one-level-deep reference.

Suggestions

Move the full bootstrap_ci implementation and judgy alternative into a references/ file (e.g. CONFIDENCE_INTERVALS.md) and link to it from Step 8, keeping only the formula and a short snippet inline.

Trim a few explanatory lines in Practical Guidance (e.g. restating the denominator effect) since the formula already conveys the point.

DimensionReasoningScore

Conciseness

Mostly lean with tables, formulas, and executable code rather than lectures on known concepts, though the bootstrap CI listing and Practical Guidance sections have a few lines that could be trimmed.

4 / 5

Actionability

Provides copy-paste-ready executable code (sklearn splits, confusion_matrix, bootstrap function, judgy usage) and concrete formulas covering the common calibration cases.

5 / 5

Workflow Clarity

Steps 1-8 are explicitly sequenced with validation checkpoints (disagreement inspection, TPR/TNR stopping criteria, test-set-run-once discipline) and an iterate-rerun feedback loop, so the batch-operation cap does not apply.

5 / 5

Progressive Disclosure

Well-organized into clear sections with no nested references and no navigation problems, but the ~150-line single file inlines the long bootstrap CI implementation that could reasonably live in a separate reference file.

4 / 5

Total

18

/

20

Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, third-person description that clearly answers what and when, with a useful negative boundary. Minor gaps in enumerated calibration actions and natural-term synonyms keep specificity and trigger quality just below maximum.

DimensionReasoningScore

Specificity

Names the domain and several concrete actions ('data splits, TPR/TNR, and bias correction'), but omits core calibration activities like prompt iteration and few-shot selection, leaving minor coverage gaps.

4 / 5

Completeness

Explicitly states both what ('Calibrate an LLM judge against human labels...') and when ('Use after writing a judge prompt... when you need to verify alignment'), plus a concrete negative boundary.

5 / 5

Trigger Term Quality

Includes natural technical triggers ('LLM judge', 'human labels', 'verify alignment', 'judge prompt') with good coverage, though a few common synonyms or phrasings a user might say are missing.

4 / 5

Distinctiveness Conflict Risk

Occupies a clear niche (LLM judge calibration) with distinct triggers and an explicit 'Do NOT use for code-based evaluators' boundary that minimizes conflict risk.

5 / 5

Total

18

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository
hamelsmu/evals-skills
Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.