CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

eval-integrity

Assess integrity of an eval, benchmark, metric, A/B test, holdout, LLM-judge, or user-study claim when the question is whether measurement, comparison, or evidence is independent and trustworthy.

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

The canonical home for this skill is eval-integrity in code-yeongyu/oh-my-openagent

SKILL.md
Quality
Evals
Security

Quality

Content

96%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

This is a strong instruction-only skill: a lean, fully actionable audit procedure with an explicit output contract, a mapping checklist, defined risk probes, and verdict decision rules. The only structural observation is that everything lives inline in a ~100-line SKILL.md, which is appropriate for the content but sits just below the ideal progressive-disclosure pattern.

DimensionReasoningScore

Conciseness

The body is tight procedural prose with no padding or explanations of concepts Claude already knows — e.g., "Contamination: benchmark items, answers, or near duplicates reached training, prompt design, retrieval, tuning, or post-hoc selection" scopes exactly what to check rather than teaching what contamination is. Even guardrail lines like "A favorable score is not itself evidence of integrity" earn their tokens; there is nothing trimmable, so it matches the anchor-5 example rather than anchor 4's 'minor instances of over-explanation'.

5 / 5

Actionability

For an instruction-only skill the guidance is fully executable: an exact output format ("Begin with exactly one verdict line in the form `**Verdict: PASS**`"), a seven-row role-mapping table, four explicit ground-truth tests, eight named risk categories, and a numbered five-part report structure (Evidence / Impact / Minimal independence fix / Missing evidence / Residual risk). It sits above anchor 4 because there are no gaps in what to do or produce, and code absence is not penalized per the instruction-skill note.

5 / 5

Workflow Clarity

The audit method is a clearly numbered four-step sequence (state claim and map → check ground truth → probe independence failures → reach verdict) with explicit checkpoints and gates: "Missing provenance for data, splits, labels, prompts, evaluator selection, or scoring is **INCONCLUSIVE**, not evidence" and the PASS/FAIL/INCONCLUSIVE decision rules. The skill is read-only, so the destructive/batch validation cap does not apply, and nothing matches anchor 4's 'minor validation gaps'.

5 / 5

Progressive Disclosure

The single file is well organized (Route correctly / Audit method with four numbered subsections / Report) with no buried or nested references and no content that clearly belongs in a separate file — matching anchor 4's 'good structure; most content is appropriately placed'. It falls short of anchor 5 because at ~100 body lines it exceeds the simple <50-line self-contained case and provides no reference split at all, so navigation relies entirely on inline sections.

4 / 5

Total

19

/

20

Passed

Description

70%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description communicates a clear, well-scoped niche with an explicit when-condition and strong domain enumeration, so it triggers correctly for the intended intent. Its main weakness is that it states a single generic action ('assess integrity') instead of the concrete audit steps, and it omits several natural trigger phrases users would say when questioning an evaluation.

Suggestions

Replace the single generic action with the concrete audit steps, e.g., 'Map evaluation roles, check ground-truth independence, probe contamination, circular judging, and control validity, and deliver a PASS/FAIL/INCONCLUSIVE verdict for eval, benchmark, metric, A/B test, holdout, LLM-judge, or user-study claims.'

Add natural trigger phrases users would actually say, such as 'contaminated', 'data leakage', 'circular', 'fair comparison', or 'biased judge', to improve trigger-term coverage.

Make the when-clause more user-facing, e.g., 'Use when the user questions whether a benchmark, A/B test, or LLM-judge result is contaminated, leaked, circular, or otherwise trustworthy.'

DimensionReasoningScore

Specificity

The description names a rich domain — "an eval, benchmark, metric, A/B test, holdout, LLM-judge, or user-study claim" — but offers only one action, "Assess integrity", which fits anchor 3 ('names domain and 1-2 concrete actions, but not comprehensive'). It does not meet anchor 4, which requires several specific actions (e.g., map roles, check ground truth, probe contamination, deliver a verdict).

3 / 5

Completeness

Both parts are explicit: the what is "Assess integrity of an eval, benchmark, metric, A/B test, holdout, LLM-judge, or user-study claim" and the when is "when the question is whether measurement, comparison, or evidence is independent and trustworthy". This satisfies anchor 4 ('has both what and when; when could be more explicit or specific') but not anchor 5, because the when clause is a meta-condition about the question rather than concrete user trigger phrases like 'Use when the user asks whether...'.

4 / 5

Trigger Term Quality

Good natural keyword coverage across evaluation types ("eval, benchmark, metric, A/B test, holdout, LLM-judge, or user-study"), matching anchor 4's 'good keyword coverage; a few natural terms missing'. It is below anchor 5 because common user phrasings for this intent — 'contamination', 'data leakage', 'fair comparison', 'biased judge' — are absent.

4 / 5

Distinctiveness Conflict Risk

The description carves a clear niche — auditing evaluation integrity — and its scoping clause ('when the question is whether measurement, comparison, or evidence is independent and trustworthy') separates it from producing results, matching anchor 4's 'mostly distinct; minor overlap risk with closely related skills'. It is not anchor 5 because broad terms like "benchmark", "metric", and "A/B test" also appear in data-analysis and experiment-design contexts and the disambiguation rests entirely on the trailing qualifier.

4 / 5

Total

15

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

code-yeongyu/lazycodex

Path

plugins/omo/skills/eval-integrity/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.