CtrlK
BlogDocsLog inGet started
Tessl Logo

eval-iterate

Iterates on a failing AI/LLM eval (an L2 suite, a golden-set / judge eval, or any eval gating a PR) until it is green AND confirmed, not just luckily passing once. Classifies the failure as a code bug, an eval-definition bug (stale golden item / criteria drift), judge-drift (silent grader version change), or flaky. Applies the minimal fix, then requires N consecutive confirming re-runs — 2 deterministic, 5 for anything a model call grades, since a flat "2" is statistically weak for a stochastic judge. Hard-capped at 5 fix iterations. Refuses to game the eval (Goodhart's Law: skip/delete/overwrite-in-place a case, loosen a threshold) without a second independent check plus `confidence(analysis) >= 90%` and a logged rationale. Composes `ai-engineering`, `confidence`, `critical`, `verify-behavior`. Use when an eval is failing on a PR and needs a real, non-gamed green. Triggers on "this eval is failing", "iterate on this eval", "fix this eval", "get this eval green", "optimize this eval", "/eval-iterate".

72

Quality

91%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

No security issues found

Scanned

Repository
mthines/agent-skills
Audited
Security analysis
Snyk

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.