CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

done-gate-calibrate

Measure and improve how often the done-gate is fooled, using a seeded false-done benchmark with hidden ground truth, genuine-fix controls, honest-partial controls, and fresh blind report sets; run in rounds of test, improvement plan, implement, retest. Use after changing any agent, prompt, test, or verifier, and on a weekly schedule.

70

Quality

88%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

78%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A tight, well-structured calibration loop: the benchmark design (controls, blind sets, tune/report separation) and the round workflow with its built-in feedback loop are clear and free of waste. The main weakness is operational specificity — how scenarios are actually run, scored, and spot-checked is left entirely implicit, with no commands, report template, or scenario layout beyond the 'truth/' directory.

Suggestions

Add the concrete execution mechanics: the command(s) or harness used to run scenarios and produce the five metrics, or at minimum the expected file layout of a scenario branch (bug base, claim commit, claim JSON, deploy target).

Replace 'spot-check' with a defined check, e.g. which scenarios to re-run or manually inspect after implementing an improvement and what result counts as passing.

Provide a minimal round-report template (the exact before/after metric fields) so the exit criteria's 'before/after numbers, the misses explained' is directly producible.

DimensionReasoningScore

Conciseness

The ~16-line body is lean and efficient with no padding and no explanation of concepts Claude already knows; every line prescribes behavior ('Ground truth lives in truth/, which the verifier never reads', 'Never tune on the set you report'). It matches the 5 anchor.

5 / 5

Actionability

It names concrete metrics ('false-done (target 0), false-reject, missed honesty flag, false honesty flag, errors') and control conditions, but provides no commands or mechanics for executing them: 'Run every scenario', 'Implement, spot-check' and the scoring step all assume an unstated harness, and the scenario file layout is unspecified beyond 'truth/'. This sits at anchor 3 (concrete but incomplete, key execution details missing) rather than 4, which requires mostly executable guidance with only minor gaps.

3 / 5

Workflow Clarity

The four-step round is clearly sequenced with an explicit feedback loop (for each miss: root cause, improvement, target scenario, then implement and retest) plus exit criteria, matching anchor 4. It falls short of 5 because 'spot-check' is undefined and there is no explicit validation step or pass threshold beyond 'target 0' for false-done.

4 / 5

Progressive Disclosure

The skill is under 50 lines, needs no external references, and organizes its content into three well-labeled sections (Benchmark, Round, Exit criteria), meeting the simple-skill exception for a 5. No bundle files exist and the only path mentioned ('truth/') is scenario data, not a skill reference.

5 / 5

Total

17

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: specific and comprehensive about both the mechanism (seeded benchmark, control conditions, blind sets, calibration rounds) and the triggers (after agent/prompt/test/verifier changes, weekly). The only gap is modest synonym coverage in its trigger terms, relying on the project-specific phrase 'done-gate is fooled' rather than common paraphrases.

DimensionReasoningScore

Specificity

The description lists multiple concrete actions with comprehensive coverage: 'Measure and improve how often the done-gate is fooled', 'seeded false-done benchmark with hidden ground truth, genuine-fix controls, honest-partial controls, and fresh blind report sets', and 'rounds of test, improvement plan, implement, retest'. It matches the 5 anchor; the 4 anchor would require minor coverage gaps, and none are present.

5 / 5

Completeness

It explicitly answers both what (measure and improve via a seeded benchmark with controls, run in rounds) and when ('Use after changing any agent, prompt, test, or verifier, and on a weekly schedule'). The 'when' is fully explicit with concrete triggers, matching the 5 anchor rather than the 4 anchor's weaker 'when'.

5 / 5

Trigger Term Quality

'Use after changing any agent, prompt, test, or verifier' plus 'done-gate', 'benchmark', and 'verifier' give good keyword coverage a user of this skill would naturally say, but common phrasings like 'claims done', 'false completion', or 'verify the agent finished' are missing. It sits above anchor 3 (relevant keywords present) but short of anchor 5's synonym-level coverage.

4 / 5

Distinctiveness Conflict Risk

Done-gate calibration with hidden ground truth, blind report sets, and honesty-flag metrics is a clear niche with distinct triggers and minimal conflict risk with other skills. The description matches the 5 anchor.

5 / 5

Total

19

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

itsablabla/qa-bug-agent-skills

Path

skills/done-gate-calibrate/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.