CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

diagnose-benchmark-failure

Use when a judged benchmark or acceptance scenario fails and you need the cause before editing. Reads the judge's rendering, the raw transcript, the primitive the agent saw and the scenario, then reports a classified diagnosis without fixing.

72

Quality

91%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

An excellent, highly actionable diagnostic workflow: concrete commands for every step, explicit validation checkpoints, and a real failure-mode taxonomy. Its weaknesses are modest — some rule/step redundancy and a monolithic single-file layout that could offload the taxonomy and report template to references.

Suggestions

Trim the repetition between the Rules section and Step-by-step (e.g., state fail-closed and cross-check rules once, where they are applied).

Consider moving the failure-mode taxonomy and/or the report template to a one-level-deep reference file (e.g. references/taxonomy.md) to shrink the always-loaded body.

Keep the exact jq/resume commands as-is — they are the skill's strongest asset.

DimensionReasoningScore

Conciseness

The body is dense and earns most of its tokens (exact jq selectors, per-CLI resume commands, a symptom-cause-fix taxonomy), but the Rules section restates content repeated in Step-by-step — fail-closed (rule 4 vs step 1), capability cross-check (rule 5 vs step 3), and quote-don't-paraphrase (rule 3 vs step 9). Not 5 because those duplications could be trimmed; not 3 because there is no generic-concept padding and every section is project-specific.

4 / 5

Actionability

Fully executable guidance: copy-paste jq histogram commands for both codex and claude transcript formats, `find <run-dir> -name '*.jsonl'`, per-CLI resume commands with environment setup, and a complete verbatim report template with citation format.

5 / 5

Workflow Clarity

Nine explicitly sequenced steps with validation checkpoints throughout: fail closed on a missing artefact kind, histogram printed BEFORE hypothesis formation, claim-vs-transcript cross-checking, mandatory two-copy snapshot diff, and re-derivation of the verdict to catch judge calibration drift. The skill is read-only, so the destructive-operation cap does not apply.

5 / 5

Progressive Disclosure

Clear, navigable sections (Why, Rules, Inputs, Step-by-step, taxonomy, Output template) in a single file with no buried references. Not 5 because the ~330-line body inlines the failure-mode taxonomy and the full report template — content that could sit in one-level-deep reference files; not 3 because the structure is well-organized and the split is a minor, defensible choice (both are needed on every invocation).

4 / 5

Total

18

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: specific, third-person, explicit 'Use when...' trigger, and clearly distinguishable from adjacent skills. The only gap is a few natural synonyms users might use when a benchmark run fails.

Suggestions

Add one or two natural trigger synonyms such as 'failed run' or 'test failure' to widen the trigger surface without adding length.

DimensionReasoningScore

Specificity

The description lists multiple concrete actions — 'Reads the judge's rendering, the raw transcript, the primitive the agent saw and the scenario, then reports a classified diagnosis without fixing' — covering the skill's behavior comprehensively in third person.

5 / 5

Completeness

It explicitly answers both questions: 'Use when a judged benchmark or acceptance scenario fails and you need the cause before editing' (when) and the artefact-reading plus classified-diagnosis-report behavior (what), with concrete trigger phrases.

5 / 5

Trigger Term Quality

'Use when a judged benchmark or acceptance scenario fails' provides good natural trigger phrases, but common synonyms a user might say ('test failure', 'failed run', 'why did this run fail') are absent. Not 5 because anchor 5 requires comprehensive synonyms/extensions; not 3 because existing keyword coverage is strong and domain-accurate.

4 / 5

Distinctiveness Conflict Risk

A clear niche — diagnosing failed judged benchmark runs — with distinct triggers ('judged benchmark', 'acceptance scenario fails', 'classified diagnosis') that no general skill would claim; minimal conflict risk.

5 / 5

Total

19

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

15

/

16

Passed

Repository

korchasa/flowai-plugins

Path

plugins/flowai-engineering/skills/diagnose-benchmark-failure/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.