diagnose-benchmark-failure

Use when a flowai benchmark fails and you need the cause from run artifacts before editing. Reads judge-evidence.md, the sandbox SKILL.md, and scenario mod.ts; classifies the failure against a known taxonomy; produces an evidence-grounded report (no fixes). Do NOT trigger for passing benchmarks or generic skill iteration.

Quality

88%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is highly actionable and workflow-clear, with explicit fail-closed validation and a concrete output template. It is slightly verbose in its motivational intro and monolithic where a separate taxonomy reference file could improve progressive disclosure.

Suggestions

Trim the 'Why this skill exists' rationale to one line or fold it into the Rules section to reduce motivational padding.

Consider moving the failure-mode taxonomy into a references/ file (e.g. TAXONOMY.md) and summarizing the codes inline, improving progressive disclosure now that no bundle files exist.

Add a brief 'Inputs' validation check that fails closed when no scenario ID can be inferred, mirroring the artifact-missing checkpoint.

Dimension	Reasoning	Score
Conciseness	The body is mostly lean and assumes Claude's competence — exact paths, a find command, and a tight taxonomy table earn their tokens — but the 'Why this skill exists' intro and a few motivational lines ('That is guessing.') are mild padding that could be trimmed, stopping short of a 5.	4 / 5
Actionability	Fully executable guidance: concrete default paths ('acceptance-tests/runs/latest/<scenario-id>/run-1/'), a ready-to-run find command, named sections to extract, and a copy-paste output template with citation format cover the common cases.	5 / 5
Workflow Clarity	A clear 7-step sequence with explicit validation checkpoints — 'If either is missing → fail closed (rule 4)' and the step-5 re-run-with- '--no-cache' fallback — plus a fail-closed rule, giving strong feedback loops for a read-only diagnostic skill.	5 / 5
Progressive Disclosure	Well-organized into labeled sections (Rules, Inputs, Step-by-step, Taxonomy, Output template) with a single one-level-deep inline reference, but the ~230-line body is monolithic with no bundle files to offload the taxonomy, leaving minor organization gaps versus the clear-overview anchor.	4 / 5
	Total	18 / 20 Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific, complete, and distinctive, using third-person voice and providing explicit positive and negative trigger guidance. Its only weak spot is that the trigger terms are somewhat specialized internal jargon rather than broadly natural synonyms.

Dimension	Reasoning	Score
Specificity	Names the domain and lists several concrete actions — 'Reads judge-evidence.md, the sandbox SKILL.md, and scenario mod.ts; classifies the failure against a known taxonomy; produces an evidence-grounded report' — with only minor abstraction in 'classifies'/'produces', short of the fully comprehensive anchor.	4 / 5
Completeness	Explicitly answers both what (reads three artifacts, classifies against a taxonomy, produces a report) and when ('Use when a flowai benchmark fails... Do NOT trigger for passing benchmarks or generic skill iteration') with concrete trigger phrases.	5 / 5
Trigger Term Quality	Natural trigger phrases like 'benchmark fails' and 'run artifacts' appear with both positive and negative guidance ('Do NOT trigger for passing benchmarks or generic skill iteration'), though the terms are somewhat domain-internal jargon rather than broad synonyms.	4 / 5
Distinctiveness Conflict Risk	Occupies a clear niche (diagnosing flowai benchmark failures from run artifacts) with distinct triggers and an explicit anti-trigger clause, giving minimal conflict risk with other skills.	5 / 5
	Total	18 / 20 Passed

Validation

81%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 13 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning
relative_links	Relative link issues: 1 suspicious	Warning
referenced_paths_exist	Referenced path issues: 1 missing, 1 deeper-than-1-level	Warning

	Total	13 / 16 Passed

Repository: korchasa/flowai-plugins
Path: plugins/flowai-engineering/skills/diagnose-benchmark-failure/SKILL.md
Commit: b4ba256

Reviewed: ~20 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.