Name: pr-failure-scan
Rating: 67.2 (1 reviews)
Author: dotnet

pr-failure-scan

Analyze a dotnet/runtime PR's CI failures, skip failures already known to Build Analysis, find matching Known Build Errors, and create or draft new KBEs for the remaining failures. Supports dry-run output to local markdown files instead of creating GitHub issues.

Quality

80%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./.github/skills/pr-failure-scan/SKILL.md

Quality

Content

77%Weight 40%Scale 1-3

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The skill body is highly actionable and very clearly sequenced with strong validation/permission checkpoints for a risky live-issue workflow. Its weaknesses are mild verbosity from repeated caveats and inline-heavy content that has no reference files to offload detail into.

Suggestions

Reduce repeated caveats — state the 'local-skill-only / do not feed back into the scheduled workflow' rule once near the top of Step 4 instead of repeating it across the draft-only subsections.

Consider extracting the full draft-file markdown template and the live-mode confirmation-question spec into a reference file (e.g. references/kbe-draft-template.md) and summarizing them in SKILL.md, improving progressive disclosure.

Tighten the dry-run drafting guidance in Step 4 by merging the three signature-preference bullets into a single concise rule.

Dimension	Reasoning	Score
Conciseness	The body is dense and mostly efficient without explaining concepts Claude already knows, but a few passages repeat caveats (e.g. the repeated 'for this local skill only' / 'do not feed this back into the scheduled workflow' reminders) and the inline draft template adds length that could be trimmed. Not 3 because of these tightening opportunities; not 1 because it is not padded or conceptual.	2 / 3
Actionability	Provides concrete, executable guidance: 'gh api user', the exact shared path '.github/workflows/shared/create-kbe.instructions.md', specific fields to collect, a full draft-file markdown template, and the exact contents required in the live-mode confirmation question. This is copy-paste-ready, matching the 3 anchor.	3 / 3
Workflow Clarity	Steps 0-6 are explicitly sequenced with strong validation checkpoints for a risky batch/live-issue operation: an explicit permission gate for non-owned PRs, 'prefer raw CI evidence and report disagreement', draft-before-live ordering, and a per-issue confirm-or-decline loop with 'only create if the user explicitly confirms'. This matches the 3 anchor including feedback loops for batch operations.	3 / 3
Progressive Disclosure	Content is organized into clear numbered sections and references one shared instruction file one level deep ('.github/workflows/shared/create-kbe.instructions.md'), but no bundle files exist and substantial detail (the draft template, all step guidance) is inlined in SKILL.md rather than split into reference files. This is a non-simple multi-step skill, so well-organized sections alone do not reach 3.	2 / 3
	Total	10 / 12 Passed

Description

82%Weight 40%Scale 1-3

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific, distinctive, and uses natural trigger terms, but it omits an explicit 'Use when...' trigger clause in the frontmatter, which caps completeness at 2. Adding an explicit 'when to use' sentence would lift it to fully comprehensive.

Suggestions

Add an explicit trigger clause to the description, e.g. 'Use when asked to triage CI failures on a dotnet/runtime PR or to create/draft Known Build Errors (KBEs) from a PR's failed checks.'

Include common user phrasings such as 'create KBEs for PR <number>' or 'dry-run KBE triage' directly in the description to strengthen natural trigger terms.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions — 'Analyze a dotnet/runtime PR's CI failures', 'skip failures already known to Build Analysis', 'find matching Known Build Errors', 'create or draft new KBEs', 'Supports dry-run output to local markdown files'. It is comprehensive rather than naming only some actions, so it is above the 2 anchor.	3 / 3
Completeness	Clearly answers 'what' but the 'when' is not stated in the frontmatter — there is no 'Use when...' clause or equivalent explicit trigger guidance. Per judging guidelines, a missing explicit trigger clause caps completeness at 2, so it does not reach 3.	2 / 3
Trigger Term Quality	Includes natural terms a user would say — 'PR's CI failures', 'Known Build Errors', 'KBEs', 'dry-run', 'GitHub issues' — covering more than just a few relevant keywords. Not score 2 because the keyword coverage is broad and user-natural rather than partial.	3 / 3
Distinctiveness Conflict Risk	The niche is highly specific — 'dotnet/runtime PR's CI failures', 'Build Analysis', 'Known Build Errors / KBEs' — making it unlikely to trigger for the wrong skill. It is clearly distinguishable, matching the 3 anchor.	3 / 3
	Total	11 / 12 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: dotnet/runtime
Path: .github/skills/pr-failure-scan/SKILL.md
Commit: ba10a6e

Reviewed: 8 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.