CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

pr-failure-scan

Analyze a dotnet/runtime PR's CI failures, skip failures already known to Build Analysis, find matching Known Build Errors, and create or draft new KBEs for the remaining failures. Supports dry-run output to local markdown files instead of creating GitHub issues.

Quality

81%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

—

The risk profile of this skill

SKILL.md
Quality
Evals
Security

Quality

Content

81%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable instruction skill with a clear step sequence, strong validation/permission gates, and appropriate delegation of shared KBE rules to a referenced file. The main improvement levers are consolidating the repeated authorization language and adding concrete commands for the CI-evidence gathering steps.

DimensionReasoningScore

Conciseness

The body is dense with repo-specific policy rather than concepts Claude already knows, and the draft-file template is concrete. However, permission rules are stated at length twice (Step 1's "Running the command itself is not enough permission" vs Step 5's "Invoking this skill or asking to analyze CI alone is not permission to publish" plus the ~30-line authorization block), and the draft-only/dry-run guidance spreads over two sections with some overlap — minor trimming opportunities that fit the 'efficient; minor instances of over-explanation' anchor rather than score 5's 'every token earns its place'.

4 / 5

Actionability

Concrete, executable guidance dominates: input-parsing examples ('/pr-failure-scan 123456', the URL and dryrun forms), 'Resolve the currently authenticated GitHub user (gh api user is fine)', exact bucket classifications, a copy-paste draft-file template, and a mandated four-section final output. It is not 5 because CI-gathering steps (check run payloads, AzDO build URLs, timeline/logs) describe what to fetch without exact gh/API commands.

4 / 5

Workflow Clarity

Steps 0-6 are clearly sequenced with explicit validation checkpoints: the identity/permission gate in Step 1, the shared verification checklist, the draft-only bucket for evidence that is 'not yet strong enough for fully automated filing', disagreement handling ('prefer the raw CI evidence and report the disagreement'), and per-issue confirmation in live mode. Error-recovery loops are present (decline -> 'stop immediately' / 'skip that issue and continue'; approval unavailable -> 'leave it as a draft and report the pending decision'), so the batch-operation cap at 3 does not apply.

5 / 5

Progressive Disclosure

No bundle files exist (references/, scripts/, assets/ are absent), and the body is well-sectioned with clearly signaled one-level-deep external references: '.github/workflows/shared/create-kbe.instructions.md' is introduced up front with an explicit list of what it owns, and the publication rules link is inline-named. Minor gaps: the draft template and dry-run drafting preferences are inline content that could live in reference files if a bundle existed, keeping the main file leaner.

4 / 5

Total

17

/

20

Passed

Description

75%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A highly specific, action-rich description with excellent distinctiveness, weakened only by the absence of an explicit 'Use when...' trigger clause and slightly incomplete keyword variations. Adding a use-when clause would resolve the completeness cap and likely raise trigger-term quality.

Suggestions

Add an explicit trigger clause, e.g. 'Use when the user asks to triage or create KBEs for a dotnet/runtime PR's CI failures, or mentions PR build/check failures.'

Include natural synonyms users might say, such as 'build failure', 'check failure', or 'CI triage', to broaden trigger-term coverage.

Consider naming the skill invocation forms ('/pr-failure-scan', 'create KBEs for PR') in the description so users searching by command can find it.

DimensionReasoningScore

Specificity

The description lists multiple concrete actions covering the full task: "Analyze a dotnet/runtime PR's CI failures", "skip failures already known to Build Analysis", "find matching Known Build Errors", "create or draft new KBEs", and "dry-run output to local markdown files instead of creating GitHub issues". This matches the anchor for multiple specific concrete actions with comprehensive coverage; it is not score 4 because there are no minor gaps in the action coverage.

5 / 5

Completeness

The 'what' is explicit and comprehensive, but there is no 'Use when...' clause or equivalent explicit trigger guidance; "Supports dry-run output..." describes a capability, not when to invoke the skill. Per the judging guidelines, a missing 'Use when...' clause caps completeness at 3, which is a clear fit ('when' only weakly implied by 'Analyze a dotnet/runtime PR's CI failures').

3 / 5

Trigger Term Quality

It includes the natural phrases users of this workflow would say — "CI failures", "PR", "Known Build Errors", "KBEs", "dry-run", "GitHub issues" — which is good keyword coverage. It is not 5 because common variations are missing (e.g., "build failure", "check failure", "flaky test") and not 3 because the terms present are exactly the domain vocabulary a dotnet/runtime contributor would use.

4 / 5

Distinctiveness Conflict Risk

It occupies a clear niche — dotnet/runtime PR CI failure triage and KBE creation — with distinct triggers ("Build Analysis", "Known Build Errors", "KBEs") that no generic skill would claim. Minimal conflict risk; not 4 because there is no meaningful overlap with any closely related skill category.

5 / 5

Total

17

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

relative_links

Relative link issues: 1 suspicious

Warning

Total

15

/

16

Passed

Repository

dotnet/runtime

Path

.github/skills/pr-failure-scan/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.