Name: analyze-ci-failures
Rating: 70.39999999999999 (1 reviews)
Author: Azure

analyze-ci-failures

Analyze CI failures on Azure SDK for .NET pull requests and post a comment with how-to-fix instructions. Use when a PR has failing checks, CI is red, or someone asks for help fixing CI.

Quality

85%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Quality

Content

82%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable skill body that supplies exactly the repo-specific knowledge Claude lacks (check-name and log-symptom mappings) without padding. The main improvement lever is progressive disclosure: the two large mapping tables could optionally move into reference files.

Suggestions

Move the 'CI Check Name → Failure Mapping' and 'Log Symptom → Root Cause Mapping' tables into a references/ file (e.g. CI_MAPPINGS.md) and link to it, keeping SKILL.md as a tighter overview.

Drop the 'When Invoked' trigger-phrase list since it duplicates the frontmatter description's triggers, saving tokens.

Add an explicit validate-retry checkpoint after posting (e.g. confirm the comment posted and re-scan for new failures on re-run) to strengthen the feedback loop.

Dimension	Reasoning	Score
Conciseness	The body is lean and avoids explaining concepts Claude already knows (no 'what is CI' padding), and the repo-specific mapping tables earn their tokens; the only redundancy is the 'When Invoked' trigger list echoing the description.	4 / 5
Actionability	Provides concrete, executable guidance throughout: exact curl URLs with api-version, named MCP tools (azure-sdk-mcp:azsdk_analyze_pipeline, github-mcp-server-get_job_logs), script paths, and exact log strings to grep.	5 / 5
Workflow Clarity	A clear three-step sequence (gather, identify, post) with strong validation guardrails (quote decisive error, do not infer from check name, dedup before posting, mark unconfirmed if logs missing), but it is a diagnostic procedure rather than a classic validate-fix-retry feedback loop.	4 / 5
Progressive Disclosure	Well-organized into clearly headed sections (When Invoked, Inputs, Workflow, CI Check Mapping, Log Symptom Mapping, New Package Checklist) with no nested references, but at ~95 lines with two dense reference tables it sits just below the ideal one-level-deep split.	4 / 5
	Total	17 / 20 Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description that clearly states a specific domain, two concrete actions, and explicit trigger conditions in third person. Minor gaps are limited coverage of action verbs and a few missing trigger synonyms.

Suggestions

Broaden trigger coverage with synonyms users actually say, e.g. 'pipeline is failing', 'build broken', or 'checks failing'.

Consider adding one more concrete action verb (e.g. 'classify' or 'diagnose') to push specificity from two to several listed actions.

Dimension	Reasoning	Score
Specificity	Names the domain ('Azure SDK for .NET pull requests') and two concrete actions ('Analyze CI failures', 'post a comment with how-to-fix instructions'), landing between the 1-2-action anchor 3 and the several-actions anchor 4; the actions are specific but only two are listed.	4 / 5
Completeness	Explicitly answers both what (analyze CI failures and post fix-instruction comments) and when ('Use when a PR has failing checks, CI is red, or someone asks for help fixing CI') with concrete trigger phrases.	5 / 5
Trigger Term Quality	Includes natural phrases a user would say ('failing checks', 'CI is red', 'asks for help fixing CI'), but misses common synonyms like 'pipeline', 'build broken', or 'checks failing', so it sits just below comprehensive.	4 / 5
Distinctiveness Conflict Risk	Scoped to a clear niche ('Azure SDK for .NET pull requests') with distinct CI-focused triggers, giving minimal overlap risk with other skills.	5 / 5
	Total	18 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: Azure/azure-sdk-for-net
Path: .github/skills/analyze-ci-failures/SKILL.md
Commit: 50a6ffe

Reviewed: ~11 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.