CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

analyze-ci-failures

Analyze CI failures on Azure SDK for .NET pull requests and post a comment with how-to-fix instructions. Use when a PR has failing checks, CI is red, or someone asks for help fixing CI.

Quality

85%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

—

The risk profile of this skill

SKILL.md
Quality
Evals
Security

Quality

Content

81%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A strong, lean operational skill: the workflow is explicitly sequenced with validation, duplicate-prevention, and escalation loops, and the mapping tables supply genuinely non-inferable repo knowledge. The main gap is on the fix side of its 'how-to-fix' mission — it demands concrete fix commands in the output comment but never provides a template for one.

Suggestions

Add one or two templated fix commands matching the mapping tables' root causes (e.g., the exact regenerate/export command for 'Generated code is not up to date' and the changelog-entry format CHANGELOG.md expects), so the 'concrete fix commands with <service>/<package> filled in' requirement has a model to follow.

Trim the repeated meta-justification ('The check names are repo-specific and not discoverable from general knowledge' / 'not inferrable from general knowledge') to a single occurrence or drop it — the executing model does not need the tables provenance explained.

Consider moving the CI check-name and log-symptom mapping tables into a references/ file (e.g., references/failure-mappings.md) with a one-line pointer from the workflow step, keeping SKILL.md as a tighter overview.

DimensionReasoningScore

Conciseness

The body is dense, repo-specific operational data (exact tool names, a ready-to-use DevOps timeline API URL, grep-able log strings) with no explanation of concepts Claude already knows. Minor trimmable padding remains — the justification 'repo-specific and not discoverable from general knowledge' is repeated before both mapping tables, and the caller-contract sentences are wordy.

4 / 5

Actionability

Diagnosis guidance is highly concrete ('github-mcp-server-get_job_logs with return_content: true', exact timeline API URL, exact symptom strings like 'MembersMustExist' and 'changelog entry has the following sections with no content', 210-char path limit). The fix side is thinner: it requires 'concrete fix commands with <service>/<package> filled in' but never templates one (e.g., the regenerate command for 'Generated code is not up to date'), so guidance is mostly executable with minor gaps.

4 / 5

Workflow Clarity

A clear gather → identify → post sequence with explicit validation checkpoints ('verify the PR is still open, non-draft, and at that head… Stop without commenting if it is superseded', duplicate check against 'the same PR/head/CI run/completed attempt', mark root cause 'unconfirmed' when logs are unavailable), a feedback loop ('If the issues are insufficient, fetch the failed record's log.url'), and a checklist for new-package PRs.

5 / 5

Progressive Disclosure

A single self-contained SKILL.md with clean section headers and no nested references (no bundle files exist), and all inline content is needed on every invocation. However, the ~50 lines of lookup tables and checklist could arguably live in references/ files, and there is no pointer or navigation structure — good structure with minor organization gaps rather than an ideal overview-plus-references split.

4 / 5

Total

17

/

20

Passed

Description

83%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A well-crafted description: it states a precise what, an explicit when with concrete trigger phrases, and a clearly bounded niche. The only improvements are broader trigger synonym coverage and slightly richer enumeration of the skill's concrete capabilities.

DimensionReasoningScore

Specificity

Names a precise domain ('Azure SDK for .NET pull requests') and two fully concrete actions ('Analyze CI failures', 'post a comment with how-to-fix instructions') that cover the skill end-to-end. It sits between anchor 3 (1-2 concrete actions) and anchor 4 (several specific actions, minor gaps) — the domain precision and complete coverage put it noticeably above the midpoint, though only two actions are listed.

4 / 5

Completeness

Explicitly answers both questions: what ('Analyze CI failures… post a comment with how-to-fix instructions') and when ('Use when a PR has failing checks, CI is red, or someone asks for help fixing CI') with concrete trigger conditions, mirroring the anchor-5 example's structure. Not 4, because the when-clause is already fully explicit and specific.

5 / 5

Trigger Term Quality

Includes natural phrases users would actually say: 'failing checks', 'CI is red', 'help fixing CI'. Good coverage, but common variations like 'build failed', 'pipeline failure', or 'fix the build' are missing, so it falls short of anchor 5's comprehensive synonym coverage.

4 / 5

Distinctiveness Conflict Risk

Clear niche — CI failure analysis scoped to Azure SDK for .NET pull requests — with mostly distinct triggers. Minor overlap risk remains with generic fix-CI/build skills because the trigger phrases themselves ('CI is red', 'failing checks') are generic CI terms not unique to this repo.

4 / 5

Total

17

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

Azure/azure-sdk-for-net

Path

.github/skills/analyze-ci-failures/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.