pipeline-test-triage

Triage test failures in an Azure DevOps pipeline run for Fluid Framework. Use when the user shares an ADO build/pipeline URL or build ID and wants failures analyzed, when they ask "why did this pipeline fail", "summarize the test failures", "is this test flaky or a real bug", "compare these runs", or wants bugs filed for genuine failures. Covers the Real Service End to End Tests pipeline as the worked example but applies to any FF test pipeline.

Quality

98%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Quality

Content

96%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is an exceptionally actionable, well-sequenced operational playbook with copy-paste-ready commands, exact API recipes, and clear validation checkpoints. Its only meaningful weakness is progressive disclosure: the ~300-line body inlines detail (recipes, templates, reference specifics) that could be split into separate reference files alongside the existing script.

Suggestions

Extract the Tier 2/3 REST recipes and auth block into a references/ file (e.g. references/rest-recipes.md), keeping SKILL.md as a tiered overview that links to it — this would lift progressive_disclosure toward the top anchor and reduce inline bulk.

Move the bug-description template and ADO text gotchas into a references/filing-bugs.md, linked from the "Filing bugs" section, so the template lives one level deep rather than inline.

Move the "Reference specifics — Real Service End to End Tests" appendix (definitionId, legs, flaky signatures) into references/real-service-e2e.md, signaling it as the pipeline-specific reference rather than inlining it.

Dimension	Reasoning	Score
Conciseness	Dense, high-signal operational guidance with no padded concept explanations (no "what is ADO" or library tutorials); every line carries actionable specifics and it assumes Claude's competence. A few transitional caveats could be trimmed but they are minor.	5 / 5
Actionability	Provides copy-paste-ready, fully executable guidance throughout: exact MCP tool calls with parameters, exact REST URLs with api-version values, a working PowerShell token recipe, a real referenced script (scripts/Parse-AdoTestLog.ps1), a bug-description template, and a deep-link assembly recipe covering the common cases.	5 / 5
Workflow Clarity	Multi-step process is clearly sequenced via a Tier 1→4 escalation with numbered sub-steps and explicit checkpoints ("Descend a tier only when the one above can't answer", "Always check for retries before declaring a build clean"); destructive bug-filing is gated with user confirmation, and validation feedback loops are present.	5 / 5
Progressive Disclosure	Good header structure and one clearly-signaled one-level-deep reference (scripts/Parse-AdoTestLog.ps1 at line 113), but the body is largely monolithic — the Tier 2/3 recipes, bug template, and reference specifics are all inlined in SKILL.md rather than split into reference files, which is the gap below the top anchor.	4 / 5
	Total	19 / 20 Passed

Description

100%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is exemplary: it states concrete actions, lists natural-language trigger phrases users would actually say, explicitly covers both what and when, and carves out a distinct niche. Third-person voice is maintained throughout with no first/second-person usage.

Dimension	Reasoning	Score
Specificity	Names the domain and multiple concrete actions — "Triage test failures", "wants failures analyzed", "bugs filed for genuine failures" — with comprehensive coverage of the triage workflow, matching the anchor for listing multiple specific concrete actions.	5 / 5
Completeness	Explicitly answers both what (triage, classify, file bugs) and when via a concrete "Use when..." clause listing multiple trigger scenarios, matching the anchor for clearly answering both with concrete trigger phrases.	5 / 5
Trigger Term Quality	Quotes natural user phrases verbatim ("why did this pipeline fail", "summarize the test failures", "is this test flaky or a real bug", "compare these runs") plus triggers like sharing an ADO URL/build ID, giving comprehensive coverage of terms users would actually say.	5 / 5
Distinctiveness Conflict Risk	Highly specific to Azure DevOps/Fluid Framework pipeline test triage with distinct triggers (ADO build URL/ID, flaky-vs-bug questions), creating a clear niche with minimal conflict risk against other skills.	5 / 5
	Total	20 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
relative_links	Relative link issues: 2 missing, 1 deeper-than-1-level	Warning

	Total	15 / 16 Passed

Repository: microsoft/FluidFramework
Path: .claude/skills/pipeline-test-triage/SKILL.md
Commit: f02e424

Reviewed: 3 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.