CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

pipeline-test-triage

Triage test failures in an Azure DevOps pipeline run for Fluid Framework. Use when the user shares an ADO build/pipeline URL or build ID and wants failures analyzed, when they ask "why did this pipeline fail", "summarize the test failures", "is this test flaky or a real bug", "compare these runs", or wants bugs filed for genuine failures. Covers the Real Service End to End Tests pipeline as the worked example but applies to any FF test pipeline.

Quality

92%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

The canonical home for this skill is pipeline-test-triage in microsoft/FluidFramework

SKILL.md
Quality
Evals
Security

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A high-quality operational runbook: dense with verified, non-inferable specifics, fully executable commands, and a clearly sequenced tiered workflow with explicit validation checkpoints and error recovery. The only weaknesses are minor repetition and the near-total inlining of content that could partially live in reference files.

DimensionReasoningScore

Conciseness

Nearly every passage carries non-obvious, verified, org-specific detail (resource GUIDs, api-version gotchas, log-size heuristics) with no padding of concepts Claude already knows. Minor duplication could be trimmed: the green-build/retried-attempt warning appears both as a taxonomy callout ("Green builds hide failures too") and again in Tier 3 ("⚠️ Green builds can still hide failures"), and the bulk-log-cost point is made twice ("treat it as a last resort" intro + "just know it's the costly path").

4 / 5

Actionability

Fully executable throughout: exact REST URLs with required api-versions ("vstmr.dev.azure.com/...resultsbybuild?...api-version=7.1-preview.1"), concrete MCP invocations with parameters ("ado-pipelines_build action=get_status buildId={ID} project=internal"), runnable PowerShell token/deep-link snippets, a real helper script ("scripts/Parse-AdoTestLog.ps1 -Path <temp-file> [-Context 30]"), and a ready-to-paste bug Description template. Not 4 because the commands cover the common cases end-to-end with no gaps requiring invention.

5 / 5

Workflow Clarity

The tiered escalation is explicitly gated ("Descend a tier only when the one above can't answer the question"), each tier has numbered steps, and validation checkpoints are pervasive: snapshot-before-click, match logs "on the first line / the test:realsvc:<driver>:report command rather than trusting size", deterministic-vs-flaky verdict criteria, "Always check for retries before declaring a build clean", and "Confirm scope with the user first" before the write operation (bug filing). Error-recovery loops are also present for auth failures (tenant mismatch → az login, az.cmd not on PATH → full path).

5 / 5

Progressive Disclosure

Good structure: clear section headers, a navigable tier table, and the one bundle file that exists (scripts/Parse-AdoTestLog.ps1, verified present) is clearly referenced with its invocation. Not 5 because virtually all ~300 lines live inline in SKILL.md — the "Reference specifics — Real Service End to End Tests" section and the bug template are natural candidates for references/ files; not 3 because what exists is well-organized and the single reference is clearly signaled.

4 / 5

Total

18

/

20

Passed

Description

95%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: concrete actions, natural quoted trigger phrases, and an explicit Use-when clause pinning exactly when to invoke it. The only gap is that the classification taxonomy that constitutes the skill's main deliverable is summarized as failure triage rather than spelled out.

DimensionReasoningScore

Specificity

Lists several concrete actions — "Triage test failures in an Azure DevOps pipeline run", "compare these runs", "wants bugs filed for genuine failures" — matching the several-actions-with-minor-gaps anchor. Not 5 because the skill's core deliverable (classifying service flakiness vs. infra vs. deterministic bugs) is left unstated and "wants failures analyzed" is generic; not 3 because more than 1-2 specific actions are named.

4 / 5

Completeness

Explicitly answers both: what ("Triage test failures in an Azure DevOps pipeline run for Fluid Framework") and when ("Use when the user shares an ADO build/pipeline URL or build ID and wants failures analyzed, when they ask...") with concrete quoted trigger phrases — a direct match for the top anchor.

5 / 5

Trigger Term Quality

Quotes the exact natural phrasings a user would say — "why did this pipeline fail", "summarize the test failures", "is this test flaky or a real bug", "compare these runs" — plus artifact triggers ("ADO build/pipeline URL or build ID") covering the build/pipeline synonyms for this domain. No natural term for this task is missing, so it fits the comprehensive-coverage anchor rather than the "a few natural terms missing" anchor at 4.

5 / 5

Distinctiveness Conflict Risk

Occupies a clear niche (ADO pipeline test-failure triage for Fluid Framework, named project/repo) with distinct triggers (ADO URL/build ID, flaky-vs-real-bug phrasing); minimal risk of firing for a different skill. Not 4 because it is not merely "mostly distinct" — the scoped domain and triggers leave essentially no overlap with generic testing or CI skills.

5 / 5

Total

19

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

relative_links

Relative link issues: 2 missing, 1 deeper-than-1-level

Warning

Total

15

/

16

Passed

Repository

microsoft/FluidFramework

Path

.github/skills/pipeline-test-triage/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.