ci-pipeline-monitor

Monitors .NET runtime CI test pipelines on Azure DevOps. Use this skill when asked to monitor CI pipeline test results, triage CI test failures across ADO pipelines, or generate CI test monitoring reports.

Quality

81%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

High

Do not use without reviewing

Quality

Content

81%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is highly actionable with an exceptionally clear, validated workflow and feedback loops. Its main weakness is progressive disclosure: it references seven files/directories that are absent from the bundle, leaving the structure unresolvable, and inlines a large schema and retry detail that belong in reference files.

Suggestions

Ship the referenced bundle files (references/prerequisites.md, references/triage-workflow.md, references/validation-checks.md, references/verbatim-rules.md, log-template.md, report-template.md, pipelines.md) so the one-level-deep navigation actually resolves.

Move the full CREATE TABLE schema into a reference file (e.g. references/db-schema.md) and keep only a brief table summary in SKILL.md to cut tokens and improve discovery.

Condense the Step 5a retry procedure into a reference (e.g. references/validation-checks.md) and reference it from the main workflow rather than inlining the full retry loop.

Dimension	Reasoning	Score
Conciseness	The body is dense and largely operational with minimal concept explanation, but the SQL schema (60+ lines) and repeated inline restatements of script behavior add tokens that could be trimmed or moved to reference files.	4 / 5
Actionability	Provides fully executable, copy-paste-ready commands for every step, a complete SQL schema, specific API URLs with api-version, and concrete retry/validation procedures covering the common cases.	5 / 5
Workflow Clarity	A clearly sequenced 7-step pipeline with explicit validation (Step 5: validate_results.py exits 1 on failure), an explicit feedback loop (Step 5a: fix -> re-validate, up to 3 retries, stop when failures stop decreasing), and an enumerated bisection checklist.	5 / 5
Progressive Disclosure	The body relies heavily on one-level-deep references (pipelines.md, references/prerequisites.md, references/triage-workflow.md, references/validation-checks.md, references/verbatim-rules.md, log-template.md, report-template.md), but no referenced bundle files or directories (references/, scripts/, assets/) actually exist in the skill bundle, so the navigation is broken and the bulk (full SQL schema, detailed retry steps) is inlined rather than split out.	2 / 5
	Total	16 / 20 Passed

Description

82%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is well-structured with a clear what/when split and an explicit trigger clause; it is distinctive and uses third person. It is slightly light on breadth of distinct concrete actions, capping specificity at 3.

Suggestions

Broaden the action list to make specificity more comprehensive, e.g. 'fetch failed test results, download Helix console logs, group failures by root cause, search GitHub for matching issues'.

Add common synonyms users might say ('CI test runs', 'pipeline failures', 'flaky-test triage') to round out trigger-term coverage.

Dimension	Reasoning	Score
Specificity	Names the domain (.NET runtime CI on Azure DevOps) and two concrete actions ('monitor CI pipeline test results', 'triage CI test failures') plus a third ('generate CI test monitoring reports'), but the actions cluster around one capability rather than a comprehensive set of distinct operations.	3 / 5
Completeness	Explicitly states what it does ('Monitors .NET runtime CI test pipelines on Azure DevOps') and an explicit 'Use this skill when asked to...' clause with concrete trigger phrases, satisfying both what and when.	5 / 5
Trigger Term Quality	Includes natural phrases users would say ('monitor CI pipeline test results', 'triage CI test failures', 'generate CI test monitoring reports'); minor gaps in synonyms/extensions (no 'CI tests', 'pipeline failures', 'flaky tests').	4 / 5
Distinctiveness Conflict Risk	Clear niche — .NET runtime CI on a specific ADO project (dnceng-public/public) — with distinct triggers unlikely to fire for unrelated skills; minimal conflict risk.	5 / 5
	Total	17 / 20 Passed

Validation

87%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

Criteria	Description	Result
relative_links	Relative link issues: 9 missing	Warning
referenced_paths_exist	Referenced path issues: 34 missing	Warning

	Total	14 / 16 Passed

Repository: dotnet/runtime
Path: .github/skills/ci-pipeline-monitor/SKILL.md
Commit: b030ed4

Reviewed: 2 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.