review-test-failures

Classifies PR CI/test failures as likely PR-caused or unrelated, compares against base-branch baseline, and emits an overall merge-readiness verdict. Uses gathered GitHub/AzDO/Helix context and the shared MAUI CI facts.

Quality

70%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./.github/skills/review-test-failures/SKILL.md

Quality

Content

75%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable with a concrete output template and a strong deterministic-ceiling validation checkpoint, and it delegates canonical facts to a one-level-deep reference. Its main weakness is conciseness — the inline gate-field documentation is long and repetitive — and two bundle scripts are not linked by path.

Suggestions

Compress the repeated "any value > 0 caps the ceiling at Needs human investigation" phrasings into a single stated rule with a compact per-field table.

Reference the merger/publisher scripts by explicit path (scripts/Merge-TestVisualsIntoComment.ps1, scripts/Publish-TestVisualAssets.ps1) instead of describing them only as "trusted merger/publisher".

Consider moving the exhaustive gate-field reference (lines ~36–220) into a companion reference file so SKILL.md stays a lean overview.

Dimension	Reasoning	Score
Conciseness	Mostly efficient and full of genuine esoteric domain knowledge, but the long inline gate-field documentation repeats the "any value > 0 caps the ceiling at Needs human investigation" emphasis many times and could be tightened.	3 / 5
Actionability	Provides a fully concrete, copy-paste-ready output template with exact badges, a per-failure verdict taxonomy, named context fields, and explicit citation requirements covering the common cases.	5 / 5
Workflow Clarity	A clear classify → baseline-compare → synthesize-verdict → format-output sequence with an explicit validation checkpoint (the deterministic gate.verdictCeiling as a hard ceiling and a more-conservative feedback path), though the steps are spread across sections rather than a single ordered list.	4 / 5
Progressive Disclosure	Good structure with one-level-deep, clearly signaled references (maui-ci-facts.md, scripts/Gather-TestFailureContext.ps1); however the two other bundle scripts (Merge-TestVisualsIntoComment.ps1, Publish-TestVisualAssets.ps1) are alluded to as "trusted merger/publisher" without paths, a minor navigation gap.	4 / 5
	Total	16 / 20 Passed

Description

66%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific and distinctive, naming concrete classification/comparison/verdict actions and natural CI/merge terms, but it omits an explicit "Use when..." trigger clause, capping completeness. Adding trigger guidance would lift the weakest dimension.

Suggestions

Append an explicit "Use when reviewing PR CI/test failures, deciding merge readiness, or triaging whether failing checks are PR-caused vs. flaky/infra/baseline" clause.

Drop the jargon tokens (AzDO, Helix, MAUI CI) from the description in favor of plain trigger phrases, or pair each with a natural synonym.

Add the file extension / system triggers users actually say (e.g. ".github CI checks", "Helix test runs") to round out keyword coverage.

Dimension	Reasoning	Score
Specificity	Names several concrete actions — "Classifies PR CI/test failures", "compares against base-branch baseline", "emits an overall merge-readiness verdict" — with only minor abstraction in the verbs used.	4 / 5
Completeness	Clearly answers "what" (classify, compare, emit verdict) but lacks any explicit "Use when..." trigger guidance, which caps completeness at 3 per the rubric.	3 / 5
Trigger Term Quality	Includes natural terms users would say ("PR CI/test failures", "merge-readiness verdict", "base-branch baseline") alongside some jargon (AzDO, Helix, MAUI CI), leaving a few plain-language variants missing.	4 / 5
Distinctiveness Conflict Risk	Occupies a clear niche (deterministic merge-readiness verdict from CI failures) with minor overlap risk against the closely related interactive azdo-build-investigator skill mentioned in the body.	4 / 5
	Total	15 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: dotnet/maui
Path: .github/skills/review-test-failures/SKILL.md
Commit: 90afade

Reviewed: ~18 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.