CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

review-test-failures

Analyze dotnet/maui PR failures across maui-pr, maui-pr-devicetests, and maui-pr-uitests against the latest five completed runs on the PR's target branch. Report only whether failures are PR-related, with failure links grouped by pipeline. When no current results exist, skip evaluation and request /azp run.

Quality

80%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./.github/skills/review-test-failures/SKILL.md
SKILL.md
Quality
Evals
Security

Quality

Content

77%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

An exceptionally actionable, well-sequenced analysis workflow with exact field names, decision rules, and a copy-paste output template. Its weaknesses are token efficiency — the same evidence-handling rules are restated several times and a paragraph about the hosted workflow's infrastructure does not instruct the agent — and bundle wiring, where the primary content reference points outside the skill and half the bundled scripts are never mentioned.

Suggestions

Consolidate the repeated 'missing results are not passes' rule into a single statement in the evidence section and reference it from the other sections instead of restating it four times.

Remove or move the hosted-workflow failure-notice/deduplication paragraph (lines 16-23) — it describes other jobs' behavior, not instructions for the agent executing this skill.

Fix the MAUI CI facts reference: either move the document inside the skill bundle (e.g. references/maui-ci-facts.md) or remove the link, and mention the other bundled scripts (Merge-TestVisualsIntoComment.ps1, Publish-TestVisualAssets.ps1) where relevant so they are discoverable.

DimensionReasoningScore

Conciseness

The body is dense and mostly prescriptive domain rules, but repeats the same rule several times — "Never treat missing current results as passing" (line 273), "do not... treat absent, skipped, or filtered tests as passes" (line 118), "Results missing from the bundle are not proof that CI has no results" (line 44), "Zero failed tests is not zero test results" (line 52) — and inlines an infrastructure paragraph about the hosted workflow's failure-notice job (lines 16-23) that describes other jobs' behavior rather than instructing the agent. Not 4: the repetition and the non-actionable infrastructure paragraph are more than minor trimmable instances; not 2: the bulk is justified, non-redundant domain guidance rather than padding or explanations of concepts Claude already knows.

3 / 5

Actionability

Quotes: exact context fields to read ("pr.headRefOid, pr.baseRefName, scope.diff, builds, and failures", "history.pipelines", "buildInsights.reports"), a four-row attribution table with required evidence, an exact budget ("at most 250 words"), exact badge parameters ("style=flat-square, labelColor=30363d, and blue 1f6feb"), and a complete copy-paste markdown template with literal HTML entities. As an instruction-only skill the guidance is fully executable, covering the common cases (no failures / pending / no results / missing context).

5 / 5

Workflow Clarity

The five sections form an explicit gated sequence — check for results first (with stop conditions per failure mode, e.g. "If context cannot be found or read, stop with a short report"), read evidence with verification checkpoints ("Verify current builds belong to the captured PR head", "Inspect collection errors and report truncated flags"), compare history, classify via decision table, then publish exactly once. Error-recovery branches and validation checkpoints are present throughout. Not 4: checkpoints are explicit and systematic, not merely 'most' present.

5 / 5

Progressive Disclosure

The body has clear section structure, but the only content reference — "consult the pipeline, data-source, and device-test sections of [MAUI CI facts](../../docs/maui-ci-facts.md)" — points two directories outside the skill bundle and does not resolve, and the bundle's other scripts (Merge-TestVisualsIntoComment.ps1, Publish-TestVisualAssets.ps1 and their tests) are never referenced from the body, leaving them undiscoverable. All detailed evidence-handling rules are inlined in one ~280-line file. Not 4: the broken external path and orphaned bundle files are more than minor organization gaps; not 2: sections are well-organized and the one in-body script reference (Gather-TestFailureContext.ps1) is clearly signaled.

3 / 5

Total

16

/

20

Passed

Description

75%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A highly specific, distinctive description that precisely states what the skill does, including its comparison methodology and no-results behavior. Its main weakness is the absence of explicit 'when to use' trigger guidance, which both caps completeness and leaves natural trigger phrases like 'test failures' or 'CI failures' unstated.

Suggestions

Add an explicit trigger clause, e.g. 'Use when reviewing test/CI failures on a dotnet/maui pull request or when a maintainer asks whether pipeline failures are related to a PR.'

Include natural trigger synonyms such as 'test failures', 'CI failures', and 'flaky tests' so the description matches how users actually phrase the request.

Consider mentioning the /review tests invocation in the description, since the body identifies it as the skill's primary entry point.

DimensionReasoningScore

Specificity

Quotes: "Analyze dotnet/maui PR failures across maui-pr, maui-pr-devicetests, and maui-pr-uitests against the latest five completed runs on the PR's target branch", "Report only whether failures are PR-related, with failure links grouped by pipeline", "skip evaluation and request /azp run". Multiple concrete actions (analyze against a precise comparison window, report attribution with grouped links, conditional skip/request) comprehensively cover the skill's domain. Not 4: coverage has no gaps — the analysis, output contract, and no-results edge case are all explicit.

5 / 5

Completeness

The 'what' is clear and detailed (analyze failures across three named pipelines, report PR-relatedness with links grouped by pipeline, skip and request /azp run when no results), but there is no 'Use when...' clause or equivalent explicit trigger guidance — "When no current results exist" is a behavioral condition, not a usage trigger. Per the rubric guideline, a missing explicit trigger caps completeness at 3.

3 / 5

Trigger Term Quality

Quotes: "PR failures", "PR-related", "maui-pr", "maui-pr-devicetests", "maui-pr-uitests", "target branch", "/azp run". These are the natural vocabulary a dotnet/maui maintainer would use, giving good keyword coverage. Not 5: common variations users would say such as "test failures", "CI failures", or "flaky tests" are absent; not 3: the domain terms present are exact and specific rather than partially covered.

4 / 5

Distinctiveness Conflict Risk

Quotes: "dotnet/maui", "maui-pr", "maui-pr-devicetests", and "maui-pr-uitests", "/azp run". The combination of repository, exact pipeline names, and Azure DevOps command gives this a clear niche with distinct triggers and minimal conflict risk against generic test-analysis skills. Not 4: no meaningful overlap risk remains — nothing this specific could plausibly route another skill.

5 / 5

Total

17

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

relative_links

Relative link issues: 1 suspicious

Warning

Total

15

/

16

Passed

Repository

dotnet/maui

Path

.github/skills/review-test-failures/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.