Name: e2e-failure-analyzer
Rating: 58.4 (1 reviews)
Author: posit-dev

e2e-failure-analyzer

Analyze e2e test failures from a GitHub Actions run. Provide a run ID or URL to download reports, extract traces/screenshots/logs, identify root causes, and get suggested actions. Works with both posit-dev/positron and posit-dev/positron-builds repos.

Quality

67%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./.claude/skills/e2e-failure-analyzer/SKILL.md

Quality

Content

67%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The skill body is highly actionable and well-structured with real bundle scripts and one-level-deep references, but it over-explains diagnostic reasoning inline that belongs in the referenced rubric, hurting token efficiency.

Suggestions

Move the multi-paragraph DOM-presence/console-digest explanation and screenshot-comparison rationale into rubric.md, leaving a one-line pointer in the body to cut tokens.

Add an explicit verify-before-delete checkpoint to the Cleanup section (confirm output dir contents before the rm -rf) since it is a destructive batch operation.

Ensure rubric.md (referenced repeatedly as the source of truth) exists in the bundle, or inline the one decision rule the body depends on.

Dimension	Reasoning	Score
Conciseness	The body is ~220 lines with several verbose explanatory passages (the DOM-presence/console-digest section, screenshot-reading paragraphs, and history-line examples) that re-teach reasoning Claude could infer from the referenced rubric.md, indicating noticeable padding.	2 / 5
Actionability	Every step gives copy-paste-ready node commands with explicit flags, placeholders, and expected output JSON fields, covering both Path A and Path B plus history queries and cleanup.	5 / 5
Workflow Clarity	Steps are clearly sequenced (Step 1, Path A, Path B, Step 6, Step 7, Cleanup) with fallbacks and an external rubric as decision source of truth, but validation checkpoints are implicit (the destructive rm -rf cleanup has no verify-before-delete step), capping it below 5.	4 / 5
Progressive Disclosure	Structure is good: an overview body points one level deep to scripts/README.md, rubric.md, and named scripts that are real files, with most bulk detail offloaded; however rubric.md is referenced heavily but not present in the bundle, a minor navigation gap.	4 / 5
	Total	15 / 20 Passed

Description

67%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific, third-person, and names concrete actions plus a clear input trigger (run ID/URL). Its main gap is the absence of an explicit 'Use when ...' clause and fuller natural trigger phrasing, which cap completeness and trigger-term quality below 5.

Suggestions

Add an explicit 'Use when ...' trigger clause naming the natural situations (e.g. 'Use when a CI run has failed and you need to triage why e2e tests broke').

Surface a couple of natural trigger phrases in the description itself (e.g. 'e2e test failure', 'flaky test', 'CI red') so it matches user wording.

Dimension	Reasoning	Score
Specificity	It names several concrete actions ('download reports, extract traces/screenshots/logs, identify root causes, and get suggested actions') plus the two-repo scope, which is broad coverage with only minor gaps in how extraction works.	4 / 5
Completeness	It states clearly what the skill does and implies when via 'Provide a run ID or URL', but lacks an explicit 'Use when ...' trigger clause, which is why it is not a 5.	4 / 5
Trigger Term Quality	Relevant terms like 'e2e test failures', 'run ID', 'URL', and 'GitHub Actions' appear, but natural user phrasing is sparse and the named workflows ('Test: Merge to branch' etc.) live in the body rather than here.	3 / 5
Distinctiveness Conflict Risk	The narrow e2e-failure-analysis niche and explicit two-repo scope make it mostly distinct, though it sits adjacent to the sibling debug-e2e-test skill mentioned in the body, leaving minor overlap risk.	4 / 5
	Total	15 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
relative_links	Relative link issues: 5 missing	Warning

	Total	15 / 16 Passed

Repository: posit-dev/positron
Path: .claude/skills/e2e-failure-analyzer/SKILL.md
Commit: 061594d

Reviewed: ~11 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.