CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

triage-e2e-failure

Diagnose a failing or recently-failed e2e smoke test in this repo by walking through the forensics bundle (summary.json + prompt.txt + response.txt + snapshot.txt + screenshot.png) following the procedure in e2e/runbook.md. Accepts a GitHub Actions run URL, a local artifact zip, or an unpacked bundle directory. Identifies whether the failure is from app version drift, selector drift, an auth challenge, a Cloudflare interstitial, an assertion mismatch, or something else, and points at the likely fix. Use this skill whenever the user mentions an e2e test failure, smoke test failure, GHA failure for the claude-desktop or chatgpt-web jobs, a red CI run, a failing forensics bundle, or asks to investigate a failed run. Also use when invoked as `/triage-e2e-failure <run-url-or-path>`.

Quality

96%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

SKILL.md
Quality
Evals
Security

Quality

Content

92%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-built procedural skill: fully executable commands and error-matching heuristics, a clearly sequenced workflow with explicit stop conditions and confirmation gates, and disciplined single-file organization that defers to repo source-of-truth files instead of duplicating them. The only trimming opportunity is the provenance/justification prose at the top and bottom.

Suggestions

Trim the provenance paragraph ('The runbook explicitly says... same steps, just consistent and faster') to a single pointer to e2e/runbook.md — the quoted procedure is restated in full by the Procedure section anyway.

Shorten or remove the 'File layout' section; a one-line note that the skill ships no bundled assets would convey the same information with fewer tokens.

DimensionReasoningScore

Conciseness

The body is dense and assumes competence (exact error strings, timeout values, field-by-field decision guidance with no concept explanations), but the provenance paragraph — 'The runbook explicitly says... This skill is the codification of that procedure — same steps, just consistent and faster' — and the closing 'File layout' section restate context rather than instruct, matching the 'efficient; minor instances of over-explanation that could be trimmed' anchor. It is not level 5 because a few sentences do not earn their place, and not level 3 because there is no genuinely unnecessary explanation of things Claude already knows.

4 / 5

Actionability

Guidance is fully executable: copy-paste commands ('gh run download <runId> --dir "$TMP"', 'unzip -d "$TMP" <path>'), exact error substrings to categorize ('Response missing start delimiter "OUTPUT_BEGIN"', 'selector ... did not match'), concrete timeout thresholds (180_000 read / 240_000 write), named locator files with SELECTOR-TODO markers, and a fill-in report template. This matches the anchor-5 'fully executable; specific examples cover the common cases' — deferring version pins to the repo runbook is appropriate design for a repo-internal skill, not a gap.

5 / 5

Workflow Clarity

A clear 7-step sequence with explicit validation checkpoints and error-recovery routing: 'If the bundle is missing summary.json, stop'; 'if true, tell the user the test didn't actually fail... and stop'; error-type-to-step routing in step 2; 'Don't silently propose a fix — show the user the old locator, the new one... and let them confirm'; and a 'When you can't tell' section handling ambiguous evidence. This matches the anchor-5 pattern of explicit validation steps with feedback loops; the level 4 anchor's 'minor validation gaps' is not evident.

5 / 5

Progressive Disclosure

The bundle is intentionally a single SKILL.md (verified: no references/, scripts/, or assets/ exist, matching the documented 'File layout'), and the body explicitly justifies not duplicating repo content: 'the source of truth for what each bundle field means is e2e/forensics.ts and e2e/runbook.md in the repo itself. Read those when in doubt rather than duplicating their content here.' External pointers are one level deep and clearly signaled, sections are well-organized, and nothing that belongs in a separate file is inlined — the procedural detail is the working content itself.

5 / 5

Total

19

/

20

Passed

Description

100%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

An exemplary description: third-person voice, comprehensive concrete actions, an explicit trigger clause with natural user phrasing and synonyms, and a sharply defined niche. Length is justified by concrete content rather than padding.

DimensionReasoningScore

Specificity

Concrete actions are enumerated comprehensively: 'Diagnose a failing or recently-failed e2e smoke test... by walking through the forensics bundle (summary.json + prompt.txt + response.txt + snapshot.txt + screenshot.png)', 'Identifies whether the failure is from app version drift, selector drift, an auth challenge, a Cloudflare interstitial, an assertion mismatch, or something else, and points at the likely fix', plus three accepted input forms. This matches the anchor for multiple specific concrete actions with comprehensive coverage, not the level 4 anchor with 'minor gaps in coverage' — every major capability is named.

5 / 5

Completeness

Both 'what' and 'when' are explicitly and concretely answered: the what covers diagnosis procedure, failure taxonomy, and fix pointer; the when is an explicit 'Use this skill whenever the user mentions...' clause listing concrete trigger phrases plus the /triage-e2e-failure invocation. This is the anchor-5 pattern; the level 4 anchor ('when' could be more explicit) does not apply since the trigger list is fully explicit.

5 / 5

Trigger Term Quality

Natural phrases a user would actually say are exhaustively covered: 'e2e test failure, smoke test failure, GHA failure for the claude-desktop or chatgpt-web jobs, a red CI run, a failing forensics bundle, or asks to investigate a failed run', plus slash-command invocation. Synonyms and domain-specific variations (red CI run, failed run, smoke test) are all present, matching the comprehensive-coverage anchor rather than the level 4 anchor where 'a few natural terms missing'.

5 / 5

Distinctiveness Conflict Risk

The niche is unambiguous — forensics-bundle triage for the claude-desktop/chatgpt-web e2e jobs with named files (summary.json, snapshot.txt) and named failure modes. Triggers like 'GHA failure for the claude-desktop or chatgpt-web jobs' and 'failing forensics bundle' would not plausibly fire any other skill, matching the clear-niche/minimal-conflict anchor.

5 / 5

Total

20

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

metadata_version

'metadata.version' is missing

Warning

Total

15

/

16

Passed

Repository

boarlabsxyz/awesome-mcp

Path

.claude/skills/triage-e2e-failure/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.