deflake

Finds flaky tests on the main branch by analyzing GitHub Actions failures, ranks them by frequency, and enters parallel plan mode to design deflake strategies. Use when you want to find and fix the flakiest tests.

Quality

78%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./.claude/skills/deflake/SKILL.md

Quality

Content

75%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, actionable multi-phase workflow for finding and fixing flaky tests with concrete commands, templates, and an explicit user-approval feedback loop. The main gap is the absence of a post-fix verification step to confirm the flake is resolved.

Suggestions

Add a Phase 3.4 verification step that re-runs the affected test (or re-checks CI) after a fix lands to confirm the flake is actually resolved.

Provide a copy-paste-ready prompt template for the parallel investigation agents rather than describing the agent's tasks as a numbered list.

Tighten the six-step agent investigation list; some sub-bullets (e.g. the root-cause enumeration) could be condensed without losing clarity.

Dimension	Reasoning	Score
Conciseness	The body is mostly lean task-specific guidance (script command, report template, agent investigation steps, deflake principles) with only minor over-explanation, e.g. the six-step agent investigation list is detailed but justified for parallel subagents.	4 / 5
Actionability	Provides a copy-paste script invocation `python3 .claude/skills/deflake/collect-flakes.py`, concrete markdown templates, and explicit agent spawn instructions (subagent_type: `general-purpose`); minor gaps are the template placeholders and the agent prompt being described as steps rather than a ready-to-send prompt.	4 / 5
Workflow Clarity	Clear three-phase sequence with explicit checkpoints (categorization gate, '--report' stop condition, user-approval gate before implementation); the notable gap is no 'verify the fix actually deflaked the test' step after Phase 3.3.	4 / 5
Progressive Disclosure	Well-organized with clear section headers (Phase 1/2/3, Deflake Principles, Anti-patterns) and a single one-level-deep script reference signaled in a code block; no bundle files are present, but the inline content is appropriately structured rather than monolithic.	4 / 5
	Total	16 / 20 Passed

Description

82%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A clear, complete description with an explicit 'Use when' trigger and a distinct niche. The only issue is second-person voice ('you want to') in the trigger clause, which the rubric penalizes on specificity.

Suggestions

Rewrite the trigger in third person / gerund form to match the good examples, e.g. 'Use when finding and fixing flaky tests on the main branch.'

Add a synonym such as 'intermittent test failures' or 'CI flakiness' to broaden natural trigger coverage.

Dimension	Reasoning	Score
Specificity	Names several concrete actions ('analyzing GitHub Actions failures', 'ranks them by frequency', 'enters parallel plan mode to design deflake strategies'), which would anchor at 4, but the second-person 'you' in 'Use when you want to find and fix' triggers the -1 voice penalty per the rubric.	3 / 5
Completeness	Explicitly answers both what ('Finds flaky tests... ranks them by frequency... enters parallel plan mode to design deflake strategies') and when ('Use when you want to find and fix the flakiest tests') with a concrete trigger phrase.	5 / 5
Trigger Term Quality	Includes natural terms users would say ('flaky tests', 'flakiest tests', 'GitHub Actions failures', 'main branch'); a few common synonyms like 'intermittent test failures' or 'CI flakiness' are missing, so it does not reach comprehensive coverage.	4 / 5
Distinctiveness Conflict Risk	Occupies a clear niche (deflaking tests on the main branch via GitHub Actions) with distinct triggers that are unlikely to fire for unrelated skills.	5 / 5
	Total	17 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: stacklok/toolhive
Path: .claude/skills/deflake/SKILL.md
Commit: ccabad6

Reviewed: about 13 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.