deflake

Finds flaky tests on the main branch by analyzing GitHub Actions failures, ranks them by frequency, and enters parallel plan mode to design deflake strategies. Use when you want to find and fix the flakiest tests.

Quality

75%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./.claude/skills/deflake/SKILL.md

Quality

Content

65%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable multi-phase skill that offloads collection to a script and gives explicit approval gates. Its main gaps are a missing post-fix verification loop and the absence of bundle reference files to confirm progressive disclosure.

Suggestions

Add a final verification step in Phase 3.3 — e.g., run the fixed test N times or re-run the collection script — to confirm the flake is gone before declaring success.

Ship the referenced collect-flakes.py in the scripts/ bundle (or a reference doc for the report/plan templates) so the offloaded logic and external references are real and discoverable.

Tighten the six-step parallel-agent investigation block and the Deflake Principles bullets to reduce verbosity while preserving the actionable discrimination guidance.

Dimension	Reasoning	Score
Conciseness	Mostly efficient and free of basic-concept padding, but the six-step agent investigation block and the multi-bullet Deflake Principles section are thorough rather than terse and could be tightened without losing clarity.	2 / 3
Actionability	Provides an executable command ("python3 .claude/skills/deflake/collect-flakes.py"), concrete subagent_type and investigation steps, and copy-ready report and plan templates — fully executable and specific.	3 / 3
Workflow Clarity	Phases are clearly sequenced with explicit gates ("Wait for all agents to complete", "Do NOT ... until the user approves"), but the workflow caps at 2 because it lacks a verification step confirming a fix actually deflaked a test after implementation — a validation gap for a batch/destructive code-changing operation.	2 / 3
Progressive Disclosure	Sections are well-organized and data collection is offloaded to a script, but no reference docs split out deeper material and the referenced collect-flakes.py is not present in the bundle to verify, so structure is present but not clearly signaled across files.	2 / 3
	Total	9 / 12 Passed

Description

85%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, specific description with a clear what-and-when structure and a distinct niche. Its only weakness is narrow trigger-term coverage, relying on one concept's variants rather than a broader set of natural phrasings.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions — "analyzing GitHub Actions failures", "ranks them by frequency", and "enters parallel plan mode to design deflake strategies" — matching the multi-action anchor rather than the single-domain level 2.	3 / 3
Completeness	Explicitly answers both what it does (find/rank/plan flaky tests via GHA failures) and when to use it ("Use when you want to find and fix the flakiest tests."), matching the level-3 anchor with an explicit trigger clause.	3 / 3
Trigger Term Quality	Contains natural terms ("flaky tests", "flakiest tests", "deflake") a user would say, but coverage is narrow — morphological variants of one concept with no common alternatives like "flakes", "flakiness", or "unstable tests", fitting the "some relevant keywords but missing common variations" anchor.	2 / 3
Distinctiveness Conflict Risk	The niche is highly specific — flaky tests on the main branch via GitHub Actions failures with parallel plan-mode deflaking — giving it distinct triggers unlikely to fire for other skills.	3 / 3
	Total	11 / 12 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: stacklok/toolhive
Path: .claude/skills/deflake/SKILL.md
Commit: cd373ec

Reviewed: 2 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.