test-flakiness

Detect non-deterministic (flaky) tests by reading CI run logs or test result history. Aggregates pass rates per test, identifies intermittent failures, recommends quarantine or fix, and maintains a flaky test registry. Best run during Polish phase or after multiple CI runs.

Quality

88%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Quality

Content

77%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

Highly actionable with a well-sequenced, validated workflow, but it carries minor conceptual padding and keeps all detail inline rather than splitting reference material into bundle files.

Suggestions

Trim the opening paragraph explaining why flaky tests are harmful — Claude already knows what flaky tests are; lead with the detection procedure instead.

Extract the engine-specific parsing details (Godot/Unity/Unreal/JUnit) and the cause-classification table into a references/ file, keeping SKILL.md as an overview with one-level-deep links.

Move the placeholder in-conversation report template into a reference file to reduce inline token weight while preserving the format for reuse.

Dimension	Reasoning	Score
Conciseness	Mostly efficient and actionable, but the opening framing ('Flaky tests are worse than no tests in some ways — they train the team to ignore red CI runs') explains a concept Claude already knows, and the placeholder report template adds inline tokens that could be trimmed.	2 / 3
Actionability	Provides concrete, executable guidance: real shell/grep commands, engine-specific pass/fail patterns, exact skip annotations (@pytest.mark.skip, [Ignore], GdUnitSkip), and named fix helpers (is_equal_approx, await get_tree().process_frame).	3 / 3
Workflow Clarity	A clear seven-step sequence with explicit checkpoints — the no-log-data branch stops and asks, sub-3-run results are flagged 'suspected', and writes require explicit approval with COMPLETE/BLOCKED verdicts.	3 / 3
Progressive Disclosure	Well-organized into numbered sections but monolithic at ~200 lines with no bundle references; engine-specific parsing detail and the cause-classification table are inline content that could live in separate files.	2 / 3
	Total	10 / 12 Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A tight, third-person description that covers capabilities, natural trigger terms, and an explicit run-condition cue. It is concise without padding and clearly distinguishes the skill's niche.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions — 'reading CI run logs or test result history', 'Aggregates pass rates per test', 'identifies intermittent failures', 'recommends quarantine or fix', 'maintains a flaky test registry' — matching the highest anchor.	3 / 3
Completeness	Answers both what (detect/aggregate/recommend/maintain) and when with explicit trigger guidance ('Best run during Polish phase or after multiple CI runs'), the equivalent of a 'Use when...' clause.	3 / 3
Trigger Term Quality	Contains natural user-utterable terms like 'flaky tests', 'CI run logs', 'intermittent failures', and 'quarantine' that a user would actually say when needing this skill.	3 / 3
Distinctiveness Conflict Risk	Occupies a clear niche (non-deterministic test detection with quarantine workflow) unlikely to trigger for unrelated skills.	3 / 3
	Total	12 / 12 Passed

Validation

87%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

Criteria	Description	Result
allowed_tools_field	'allowed-tools' contains unusual tool name(s)	Warning
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	14 / 16 Passed

Repository: Donchitos/Claude-Code-Game-Studios
Path: .claude/skills/test-flakiness/SKILL.md
Commit: 984023d

Reviewed: 3 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.