flaky-test-detector

Identifies non-deterministic or unreliable tests through static code analysis and test result analysis. Use when Claude needs to find flaky tests, analyze test reliability, or investigate intermittent test failures. Supports Python (pytest, unittest) and Java (JUnit, TestNG) test frameworks. Trigger when users mention "flaky tests", "intermittent failures", "non-deterministic tests", "unreliable tests", or ask to "find flaky tests", "analyze test stability", or "why tests fail randomly".

1.04x

Quality

92%

Does it follow best practices?

Impact

97%

1.04x

Average score across 3 eval scenarios

Securityby

Passed

No known issues

Evaluation results

100%

CI/CD Flakiness Investigation

Test result analysis workflow

Criteria

Without context

With context

Script invocation

80%

100%

analysis_output.txt created

100%

Report summary section

100%

High priority section

100%

Category grouping

100%

Flakiness scores reported

70%

100%

Pass/fail pattern reported

100%

Pass rate reported

87%

100%

Alternating pattern noted

100%

Duration variance noted

100%

Specific test details

100%

Test Suite Reliability Review

Python static analysis and remediation

Criteria

Without context

With context

Report structure per issue

100%

Line numbers included

100%

Issues grouped by category

100%

Shared state identified

100%

Fixed: setUp/fixture used

87%

100%

Fixed: tmp_path or tempfile

100%

Fixed: time mock

100%

Fixed: network mock

100%

Fixed: sleep removed

100%

Fixed: random seeded

100%

Confidence or priority noted

100%

File cleanup issue noted

100%

92%

Java Test Suite Stabilization

Java test flakiness detection and fix

Criteria

Without context

With context

Report structure per issue

100%

Line numbers in report

100%

Issues grouped by category

100%

Fixed: @Before setup

100%

Fixed: @After teardown

50%

100%

Fixed: Thread.sleep removed

100%

Fixed: Clock/time mock

100%

Fixed: resource management

100%

Fixed: race condition

100%

Fixed: network mock

100%

Confidence or severity noted

100%

Shared static state identified

100%

Repository: ArabelaTso/Skills-4-SE
Commit: 0f00a4f

Evaluated: 29 days ago
Agent: Claude Code
Model: Claude Sonnet 4.6

Table of Contents

CI/CD Flakiness Investigation Test Suite Reliability Review Java Test Suite Stabilization

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.