CtrlK
BlogDocsLog inGet started
Tessl Logo

fix-random-ci-test-failure

Investigate and fix flaky/random CI test failures in dotnet/macios. Trigger on GitHub issues describing intermittent test failures, CI postmortem issues, or when asked to fix a flaky test. Analyzes test code, identifies root causes (shared state, environment dependencies, race conditions), and applies fixes.

70

Quality

85%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Low

Low-risk findings worth noting

SKILL.md
Quality
Evals
Security

Quality

Content

78%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, repo-specific flaky-test workflow with concrete commands and a diagnostic feedback loop. Main gaps are slight redundancy between the root-cause and fix sections and the absence of an explicit test-runs-green verification checkpoint before the PR.

Suggestions

Add an explicit verification checkpoint in the Create-PR step — e.g., 'Run the affected test suite locally / on CI and confirm it passes before opening the PR' — to close the validation gap in workflow_clarity.

Tighten the redundancy between section 3 (root-cause categories) and section 4 (apply the fix) by cross-referencing instead of restating symptoms and fixes in both places.

Where fix guidance is currently pattern-level (e.g., the unique-identifier patterns), add one minimal copy-paste code snippet to lift actionability toward fully executable.

DimensionReasoningScore

Conciseness

Lean and repo-specific with no generic concept explanation, but the root-cause categories in section 3 partly restate the fixes in section 4, leaving minor instances that could be trimmed.

4 / 5

Actionability

Provides concrete commands and patterns (grep command, Process.GetCurrentProcess().Id, Guid.NewGuid(), TestContext.Out.WriteLine, branch naming, Fixes #NNNN), but some guidance is pattern-level rather than literal copy-paste code, leaving minor gaps.

4 / 5

Workflow Clarity

Clear numbered six-step sequence with a diagnostic feedback loop in section 5 and a flakiness-vs-regression confirmation check, but lacks an explicit 'run the test and verify it passes' checkpoint before opening the PR.

4 / 5

Progressive Disclosure

A single-file skill under 100 lines with well-organized sections and clear navigation; in-repo path references (KeyChainTest.cs, RecordTest.cs) are appropriate inline pointers, meeting the simple-skill exception for a 5.

5 / 5

Total

17

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, specific description that clearly states both the capability and explicit trigger conditions, scoped narrowly to dotnet/macios flaky CI failures. Trigger term coverage is the only minor gap, lacking a few natural variations.

DimensionReasoningScore

Specificity

Lists multiple concrete actions — 'Investigate and fix flaky/random CI test failures', 'Analyzes test code, identifies root causes (shared state, environment dependencies, race conditions), and applies fixes' — covering the full workflow comprehensively.

5 / 5

Completeness

Explicitly answers both what (investigate/analyze/identify/fix flaky CI failures) and when ('Trigger on GitHub issues describing intermittent test failures, CI postmortem issues, or when asked to fix a flaky test') with concrete trigger phrases.

5 / 5

Trigger Term Quality

Good natural coverage with synonyms ('flaky/random', 'intermittent') plus the 'ci-postmortem' label and 'fix a flaky test', but a few common phrasings a user might say are absent; not as exhaustive as a 5-anchor list.

4 / 5

Distinctiveness Conflict Risk

Narrowly scoped to 'flaky/random CI test failures in dotnet/macios' — a clear niche with distinct triggers and minimal overlap with other skills.

5 / 5

Total

19

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository
dotnet/macios
Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.