fix-ci-tests

Diagnose and fix CI failures on a GitHub PR by analyzing failing checks, reading logs, and applying fixes

Quality

72%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

—

The risk profile of this skill

Fix and improve this skill with Tessl

tessl review fix ./.claude/skills/fix-ci-tests/SKILL.md

Quality

Content

77%Scale 1-3

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

This is a strong, highly actionable skill with excellent workflow clarity and concrete executable commands throughout. The security warning about treating CI logs as untrusted data is a thoughtful addition. The main weakness is that the document is quite long and could benefit from splitting detailed subsections (like GraphQL comment resolution or fuzz fix workflows) into separate referenced files to improve progressive disclosure and reduce token cost.

Suggestions

Consider extracting the detailed GraphQL queries for resolving review comments into a separate reference file to reduce the main skill's token footprint.

The CI job mapping table and failure category table could be moved to a referenced REFERENCE.md file, keeping only a brief summary inline.

Dimension	Reasoning	Score
Conciseness	The skill is fairly long but most content is project-specific (CI job mappings, failure categories, fuzz workflow, GraphQL queries for resolving comments) which Claude wouldn't know. However, some sections like the general descriptions of failure categories and the security warning are somewhat verbose, and the step-by-step for resolving review comments via GraphQL is very detailed but justified given the complexity. Overall mostly efficient with some room to tighten.	2 / 3
Actionability	Excellent actionability throughout — every step has concrete, executable bash commands and go test invocations. The failure categories map to specific actions with real commands. The GraphQL queries for resolving review threads are copy-paste ready. Fuzz failure handling includes exact file paths and reproduction commands.	3 / 3
Workflow Clarity	The 10-step workflow is clearly sequenced with explicit validation checkpoints (step 4: reproduce locally, step 7: verify all fixes with specific test commands, feedback loop 'if new failures appear, repeat from step 4'). Race condition fixes include '-count=5' verification. The workflow handles error recovery and has clear decision points (e.g., if no PR found, stop; if all checks pass, stop).	3 / 3
Progressive Disclosure	The skill is a monolithic document with no references to external files (except mentioning the 'fix-tests' skill for bash comparison failures). At ~200+ lines, some content like the detailed GraphQL queries, the CI job mapping table, or the fuzz fix workflow could be split into referenced files. However, no bundle files are provided, so everything must be inline, which partially justifies the structure.	2 / 3
	Total	10 / 12 Passed

Description

67%Scale 1-3

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is strong in specificity and distinctiveness, clearly identifying a well-scoped task around CI failure diagnosis on GitHub PRs. Its main weakness is the lack of an explicit 'Use when...' clause, which would help Claude know exactly when to select this skill. Adding more natural trigger term variations would also improve discoverability.

Suggestions

Add a 'Use when...' clause, e.g., 'Use when CI checks fail on a pull request, the build is broken, tests are red, or the user mentions pipeline/workflow failures.'

Include additional trigger terms users might naturally say: 'build broken', 'tests failing', 'pipeline failed', 'GitHub Actions', 'CI/CD', 'red checks', 'workflow errors'.

Dimension	Reasoning	Score
Specificity	Lists multiple specific concrete actions: 'diagnose and fix CI failures', 'analyzing failing checks', 'reading logs', and 'applying fixes'. These are clear, actionable capabilities.	3 / 3
Completeness	Clearly answers 'what does this do' (diagnose and fix CI failures by analyzing checks, reading logs, applying fixes) but lacks an explicit 'Use when...' clause specifying when Claude should select this skill. Per rubric guidelines, missing 'Use when' caps completeness at 2.	2 / 3
Trigger Term Quality	Includes good terms like 'CI failures', 'GitHub PR', 'failing checks', and 'logs', but misses common user variations like 'pipeline failed', 'build broken', 'tests failing', 'CI/CD', 'GitHub Actions', or 'red checks'.	2 / 3
Distinctiveness Conflict Risk	The combination of 'CI failures', 'GitHub PR', 'failing checks', and 'logs' creates a clear, distinct niche. This is unlikely to conflict with general coding skills or other GitHub-related skills.	3 / 3
	Total	10 / 12 Passed

Validation

90%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 10 / 11 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	10 / 11 Passed

Repository: DataDog/rshell
Path: .claude/skills/fix-ci-tests/SKILL.md
Commit: 7518a70

Reviewed: 6 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.