fix-ci-tests

Diagnose and fix CI failures on a GitHub PR by analyzing failing checks, reading logs, and applying fixes

Quality

77%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./.claude/skills/fix-ci-tests/SKILL.md

Quality

Content

85%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The skill body is highly actionable and well-sequenced with strong validation checkpoints and feedback loops, making it reliably executable. Its main weakness is that everything lives in one large SKILL.md with no reference files, leaving progressive disclosure underdeveloped.

Suggestions

Extract the CI job table (section 3) and/or the GraphQL review-thread resolution block (section 9) into a references file (e.g. references/ci-jobs.md, references/resolve-comments.md) and link to them from the body.

Tighten the security preamble — keep the prompt-injection warning but trim the repeated restatement of the <external-data> framing.

Consider moving the detailed fuzz-failure sub-workflow into its own reference doc and linking it, since it is a self-contained special case.

Dimension	Reasoning	Score
Conciseness	The body is dense with executable commands and tables and largely avoids explaining concepts Claude already knows, but the security preamble and some prose around fix categories are slightly over-explained, fitting 'efficient; minor instances of over-explanation that could be trimmed' rather than the fully lean 5.	4 / 5
Actionability	It provides copy-paste-ready gh and go test commands, exact fuzz corpus file paths, and complete GraphQL resolve-thread queries covering the common cases, matching 'fully executable; copy-paste ready code or commands'.	5 / 5
Workflow Clarity	Ten numbered steps with explicit reproduction-before-fix validation, a 'Verify all fixes' checkpoint, and a feedback loop ('If new failures appear, repeat from step 4') match 'clear sequence with explicit validation steps; feedback loops for error recovery'.	5 / 5
Progressive Disclosure	No references/scripts/assets bundle exists, so the ~280-line body is monolithic; the CI job table, GraphQL thread-resolution queries, and fuzz workflow could be split into separate reference files, fitting 'some structure but content that should be separate is inline' rather than the simple-skill exception.	3 / 5
	Total	17 / 20 Passed

Description

70%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific and well-targeted to a distinct CI-fixing niche with natural trigger terms, but it lacks an explicit 'Use when' clause, which caps its completeness. Adding trigger guidance would lift the weakest dimension.

Suggestions

Append an explicit 'Use when...' clause, e.g. 'Use when CI checks are failing on a GitHub PR, when the user asks to fix CI/build failures, or when a PR has red checks.'

Add a couple of common synonyms ('build broken', 'tests failing', 'red CI') to broaden trigger-term coverage.

Optionally enumerate the fix categories briefly (test, race, build, fuzz) to push specificity toward comprehensive coverage.

Dimension	Reasoning	Score
Specificity	Names the domain (CI failures on a GitHub PR) and lists several concrete actions — 'analyzing failing checks, reading logs, and applying fixes' — with only minor gaps in coverage, fitting the 'several specific actions' anchor rather than the comprehensive 5.	4 / 5
Completeness	It clearly answers 'what' (diagnose and fix CI failures) but provides no explicit 'Use when...' trigger clause, and per the judging guidelines a missing explicit trigger caps completeness at 3 even though 'when' is weakly implied.	3 / 5
Trigger Term Quality	Natural terms a user would say ('CI failures', 'GitHub PR', 'failing checks', 'logs') are present and relevant, though common synonyms like 'build broken' or 'red CI' are missing, matching 'good keyword coverage; a few natural terms missing'.	4 / 5
Distinctiveness Conflict Risk	'CI failures on a GitHub PR' is a clear niche with distinct triggers and minimal overlap with other skills, matching the 'clear niche with distinct triggers; minimal conflict risk' anchor.	5 / 5
	Total	16 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: DataDog/rshell
Path: .claude/skills/fix-ci-tests/SKILL.md
Commit: 30a7e9e

Reviewed: 2 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.