fix-ci-tests
Diagnose and fix CI failures on a GitHub PR by analyzing failing checks, reading logs, and applying fixes
77
72%
Does it follow best practices?
Impact
Pending
No eval scenarios have been run
Advisory
Suggest reviewing before use
Optimize this skill with Tessl
npx tessl skill review --optimize ./.claude/skills/fix-ci-tests/SKILL.mdQuality
Discovery
67%Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.
The description is strong in specificity and distinctiveness, clearly identifying a niche around CI failure diagnosis on GitHub PRs with concrete actions. Its main weakness is the lack of an explicit 'Use when...' clause and missing some natural trigger term variations that users might employ when requesting help with broken builds or failing pipelines.
Suggestions
Add an explicit 'Use when...' clause, e.g., 'Use when a GitHub PR has failing checks, the CI pipeline is broken, or the user mentions build failures or test errors.'
Include additional natural trigger terms like 'pipeline failed', 'build broken', 'tests failing', 'GitHub Actions', 'CI/CD', and 'red checks' to improve keyword coverage.
| Dimension | Reasoning | Score |
|---|---|---|
Specificity | Lists multiple specific concrete actions: 'diagnose and fix CI failures', 'analyzing failing checks', 'reading logs', and 'applying fixes'. These are clear, actionable capabilities. | 3 / 3 |
Completeness | Clearly answers 'what does this do' (diagnose and fix CI failures by analyzing checks, reading logs, applying fixes) but lacks an explicit 'Use when...' clause specifying when Claude should select this skill. Per rubric guidelines, missing 'Use when' caps completeness at 2. | 2 / 3 |
Trigger Term Quality | Includes good terms like 'CI failures', 'GitHub PR', 'failing checks', and 'logs', but misses common user variations like 'pipeline failed', 'build broken', 'tests failing', 'CI/CD', 'GitHub Actions', or 'red checks'. | 2 / 3 |
Distinctiveness Conflict Risk | The combination of 'CI failures', 'GitHub PR', 'failing checks', and 'logs' creates a clear niche that is unlikely to conflict with other skills. This is a well-defined, specific domain. | 3 / 3 |
Total | 10 / 12 Passed |
Implementation
77%Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
This is a strong, highly actionable skill with excellent workflow clarity and concrete executable commands for every step. Its main weakness is that it's quite long and monolithic—the detailed GraphQL examples, CI job tables, and per-failure-category guidance could benefit from being split into referenced files. The security callout is appropriate but slightly verbose.
Suggestions
Extract the detailed per-failure-category fix instructions (fuzz, race, bash comparison, platform-specific) into a separate FAILURE_TYPES.md reference file to reduce the main skill's length.
Move the GraphQL queries for resolving review threads into a separate REVIEW_COMMENTS.md reference, keeping only a brief mention and link in the main workflow.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The skill is fairly long (~250 lines) and includes some content that could be tightened—e.g., the detailed CI job table is repo-specific and useful, but the extensive GraphQL examples for resolving comments and the repeated security warnings add bulk. However, most content is genuinely instructive and not explaining things Claude already knows. | 2 / 3 |
Actionability | The skill provides fully executable bash and Go commands throughout, with specific flags, environment variables, and concrete examples for every failure category. Commands are copy-paste ready with clear placeholders, and the fuzz failure section even shows how to create corpus files. | 3 / 3 |
Workflow Clarity | The 10-step workflow is clearly sequenced with explicit validation checkpoints (step 4: reproduce locally, step 7: verify all fixes with specific commands, feedback loop 'if new failures appear, repeat from step 4'). The failure classification table maps categories to specific actions, and destructive operations like committing include explicit guardrails (never use git add -A). | 3 / 3 |
Progressive Disclosure | The content is entirely monolithic—all detail is inline in a single file with no references to supporting documents. The failure categories, GraphQL queries, and platform-specific guidance could be split into separate reference files. However, the internal structure with clear headers and tables provides reasonable navigability within the single file. | 2 / 3 |
Total | 10 / 12 Passed |
Validation
90%Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.
Validation — 10 / 11 Passed
Validation for skill structure
| Criteria | Description | Result |
|---|---|---|
frontmatter_unknown_keys | Unknown frontmatter key(s) found; consider removing or moving to metadata | Warning |
Total | 10 / 11 Passed | |
- Repository
- DataDog/rshell
- Commit
729dfbb
- Reviewed
Table of Contents
Is this your skill?
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.