coderabbit-incident-runbook
Execute CodeRabbit incident response procedures when reviews stop working or block PRs. Use when CodeRabbit is down, reviews are not posting, PRs are blocked by stale checks, or CodeRabbit is producing incorrect reviews. Trigger with phrases like "coderabbit incident", "coderabbit outage", "coderabbit down", "coderabbit broken", "coderabbit emergency", "coderabbit not reviewing".
85
83%
Does it follow best practices?
Impact
Pending
No eval scenarios have been run
Advisory
Suggest reviewing before use
Quality
Discovery
89%Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.
This is a strong skill description with excellent trigger term coverage and completeness. It clearly identifies when to use the skill and provides multiple natural language triggers. The main weakness is that the 'what' portion could be more specific about the concrete actions/procedures it performs rather than just saying 'execute incident response procedures'.
Suggestions
Add specific concrete actions the skill performs, e.g., 'bypass stale CodeRabbit checks, disable required review gates, configure fallback review workflows, restart CodeRabbit integration' to improve specificity.
| Dimension | Reasoning | Score |
|---|---|---|
Specificity | It names the domain (CodeRabbit incident response) and some actions ('execute incident response procedures'), but doesn't list specific concrete actions like 'bypass stale checks', 'disable required review gates', or 'restart CodeRabbit integration'. The actual procedures are vague. | 2 / 3 |
Completeness | Clearly answers both 'what' (execute CodeRabbit incident response procedures when reviews stop working or block PRs) and 'when' (explicit 'Use when...' clause with multiple scenarios, plus a 'Trigger with phrases like...' section). Both dimensions are well-covered. | 3 / 3 |
Trigger Term Quality | Excellent coverage of natural trigger terms including 'coderabbit incident', 'coderabbit outage', 'coderabbit down', 'coderabbit broken', 'coderabbit not reviewing', plus contextual phrases like 'PRs are blocked by stale checks' and 'reviews are not posting'. These are phrases users would naturally say. | 3 / 3 |
Distinctiveness Conflict Risk | Highly distinctive — CodeRabbit incident response is a very specific niche. The trigger terms are all CodeRabbit-specific and unlikely to conflict with other skills. The combination of 'CodeRabbit' + incident/outage/down terminology creates a clear, unique selection signal. | 3 / 3 |
Total | 11 / 12 Passed |
Implementation
77%Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
This is a solid incident response runbook with excellent actionability — real CLI commands, specific API calls, and clear severity-based workflows. The workflow is well-sequenced with proper recovery and verification steps. Main weaknesses are moderate verbosity (echo statements, communication templates) and the monolithic structure that could benefit from splitting detailed per-severity procedures into separate files.
Suggestions
Trim echo-heavy diagnostic scripts to focus on the actual commands rather than printing instructions — Claude can interpret the commands directly without verbose echo wrappers.
Consider splitting per-severity procedures (P1-P4) into separate referenced files to improve progressive disclosure and reduce the main file length.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The skill is reasonably efficient but includes some unnecessary verbosity — the communication template, the markdown-commented delay expectations, and some echo-heavy diagnostic scripts could be tightened. The severity table and overview are useful but slightly padded. | 2 / 3 |
Actionability | Provides concrete, executable bash scripts with real gh CLI commands, specific API endpoints, and copy-paste ready code for each severity level. The config check with Python parsing and the branch protection restore commands are fully actionable. | 3 / 3 |
Workflow Clarity | Clear severity-based triage sequence with explicit decision points (Step 1 triage → severity-specific response → communication → post-incident recovery). Includes validation checkpoints like 'Wait 5 minutes. If no response, use Option A or B' and post-incident verification steps. | 3 / 3 |
Progressive Disclosure | Content is well-structured with clear sections per severity level, but it's a long monolithic file (~180 lines of content) that could benefit from splitting detailed procedures into separate files. The single 'Next Steps' reference to coderabbit-data-handling is minimal navigation. | 2 / 3 |
Total | 10 / 12 Passed |
Validation
81%Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.
Validation — 9 / 11 Passed
Validation for skill structure
| Criteria | Description | Result |
|---|---|---|
allowed_tools_field | 'allowed-tools' contains unusual tool name(s) | Warning |
frontmatter_unknown_keys | Unknown frontmatter key(s) found; consider removing or moving to metadata | Warning |
Total | 9 / 11 Passed | |
- Commit
4dee593
- Reviewed
Table of Contents
Is this your skill?
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.