sharaf/codebase-test-suite-audit
Use when the user wants a test suite audit, test quality or reliability review, regression-protection review, unit/integration/e2e test review, coverage or CI signal assessment, flaky CI investigation, fixture-realism review, spec-drift review, or generated-test validation for AI/LLM/agent-written code. Produces severity-ranked findings for weak assertions, oracle gaps, brittle fixtures, over-mocking, CI trust, and generated-code test risks.
100
100%
Does it follow best practices?
Impact
100%
1.31xAverage score across 3 eval scenarios
Passed
No known issues
Quality
Discovery
100%Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.
This is a strong description that clearly defines its niche in test quality auditing and review. It excels at providing explicit trigger scenarios via the 'Use when' clause and lists concrete, specific outputs. The inclusion of AI/LLM-generated test validation adds a distinctive angle that further reduces conflict risk with generic testing skills.
| Dimension | Reasoning | Score |
|---|---|---|
Specificity | Lists multiple specific concrete actions: test suite audit, test quality review, regression-protection review, coverage assessment, flaky CI investigation, fixture-realism review, spec-drift review, and generated-test validation. Also specifies concrete outputs: severity-ranked findings for weak assertions, oracle gaps, brittle fixtures, over-mocking, CI trust, and generated-code test risks. | 3 / 3 |
Completeness | Explicitly answers both 'what' (produces severity-ranked findings for weak assertions, oracle gaps, brittle fixtures, etc.) and 'when' (opens with 'Use when the user wants...' followed by a comprehensive list of trigger scenarios). The 'Use when' clause is explicit and detailed. | 3 / 3 |
Trigger Term Quality | Excellent coverage of natural terms users would say: 'test suite audit', 'test quality', 'flaky CI', 'coverage', 'unit/integration/e2e test review', 'over-mocking', 'brittle fixtures', 'AI/LLM/agent-written code'. These are terms developers naturally use when discussing test quality concerns. | 3 / 3 |
Distinctiveness Conflict Risk | Highly distinctive niche focused specifically on test quality auditing and review, with particular emphasis on AI/LLM-generated test validation. The combination of test quality assessment, CI signal analysis, and generated-code test risks creates a clear, unique identity unlikely to conflict with general code review or testing skills. | 3 / 3 |
Total | 12 / 12 Passed |
Implementation
100%Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
This is a high-quality skill that efficiently communicates a complex multi-step audit process. Its strengths include executable discovery commands, a rigorous finding contract with classification shortcuts that prevent common overclaiming errors, and well-structured progressive disclosure to reference files. The content is appropriately dense without being verbose, and every section serves a clear purpose in guiding Claude through a test suite audit.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The content is lean and efficient. It assumes Claude's competence with testing concepts, avoids explaining what test suites or CI are, and every section earns its place—the classification shortcuts table, finding contract, and process steps all add unique, non-obvious value. | 3 / 3 |
Actionability | Provides executable bash commands for initial discovery, a concrete finding contract template with required fields, a classification shortcuts table mapping specific signals to severity levels, and a clear 6-step process. The guidance is specific and copy-paste ready. | 3 / 3 |
Workflow Clarity | The 6-step process is clearly sequenced with an explicit validation checkpoint (step 5: cross-check severities against the finding contract and fix overclaims before reporting). The classification shortcuts table and finding contract serve as built-in guardrails against common audit errors. | 3 / 3 |
Progressive Disclosure | The SKILL.md provides a complete overview with well-signaled one-level-deep references to four supporting files (report-template.md, evidence-inventory.md, audit-domains.md, guardrails-and-success.md), clearly indicating which are required vs optional and when to load them. | 3 / 3 |
Total | 12 / 12 Passed |
Validation
100%Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.
Validation — 11 / 11 Passed
Validation for skill structure
No warnings or errors.
Reviewed
Table of Contents