CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

qa-orchestrator

Orchestrate QA agent workflows — spawn test agents in parallel, collect results, triage bugs, trigger the bug fixer, and generate QA reports. The main entry point for running a QA session. Trigger on "run QA", "start QA session", "test the PR", "orchestrate QA agents", or when the user wants to run multiple QA agents together. Also triggered by /qa-orchestrator.

Quality

92%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

SKILL.md
Quality
Evals
Security

Quality

Content

85%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is a well-structured, highly actionable orchestration playbook with explicit sequencing, validation checkpoints, and feedback loops, and it avoids teaching Claude things it already knows. Its main defects are broken bundle references — the `rules/orch-*.md` files cited as CRITICAL/HIGH standards are absent, leaving the spawn hierarchy details and the entire report format undefined — and some recap duplication in the Workflow and Examples sections.

Suggestions

Add the missing `rules/orch-spawn.md`, `rules/orch-triage.md`, and `rules/orch-report.md` files to the bundle, or inline their essential content (especially the report format from orch-report.md, which Phase 6 depends on but never defines).

Trim the recap in the "Workflow" section and the "Examples" section, which restate the Mode Detection table and CI vs Interactive rules already covered above.

Make the CI-mode GitHub-override rule consistent: the Issue Tracker Detection table and the closing paragraph partially duplicate the Phase 5 non-interactive rules — consolidate into one place.

DimensionReasoningScore

Conciseness

The body is dense and efficient — tables, phase lists, exact prompts, and specific tool names with almost no explanation of concepts Claude already knows. However, the "Workflow" section (lines 240-245) and much of the "Examples" section recap rules already stated in "Mode Detection", "CI vs Interactive", and the phase descriptions, so it could be tightened; this fits "efficient; minor instances of over-explanation that could be trimmed" rather than the 3 anchor's "some unnecessary explanation".

4 / 5

Actionability

Guidance is fully executable for an instruction-only skill: exact file paths (`.qa/config.yml`, `.qa/reports/YYYY-MM-DD-HHmmss-qa-report.md`), exact MCP tool names (`mcp__github__get_pull_request`, `mcp__linear__get_issue`), copy-ready prompt templates for every user interaction, concrete agent-selection conditions, and a prioritized spawn strategy. This matches the 5 anchor's "copy-paste ready... specific examples cover the common cases"; the 4 anchor would imply missing key details.

5 / 5

Workflow Clarity

Mode A's six phases are explicitly sequenced ("Follow these phases IN SEQUENCE") with validation checkpoints and feedback loops: stop if the test plan is empty, warn on missing env vars, detect STOPPED_EARLY blockers, and after bug fixes "re-run ONLY failing scenarios to verify" with an explicit "If still broken: ask user for another iteration" recovery path. This matches the 5 anchor's "explicit validation steps; feedback loops for error recovery"; the 4 anchor would require missing checkpoints.

5 / 5

Progressive Disclosure

Structure is good — the body stays an overview, templates are clearly signaled one level deep in `references/` (config.md, test-plan.md, env-qa.md all exist) — but three files referenced as load-bearing (`rules/orch-spawn.md` marked CRITICAL, `rules/orch-triage.md`, `rules/orch-report.md` which defines the report format used in Phase 6) do not exist anywhere in the bundle, so the report format is never actually defined. This falls between the 3 anchor (structure present but organization issues) and the 4 anchor ("minor organization gaps") — missing referenced files are more than a minor gap, so 3.

3 / 5

Total

17

/

20

Passed

Description

100%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: it names the domain and every concrete action in the orchestration pipeline, provides explicit natural-language trigger phrases plus the slash-command name, and clearly separates itself from adjacent skills. It stays concise with no filler despite covering what and when completely.

DimensionReasoningScore

Specificity

The description lists multiple specific concrete actions covering the full pipeline: "spawn test agents in parallel, collect results, triage bugs, trigger the bug fixer, and generate QA reports". This matches the comprehensive-coverage anchor; it is not the level below (4), which would require minor gaps in the action list.

5 / 5

Completeness

It explicitly answers both questions: what it does ("spawn test agents in parallel, collect results, triage bugs, trigger the bug fixer, and generate QA reports") and when to use it ("Trigger on 'run QA', 'start QA session', 'test the PR'..."). This matches the 5 anchor with concrete trigger phrases; the 4 anchor would require the 'when' to be less explicit.

5 / 5

Trigger Term Quality

Natural trigger phrases users would actually say are comprehensively covered: "run QA", "start QA session", "test the PR", "orchestrate QA agents", "run multiple QA agents together", plus the explicit "/qa-orchestrator" command name as a synonym. Nothing users commonly say for this need is missing, so it fits the comprehensive anchor rather than the 4 anchor ("a few natural terms missing").

5 / 5

Distinctiveness Conflict Risk

It occupies a clear niche (QA agent orchestration as "the main entry point for running a QA session") with distinct trigger phrases that would not naturally invoke a generic testing or bug-fixing skill. The QA-session framing keeps conflict risk minimal, fitting the 5 rather than the 4 anchor ("minor overlap risk with closely related skills").

5 / 5

Total

20

/

20

Passed

Validation

87%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

metadata_field

'metadata' should map string keys to string values

Warning

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

14

/

16

Passed

Repository

ravnhq/ai-toolkit

Path

.claude-plugin/plugins/qa-orchestrator/skills/qa-orchestrator/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.