positron-qa-verify

Generates clear, actionable verification guides for QA testing of Positron bug fixes and features

Quality

70%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./.claude/skills/positron-qa-verify/SKILL.md

Quality

Content

82%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is a well-structured, highly actionable workflow with concrete executable commands and appropriately deferred references to real bundle files. Its main weakness is the absence of explicit validation/error-recovery checkpoints in the workflow and a small amount of inline content that could live in the reference file.

Suggestions

Add explicit validation checkpoints, e.g. 'If no PR is found for the issue, note this in the guide and proceed using issue content only' and 'Confirm the issue number resolves before fetching related PRs'.

Move the Step 7 comment-format template (single vs multiple scenario formatting) into references/verification_guide.md and link to it, keeping SKILL.md as a pure overview.

Clarify the error path when gh calls fail under the 'fail fast' mode (e.g. record which step failed and what empty values to substitute) so recovery is unambiguous.

Dimension	Reasoning	Score
Conciseness	The body is lean and mostly concrete commands with minimal padding, assuming Claude knows gh/GitHub basics; a few lines (e.g. the inline comment-format template in Step 7) could be trimmed or deferred.	4 / 5
Actionability	It provides copy-paste-ready, fully-specified gh commands (with --json field lists), a concrete script invocation with sample JSON output, and explicit conditional logic, covering the common cases.	5 / 5
Workflow Clarity	Steps 1-7 are clearly sequenced with parallel execution and the <500-line conditional flagged; however, explicit validation checkpoints (e.g. what to do if no PR is found) are only weakly present, so it stops short of 5.	4 / 5
Progressive Disclosure	The SKILL.md is a clean overview that defers guide format to references/verification_guide.md (which exists) and version detection to scripts/detect_versions.sh (which exists), both one level deep and clearly signaled; the inline comment-format template is a minor organization gap keeping it from 5.	4 / 5
	Total	17 / 20 Passed

Description

57%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description states a clear, specific purpose tied to a distinctive niche, but omits any 'Use when...' trigger guidance and lists only one concrete action. Adding explicit trigger phrases and a broader set of concrete actions would lift specificity, trigger_term_quality, and completeness.

Suggestions

Add a 'Use when...' clause with concrete trigger phrases (e.g. 'Use when assigned a QA verification ticket, or when you need test scenarios for a Positron bug fix or feature').

Broaden the action list to be more comprehensive (e.g. 'extracts edge cases from issue/PR discussion, drafts test scenarios, generates a verification guide and comment template').

Include natural synonyms users say (test plan, test cases, manual testing, regression checks) alongside 'QA testing' and 'verification'.

Dimension	Reasoning	Score
Specificity	Names a clear domain ('QA testing of Positron bug fixes and features') and one concrete action ('Generates ... verification guides'), but lists only a single action rather than comprehensive coverage.	3 / 5
Completeness	It clearly answers the 'what' (generates verification guides) but provides no 'Use when...' clause or equivalent explicit trigger guidance, which per the judging guidelines caps completeness at 3.	3 / 5
Trigger Term Quality	Relevant natural keywords like 'QA testing', 'bug fixes', and 'verification guides' are present, but common synonyms users might say (e.g. 'test plan', 'test cases', 'manual testing', 'regression') are missing.	3 / 5
Distinctiveness Conflict Risk	The Positron-specific QA verification niche is highly distinctive; no other skill would plausibly trigger for this combination of domain and action, so conflict risk is minimal.	5 / 5
	Total	14 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: posit-dev/positron
Path: .claude/skills/positron-qa-verify/SKILL.md
Commit: 061594d

Reviewed: 4 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.