bat-adhoc

Run bot acceptance tests to validate MCP tools work correctly from a real AI agent's perspective. Use when testing PRs, detecting regressions, or verifying tool changes end-to-end with Claude/Gemini CLIs.

1.27x

Quality

88%

Does it follow best practices?

Impact

97%

1.27x

Average score across 2 eval scenarios

Securityby

Passed

No findings from the security scan

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is well-structured and highly actionable with executable examples and a clear validation-gated workflow. Progressive disclosure is good but not ideal since substantial detail remains inline rather than being split into bundled reference files.

Suggestions

Move the full output-structure JSON schema and detailed regression-comparison metric tables into a bundled reference file (e.g. references/OUTPUT_FORMAT.md), keeping SKILL.md as a concise overview with a one-level pointer.

Tighten the Cost Awareness and Regression Comparison prose into bullet points to trim tokens without losing the actionable guidance.

Add a short 'Prerequisites' note clarifying that tests/uat/run_uat.py and tests/uat/README.md live in the target repo, so the referenced paths are unambiguous.

Dimension	Reasoning	Score
Conciseness	The body is efficient and avoids explaining concepts Claude already knows, with mostly earned tokens; minor sections (e.g., the robustness tip, cost-awareness prose) could be trimmed, placing it at the score-4 'efficient with minor over-explanation' anchor rather than the fully lean score-5.	4 / 5
Actionability	Provides copy-paste-ready commands ('python tests/uat/run_uat.py', '--agents', '--branch') and a complete executable scenario JSON example covering a common case (error signaling), matching the score-5 anchor.	5 / 5
Workflow Clarity	The 6-step workflow is clearly sequenced with explicit validation checkpoints ('Check all_passed ... If true, you're done', 'Dig deeper on failure', 'Regression check') and feedback loops for error recovery, satisfying the score-5 anchor; validation is present so the destructive/batch cap does not apply.	5 / 5
Progressive Disclosure	Good section structure with a clearly signaled one-level reference to 'tests/uat/README.md' for full docs, but a fair amount of reference-level material (output schema, comparison metrics) lives inline in SKILL.md, leaving minor organization gaps versus the score-5 'appropriately split' anchor.	4 / 5
	Total	18 / 20 Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is strong: third-person voice, explicit what-and-when structure, and a distinctive niche. Its only weakness is slightly limited keyword/synonym coverage, which keeps trigger_term_quality and specificity at 4 rather than 5.

Dimension	Reasoning	Score
Specificity	Names the domain ('bot acceptance tests', 'MCP tools') and several concrete actions (testing PRs, detecting regressions, verifying tool changes end-to-end), matching the score-4 anchor of several specific actions with minor coverage gaps rather than the comprehensive score-5 list.	4 / 5
Completeness	Explicitly states both what ('Run bot acceptance tests to validate MCP tools work correctly from a real AI agent's perspective') and when ('Use when testing PRs, detecting regressions, or verifying tool changes end-to-end with Claude/Gemini CLIs') with concrete trigger phrases, matching the score-5 anchor exactly.	5 / 5
Trigger Term Quality	Includes natural phrases users would say ('testing PRs', 'detecting regressions', 'verifying tool changes', 'Claude/Gemini CLIs') but is missing common synonyms such as 'acceptance tests' or 'MCP tool testing', placing it just below the comprehensive score-5 anchor.	4 / 5
Distinctiveness Conflict Risk	Occupies a clear niche (bot acceptance testing of MCP tools via real agent CLIs) with distinct triggers and minimal overlap with other skills, matching the score-5 anchor.	5 / 5
	Total	18 / 20 Passed

Validation

87%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

Criteria	Description	Result
allowed_tools_field	'allowed-tools' contains unusual tool name(s)	Warning
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	14 / 16 Passed

Repository: homeassistant-ai/ha-mcp
Path: .claude/skills/bat-adhoc/SKILL.md
Commit: 47582a7

Reviewed: ~23 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.