builder-smoke-test

Smoke test the Agent Builder feature branch end-to-end against a hermetic project scaffolded by the skill (linked to the current worktree). Covers workspace reconciliation, stored agents/skills CRUD, ownership, visibility, stars, registry/library Copy flow, picker allowlists, model policy, RBAC role gating, role impersonation UI, builder defaults, infrastructure diagnostics, channels, and Studio + Agent Builder UI. Trigger when validating the agent-builder feature branch, PRs that touch packages/server, packages/playground, packages/playground-ui agent-builder routes, or builder EE code paths.

Quality

92%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Critical

Do not install without reviewing

Quality

Content

85%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable QA-orchestration skill: executable commands, explicit validation gates and feedback loops, and clean one-level-deep reference split. Its main weakness is length and redundancy — the cookie-extraction and multi-user-seeding procedures are described twice — which keeps conciseness from the top level.

Suggestions

Collapse the duplicated guidance: keep the detailed "Extracting the session cookie for curl (auth on)" and "Seeding non-owner skills" sections, and shorten the corresponding Execution-flow steps 3 and 4 to one-line pointers so the procedures appear once.

Trim the verbose repeated env-handling rationale — the `.env`-is-source-of-truth point is made in "How `mastra dev` reads env", "Auth modes", "Known rough edges", and the `.env` policy note; consolidate to one place and cross-reference.

Consider moving the large parameter table and scope-shortcut table into a `references/usage.md` (or `references/cli.md`) so the SKILL.md overview stays lean, leaving only the mandatory checklist and execution flow inline.

Dimension	Reasoning	Score
Conciseness	The body is dense and largely operational (error-code tables, env handling, design decisions) rather than padded with concepts Claude knows, but it is long and repeats material — the session-cookie and non-owner-seeding procedures are summarized in "Execution flow" and then re-stated verbatim in dedicated sections — so it could be tightened.	2 / 3
Actionability	Provides fully executable bash (exact script paths like `bash .claude/skills/builder-smoke-test/scripts/preflight.sh --expect off`, `curl -H "Cookie: $COOKIE"`, `export BASE=…`) plus a per-error-code remediation table, matching the copy-paste-ready anchor.	3 / 3
Workflow Clarity	Multi-step process is clearly sequenced with explicit validation checkpoints — the Mandatory Test Checklist table, numbered Execution flow, Canonical order rationale, and preflight/wait-for-server gating — and includes feedback loops (error code → remediation → re-run preflight; "Verify before filing") for destructive/batch operations.	3 / 3
Progressive Disclosure	SKILL.md acts as an overview whose section table maps each of 15 sections to a one-level-deep `references/<section>.md` file (all 18 reference files and 4 scripts exist on disk), with a Required/Standard/Extended tiering scheme and a final References index for easy navigation.	3 / 3
	Total	11 / 12 Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, third-person description that names a comprehensive set of concrete capabilities and pairs them with an explicit, natural trigger clause scoped to the agent-builder feature branch. It clearly answers both what the skill does and when to use it with no fluff or over-claims.

Dimension	Reasoning	Score
Specificity	Lists many concrete capabilities — "workspace reconciliation, stored agents/skills CRUD, ownership, visibility, stars, registry/library Copy flow, picker allowlists, model policy, RBAC role gating, role impersonation UI, builder defaults, infrastructure diagnostics, channels" — matching the multiple-specific-actions anchor.	3 / 3
Completeness	Explicitly answers both what ("Smoke test the Agent Builder feature branch end-to-end… Covers …") and when ("Trigger when validating the agent-builder feature branch, PRs that touch …"), satisfying the explicit-trigger anchor.	3 / 3
Trigger Term Quality	The "Trigger when validating the agent-builder feature branch, PRs that touch packages/server, packages/playground, packages/playground-ui agent-builder routes, or builder EE code paths" clause gives natural terms a branch-QA user would actually say, with good coverage of the relevant package surfaces.	3 / 3
Distinctiveness Conflict Risk	It carves a clear niche — branch QA of the agent-builder EE surface — with triggers tied to specific packages/code paths, making it unlikely to fire for an unrelated skill.	3 / 3
	Total	12 / 12 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
skill_md_line_count	SKILL.md is long (519 lines); consider splitting into references/ and linking	Warning

	Total	15 / 16 Passed

Repository: mastra-ai/mastra
Path: .claude/skills/builder-smoke-test/SKILL.md
Commit: 3b78ed9

Reviewed: 1 day ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.