builder-smoke-test

Smoke test the Agent Builder feature branch end-to-end against a hermetic project scaffolded by the skill (linked to the current worktree). Covers workspace reconciliation, stored agents/skills CRUD, ownership, visibility, stars, registry/library Copy flow, picker allowlists, model policy, RBAC role gating, role impersonation UI, builder defaults, infrastructure diagnostics, channels, and Studio + Agent Builder UI. Trigger when validating the agent-builder feature branch, PRs that touch packages/server, packages/playground, packages/playground-ui agent-builder routes, or builder EE code paths.

Quality

92%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Critical

Do not install without reviewing

Quality

Content

92%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable orchestration skill with explicit workflow sequencing, validation gates, and a clean one-level reference split. The main lever for improvement is tightening or relocating some of the longer cross-cutting procedural sections to reduce SKILL.md length.

Suggestions

Move the detailed "Resolving missing env vars" 6-step procedure and its error-code table into references/setup.md, leaving a one-line pointer and the error-code names in SKILL.md.

Consider relocating the "Extracting the session cookie" and "Seeding non-owner skills" walkthroughs to references/auth.md and references/skills.md respectively, keeping only the trigger conditions and the do-not-pivot warnings inline.

Trim the explanatory framing in the opening ("This skill is for branch QA — it complements...") to a single sentence since the description frontmatter already conveys the purpose.

Dimension	Reasoning	Score
Conciseness	Dense and operational with little concept-explanation padding, but the env-resolution procedure, error-code table, and cookie-extraction detail are long enough that some could be trimmed or moved into references/setup.md.	4 / 5
Actionability	Fully executable: copy-paste curl commands, concrete script invocations ("bash .claude/skills/builder-smoke-test/scripts/preflight.sh --expect off"), an endpoint table, and a parameter table with defaults covering the common cases.	5 / 5
Workflow Clarity	Clear sequenced execution flow (1–7) with explicit validation checkpoints (preflight before each section, wait-for-server probing /api/agents), an error-code recovery table, and feedback loops for destructive/batch test operations.	5 / 5
Progressive Disclosure	Clear overview in SKILL.md with well-signaled one-level-deep references — the section table maps each of the 15 sections to references/<section>.md, and all referenced files (15 references + 4 scripts) exist on disk.	5 / 5
	Total	19 / 20 Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, third-person description that concretely enumerates the tested surfaces and gives explicit, natural trigger conditions tied to the feature branch and affected packages. Only minor gap is a few missing user-facing synonyms for triggering.

Dimension	Reasoning	Score
Specificity	Lists multiple specific concrete surfaces with comprehensive coverage: "workspace reconciliation, stored agents/skills CRUD, ownership, visibility, stars, registry/library Copy flow, picker allowlists, model policy, RBAC role gating, role impersonation UI, builder defaults, infrastructure diagnostics, channels, and Studio + Agent Builder UI".	5 / 5
Completeness	Explicitly answers both what ("Smoke test the Agent Builder feature branch end-to-end..." plus the full coverage list) and when ("Trigger when validating the agent-builder feature branch, PRs that touch...") with concrete trigger phrases.	5 / 5
Trigger Term Quality	Good natural trigger coverage — "Trigger when validating the agent-builder feature branch, PRs that touch packages/server, packages/playground, packages/playground-ui agent-builder routes, or builder EE code paths" — but a few natural synonyms (e.g. "QA the builder", "smoke test the builder") are not present.	4 / 5
Distinctiveness Conflict Risk	Clear niche (agent-builder feature-branch smoke testing) with distinct, specific triggers tied to package paths and EE code surfaces; minimal overlap risk with other skills.	5 / 5
	Total	19 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
skill_md_line_count	SKILL.md is long (519 lines); consider splitting into references/ and linking	Warning

	Total	15 / 16 Passed

Repository: mastra-ai/mastra
Path: .claude/skills/builder-smoke-test/SKILL.md
Commit: 748d4ed

Reviewed: 1 day ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.