benchmark

Measure Z3 performance on a formula or file. Collects wall-clock time, theory solver statistics, memory usage, and conflict counts. Results are logged to z3agent.db for longitudinal tracking.

Quality

80%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

86%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A strong, executable body: commands are copy-paste ready and verified against the bundled script, and structure/navigation are appropriate. The main weaknesses are Expectation/Result redundancy in the step scaffolding and the absence of explicit validation or failure-path handling (z3 timeout, non-zero exit) in the workflow.

Suggestions

Merge each step's Expectation and Result blocks into a single expected-output statement to remove the repeated "the script runs and prints X" phrasing.

Add an explicit failure-path branch, e.g. "If a run times out or exits non-zero, re-run with a larger --timeout or report the failure before comparing against historical runs."

Trim filler lines such as "If performance is acceptable, no action needed" and state Step 3's regression guidance as a single conditional.

Dimension	Reasoning	Score
Conciseness	The body avoids over-explaining known concepts and the parameters table earns its place, but the Action/Expectation/Result scaffolding is redundant in places — e.g. Step 1's "invokes `z3 -st`, parses the statistics block, and prints a performance summary" vs. "Result: Timing and statistics are displayed" — and "If performance is acceptable, no action needed" is filler.	4 / 5
Actionability	Every step ships copy-paste-ready commands ("python3 scripts/benchmark.py --file problem.smt2 --runs 5") that match the actual script's argparse interface, plus a complete parameters table with defaults and an enumerated output description.	5 / 5
Workflow Clarity	Steps 1–3 are clearly sequenced with conditional guidance ("If slow, try simplify to reduce the formula or adjust tactic strategies"), but validation is implicit in the Expectation blocks rather than explicit verify steps, and failure paths such as z3 timeouts or non-zero exits go unaddressed.	4 / 5
Progressive Disclosure	Scored against the actual bundle: scripts/benchmark.py exists and is correctly referenced, the shared ../../shared/z3db.py is a one-level external dependency, and everything inlined (parameters table, output list) is appropriately sized for SKILL.md with nothing that should be split into a separate file.	5 / 5
	Total	18 / 20 Passed

Description

75%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A specific, distinctive description with concrete capability enumeration, undermined by the complete absence of "when to use" trigger guidance and thin coverage of natural synonyms (SMT, solver, .smt2). Adding a "Use when..." clause would lift both completeness and trigger term quality.

Suggestions

Append a trigger clause such as "Use when the user asks to benchmark, profile, or time Z3 on SMT-LIB2 formulas or .smt2 files, or to compare solver performance over time."

Include natural synonyms and file extensions (SMT, SMT-LIB, solver, .smt2) so the description matches how users actually phrase these requests.

Clarify that results support regression detection/comparison over time, since "longitudinal tracking" is opaque to a user deciding whether to invoke this skill.

Dimension	Reasoning	Score
Specificity	The description enumerates concrete, measurable outputs — "Collects wall-clock time, theory solver statistics, memory usage, and conflict counts" plus "logged to z3agent.db" — giving comprehensive coverage of this skill's capabilities with no gaps.	5 / 5
Completeness	The "what" is clear and specific (measure Z3 performance, collect named metrics, log to z3agent.db), but there is no "Use when..." clause or equivalent trigger guidance, which caps completeness at 3.	3 / 5
Trigger Term Quality	It includes relevant terms like "Z3", "performance", "formula", and "file", but misses natural synonyms and extensions users would say — "SMT", "SMT-LIB", "solver", "benchmark", or ".smt2" — so a few natural terms are missing.	4 / 5
Distinctiveness Conflict Risk	"Measure Z3 performance on a formula or file" names a specific tool and a specific activity, carving out a clear niche with minimal risk of triggering for unrelated skills.	5 / 5
	Total	17 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: Z3Prover/z3
Path: .github/skills/benchmark/SKILL.md
Commit: dabaa0b

Reviewed: 7 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.