plugin-eval

Help engineers evaluate a local skill or plugin, explain why it scored that way, show what to fix first, measure real token usage, benchmark starter scenarios, or decide what to run next. Use when the user says things like "evaluate this skill", "give me an analysis of the game dev skill", "why did this score that way", "what should I fix first", "measure the real token usage of this skill", or "what should I run next?".

Quality

85%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

77%Weight 40%Scale 1-3

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured router skill with concrete, executable commands and a clear sequenced workflow. Its weaknesses are redundant repeated sections (Chat Requests / Matching Commands duplicate Start Here) and three reference paths that point to files not present in the bundle.

Suggestions

Remove the "Chat Requests To Recognize" and "Matching Commands" sections, or collapse them into Start Here, since both duplicate phrases and commands already shown inline in step 3.

Fix the dangling references: either add the missing files under references/ (chat-first-workflows.md, technical-design.md, evaluation-result-schema.md) or drop the References section until those files exist.

Disambiguate the "Evaluate this skill." route from sibling evaluate-skill/evaluate-plugin skills - e.g., note in the routing that plain evaluation hands off to those skills while this one owns analysis/scoring/fix-first/token/benchmark flows.

Dimension	Reasoning	Score
Conciseness	The body is mostly lean and uses lists/commands, but "Chat Requests To Recognize" repeats the same phrases already embedded in Start Here step 3, and "Matching Commands" repeats commands already shown inline. Matches score-2 (mostly efficient but includes unnecessary duplication); not 3 because those redundant sections don't earn their tokens, not 1 because there is no concept-explaining fluff.	2 / 3
Actionability	Provides copy-paste-ready executable commands throughout - `plugin-eval start <path> --request "..." --format markdown`, `plugin-eval analyze <path> --format markdown`, `plugin-eval explain-budget`, etc. - with concrete routing. Matches score-3 (fully executable commands, specific examples); not 2 because guidance is complete rather than pseudocode or abstract.	3 / 3
Workflow Clarity	Start Here steps 1-7 give a clearly sequenced routing decision flow with an explicit ambiguity checkpoint ("ask one short clarifying question"). Matches score-3 for a clear, well-sequenced process; the missing-validation cap does not apply because this is non-destructive routing, not batch/destructive ops. Not 2 because the sequence and routing logic are explicit.	3 / 3
Progressive Disclosure	References are one-level-deep and signaled in a dedicated References section, but they point to `../../references/chat-first-workflows.md`, `technical-design.md`, and `evaluation-result-schema.md` - none of which exist in the bundle (no references/ directory present), so navigation does not resolve. Matches score-2 (references present but navigation/discovery broken); not 3 because easy navigation fails on dangling paths, not 1 because organization is clean with no deep nesting.	2 / 3
	Total	10 / 12 Passed

Description

92%Weight 40%Scale 1-3

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, third-person description that concretely lists capabilities and pairs them with natural-language trigger phrases covering both what and when. Its only weakness is overlap risk with sibling evaluate-skill/evaluate-plugin skills on the generic "evaluate this skill" trigger.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions - "evaluate a local skill or plugin", "explain why it scored that way", "show what to fix first", "measure real token usage", "benchmark starter scenarios", "decide what to run next" - matching the score-3 anchor for multiple specific concrete actions. Not 2 because the action list is comprehensive rather than partial.	3 / 3
Completeness	Explicitly answers both what (evaluate/explain/fix-first/measure/benchmark/next) and when ("Use when the user says things like..."). Matches the score-3 anchor for clearly answering both with explicit triggers; not 2 because the when clause is explicit, not implied.	3 / 3
Trigger Term Quality	Embeds natural phrasings users would actually say - "evaluate this skill", "give me an analysis of the game dev skill", "why did this score that way", "what should I fix first", "measure the real token usage of this skill", "what should I run next?". Good coverage of natural terms; not 2 because common variations are well represented.	3 / 3
Distinctiveness Conflict Risk	The niche (scoring explanation, fix-first, token measurement, benchmarking) is distinct, but the trigger "Evaluate this skill." overlaps with the sibling evaluate-skill/evaluate-plugin skills the body hands off to. Matches score-2 (somewhat specific but could overlap with similar skills); not 3 because of that overlap, not 1 because the broader analysis/token/benchmark triggers are clearly niche-scoped.	2 / 3
	Total	11 / 12 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: openai/plugins
Path: plugins/plugin-eval/skills/plugin-eval/SKILL.md
Commit: 11c74d6

Reviewed: 17 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.