compare-skill-model-performance

Run task evals across multiple Claude models, compare results side-by-side, and optimise. Use when you want to benchmark a skill across models, compare haiku vs sonnet vs opus performance, run multi-model comparison or benchmark reports, identify model-specific gaps versus universal plugin gaps, evaluate whether a skill works for all model tiers, or validate a skill before publishing it to the registry.

1.14x

Quality

86%

Does it follow best practices?

Impact

89%

1.14x

Average score across 29 eval scenarios

Securityby

Passed

No findings from the security scan

The canonical home for this skill is tessl/skill-optimizer

Quality

Content

77%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

This is a well-structured, highly actionable skill with excellent workflow clarity — the 7-phase sequential process with explicit validation gates and error recovery is exemplary. Its main weaknesses are moderate verbosity (some sections over-explain concepts Claude would infer) and the lack of progressive disclosure via supporting files, keeping everything in one long document. The diagnostic framework in Phase 6 is particularly valuable but contributes to the overall length.

Suggestions

Trim the run-label section — the 6 examples and composition guidance are excessive; 2 examples and a one-line rule would suffice.

Extract the Phase 5 output table templates and Phase 6 diagnostic classification patterns into a separate REFERENCE.md to reduce the main file's length and improve progressive disclosure.

Remove explanatory asides like 'high baselines (≥80%) mean the skill adds little' — Claude can interpret score semantics without coaching.

Dimension	Reasoning	Score
Conciseness	The skill is reasonably efficient for its complexity but includes some unnecessary verbosity — e.g., explaining what baselines mean ('high baselines ≥80% mean the skill adds little'), spelling out obvious polling logic, and over-explaining label composition with many examples. Some sections could be tightened without losing clarity.	3 / 5
Actionability	Highly actionable throughout — every phase includes exact CLI commands with flags, specific bash snippets for finding plugins and generating scenarios, concrete table formats for output, and precise classification patterns for diagnosis. Copy-paste ready commands cover the full workflow.	5 / 5
Workflow Clarity	Excellent multi-step workflow with 7 clearly sequenced phases, explicit validation checkpoints (verify scenarios exist, verify login, poll each run to completion before starting next), error recovery (retry failed runs), and clear stop conditions. The explicit instruction to NOT run models concurrently is a valuable guardrail.	5 / 5
Progressive Disclosure	The skill is a single monolithic file (~200 lines) with no bundle files or references to supporting documents. The phase-based structure provides internal organization, but the output formatting templates (tables, criterion breakdowns) and the diagnostic classification patterns could reasonably be split into separate reference files for cleaner navigation.	3 / 5
	Total	16 / 20 Passed

Description

95%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

This is a strong skill description that clearly communicates its purpose, provides comprehensive trigger terms, and explicitly states both what the skill does and when to use it. The inclusion of specific Claude model names (haiku, sonnet, opus) and multiple natural trigger phrases makes it highly discoverable and distinctive. Minor improvement could be made by specifying concrete output formats or additional actions beyond 'optimise'.

Dimension	Reasoning	Score
Specificity	Lists several specific actions: 'run task evals across multiple Claude models', 'compare results side-by-side', and 'optimise'. It also mentions identifying model-specific gaps and validating skills before publishing. Minor gaps in describing exactly what outputs are produced or how optimization works.	4 / 5
Completeness	Clearly answers both 'what' (run task evals across models, compare results, optimise) and 'when' with an explicit 'Use when' clause listing six concrete trigger scenarios including benchmarking, comparing model performance, identifying gaps, evaluating across tiers, and validating before publishing.	5 / 5
Trigger Term Quality	Excellent coverage of natural trigger terms: 'benchmark', 'compare', 'haiku vs sonnet vs opus', 'multi-model comparison', 'benchmark reports', 'model-specific gaps', 'model tiers', 'validate', 'publishing', 'registry', 'task evals'. These are terms users would naturally use when wanting this functionality.	5 / 5
Distinctiveness Conflict Risk	Highly distinctive with a clear niche: multi-model evaluation and comparison specifically for Claude model tiers (haiku/sonnet/opus). The mention of specific model names, registry publishing, and cross-model benchmarking creates a very distinct identity unlikely to conflict with other skills.	5 / 5
	Total	19 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 11 / 11 Passed

Validation for skill structure

No warnings or errors.

Repository: tesslio/product-plugins
Path: skill-optimizer/skills/compare-skill-model-performance/SKILL.md
Commit: 37f0a21

Reviewed: 1 day ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.