new-provider

Add a new LLM provider to hermesllm. Use when integrating a new AI provider.

Quality

74%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./.claude/skills/new-provider/SKILL.md

Quality

Content

86%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is an exemplary lean, path-specific checklist with a verification step. Its only gaps are underspecified steps (match arms, string parsing) and the absence of an error-recovery loop after the test command.

Suggestions

Show the pattern for step 2, e.g., a one-line match arm example in the TryFrom<&str> impl so "implement string parsing" is copy-paste ready.

Clarify "update all match arms" with the file locations or a grep command (e.g., grep for the enum name) so Claude can enumerate the match sites mechanically.

Add a brief feedback loop after the test step: if cargo test fails, fix compile errors in the new match arms before proceeding.

Dimension	Reasoning	Score
Conciseness	The body is a lean numbered list with exact file paths and one verification command; it assumes Claude's competence with zero padding, matching the anchor-5 example of every token earning its place.	5 / 5
Actionability	Steps cite concrete paths (crates/hermesllm/src/providers/id.rs, provider_models.yaml) and a runnable command (cd crates && cargo test --lib), but instructions like "update all match arms" and "implement string parsing" leave key details unspecified, matching anchor 4 rather than a fully copy-paste-ready 5.	4 / 5
Workflow Clarity	The six steps are clearly sequenced and end with an explicit validation checkpoint (cargo test), matching anchor 4; there is no feedback loop for what to do when tests fail, which anchor 5 requires.	4 / 5
Progressive Disclosure	No bundle files exist and the skill is well under 50 lines with a well-organized numbered structure, so the simple-skill exception applies and progressive disclosure scores 5 on well-organized sections alone.	5 / 5
	Total	18 / 20 Passed

Description

62%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description correctly uses third person, includes both 'what' and 'when', and is well-scoped to hermesllm. Its weaknesses are thin action coverage and a limited set of natural trigger phrases that users might actually say when requesting this skill.

Suggestions

List a couple of the concrete sub-actions in the description (e.g., wiring up API request/response types, model lists) to raise specificity from one action to several.

Broaden trigger terms to include natural user phrasings like "add a model provider", "integrate an LLM API", or named providers to improve trigger term quality and distinctiveness.

Make the 'when' clause more concrete about the situations that call for the skill, e.g., "Use when adding a new AI provider (OpenAI, Anthropic, Google, or a custom endpoint) to hermesllm".

Dimension	Reasoning	Score
Specificity	"Add a new LLM provider to hermesllm" names the domain and one concrete action but stops there; it does not list several specific actions, matching anchor 3 rather than anchor 4.	3 / 5
Completeness	It has a clear 'what' (add a new LLM provider to hermesllm) and an explicit 'when' clause ("Use when integrating a new AI provider"), matching anchor 4; the 'when' could be more specific about the situations that call for this skill, so it falls short of anchor 5.	4 / 5
Trigger Term Quality	Terms like "LLM provider", "AI provider", and "integrating" are relevant, but common natural variations users would actually say (specific provider names, "model", "backend", "OpenAI-compatible") are missing, so coverage matches anchor 3 rather than 4.	3 / 5
Distinctiveness Conflict Risk	The description is scoped to a named codebase (hermesllm) and a narrow task, making it mostly distinct with only minor overlap risk against other provider/model-related skills, matching anchor 4.	4 / 5
	Total	14 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: katanemo/plano
Path: .claude/skills/new-provider/SKILL.md
Commit: 72002a6

Reviewed: 20 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.