new-provider

Add a new LLM provider to hermesllm. Use when integrating a new AI provider.

Quality

73%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./.claude/skills/new-provider/SKILL.md

Quality

Content

80%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is exceptionally lean and well-targeted at specific files and enums, with appropriate single-file structure for a short skill. Its main weakness is workflow clarity around the batch 'update all match arms' step, which lacks a verification checkpoint and an error-recovery loop.

Suggestions

Add a checkpoint after step 4 (e.g. 'Run cargo check to confirm all match arms are exhausted before moving on') so the batch update has an explicit validation gate.

Add a brief feedback loop after the final test command: 'If tests fail, fix the reported errors and re-run until they pass.'

Make step 3/4 more actionable by naming the typical request/response struct pattern or pointing to an existing provider's impl as a template.

Dimension	Reasoning	Score
Conciseness	The body is lean and assumes Claude's competence: a one-line restatement of the task followed by six numbered file-level steps and a verification command, with no padding or explanation of concepts Claude already knows.	5 / 5
Actionability	Steps point at concrete file paths and enums (e.g. 'crates/hermesllm/src/providers/id.rs', 'provider_models.yaml') plus an executable test command, but step 4 ('update all match arms') and the API-type creation in step 3 are described at a high level without code, leaving minor gaps.	4 / 5
Workflow Clarity	The six steps form a clear sequence and end with a validation command ('cargo test --lib'), but step 4 'update all match arms' is a batch operation with no checkpoint to confirm completeness, and there is no feedback loop describing what to do if compilation/tests fail.	3 / 5
Progressive Disclosure	At 17 lines with no bundle files, the skill is a single well-organized numbered list with no need for external references; the simple-skill guideline lets a short, cleanly structured body score 5.	5 / 5
	Total	17 / 20 Passed

Description

67%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description clearly states both what the skill does and when to use it, with a concrete domain anchor. It is held back by thin action coverage and limited trigger-term variation.

Suggestions

List 2-3 concrete actions (e.g. 'add provider enum variant, wire request/response types, register models') so the what reads as a capability list rather than a single verb.

Broaden trigger terms to natural synonyms users would say, such as 'connect an LLM provider', 'add an LLM backend', or 'wire up a new API'.

Tighten the 'when' clause with a concrete trigger phrase, e.g. 'Use when the user asks to add or connect a new LLM provider to hermesllm.'

Dimension	Reasoning	Score
Specificity	The description names the domain ('Add a new LLM provider to hermesllm') and a concrete action (integrating a new AI provider), but provides only one action rather than a comprehensive list of capabilities, fitting the 'lists several specific actions; minor gaps' anchor slightly below it.	4 / 5
Completeness	It answers 'what' clearly (add a new LLM provider) and 'when' explicitly via 'Use when integrating a new AI provider', matching the anchor where both are present though the 'when' could be more specific.	4 / 5
Trigger Term Quality	It includes the natural phrase 'integrating a new AI provider' but misses common synonyms and variations a user might say (e.g. 'connect a provider', 'add an LLM backend', 'wire up an API'), landing at 'some relevant keywords but missing common variations'.	3 / 5
Distinctiveness Conflict Risk	The narrow scope of adding a provider to a specific codebase (hermesllm) makes it largely distinct with only minor overlap risk against other code-integration skills.	4 / 5
	Total	15 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: katanemo/plano
Path: .claude/skills/new-provider/SKILL.md
Commit: 003c36a

Reviewed: 3 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.