llm-provider

Adds a new LLM provider implementing LLMProvider interface with call() and stream() methods. Integrates with provider factory in src/llm/index.ts, config detection in src/llm/config.ts, and error handling via tracking and recovery. Use when adding a new model backend, integrating a third-party LLM API, or extending LLM platform support. Do NOT use for fixing bugs in existing providers, modifying existing provider behavior, or changing the LLMProvider interface.

Quality

88%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

77%Weight 40%Scale 1-3

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable with a well-sequenced, validated workflow, but it is monolithic and repeats some guidance across the Critical, Steps, and Common Issues sections. Splitting the provider template and troubleshooting into reference files and de-duplicating the repeated rules would lift conciseness and progressive disclosure.

Suggestions

De-duplicate guidance that appears in the Critical section, the Steps, and Common Issues (e.g., the trackUsage mandate and the 'options.model || this.defaultModel' pattern) so each rule is stated once.

Extract the full provider-class template and/or the Common Issues troubleshooting into one-level-deep reference files (e.g., references/provider-template.md, references/troubleshooting.md) and link to them from the body to improve progressive disclosure.

Tighten the Critical section into a short checklist of invariants, keeping the concrete code only in the Step it supports, to reduce token overhead.

Dimension	Reasoning	Score
Conciseness	Dense and free of concept-explanation fluff, but the 'Critical' rules, the Step demonstrations, and the 'Common Issues' fixes overlap (e.g., the trackUsage mandate and the model-parameter pattern are explained in three places), adding mild redundancy that could be tightened.	2 / 3
Actionability	Provides complete executable TypeScript, exact file paths, specific line numbers (9, 17, 59, 115), and concrete verify commands ('npx tsc --noEmit', 'npm run test -- ...'), making the guidance copy-paste ready.	3 / 3
Workflow Clarity	A clear 7-step sequence where nearly every step carries an explicit 'Verify:' checkpoint, plus an integration-test feedback loop in Step 7 and lock-step warnings for multi-file edits.	3 / 3
Progressive Disclosure	No bundle files exist and the skill is a single well-organized SKILL.md, but it is monolithic (~240 lines) with full provider templates and a Common Issues section that could be split into one-level-deep reference files rather than kept inline.	2 / 3
	Total	10 / 12 Passed

Description

100%Weight 40%Scale 1-3

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific, complete, and well-bounded, clearly stating what the skill does, when to use it, and when not to. Trigger terms are natural and the exclusion clause sharply limits conflict risk.

Dimension	Reasoning	Score
Specificity	Enumerates multiple concrete integration actions — 'implementing LLMProvider interface with call() and stream() methods', 'provider factory in src/llm/index.ts', 'config detection in src/llm/config.ts', and 'error handling via tracking and recovery' — matching the score-3 anchor of listing several specific concrete actions.	3 / 3
Completeness	Explicitly answers both what (adds a provider implementing the interface with factory/config/error integration) and when ('Use when adding a new model backend...'), and adds a 'Do NOT use for...' exclusion clause.	3 / 3
Trigger Term Quality	Provides natural phrases a developer would say ('adding a new model backend', 'integrating a third-party LLM API', 'extending LLM platform support'), covering multiple common variations rather than a single keyword.	3 / 3
Distinctiveness Conflict Risk	Clear niche (new LLM provider backends) sharpened by the explicit 'Do NOT use for fixing bugs in existing providers, modifying existing provider behavior, or changing the LLMProvider interface' boundary, making wrong-skill triggering unlikely.	3 / 3
	Total	12 / 12 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: caliber-ai-org/ai-setup
Path: .claude/skills/llm-provider/SKILL.md
Commit: 7950921

Reviewed: 13 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.