llm-provider

Adds a new LLM provider implementing LLMProvider interface with call() and stream() methods. Integrates with provider factory in src/llm/index.ts, config detection in src/llm/config.ts, and error handling via tracking and recovery. Use when adding a new model backend, integrating a third-party LLM API, or extending LLM platform support. Do NOT use for fixing bugs in existing providers, modifying existing provider behavior, or changing the LLMProvider interface.

Quality

89%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

85%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

Highly actionable and workflow-sound content: complete executable code, per-step validation checkpoints, and a thorough troubleshooting section, with no filler or redundant explanation. Its main structural weakness is that everything lives inline in one long SKILL.md rather than splitting tests, examples, and troubleshooting into reference files.

Suggestions

Move the full test file template (Step 6) and the integration test invocation (Step 7) into a references/ file (e.g. references/testing.md), keeping only a short skeleton and the test command in SKILL.md.

Extract the Examples section (LM Studio, Ollama) into references/examples.md and link to it, since these are illustrative variants rather than core workflow steps.

Move the Common Issues cause/fix table into references/troubleshooting.md, leaving the two or three most likely failures inline so the main workflow stays lean.

Dimension	Reasoning	Score
Conciseness	Dense with project-specific detail (line numbers, file paths, exact commands) and avoids explaining concepts Claude already knows; only minor trimming opportunities exist, such as the 30-line test template and explanatory asides like "This is mandatory — it captures token metrics for CLI telemetry and cost analysis".	4 / 5
Actionability	Every step ships copy-paste-ready TypeScript, exact verification commands ("npx tsc --noEmit", "npm run test -- src/llm/__tests__/your-provider.test.ts"), and specific edit locations, fully matching the anchor for executable guidance covering common cases.	5 / 5
Workflow Clarity	Seven clearly sequenced steps each end with an explicit Verify checkpoint (type checks, unit tests, integration test with env var), and the Common Issues section provides cause/fix error-recovery loops — matching the anchor-5 example structure.	5 / 5
Progressive Disclosure	Headers organize the file well, but the ~240-line body is fully monolithic: the complete test file template, both worked examples, and the troubleshooting section are all inlined in SKILL.md with no reference files at all (the bundle contains none), which is content that could be split into separate files.	3 / 5
	Total	17 / 20 Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description that clearly states concrete capabilities, integration points, and explicit positive and negative triggers in third person. The only weakness is the slightly vague phrase "error handling via tracking and recovery" and some missing natural trigger synonyms.

Dimension	Reasoning	Score
Specificity	Lists several concrete actions — "implementing LLMProvider interface with call() and stream() methods", "Integrates with provider factory", "config detection" — but "error handling via tracking and recovery" is somewhat generic, leaving minor gaps versus the comprehensive anchor 5.	4 / 5
Completeness	Explicitly answers both "what" (concrete implementation and integration actions) and "when" with concrete trigger phrases, plus negative triggers ("Do NOT Use for fixing bugs in existing providers"), matching the anchor-5 example exactly.	5 / 5
Trigger Term Quality	"Use when adding a new model backend, integrating a third-party LLM API, or extending LLM platform support" provides good natural keyword coverage, though common variations users might say (e.g. "add support for provider X") are missing.	4 / 5
Distinctiveness Conflict Risk	Clear niche (adding a new provider to this codebase) with distinct triggers, and the explicit "Do NOT Use" clause eliminates overlap with bug-fixing and interface-modification work, giving minimal conflict risk.	5 / 5
	Total	18 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: caliber-ai-org/ai-setup
Path: .claude/skills/llm-provider/SKILL.md
Commit: f5dbc00

Reviewed: 14 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.