CtrlK
BlogDocsLog inGet started
Tessl Logo

llm-provider

Adds a new LLM provider implementing LLMProvider interface with call() and stream() methods. Integrates with provider factory in src/llm/index.ts, config detection in src/llm/config.ts, and error handling via tracking and recovery. Use when adding a new model backend, integrating a third-party LLM API, or extending LLM platform support. Do NOT use for fixing bugs in existing providers, modifying existing provider behavior, or changing the LLMProvider interface.

73

Quality

92%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

The canonical home for this skill is llm-provider in caliber-ai-org/ai-setup

SKILL.md
Quality
Evals
Security

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

An exceptionally actionable, well-sequenced skill body: executable code, exact file/line targets, per-step verification, and a genuine error-recovery section. Its weaknesses are mild — some redundancy between the inline test template/Examples and the step instructions, and a monolithic single-file layout where a references/ file could hold the test template and examples.

Suggestions

Move the full Step 6 test template and the two worked Examples into a references/ file (e.g., references/examples.md) and keep one-line pointers in SKILL.md, trimming roughly 60–80 lines from the main body.

Tighten the Examples section to the deltas from the standard steps (e.g., 'LM Studio = OpenAICompatProvider + LM_STUDIO_BASE_URL detection') instead of repeating config.ts code blocks already templated in Step 4.

DimensionReasoningScore

Conciseness

Assumes Claude's competence throughout — no explanation of what an LLM provider or TypeScript interface is — and every token is project-specific (file paths, exact line numbers, verify commands). It falls short of the score-5 'lean and efficient' anchor because the full ~30-line test template in Step 6 and the two worked Examples partially restate patterns already shown in Steps 1–5, padding that could be trimmed.

4 / 5

Actionability

Copy-paste ready throughout: a complete executable TypeScript class template in Step 1, exact edits with line numbers ('Line 9: Add to DEFAULT_MODELS', 'Line 59: In resolveFromEnv()'), and concrete verify commands per step ('npx tsc --noEmit', 'npm run test -- src/llm/__tests__/ -t config'). Specific examples cover the common cases (LM Studio, seat-based Ollama), matching the 'fully executable; copy-paste ready' anchor.

5 / 5

Workflow Clarity

Seven clearly sequenced steps each ending in an explicit 'Verify:' command forming validation checkpoints, a 'Common Issues' section structured as cause/fix feedback loops, and a Critical section flagging the lock-step invariant ('Missing any one breaks the build'). This matches the score-5 anchor (explicit validation steps plus error-recovery guidance); score 4 would require missing checkpoints, which are not missing.

5 / 5

Progressive Disclosure

No bundle files exist, so the entire skill is one ~240-line file; it is well-sectioned (Critical / Instructions / Examples / Common Issues) and easy to navigate, matching 'good structure; most content is appropriately placed'. It does not reach score 5 because the under-50-lines exception does not apply and content such as the full test template and worked examples is inlined rather than split into one-level-deep reference files; it is above score 3 since nothing is buried and navigation is clear.

4 / 5

Total

18

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: third-person voice, concrete capabilities anchored to real source files, an explicit 'Use when...' clause with natural trigger phrases, and an unusually good 'Do NOT use' boundary clause. The only gap is trigger-term breadth — no named provider examples or synonyms a user might naturally say.

DimensionReasoningScore

Specificity

Multiple concrete actions with exact artifacts: 'implementing LLMProvider interface with call() and stream() methods', 'Integrates with provider factory in src/llm/index.ts, config detection in src/llm/config.ts, and error handling via tracking and recovery'. This comprehensively covers the what (interface, factory, config, error handling) with file-level specificity, matching the 'multiple specific concrete actions; comprehensive coverage' anchor rather than the score-4 anchor with 'minor gaps in coverage'.

5 / 5

Completeness

Explicitly answers both: what — 'Adds a new LLM provider implementing LLMProvider interface... Integrates with provider factory... config detection... error handling' — and when — 'Use when adding a new model backend, integrating a third-party LLM API, or extending LLM platform support'. Concrete trigger phrases on both sides match the score-5 anchor; it is not score 4 because the 'when' clause is already explicit and specific, not merely present.

5 / 5

Trigger Term Quality

Natural phrases users would say are present: 'adding a new model backend', 'integrating a third-party LLM API', 'extending LLM platform support'. Falls short of the score-5 anchor because it lacks synonyms and concrete instances users commonly mention (e.g., 'add support for OpenAI/Ollama/LM Studio', 'local model'); it is clearly above score 3 since several natural trigger phrases are covered, not just domain keywords.

4 / 5

Distinctiveness Conflict Risk

Clear niche plus explicit negative triggers: 'Do NOT use for fixing bugs in existing providers, modifying existing provider behavior, or changing the LLMProvider interface'. The exclusion clause sharply separates this from provider-debugging and interface-refactoring skills, matching the 'clear niche with distinct triggers; minimal conflict risk' anchor.

5 / 5

Total

19

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository
caliber-ai-org/ai-setup
Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.