validate-model

Validate a model entry (or every model in a provider) in apps/sim/providers/models.ts against the provider's live API docs (no hallucination — reports what cannot be verified)

Quality

76%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./.agents/skills/validate-model/SKILL.md

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The SKILL.md body is a highly actionable, well-sequenced validation protocol with strong feedback loops and concrete templates. Its only weaknesses are mild redundancy in the hard-rules restatement and a dense inline checklist rather than a leaner overview with more delegated detail.

Suggestions

Collapse the "Hard rules" block into the relevant steps (or trim it to a one-line pointer) to remove the redundancy with Steps 1-6 and the severity definitions.

Tighten the long parenthetical asides on `reasoningEffort` and `thinking.streamed` into shorter lines, moving edge-case rationale to a referenced detail file.

Consider extracting the full per-field checklist into a referenced file so SKILL.md reads as a concise overview pointing to the detailed checklist.

Dimension	Reasoning	Score
Conciseness	The body is efficient and assumes Claude's competence (no basic-concept padding), with only minor redundancy between the "Hard rules" block and the later numbered steps plus a few long parenthetical checklist asides, fitting anchor 4 rather than the lean anchor 5.	4 / 5
Actionability	It provides concrete file paths, named commands (e.g. `bun run lint`, `bun run agent-stream-docs:generate`), a full copy-paste report template with example rows, and explicit severity definitions, matching the fully-executable anchor 5.	5 / 5
Workflow Clarity	A clear six-step sequence (read → fetch → consumption map → checklist → report → offer-to-fix) is paired with explicit validation checkpoints and a confirm→diff→edit→lint→re-run-failed-rows feedback loop, matching anchor 5; the batch/destructive cap does not apply because verification is present.	5 / 5
Progressive Disclosure	Sections are well organized and the canonical URL table and Consumption Matrix are delegated via clearly signaled one-level-deep references to `add-model/SKILL.md` Step 1/2, but the body itself is a dense inline protocol with no own bundle files, fitting anchor 4 rather than the clean-overview anchor 5.	4 / 5
	Total	18 / 20 Passed

Description

65%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific and highly distinct, but lacks an explicit "Use when..." trigger clause, which caps its completeness. Trigger-term coverage is good though not exhaustive.

Suggestions

Add an explicit "Use when..." clause, e.g. "Use when adding or updating models in models.ts, or when you suspect model pricing or capabilities are stale."

Surface a few natural synonyms ("check", "audit") alongside "validate" so the description matches more phrasings a user might say.

Optionally name the concrete sub-checks (pricing, capabilities, context window) to lift specificity toward multiple distinct actions.

Dimension	Reasoning	Score
Specificity	The description names the domain ("model entry... in apps/sim/providers/models.ts") and one concrete action ("Validate... against the provider's live API docs"), but does not enumerate multiple distinct actions, matching anchor 3 rather than the multi-action anchor 4.	3 / 5
Completeness	It clearly states what the skill does, but provides no explicit "Use when..." trigger clause; per the rubric guideline a missing explicit trigger caps completeness at 3 (anchor 3: clear "what", weakly implied "when").	3 / 5
Trigger Term Quality	Natural terms a developer would say are well covered ("validate", "model entry", "provider", "live API docs"), aligning with anchor 4; a few synonyms ("check", "audit", "pricing") are missing, keeping it below 5.	4 / 5
Distinctiveness Conflict Risk	The pinned file path and narrow purpose (validating model entries against live API docs) give it a clear niche with minimal overlap risk, matching anchor 5.	5 / 5
	Total	15 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: simstudioai/sim
Path: .agents/skills/validate-model/SKILL.md
Commit: 7945b29

Reviewed: 3 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.