validate-model

Validate a model entry (or every model in a provider) in apps/sim/providers/models.ts against the provider's live API docs (no hallucination — reports what cannot be verified)

Quality

67%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./.agents/skills/validate-model/SKILL.md

Quality

Content

77%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is a strong, highly actionable validation workflow with concrete commands, a literal report template, and well-placed validation/feedback checkpoints. Its weakness is conciseness and progressive disclosure: some redundancy and reference-style content (snapshot table, common-bugs list) are inline rather than split into bundle files.

Suggestions

Move the consumption-map snapshot table and/or the 'Common bugs this skill catches' list into a references/ file (e.g. CONSUMPTION.md) and link to it, improving progressive disclosure and reducing inline length.

De-duplicate severity definitions — keep them in one place (the 'Severity definitions' section) and have the report example reference them rather than restating each level inline.

Tighten the 'What I cannot verify this looks like' section; the example sentence is illustrative but long and slightly overlaps the earlier hard rules on unverified marking.

Dimension	Reasoning	Score
Conciseness	The body is action-dense and avoids explaining generic concepts Claude already knows, but at ~165 lines it carries redundancy (e.g. severity meanings appear both inline in the report example and again in the 'Severity definitions' section) that could be tightened, matching the level-2 'mostly efficient but could be tightened' anchor.	2 / 3
Actionability	It provides copy-paste-ready, executable guidance — concrete `rg` greps, exact field names in a per-model checklist, a literal report table template, and commands like `bun run lint` and `bun run agent-stream-docs:generate` — fully meeting the level-3 anchor for executable, specific examples.	3 / 3
Workflow Clarity	A clear Step 1–6 sequence with explicit validation checkpoints ('Re-grep before trusting the snapshot', 'Never auto-fix without printing the diff', re-run only failed rows) and feedback loops for the destructive edit step, satisfying the level-3 anchor for sequenced workflows with validation and error-recovery loops.	3 / 3
Progressive Disclosure	Sections are well organized and the only cross-reference is one level deep (`.agents/skills/add-model/SKILL.md`), but no bundle files exist and reference-style material (the consumption-map snapshot table, the common-bugs list) lives inline in a >50-line file, fitting the level-2 'content that should be separate is inline' anchor rather than a clean one-level-deep split.	2 / 3
	Total	10 / 12 Passed

Description

57%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific and distinctive, clearly scoping a niche task with a precise file target and a notable anti-hallucination behavior. Its main weakness is the absence of an explicit 'Use when...' trigger clause and natural user phrasings, which caps both completeness and trigger-term quality at 2.

Suggestions

Add an explicit trigger clause, e.g. 'Use when checking or updating model pricing, capabilities, or context windows in models.ts.'

Surface natural user phrasings ('verify model specs', 'check pricing', 'refresh stale model entries') alongside the technical 'validate a model entry' wording.

Consider listing a couple of concrete sub-actions (e.g. 'verify pricing, capabilities, and context windows') to lift specificity toward the level-3 anchor.

Dimension	Reasoning	Score
Specificity	Quotes 'Validate a model entry (or every model in a provider) in apps/sim/providers/models.ts against the provider's live API docs' — names a concrete domain and action with a precise file target, but describes essentially one validation action rather than the multiple distinct concrete actions the level-3 anchor requires.	2 / 3
Completeness	The 'what' is explicit ('Validate a model entry ... against the provider's live API docs') but there is no 'Use when...' clause or equivalent explicit trigger guidance, so per the judging guidelines completeness is capped at 2 with 'when' only implied.	2 / 3
Trigger Term Quality	'Validate a model entry' and 'live API docs' are relevant but technical terms; common user phrasings like 'check model pricing' or 'verify model specs' are absent, matching the level-2 'some relevant keywords but missing common variations' anchor.	2 / 3
Distinctiveness Conflict Risk	The description pins a tight niche — validating model entries in a specific file path against live API docs with an unverified-reporting behavior — making it clearly distinguishable and unlikely to trigger for the wrong skill.	3 / 3
	Total	9 / 12 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: simstudioai/sim
Path: .agents/skills/validate-model/SKILL.md
Commit: 6f514c1

Reviewed: about 16 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.