validate-model

Validate a model entry (or every model in a provider) in apps/sim/providers/models.ts against the provider's live API docs (no hallucination — reports what cannot be verified)

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A highly actionable, well-sequenced skill with strong validation checkpoints and concrete executable guidance throughout. Slight conciseness trimming and moving the inlined reference material into a bundled file would push it to top marks.

Suggestions

Trim the closing 'What I cannot verify this looks like' section — the hard rules and unverified-severity definition already convey this behavior.

Consider moving the consumption-matrix snapshot and per-field checklist into a bundled reference file under references/ to keep SKILL.md as a lean overview.

Dimension	Reasoning	Score
Conciseness	Mostly lean and assumes Claude's competence — concrete commands, a consumption-matrix snapshot, and a tight checklist — with a few sections (e.g., the 'What I cannot verify' closing and some repeated hard-rules) that could be trimmed.	4 / 5
Actionability	Provides copy-paste-ready ripgrep commands, an explicit checklist of every field to verify, a concrete report table format with example rows, and exact severity definitions — fully executable guidance covering common cases.	5 / 5
Workflow Clarity	A six-step sequence with explicit validation checkpoints (live-fetch or mark UNVERIFIED, two-source pricing cross-check, print diff before fixing, re-lint after edits, re-run failed rows), feedback loops, and a mandatory report format.	5 / 5
Progressive Disclosure	Well-organized with clear section headers and a one-level-deep reference to the add-model skill's URL table, but no bundle files exist and the inlined consumption matrix/checklist could arguably live in a reference file; structure is good with minor organization gaps.	4 / 5
	Total	18 / 20 Passed

Description

83%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A specific, distinctive description that clearly conveys the skill's narrow purpose around model validation and hallucination avoidance. It would benefit from an explicit 'Use when...' trigger clause and a few more natural synonyms.

Suggestions

Add an explicit trigger clause such as 'Use when verifying or auditing provider model entries, or before trusting pricing/capability claims in models.ts'.

Include a couple of natural synonyms users might say (e.g., 'check model pricing', 'audit provider models') to broaden trigger coverage.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions — 'Validate a model entry (or every model in a provider)', 'reports what cannot be verified' against live API docs — describing a precise, comprehensive capability set.	5 / 5
Completeness	Clearly states the 'what' (validate model entries against live API docs) and implies the 'when' (when verifying a provider's models for hallucinated pricing/capabilities), but the 'when' is implicit rather than an explicit 'Use when...' trigger clause.	4 / 5
Trigger Term Quality	Includes solid natural terms ('validate', 'model entry', 'provider', 'live API docs', 'no hallucination') but lacks common synonyms or file extensions a user might say; coverage is good but a few natural variants are missing.	4 / 5
Distinctiveness Conflict Risk	Targets a very specific niche — auditing entries in apps/sim/providers/models.ts against provider API docs — with distinct triggers unlikely to collide with other skills.	5 / 5
	Total	18 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: simstudioai/sim
Path: .agents/skills/validate-model/SKILL.md
Commit: 417ae20

Reviewed: 3 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.