CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

validate-model

Validate a model entry (or every model in a provider) in apps/sim/providers/models.ts against the provider's live API docs (no hallucination — reports what cannot be verified)

Quality

82%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

SKILL.md
Quality
Evals
Security

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

An exceptionally actionable, well-sequenced auditing procedure: exact commands, a mandatory report format, confirmation gates, and re-validation loops throughout. The only weaknesses are mild — redundant restatement of the unverified rule and cross-skill references keyed to another skill's step numbers rather than stable anchors.

Suggestions

Consolidate the unverified-handling rule into one canonical place (e.g., the severity definitions) — it currently appears in Hard rules 1–2, the ❓ checklist status, and a dedicated closing section, which costs tokens without adding guidance.

Replace references to another skill's step numbers ('add-model/SKILL.md, Step 1' / 'Step 2') with stable anchors or file paths (e.g., a URL-table file or a named section) so the references survive edits to the other skill.

The checklist rows for thinking.streamed and hosted/billing pack multiple conditions into single bullets; splitting or shortening them would tighten the densest section without losing content.

DimensionReasoningScore

Conciseness

Every section is task-specific — hard rules, a per-field checklist, a report template, severity definitions — with zero explanation of concepts Claude already knows, so it clears the 'efficient' anchor. It misses 5 because the unverified-handling rule is restated four times (Hard rules 1–2, the ❓ checklist status, the severity definitions, and the entire closing 'What "I cannot verify this" looks like' section), which is trimmable redundancy.

4 / 5

Actionability

Guidance is copy-paste executable throughout: exact commands ('bun run lint', 'bun run generate:agent-stream-docs'), exact paths ('.agents/skills/add-model/SKILL.md'), a fully filled-in report template with example rows and statuses, and a concrete per-field checklist. The common cases (price drift, dead flags, stale updatedAt) are all covered with specific examples.

5 / 5

Workflow Clarity

Six sequenced steps with an explicit per-model checklist, a mandatory report format, a confirmation gate before destructive edits ('print the diff first', 'Get user confirmation'), and feedback loops after fixing ('Run bun run lint', 'Re-run only the failed rows of the checklist'). This matches the top anchor including validation steps, error-recovery loops, and checklists.

5 / 5

Progressive Disclosure

The bulk reference material (canonical URL table, consumption matrix with re-grep commands) is correctly split into the referenced add-model skill — one level deep and clearly signaled — and the remaining sections are well-organized and easy to navigate. It stops short of 5 because the references point at another skill's internal step numbers ('add-model/SKILL.md, Step 1' / 'Step 2') rather than stable anchors, a fragile organization gap, and the skill ships no bundle structure of its own.

4 / 5

Total

18

/

20

Passed

Description

70%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A highly specific, distinctive description that names its exact target file and scope, but it omits any 'Use when…' trigger guidance and misses the natural keywords (pricing, capabilities) that describe most of the work it does. Adding an explicit trigger clause and a keyword or two about what gets validated would lift it substantially.

Suggestions

Add a 'Use when…' clause (e.g., 'Use when adding or updating model entries in models.ts, or when pricing/capability drift is suspected') to satisfy the missing trigger guidance that caps completeness at 3.

Mention the concrete fields being validated — pricing, context window, capabilities — so users searching for 'check model pricing' or 'fix stale capability flags' naturally hit this description.

Optionally note the outcome (report of critical/warning/unverified findings) so the description previews the deliverable, improving action coverage without padding.

DimensionReasoningScore

Specificity

The description pins a concrete target ("a model entry (or every model in a provider) in apps/sim/providers/models.ts") and concrete behaviors ("validate… against the provider's live API docs", "reports what cannot be verified"), so it sits at the several-specific-actions anchor. It falls short of 5 because coverage is narrow — the actual work (pricing, capability, context-window checks) is not mentioned — and above 3 because the file path and scope are far more concrete than 'Processes PDF files and extracts content'.

4 / 5

Completeness

The 'what' is clear and specific (validate model entries against live docs with unverified reporting), but there is no 'Use when…' clause or equivalent explicit trigger guidance, which the judging guidelines cap at 3. It is not 2 because the 'what' is fully present and concrete; it cannot be 4+ without an explicit 'when'.

3 / 5

Trigger Term Quality

Natural phrases a user would say are present: "Validate a model entry", "every model in a provider", "live API docs", "no hallucination". It is below 5 because common terms for this task — 'pricing', 'capabilities', 'check model specs' — are missing, and above 3 because the included keywords go beyond generic single-domain mention.

4 / 5

Distinctiveness Conflict Risk

Naming the exact repository file "apps/sim/providers/models.ts" and the provider-docs comparison gives this a clear niche with distinct triggers; no other skill would plausibly intercept these invocations, matching the minimal-conflict-risk anchor exactly.

5 / 5

Total

16

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

15

/

16

Passed

Repository

simstudioai/sim

Path

.agents/skills/validate-model/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.