Add a new LLM model to apps/sim/providers/models.ts with specs verified against the provider's live API docs (no hallucination)
58
67%
Does it follow best practices?
Run evals on this skill
Adds up to 20 points to the overall score
View guide
Low
Low-risk findings worth noting
Fix and improve this skill with Tessl
tessl review fix ./.agents/skills/add-model/SKILL.mdYou add a new model entry to apps/sim/providers/models.ts. Every numeric and capability claim MUST be derived from a live web fetch of the provider's official docs in this session. Marketing emails, training data, and your prior knowledge are not sources of truth — they routinely hallucinate pricing, context windows, and capability lists.
apps/sim/providers/{provider}/ actually consumes them (see Consumption Matrix below). Setting a flag the provider ignores is a silent bug.models.ts and match their pattern exactlybun run lint, print the verification reportIn priority order — fetch all that exist for the provider:
| Provider | Models index | Pricing | Reasoning/parameter caveats |
|---|---|---|---|
| OpenAI | platform.openai.com/docs/models | openai.com/api/pricing | platform.openai.com/docs/guides/reasoning |
| Anthropic | docs.anthropic.com/en/docs/about-claude/models | anthropic.com/pricing | docs.anthropic.com/en/docs/build-with-claude/extended-thinking |
| Google (Gemini) | ai.google.dev/gemini-api/docs/models | ai.google.dev/pricing | ai.google.dev/gemini-api/docs/thinking |
| xAI | docs.x.ai/developers/models | docs.x.ai/developers/models (per-model detail page) | docs.x.ai/developers/model-capabilities/text/reasoning |
| Mistral | docs.mistral.ai/getting-started/models/models_overview | mistral.ai/pricing | n/a |
| DeepSeek | api-docs.deepseek.com/quick_start/pricing | same | api-docs.deepseek.com/guides/reasoning_model |
| Groq | console.groq.com/docs/models | groq.com/pricing | n/a |
| Cerebras | inference-docs.cerebras.ai/models | cerebras.ai/pricing | n/a |
Secondary verification (use at least one): openrouter.ai/<provider>/<model>, artificialanalysis.ai/models/<model>, cloudprice.net/models/<provider>-<model>.
Use a precise WebFetch prompt: "Extract for {model_id}: exact model id string, context window in tokens, input price per 1M, cached input price per 1M, output price per 1M, max output tokens, supported reasoning effort levels, accepted parameters (temperature, top_p), release date. Do not fill in fields you cannot find."
| Capability | Honored by | Effect if set elsewhere |
|---|---|---|
temperature | All providers (passed through if set) | Safe but inert on always-reasoning models that reject it |
toolUsageControl | All providers (provider-level, not per-model) | n/a — set on ProviderDefinition, not models |
reasoningEffort | openai/core.ts, azure-openai, anthropic/core.ts (mapped to thinking), gemini/core.ts | Dead on xai, deepseek, mistral, groq, cerebras, openrouter, fireworks, bedrock, vertex unless their core consumes it — re-grep before assuming |
verbosity | openai/core.ts, azure-openai/index.ts only | Dead elsewhere |
thinking | anthropic/core.ts, gemini/core.ts | Dead elsewhere |
thinking.streamed | Docs generator + getThinkingStreamVisibility (models.ts); anthropic/core.ts uses 'summary' to request display: 'summarized' on agent-events runs | Mandatory on Anthropic-family thinking models (agent-stream-docs:check fails without it); other families fall back to provider defaults |
nativeStructuredOutputs | anthropic/core.ts, fireworks/index.ts, openrouter/index.ts | Dead on openai, xai, google, vertex, bedrock, azure-openai, deepseek, mistral, groq, cerebras |
maxOutputTokens | Read by UI + executor for token estimation | Always meaningful — set if provider documents a cap |
computerUse | anthropic/core.ts | Dead elsewhere |
deepResearch | UI flag for routing to deep-research SKUs | Set only on actual deep-research model IDs |
memory: false | Conversation persistence opt-out | Set only when model genuinely cannot maintain history (e.g., deep-research) |
Always re-grep before relying on this table — the codebase moves:
rg "reasoningEffort|reasoning_effort" apps/sim/providers/<provider>/
rg "verbosity" apps/sim/providers/<provider>/
rg "request\.thinking|thinking:" apps/sim/providers/<provider>/
rg "supportsNativeStructuredOutputs|nativeStructuredOutputs" apps/sim/providers/<provider>/Open apps/sim/providers/models.ts, find PROVIDER_DEFINITIONS[<provider>].models, read 2-3 sibling entries. Match field order exactly:
{
id: '<exact-api-id>',
pricing: {
input: <number>,
cachedInput: <number>, // omit if provider doesn't offer caching
output: <number>,
updatedAt: '<today YYYY-MM-DD>',
},
capabilities: {
// only flags the provider actually consumes — see matrix
},
contextWindow: <tokens>,
releaseDate: '<YYYY-MM-DD>',
recommended: true, // only if new flagship; ask user before swapping
speedOptimized: true, // only on smallest/fastest tier
deprecated: true, // only on retired models
}Model id MUST be prefixed: azure/, azure-anthropic/, vertex/, bedrock/, openrouter/. Pricing usually mirrors the upstream provider but verify on the reseller's own pricing page.
Within a family, newest first (matches existing convention: GPT-5.5 above GPT-5.4 above GPT-5.2). Across families, biggest/flagship at top of list.
recommended / speedOptimizedrecommended: true per provider — the current flagship(s).recommended from the previous flagship. Never silently flip it.speedOptimized: true only on the smallest/fastest tier (nano, flash-lite, haiku class).Adding the models.ts entry is most of the job because nearly every consumer is data-driven and picks the model up automatically: the ~40 query helpers in models.ts / providers/utils.ts, the public /models catalog (app/(landing)/models/utils.ts iterates PROVIDER_DEFINITIONS), the agent-block model dropdown, and copilot's isKnownModelId / suggestModelIdsForUnknownModel validation. The touchpoints below are the exceptions — they are not data-driven, so check each one.
getHostedModels() in apps/sim/providers/models.ts returns every model under openai, anthropic, and google:
export function getHostedModels(): string[] {
return [
...getProviderModels('openai'),
...getProviderModels('anthropic'),
...getProviderModels('google'),
]
}So a model added to any of those three providers is automatically served with Sim's rotating hosted key and billed to the workspace via shouldBillModelUsage() (providers/utils.ts). Before you insert:
openai/anthropic/google as-is — that silently enrolls it in hosted billing. Confirm hosting/billing intent with the user. (Precedent: Ollama Cloud is a deliberately separate isReseller provider specifically to stay BYOK-only/never-billed.){PREFIX}_COUNT / {PREFIX}_1..N env vars must be set, or hosted runs fail at execution time.bun run lint does not run tests. A few tests assert specific model IDs and can break or need updating when you touch a hosted or flagship model:
apps/sim/providers/utils.test.ts — asserts membership of getHostedModels() / shouldBillModelUsage()apps/sim/providers/index.test.ts and serializer tests — reference concrete model IDsrg "<new-model-id>|getHostedModels|shouldBillModelUsage" apps/sim/providers/*.test.tsIf anything matches, run the affected provider tests and update assertions as needed.
The Consumption Matrix (Step 2) tells you which capability flags are honored by existing provider code. But if the new model needs net-new request handling that the provider doesn't implement yet — a new beta header (e.g. Anthropic's anthropic-beta structured-outputs header in anthropic/index.ts), a new thinking/reasoning encoding, a Responses-API quirk — you must edit apps/sim/providers/<provider>/core.ts / index.ts. Setting a flag whose behavior isn't implemented is a silent no-op. When you do edit provider code, reuse the shared helpers rather than hand-rolling: streaming responses are assembled via createStreamingExecution (@/providers/streaming-execution) and tool schemas via adaptOpenAIChatToolSchema / adaptAnthropicToolSchema (@/providers/tool-schema-adapter).
streamed visibility + generated docsIf the entry has capabilities.thinking or capabilities.reasoningEffort, it appears in the autogenerated "Streamed thinking and tool calls" table on the Agent block docs page:
anthropic, azure-anthropic) thinking models MUST declare capabilities.thinking.streamed ('full' | 'summary' | 'none'). Verify against Anthropic's current thinking-display and streaming docs: visible thinking returned by the API is summarized, including when Sim opts models whose default display is omitted into display: 'summarized' on agent-events runs, so current Claude thinking models use 'summary'. Use 'full' only if future official API docs explicitly guarantee raw thinking deltas. bun run agent-stream-docs:check (CI) fails if the field is missing.getThinkingStreamVisibility (Gemini/OpenAI → summaries; Bedrock/Meta → none; OpenAI-compatible vendors with documented reasoning fields → full deltas). Set it explicitly only when the model deviates from its family.bun run agent-stream-docs:generate and commit the regenerated apps/docs/content/docs/en/workflows/blocks/agent.mdx — CI diffs it.streamed value (with its source URL) in the verification report when set.models[] array. Use EMBEDDING_MODEL_PRICING / RERANK_MODEL_PRICING in models.ts instead.ProviderId in providers/types.ts, a registry entry in providers/registry.ts, a provider implementation under providers/<id>/ (assemble streaming responses with createStreamingExecution and wrap tool schemas with the @/providers/tool-schema-adapter helpers), an icon in components/icons.tsx, and the PROVIDER_DEFINITIONS block. That is beyond this skill — tell the user.bun run lint
bun run agent-stream-docs:generate # only when the entry has thinking/reasoningEffortLint must pass before reporting done. If lint fails: read the error, fix the syntax/typing issue in the entry you just wrote (do not delete the entry — it's the work product), re-run lint, and note the fix in a "Lint adjustments" line in the verification report. Never report done with lint failing.
End with this exact structure:
### Verification — <model-id>
| Field | Value | Source URL | Status |
|---|---|---|---|
| `id` | `grok-4.3` | https://docs.x.ai/... | ✓ verified |
| `contextWindow` | 1,000,000 | https://docs.x.ai/... + https://openrouter.ai/... | ✓ verified (2 sources agree) |
| `input` | $1.25/M | https://docs.x.ai/... | ✓ verified |
| `cachedInput` | $0.20/M | https://cloudprice.net/... | ⚠️ single source |
| `output` | $2.50/M | https://docs.x.ai/... + https://openrouter.ai/... | ✓ verified |
| `capabilities.temperature` | `{ min: 0, max: 1 }` | matches sibling entries | — pattern-match only |
| `capabilities.reasoningEffort` | NOT SET | provider docs say API rejects it for this model | ✓ correctly omitted |
| `releaseDate` | 2026-04-30 | https://docs.x.ai/... announcement | ✓ verified |
| hosted/billing | BYOK-only (xai not in `getHostedModels`) | `providers/models.ts` | — confirmed intent |
**Disagreements**
- _none_ OR _OpenRouter says X, provider docs say Y — used Y per provider rule_
**Unverified fields**
- _none_ OR _<field>: could not find authoritative source — left as <X> based on sibling pattern; please confirm_If any row is ⚠️ single-source or "unverified," state it plainly to the user and ask whether to proceed. Do not silently merge.
Omitting a field is not the same as verifying it. Any field you cannot confirm from a live fetch must be both omitted from the entry and listed as ❓ UNVERIFIED in the report's "Unverified fields" section, with the URLs you attempted. Then ask the user to confirm before merging.
cachedInput. Mark ❓ UNVERIFIED. Ask the user for the price or the docs URL.reasoning_effort — verified by official docs only)nativeStructuredOutputs: true on xai/openai/google (dead — only anthropic/fireworks/openrouter consume it)thinking on non-Anthropic/non-Gemini providersthinking.streamed (CI agent-stream-docs:check fails), or skipping bun run agent-stream-docs:generate after adding any thinking/reasoning modelverbosity on anything other than OpenAI gpt-5.xpricing.updatedAt from a sibling instead of using today's datecachedInput price by dividing input by 4 (varies by provider — find an explicit number)recommended: true on the new model without removing it from the previous flagshipopenai/anthropic/google (silently enrolls it in hosted billing via getHostedModels())bun run lint when you touched a hosted (openai/anthropic/google) or flagship model with assertions in providers/utils.test.ts6f514c1
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.