Validate an existing Sim integration (tools, block, registry, and resolved-secret/model-input boundaries) against the service's API docs and Sim execution conventions
61
73%
Does it follow best practices?
Run evals on this skill
Adds up to 20 points to the overall score
View guide
Low
Low-risk findings worth noting
Fix and improve this skill with Tessl
tessl review fix ./.agents/skills/validate-integration/SKILL.mdYou are an expert auditor for Sim integrations. Your job is to thoroughly validate that an existing integration is correct, complete, and follows all conventions.
When the user asks you to validate an integration:
Read every file for the integration — do not skip any:
apps/sim/tools/{service}/ # All tool files, types.ts, index.ts
apps/sim/blocks/blocks/{service}.ts # Block definition
apps/sim/tools/registry.ts # Tool registry entries for this service
apps/sim/blocks/registry-maps.ts # Block + meta registry entry (BLOCK_REGISTRY / BLOCK_META_REGISTRY)
apps/sim/components/icons.tsx # Icon definition
apps/sim/lib/auth/auth.ts # OAuth config — should use getCanonicalScopesForProvider()
apps/sim/lib/oauth/oauth.ts # OAuth provider config — single source of truth for scopes
apps/sim/lib/oauth/utils.ts # Scope utilities, SCOPE_DESCRIPTIONS for modal UI
packages/deployment-config/src/env-capabilities.ts # OAuth client runtime capability source of truth
apps/sim/lib/core/config/env.ts # Runtime env schema for capability fields
packages/sim-setup/src/capability-config.ts # Exhaustive CLI input-mode mapping for OAuth fields
packages/deployment-config/src/integrations.json # Generated client-safe integration catalog
packages/deployment-config/src/service-account-providers.generated.ts # Generated provider-ID facts
packages/deployment-config/src/service-account-metadata.ts # Handwritten deployment policyIf the block, its triggers, or connector fields use a selectorKey, also apply the validate-selector skill and read
the key's entry in apps/sim/lib/selectors/manifest.ts, its server attachment and provider listing
primitive, and the shared context builder. There is no client provider selector registry.
Fetch the official API docs for the service. This is the source of truth for:
If the official docs do not clearly show the response JSON shape for an endpoint, you MUST tell the user instead of guessing.
If a response schema is unknown, the validation must explicitly call that out and require:
For every tool file, check:
snake_case: {service}_{action} (e.g., x_create_tweet, slack_send_message)name is human-readable (e.g., 'X Create Tweet')description is a concise one-liner describing what it doesversion is set ('1.0.0' or '2.0.0' for V2)required: truerequired: falserequired: true or required: false — never omitted'string', 'number', 'boolean', 'json')'hidden' — ONLY for OAuth access tokens and system-injected params'user-only' — for API keys, credentials, and account-specific IDs the user must provide'user-or-llm' — for everything else (search queries, content, filters, IDs that could come from other blocks)description that explains what it doesAuthorization: Bearer ${params.accessToken}Content-Type header is set for POST/PUT/PATCH requests.trim()-ed to prevent copy-paste whitespace errors`https://api.service.com/v1/${params.id.trim()}`await response.json())data.data vs data vs data.results)?? null?? []optional: true is set on fields that may not exist in all responsestype: 'json' and the shape is known, properties defines the inner fields (tool outputs only — block outputs do not support properties)type: 'array', items defines the item structure with properties (tool outputs only)XCreateTweetParams)ToolResponse)? in the interface (e.g., replyTo?: string)XTweetResponse shared across tweet tools)export * from './types')For every request field, determine whether it is ordinary API input, model-visible text/structured content, opaque model input, or a value persisted into Sim-owned durable storage.
Treat model-input provenance as opt-in. Require official documentation or an unambiguous local execution path proving that the exact field reaches an AI model. If the evidence is ambiguous, leave the integration unchanged; do not infer a model boundary merely from natural-language, search, extraction, or "AI-powered" marketing terminology.
request.modelInput with mode: 'project' and a
minimal exact selector; nested/JSON-string adapters preserve shape through applyProjectedrequest.modelInput, projected before the existing formatter parses it, and has deterministic
formatter behavior when a whole-value placeholder is invalid for the serialized grammarprivateProvenance (or mode: 'private-provenance'), and the route validates
validateOpaqueModelInputProvenance before model egress; storage keys, paths, signed URLs,
and ordinary remote URLs are not treated as byte provenance, while tracked stored bytes are
authorized independently at the owning model-egress boundaryfile_writerequest.secretProvenance; authenticated receivers validate the exact selection
and scope, strip private metadata, and persist, import, or propagate it at the owning boundaryoperation.modelInput / operation.secretProvenance, while proven
model-visible external fields use request projection and other external inputs remain unchangedtransformResponse or tool-local helper blanket-sanitizes ordinary third-party results;
only execution-scoped, activated Sim provenance is projected at shared model/log boundaries{{...}} resolution path and a later
persistence/model/log crossing; there is no generic handling for arbitrary filenames,
metadata, provider results, or API payloads{{NAME}} projection, unproven identical public text, nested and serialized
shape handling, unchanged ordinary external inputs, malformed/incomplete metadata, headerless
legacy requests, metadata stripping, and durable legacy/stale/scope cases when applicableTreat a missing or bypassed model, durable, or internal-execution provenance boundary as critical. Do not fix it with a tool-specific string replacer or by sanitizing every provider result; repair the shared request, authenticated internal-route, persistence, or re-entry boundary that owns the data.
This is the most important validation — the block must be perfectly aligned with every tool it references.
For each tool in tools.access:
tools.config.tool function correctly maps to it)accessToken) has a corresponding subBlock input that is:
condition)required: true (or conditionally required)id values are unique across the entire block — no duplicates even across different conditionstools.config.tool function returns the correct tool ID for every possible operation valuetools.config.params function correctly maps subBlock IDs to tool param names when they differtools.access{ field: 'operation', value: 'x_create_tweet' }{ field: 'operation', value: ['x_create_tweet', 'x_delete_tweet'] }{ field: 'operation', value: 'delete', not: true }{ field: 'op', value: 'send', and: { field: 'type', value: 'dm' } }dependsOn is set for fields that need other values (selectors depending on credential, cascading dropdowns)dropdownshort-inputlong-inputswitch (a Yes/No dropdown only when the tool needs a third "unset" state)oauth-input with correct serviceIdvalue: () => 'default' is set for dropdowns with a sensible defaultmode: 'advanced':
mode: 'advanced'mode: 'advanced'wandConfig with generationType: 'timestamp'wandConfig with a descriptive promptwandConfig with format examples in the promptwandConfig prompts end with an explicit Return ONLY the <format> instruction so the generated value can be pasted directly into the fieldwandConfig.placeholder describes what to type in natural languagetools.access lists every tool ID the block can use — none missingtools.config.tool returns the correct tool ID for each operationtools.config.params (runs at execution time), NOT in tools.config.tool (runs at serialization time before variable resolution — coercing there destroys dynamic references like <Block.output>)tools.config.params handles:
Number() conversion for numeric params that come as strings from inputsBoolean / string-to-boolean conversion for toggle paramsundefined conversion for optional dropdown values'string', 'number', 'boolean', 'json')type: 'json' outputs describe inner fields in the description string: 'User profile (id, name, username, bio)' or '[{address, status, type}]' for arraysproperties: {...} field on block outputs. Block-level OutputFieldDefinition (from @sim/workflow-types/blocks) only accepts { type, description?, condition?, hiddenFromDisplay? }. Nested properties is a tool-level construct (OutputProperty) — adding it to a block output will fail TypeScript at build timetype: 'json' with vague descriptions like 'Response data'condition if supported, or document which operations return themtype is snake_case (e.g., 'x', 'cloudflare')name is human-readable (e.g., 'X', 'Cloudflare')description is a concise one-linerlongDescription provides detail for docsdocsLink points to 'https://docs.sim.ai/integrations/{service}'category is 'tools'bgColor uses the service's brand color hexicon references the correct icon component from @/components/iconsauthMode is set correctly (AuthMode.OAuth or AuthMode.ApiKey)blocks/registry-maps.ts (BLOCK_REGISTRY / BLOCK_META_REGISTRY) alphabetically{Service}BlockMeta is exported in the same file as the blockicon, title, prompt, modules, category, and tagsalsoIntegrations is set on any template whose prompt references another serviceskills present (3–5 mainstream, 2–3 niche), each grounded in tools.access — flag any skill implying an unsupported actionname (≤64 chars, unique), a one-line description, and markdown content with # Title + ## Steps + an output/guidance sectioninputs section lists all subBlock params that the block acceptscanonicalParamId, inputs list the canonical ID (not the raw subBlock IDs)selectorKey is classified in the browser-safe manifest and has exactly one
server attachmentdependsOn context and matches list/search/detail,
pagination, scope, and stale-time behavior{{KEY}} references remain unresolved in the browserfixed, credential-bound, or explicitly reviewed
user-controlled destination policyScopes are centralized — the single source of truth is OAUTH_PROVIDERS in lib/oauth/oauth.ts.
lib/oauth/oauth.ts under OAUTH_PROVIDERS[provider].services[service].scopesauth.ts uses getCanonicalScopesForProvider(providerId) — NOT a hardcoded arrayrequiredScopes uses getScopesForService(serviceId) — NOT a hardcoded arrayauth.ts or block files (should all use utility functions)SCOPE_DESCRIPTIONS within lib/oauth/utils.tsThe deployment UI and setup CLI do not infer OAuth client fields from scopes. They resolve the
block's generated oauthServiceId through the shared deployment capability catalog.
oauth-input.serviceIdresolveOAuthClientCapabilityId(serviceId) returns the intended provider capabilityOAUTH_CLIENT_CAPABILITIESapps/sim/lib/core/config/env.tstext or secret entry in OAUTH_CLIENT_SETUP_FIELDS; no CLI naming heuristic is requirednpx sim-setup add integration <capabilityId> is the command emitted by availability; the CLI has only the exhaustive input-mode projection, not a second runtime provider definitionserviceAccountProviderId,
the generated SERVICE_ACCOUNT_PROVIDER_BY_OAUTH_SERVICE_ID[serviceId] has the same provider IDdeploymentRequirement matches how that credential actually works:
omitted for an independent path, 'oauth-client' when it needs the OAuth client fields, or
'preview-gated' when controlled by a preview blockTreat a missing capability as critical: runtime availability intentionally throws instead of silently exposing an unusable integration.
If any tools support pagination:
pagination_token vs next_token vs cursor)nextToken, cursor, etc.) are included in tool outputsmode: 'advanced'If any tool lists, searches, exports, imports, downloads, uploads, paginates, batches, transforms arrays, or reads file/HTTP bodies, read .agents/skills/memory-load-check/SKILL.md and apply it to the integration.
Promise.all fan-outstransformResponse checks for error conditions before accessing dataresponse.ok or status codes)Group findings by severity:
Critical (will cause runtime errors or incorrect behavior):
required flagtools.accessauth.ts that tools needserviceId missing from the deployment capability catalogtools.config.tool returning wrong tool ID for an operationtools.config.tool instead of tools.config.paramsNULL dataWarning (follows conventions incorrectly or has usability issues):
mode: 'advanced'wandConfig on timestamp/complex fieldsvisibility on params (e.g., 'hidden' instead of 'user-or-llm')optional: true on nullable outputstype: 'json' without property descriptions.trim() on ID fields in request URLs?? null on nullable response fieldsgetScopesForService() / getCanonicalScopesForProvider()SCOPE_DESCRIPTIONS within lib/oauth/utils.tsSuggestion (minor improvements):
longDescription or docsLinkwandConfigAfter reporting, fix every critical and warning issue. Apply suggestions where they don't add unnecessary complexity.
Several files are generated from tool and block definitions. Editing a tool or block WITHOUT regenerating them fails CI, so run these before pushing:
bun run tool-metadata:generate # repo root — apps/sim/tools/generated/*
bun run scripts/generate-docs.ts # docs .mdx + deployment-config/integrations.json + docs icons
bun run deployment-config:generate # canonical OAuth registry + catalog → provider-ID facts
bun run integration-catalog:check # registry ↔ committed deployment metadata drift
bun run docs:check # committed docs ↔ what the generator renders today
bun run deployment-config:check # OAuth registry/catalog ↔ provider-ID fact drifttool-metadata:generate — required whenever a tool's outputs, params, or descriptions change. CI enforces this with bun run tool-metadata:check, which fails with "Generated tool metadata is stale". This is the easiest gate to miss, because nothing in the tool file hints that a generated artifact mirrors it.generate-docs — required whenever block metadata changes (bgColor, name, description, operations, outputs). Regenerates the integration .mdx, packages/deployment-config/src/integrations.json, and the docs copy of components/icons.tsx.deployment-config:generate — required for OAuth or service-account changes. Regenerates provider-ID facts from the canonical OAuth registry and integration catalog; special deployment requirements remain handwritten policy.integration-catalog:check — loads the executable block registry, derives visible integration
deployment fields, and compares them with the committed catalog. It catches missing/unexpected
entries and stale auth/service IDs without loading the executable registry in client code.docs:check — check mode of generate-docs.ts: renders every generated docs artifact in
memory and fails listing any committed file that differs. Runs in CI via check:audits.Always diff the regen output before committing — but commit all of it. These generators rewrite
every file they own, so they also true up drift that accumulated on the base branch (pages whose
source changed without a regen). That catch-up is correct output, not a regression: docs:check
fails CI on any page left stale, so reverting swept-in hunks with git checkout -- reintroduces the
failure. Review the diff to confirm each hunk is explained by a real source change (yours or an
upstream PR that skipped regeneration), and investigate anything that looks like content loss — a
page losing a section usually means its source block moved or a generator input broke, not that the
hunk should be reverted.
If an icon changed, apps/sim/components/icons.tsx is the source of truth and apps/docs/components/icons.tsx is its generated mirror — they must end up byte-identical for that component.
After fixing, confirm:
bun run lint passes with no fixes neededbun run integration-catalog:check passesbun run docs:check passesbun run deployment-config:check passesbun run --cwd apps/sim test lib/integrations/availability.server.test.ts passesSCOPE_DESCRIPTIONS within lib/oauth/utils.ts for all scopesserviceId resolves to the intended OAUTH_CLIENT_CAPABILITIES entry and all capability fields exist in the env schema.agents/skills/memory-load-check/SKILL.md when tools list/search/download/import/export/batch data{Service}BlockMeta exported with at least 7 templatesselectors.execute boundarybun run tool-metadata:generate if any tool outputs/params changed, and confirmed bun run tool-metadata:check passesbun run scripts/generate-docs.ts if any block metadata changed, and committed the full generated diff — including stale-page catch-up for other integrations (bun run docs:check fails CI on reverted generator output)bun run lint after fixes7945b29
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.