Run and extend Phoenix's executable integration documentation checks. Use when changing an integration doc, its runnable example, or the CI harness.
81%
Does it follow best practices?
Run evals on this skill
Adds up to 20 points to the overall score
Passed
No findings from the security scan
The harness runs published Phoenix integration examples against a local Phoenix
server and verifies the result in Phoenix. The enabled set is deliberately
incremental: scripts/testing/integration_docs/integrations.json lists the
documents CI executes. INTEGRATION_CHECKS.md is only an onboarding inventory.
Start Phoenix at http://127.0.0.1:6006, set the credentials required by the
selected integration, then run:
make test-integration-docsRun a single configured document with:
make test-integration-docs INTEGRATION_DOCS_ARGS="--doc docs/phoenix/integrations/llm-providers/openai/openai-tracing.mdx"The runner parses the selected page before checking credentials, installs its
documented dependencies into a disposable environment, executes the example,
and verifies spans using the Phoenix API. Reports are written under
ci-results/integration-docs/. PHOENIX_ENDPOINT can override the local
endpoint.
For Ollama checks, also start Ollama and pull the configured model before
running the harness. The current Ollama entries use llama3.2:1b; Python uses
the native Ollama client, while Node.js and Go use the OpenAI SDK against
Ollama's local OpenAI-compatible endpoint. CI reads each integration's
services declaration, starts Ollama when that shard needs it, and pulls the
declared model. Ollama checks do not require a provider API key.
paths filter starts it only for integration docs, the
routed shared snippet, and harness files. Keep it a superset of
classify_changes.py routing.docs input accepts space-separated paths from integrations.json.integrations.json and
prepared only in the shards assigned to those docs.integrations.json with a stable id, document path,
language, required environment variables, setup rules, output expectations,
and a verifier that checks the page's promised behavior.INTEGRATION_CHECKS.md and document any routing needs in
scripts/skip_config.json.The skip configuration supports AX-style always_skip, lint_only, and
skip_if_missing_env routes. A lint-only document is extracted, its documented
dependencies are installed, and its Python example is bytecode-compiled,
JavaScript is syntax-checked, TypeScript is compiled with tsc, Go is compiled with go vet, or .NET is compiled with dotnet build, without contacting the model provider. Conditional
documents run normally when their configured environment variables are set;
otherwise the report records the configured skip reason.
4. Run the document locally against Phoenix and inspect both the generated
report and the spans in its unique project before enabling it in CI.
Keep credentials in environment variables. Do not put API keys in docs, fixtures, or reports.
Arize-ai/phoenix
fe9cbda
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.