orchestra-skills

Agent skills and reference docs for diagnosing, fixing, and triaging Orchestra data pipelines with an AI assistant. The workflows assume Orchestra's cloud MCP server is connected so the agent can list runs, fetch logs and artifacts, and retry pipelines from your workspace.

This repo is a plugin marketplace with two plugins, installed independently into both Claude Code and Cursor from the manifests at the repo root (see Install):

• orchestra — diagnose/fix/triage runs, author pipeline YAML, dbt Slim CI, data-quality tests, account health.
• migrate-to-orchestra — convert pipelines from another orchestrator (Dagster, Prefect, and Airflow) into Orchestra pipeline YAML.

What is in this repo

Skills

Each skill auto-triggers when your prompt matches it — just describe the problem in natural language. The "Try saying" column shows a prompt that activates each one. Skills are grouped below by what they're for.

Diagnose & fix pipelines

Author & maintain pipelines

Account health & governance

dbt state-aware orchestration

Data-quality testing

To get going: connect Orchestra's cloud MCP server (see Install below), install the orchestra plugin so the skills are discoverable by your client (see Install), then just ask.

Migrate to Orchestra

A separate plugin (migrate-to-orchestra) for converting pipelines from another orchestrator into Orchestra pipeline YAML. Point your client at the source project (Dagster, Prefect, or Airflow code) and describe what you want migrated — each skill auto-triggers off the APIs it recognizes. Start with dagster-definitions-to-orchestra (Dagster), prefect-flow-structure-to-orchestra (Prefect), or airflow-dag-structure-to-orchestra (Airflow) for any whole-job/whole-flow/whole-DAG conversion; it establishes the pipeline root that the task-level skills below build on.

Not yet in .tessl-plugin/plugin.json — intentionally excluded from Tessl publishing for now.

Pipeline structure & cross-cutting concerns

Integration & task conversion

Prefect

Integration & task conversion (Prefect)

Airflow

Integration & task conversion (Airflow)

To get going: install the migrate-to-orchestra plugin (see Install), open a Claude/Cursor session in the source Dagster, Prefect, or Airflow project, and describe the migration — no upload step needed, the skills read the project's own source files directly.

Reference library

Start at skills/orchestra/references/orchestra/README.md. Highlights:

• Pipeline — authoring schema + examples, failure classification, remediation playbooks, and an optional local fix-history template (knowledge-store.md)
• State-aware orchestration (dbt SAO) — source-freshness and build_after schemas, enabling use_state_orchestration, and a per-warehouse freshness matrix for Snowflake, BigQuery, Databricks, MotherDuck/DuckDB, Redshift, Microsoft Fabric, and Postgres (plus an other fallback) (dbt-sao/)
• MCP — cloud MCP setup and tool quick reference

Pipeline validation hook

The orchestra plugin installs a PostToolUse hook. After the agent writes or edits a YAML file that is an Orchestra pipeline, it is validated against the public /pipelines/schema endpoint and any errors go straight back to the agent, which fixes them before you ever start a run. Other YAML is left alone.

The hook itself needs no API key, no CLI install and no MCP connection, so it works before a workspace is configured. Set ORCHESTRA_API_KEY and integration connection references get checked too. If the endpoint is unreachable, rate limited, or PyYAML is missing, the hook stays silent and lets the edit stand.

Install for humans

Prerequisites

• An Orchestra API key (Orchestra UI → Settings → API Keys) — needed for the MCP tools and the skills that call them. The pipeline validation hook works without one.

1. Connect Orchestra's cloud MCP server. Point your client at the hosted endpoint following the cloud MCP docs (~/.claude/mcp.json for Claude Code, or Cursor MCP settings) and authenticate with your ORCHESTRA_API_KEY — no local install required. Restart/reload so tools such as list_pipeline_runs and list_task_run_logs appear.
2. Install the plugin(s) you need so the skills are discoverable by your client — the two install independently:
   • Claude Code — add this repo as a marketplace, then install one or both plugins:

     /plugin marketplace add orchestra-hq/orchestra-skills
     /plugin install orchestra@orchestra-marketplace
     /plugin install migrate-to-orchestra@orchestra-marketplace

     (or point at a local clone: /plugin marketplace add /path/to/orchestra-skills).
   • Cursor — add the marketplace and install orchestra and/or migrate-to-orchestra from .cursor-plugin/marketplace.json per Cursor's plugin docs. Each skill auto-triggers from a matching prompt once installed.
3. For agent behavior in this repo, read AGENTS.md.

Typical workflows

Failed run — Paste a pipeline run URL, run UUID, pipeline name, or error snippet. identify-pipeline-error parses the input, loads the pipeline run and failed task runs, identifies the failing task and its cause, then routes to the right fixer: a dbt code issue → fix-pipeline-dbt-task, a Python code/schema issue → fix-pipeline-python-task, an Orchestra-platform/config issue → fix-orchestra-pipeline. Data, vendor/ingestion, auth, network, and other causes are reported with the right next action by identify-pipeline-error itself.

Author pipeline YAML — Describe the desired stages/tasks and create a version: v1 pipeline YAML. The authoring skill validates (via orchestra-cli or MCP) and remediates validation errors until clean.

Review before merge — Use the triage skill when you want a branch fix, validation run, and triage summary, then explicit approval before merge and production retry.

Downstream symptom — Triage can start from a downstream issue (stale dashboard, bad dbt output) and walk upstream through the pipeline graph.

Contributing

• Skills live under skills/orchestra/skills/ (the orchestra plugin) and skills/migrate-to-orchestra/skills/ (the migrate-to-orchestra plugin), each its own plugin bundle with its own .claude-plugin/plugin.json/.cursor-plugin/plugin.json. Shared Orchestra pipeline/schema material lives under skills/orchestra/references/orchestra/ and is the single source of truth for pipeline YAML schema/validation — migration skills should link to it rather than re-deriving schema tables locally, to avoid two sources drifting apart.
• To add a skill, create skills/<plugin>/skills/<skill-name>/SKILL.md with name + description frontmatter, put any supporting references//templates/ in the same folder, and add it to the relevant category table under Skills (or Migrate to Orchestra). The plugin exposes it automatically — bump the version in that plugin's .claude-plugin/plugin.json and .cursor-plugin/plugin.json. Also add its path to the skills array in .tessl-plugin/plugin.json and bump its version — this isn't auto-generated, so a skill left out here silently stops showing up in Tessl's published listing (Tessl has no plugin concept, so both plugins show up in one flat list there — the skills/<plugin>/... path prefix is the only separation). CI (Validate Skills) checks the frontmatter, that SKILL.md stays under ~500 lines, that the manifests are valid JSON, and that every skill on disk lives inside some plugin's skills/ directory. Write skills to be client-agnostic — describe capabilities (e.g. "if your client can schedule a wake-up…") rather than naming a specific tool.
• To add a new orchestrator's migration skills alongside Dagster's, Prefect's, and Airflow's, add them under skills/migrate-to-orchestra/skills/<orchestrator>-*-to-orchestra/ following the existing naming convention — one plugin covers all source orchestrators, so no new plugin/marketplace entry is needed. If a shared/reused skill from another orchestrator's set (e.g. a connections-style skill) doesn't have a same-orchestrator equivalent, author or complete a self-contained one rather than pointing the new orchestrator's skills at another orchestrator's vocabulary.
• Recording fixes is optional and deferred to your client's persistent memory — never commit workspace-specific fix history. Extend pipeline/diagnosis-patterns.md only with generic, reusable patterns.
• Evals. Skill evals live under evals/ — an eval-driven harness that runs a skill with and without it via the headless claude CLI and grades the output. Currently wired up for write-snowflake-dq-tests. See evals/README.md for setup and how to run, grade, and add a suite.
• Do not commit API keys, .env files, or other secrets.

Agents editing this repo should follow AGENTS.md.