CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

databricks-pipelines

Develop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks. Use when building batch or streaming data pipelines with Python or SQL, including Auto CDC from event streams or periodic complete snapshots. Invoke BEFORE starting implementation.

Quality

87%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

The canonical home for this skill is databricks-pipelines in databricks/databricks-agent-skills

SKILL.md
Quality
Evals
Security

Quality

Content

86%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-engineered skill body: an index-style overview that keeps only decision logic, traps, and error mappings inline while pushing per-feature, per-language detail to 37 real, well-named reference files. Commands and syntax are executable, and risky operations (full refresh, DLT migration) carry explicit guardrails. Minor room to tighten a few wordy passages and inline a clearer end-to-end run sequence.

DimensionReasoningScore

Conciseness

The body is dense and almost entirely product-specific knowledge Claude cannot know (SDP naming rules, `CREATE OR REFRESH` vs `CREATE OR REPLACE`, `dbfs:` prefix requirement, `__START_AT__`/`__END_AT__` columns, type restrictions on cluster keys) with no filler or textbook explanation. Not 5 — a few passages could be trimmed, e.g. the "Gold layer: preserve key business dimensions" paragraph is wordy and the language-selection table repeats its Python signals across two rows.

4 / 5

Actionability

Concrete, copy-paste-ready commands and syntax throughout: `databricks pipelines init --output-dir . --config-file init-config.json`, the full bundle validate/deploy/run/status command block, `databricks workspace import-dir ... --overwrite`, exact decorators (`@dp.append_flow(once=True)`), exact SQL forms (`CREATE OR REFRESH STREAMING TABLE`, `AUTO CDC INTO ... FROM STREAM`), and an error→fix table covering the common failure cases.

5 / 5

Workflow Clarity

A coherent path exists: parent-skill bootstrap → decision tree → language selection → workflow choice (A/B/C) → scaffold → deploy/upload → run → poll, with explicit checkpoints (`bundle validate` before deploy, "Always poll the update", full-refresh data-loss warning gated on explicit user approval, error-extraction mapping). Not 5 — the body presents these as linked sections rather than a single end-to-end numbered sequence with feedback loops, delegating step-level detail to the reference files. The destructive-operation validation is present, so the workflow cap at 3 does not apply.

4 / 5

Progressive Disclosure

Excellent structure against the actual bundle: all 37 referenced files in `references/` exist on disk (verified, including the `#cluster-key-data-types` anchor in performance.md), references are one level deep, every API-table row maps a feature to the right per-(feature, language) file, workflows A/B/C each link to their own step-by-step file, and a Reference Index closes the body. The inline tables function as a navigation index rather than bulk content, and detail is appropriately split out.

5 / 5

Total

18

/

20

Passed

Description

83%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: it names the product and its deprecated synonym, states concrete capabilities, and gives an explicit, well-aimed 'Use when' trigger with an invocation-timing directive. The only gaps are the missing "DLT" acronym and slightly broad "batch or streaming data pipelines" phrasing that overlaps adjacent Spark skills.

Suggestions

Add the "DLT" acronym alongside "Delta Live Tables" since users very commonly say "DLT".

Narrow the 'when' clause with a distinguishing qualifier (e.g., "declarative/managed pipelines" rather than "data pipelines" generally) to reduce overlap with plain Spark/structured-streaming skills.

DimensionReasoningScore

Specificity

"Develop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks" plus "building batch or streaming data pipelines with Python or SQL, including Auto CDC from event streams or periodic complete snapshots" lists several specific capabilities (batch/streaming, Python/SQL, both Auto CDC source types). Not 3 — it goes beyond 1-2 concrete actions; not 5 — coverage omits other core capabilities such as expectations, sinks, and materialized views.

4 / 5

Completeness

Explicitly answers both: what — "Develop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks"; when — "Use when building batch or streaming data pipelines with Python or SQL, including Auto CDC from event streams or periodic complete snapshots. Invoke BEFORE starting implementation." Concrete trigger phrases match the top anchor.

5 / 5

Trigger Term Quality

Includes natural terms users would say: "Databricks", "Lakeflow", "Delta Live Tables" (the deprecated synonym), "batch or streaming data pipelines", "Python or SQL", "Auto CDC", "event streams", "snapshots". Not 5 — it misses the ubiquitous acronym "DLT" and casual phrasings like "DLT pipeline".

4 / 5

Distinctiveness Conflict Risk

Clear niche (Databricks' declarative pipeline product) with distinctive trigger terms, but "batch or streaming data pipelines with Python or SQL" is broad and could overlap sibling Spark/structured-streaming skills. Not 5 — minor overlap risk remains; well above 3 since the product name is unambiguous.

4 / 5

Total

17

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

15

/

16

Passed

Repository

databricks/devhub

Path

.databricks/aitools/skills/databricks-pipelines/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.