CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

databricks-pipelines

Develop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks. Use when building batch or streaming data pipelines with Python or SQL, including Auto CDC from event streams or periodic complete snapshots. Invoke BEFORE starting implementation.

71

Quality

89%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

The canonical home for this skill is databricks-pipelines in databricks/databricks-agent-skills

SKILL.md
Quality
Evals
Security

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

An exemplary SKILL.md for a complex multi-feature domain: dense, trap-oriented, exact-API guidance with a decision tree, error-recovery table, and a well-verified one-level-deep reference bundle. The remaining weaknesses are minor: some cross-section repetition, scattered version pins, and two unreferenced basics files.

Suggestions

Add python-basics.md and sql-basics.md to the Reference Index (or the Language Selection section) so the per-language entry points are discoverable directly from SKILL.md instead of only via 2-rapid-iteration-with-cli.md and dlt-migration.md.

Consolidate cross-section repetition: the full-refresh data-loss warning, Python-only-feature notes, and full-refresh-doesn't-help-for-type-changes each appear in multiple sections — state each once and cross-reference.

Centralize the version constraints (DBR 18.3+ for SQL AUTO CDC FROM SNAPSHOT, CLI >= v1.0.0) in a single compatibility note instead of three inline mentions in the decision tree, Common Traps, and CDC table.

DimensionReasoningScore

Conciseness

The body assumes Claude's competence throughout — no explanations of what Spark, streaming, or medallion architectures are; every line is a decision rule, exact API, trap, or error→fix mapping. It falls short of anchor 5 due to cross-section redundancy (full-refresh danger appears in both Common Traps and Running a Pipeline; Python-only-feature notes appear in Common Traps, the decision tree, and four table rows) and version pins ("DBR 18.3+") repeated inline three times rather than centralized. Not 3, since padding and known-concept explanation are essentially absent.

4 / 5

Actionability

Fully executable guidance: copy-paste-ready CLI blocks ("databricks bundle validate --profile <profile>", "databricks pipelines start-update <pipeline_id>"), exact API names per (feature, language) ("SEQUENCE BY STRUCT(col1, col2)", "to_json(struct(*)) AS value", "@dp.append_flow(once=True)"), a 12-row error→fix table keyed to literal error strings, and precise migration conversions for every legacy DLT construct. Matches anchor 5 ('fully executable; copy-paste ready; specific examples cover the common cases').

5 / 5

Workflow Clarity

Clear multi-stage sequence with explicit checkpoints: workflow choice before scaffolding, "Always deploy before any run", "Always poll the update, not top-level pipeline state" with rationale, and `bundle validate` preceding deploy. Feedback loops for error recovery are present (Common Issues error→fix mappings), destructive operations are gated (full refresh "can lead to data loss... Always suggest it as a follow-up the user must explicitly approve"), and the decision tree plus language-selection table act as checklists. Matches anchor 5; the only validation detail delegated to references (output-data verification, re-upload loop) is signaled with exact links.

5 / 5

Progressive Disclosure

Verified against the actual bundle: all ~25 distinct reference paths cited in the body exist in references/, and all spot-checked deep anchors resolve (performance.md#cluster-key-data-types, 2-rapid-iteration-with-cli.md#step-4..., temporary-view-sql.md#using-expectations..., 1-project-initialization-with-dab.md#workflow-b-... and #running-a-pipeline-workflow-a--b). Structure is one level deep with a categorized Reference Index and per-(feature, language) navigation tables. Anchor 4 rather than 5 because python-basics.md and sql-basics.md — the natural per-language entry points — are not reachable from SKILL.md at all (only linked from 2-rapid-iteration-with-cli.md and dlt-migration.md), a minor navigation gap.

4 / 5

Total

18

/

20

Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: third-person, explicit what-and-when triggers, current and legacy product names, and concrete capability scope including both CDC modes. Its only weaknesses are minor — the ubiquitous "DLT" abbreviation is absent and the capability list (sinks, expectations) is slightly under-sampled.

DimensionReasoningScore

Specificity

"Develop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks" plus "building batch or streaming data pipelines with Python or SQL, including Auto CDC from event streams or periodic complete snapshots" names the domain and several concrete capabilities with clear scope boundaries, but coverage has minor gaps (no mention of sinks, expectations, or views). Fits anchor 4 ('several specific actions; minor gaps') rather than 5, whose example lists four distinct actions; here the core verb is a single 'Develop' qualified by source types.

4 / 5

Completeness

Both questions are explicitly answered: what — "Develop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks"; when — "Use when building batch or streaming data pipelines with Python or SQL", with concrete trigger phrases including the Auto CDC variants. This matches anchor 5 ('clearly and explicitly answers both what AND when with concrete trigger phrases'), not 4, where the 'when' would be less specific.

5 / 5

Trigger Term Quality

Strong natural keywords: "Databricks", "Delta Live Tables" (capturing the legacy name users still say), "batch or streaming data pipelines", "Python or SQL", "Auto CDC", "event streams". Missing the ubiquitous "DLT" abbreviation and common phrasings like "ETL"/"ingestion", so it matches anchor 4 ('good keyword coverage; a few natural terms missing') rather than anchor 5's comprehensive synonym coverage.

4 / 5

Distinctiveness Conflict Risk

The niche is unambiguous: product-specific names ("Lakeflow Spark Declarative Pipelines", "Delta Live Tables", "Databricks") leave minimal conflict risk with other skills; a generic pipeline request would only match if Databricks is in play. Matches anchor 5 ('clear niche with distinct triggers; minimal conflict risk'); the generic phrase "data pipelines" alone would suggest 4, but the product qualifiers dominate the description.

5 / 5

Total

18

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

15

/

16

Passed

Repository

databricks/databricks-agent-skills

Path

plugins/databricks/claude/skills/databricks-pipelines/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.