CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

databricks-pipelines

Develop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks. Use when building batch or streaming data pipelines with Python or SQL, including Auto CDC from event streams or periodic complete snapshots. Invoke BEFORE starting implementation.

72

Quality

91%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

92%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

An excellent hub document for a large 34-file bundle: concrete commands and APIs throughout, clearly sequenced workflows with validation checkpoints and destructive-operation gating, and a clean progressive-disclosure structure where every reference path was verified to exist. The only weakness is minor conciseness redundancy where a few rules are stated in two or three places.

Suggestions

Consolidate the CREATE OR REFRESH guidance: it appears both as a Common Trap ("'CREATE OR REFRESH' vs 'CREATE'") and as the first row of the Common Issues table — keep one and cross-link the other.

Trim one of the three statements of the UNION/Append-Flow rule (decision tree, Common Traps, and the Flow and Sink API table) to a single canonical location with cross-references.

DimensionReasoningScore

Conciseness

The body is dense with non-obvious, high-value specifics (error-to-fix mappings, version gates like "DBR 18.3+", exact decorators and SQL forms) and assumes Claude's competence — no concept padding like "what is streaming". It sits at anchor 4 rather than 5 because of minor trimmable redundancy: the CREATE OR REFRESH convention appears in both Common Traps and the Common Issues table, and the UNION/Append-Flow point is made three times (decision tree, traps, API table).

4 / 5

Actionability

Fully executable, copy-paste-ready guidance throughout: exact CLI commands ("databricks pipelines init --output-dir . --config-file init-config.json", "databricks bundle deploy -t dev --profile <profile>"), exact API names ("@dp.append_flow(once=True)", "dp.create_auto_cdc_from_snapshot_flow()"), exact SQL ("AUTO CDC INTO ... FROM STREAM"), and exact error keys ("__START_AT"/"__END_AT", "error.exceptions[0].message"). This matches the anchor-5 example of specific commands covering the common cases with no gaps.

5 / 5

Workflow Clarity

Sequences are explicit and gated: pick language → choose workflow (A/B/C with defaults) → scaffold → validate → deploy → run → poll that update, with explicit validation steps ("databricks bundle validate", DESCRIBE-before-submit for cluster keys, "Always poll the update, not top-level pipeline state"), error-recovery feedback loops (Common Issues table plus jq error-extraction pointer), and an approval gate on the destructive full-refresh option. The batch/destructive cap at 3 does not apply since validation is present; this fits the anchor-5 checklist-and-recovery pattern.

5 / 5

Progressive Disclosure

A hub-and-reference design executed well: the SKILL.md body holds only routing/decision knowledge and an API index whose every row links the exact (feature, language) reference file; all 34 bundle files exist and are reachable (verified — python-basics.md and sql-basics.md are linked from sibling references rather than orphaned); referenced anchors resolve (e.g., performance.md "### Cluster key data types"); and references are one level deep with a grouped Reference Index. This matches anchor 5 (clear overview, well-signaled one-level-deep references, easy navigation).

5 / 5

Total

19

/

20

Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description that clearly states what the skill does and when to invoke it, with explicit trigger guidance and concrete capability terms including the important former-name synonym. The two gaps are the missing "DLT" abbreviation in the trigger terms and feature coverage that omits sinks, expectations, materialized views, and deployment workflows.

Suggestions

Add the abbreviation "DLT" to the trigger terms (e.g., "(formerly Delta Live Tables, DLT)") since users most commonly refer to this product by that abbreviation.

Broaden the capability coverage slightly beyond CDC — e.g., "including Auto CDC, expectations, and sinks" — so the description reflects the skill's other first-class features without adding much length.

DimensionReasoningScore

Specificity

The description names several concrete actions — "Develop Lakeflow Spark Declarative Pipelines", "building batch or streaming data pipelines with Python or SQL", "Auto CDC from event streams or periodic complete snapshots" — matching the anchor for several specific actions with minor gaps. It falls short of anchor 5 ("comprehensive coverage") because major capabilities the skill body treats as first-class (sinks, expectations, materialized views, DAB deployment/migration workflows) go unmentioned.

4 / 5

Completeness

Both questions are answered explicitly: the "what" ("Develop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks... building batch or streaming data pipelines with Python or SQL, including Auto CDC...") and the "when" ("Use when building batch or streaming data pipelines...", "Invoke BEFORE starting implementation"). This mirrors the anchor-5 example structure with concrete trigger phrases; anchor 4 would require the "when" to be less explicit, which it is not.

5 / 5

Trigger Term Quality

Good natural-term coverage: "Databricks", "Lakeflow", "pipelines", "batch", "streaming", "Python", "SQL", "Auto CDC", plus the former-name synonym "Delta Live Tables". It stops short of anchor 5 because the abbreviation "DLT" — arguably the most common way users refer to this product — is absent, fitting anchor 4 ("good keyword coverage; a few natural terms missing").

4 / 5

Distinctiveness Conflict Risk

The description is pinned to a specific niche via product names ("Lakeflow Spark Declarative Pipelines", "Delta Live Tables", "Databricks", "Auto CDC") that users of this exact product naturally say, giving it a clear niche with distinct triggers and minimal conflict risk — the anchor-5 case. Only a hypothetical sibling Databricks skill could overlap, which is minor.

5 / 5

Total

18

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

15

/

16

Passed

Repository

databricks/databricks-agent-skills

Path

skills/databricks-pipelines/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.