CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

databricks-pipelines

Develop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks. Use when building batch or streaming data pipelines with Python or SQL, including Auto CDC from event streams or periodic complete snapshots. Invoke BEFORE starting implementation.

Quality

92%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

100%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

An exemplary skill body: decision logic, error recovery, and operational sequences are stated with exact commands and API names, versioned legacy content is quarantined in a migration section, and all detail is pushed to a verified, well-labeled one-level-deep reference bundle. Nothing in the body explains concepts Claude already knows, and every section either directs a decision or an action.

DimensionReasoningScore

Conciseness

The body is telegraphic and lean throughout — a compact decision tree, error-message-to-fix mappings ('UNRESOLVED_COLUMN', 'Cannot create streaming table from batch query'), exact syntax fragments, and navigation tables, with zero explanation of concepts Claude already knows. The one piece of time-sensitive/versioned content (legacy DLT syntax) is properly isolated in a dedicated migration section, satisfying the anchor where every token earns its place. Not 4 because no section is padded or could be materially trimmed without losing guidance.

5 / 5

Actionability

Concrete, executable guidance throughout: copy-paste CLI sequences ('databricks bundle validate', 'databricks bundle deploy -t dev', 'databricks pipelines start-update'), exact API mappings ('dlt.apply_changes(...)' → 'dp.create_auto_cdc_flow(...)'), exact syntax ('FROM STREAM(name)', '__END_AT IS NULL', 'dbfs:/Volumes/<catalog>/<schema>/<volume>/<path>'), and behavioral specifics like 'stored_as_scd_type is integer 2 for Type 2'. This matches the fully-executable anchor; per the scoring notes, the absence of a full inline pipeline example is not penalized because actual definitions are deliberately delegated to the language-basics references and all stated commands are runnable.

5 / 5

Workflow Clarity

Multi-step processes are explicitly sequenced with validation checkpoints: 'Sequence: databricks bundle validate, databricks bundle deploy -t dev, databricks bundle run <pipeline_name> -t dev', a dry-run checkpoint ('--validate-only is a faster way to confirm the correctness'), poll-the-update-ID monitoring with cold-start guidance ('don't cancel it'), and the destructive full-refresh operation explicitly gated behind user confirmation. Error-recovery feedback exists via Common Traps mapping specific failure messages to fixes, matching the anchor for clear sequence with explicit validation and recovery loops. Not 4 because no meaningful checkpoint is missing.

5 / 5

Progressive Disclosure

Verified against the actual bundle: all 33 reference paths cited in the body exist in references/ (including the anchored link temporary-view-sql.md#using-expectations-with-temporary-views), are one level deep, and are clearly signaled with per-feature descriptions and mandatory-reading companions. Sibling cross-links between references are targeted section pointers (e.g., auto-loader → options-csv.md), not buried nesting, and spot-checked files contain real content (e.g., options-csv.md's 34-row option table). This matches the anchor for a clear overview with well-signaled one-level-deep references and easy navigation. Not 4 because no inlined content obviously belongs in a separate file — the decision tree and migration table are the core always-needed logic.

5 / 5

Total

20

/

20

Passed

Description

83%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description that states a clear niche, includes the crucial former-product-name synonym (Delta Live Tables), and has an explicit 'Use when...' clause with concrete trigger conditions plus invocation timing. The only gaps are a singular top-level verb and a few missing natural variations (ETL, Auto Loader, DLT abbreviation), which keep specificity and trigger coverage just short of comprehensive.

DimensionReasoningScore

Specificity

Names the domain ('Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks') and several concrete capability facets — 'batch or streaming data pipelines with Python or SQL' and 'Auto CDC from event streams or periodic complete snapshots' — matching the anchor for several specific actions with minor gaps. Not 5 because the action verb is singular ('Develop') and distinct operations like Auto Loader ingestion or expectations are not enumerated; not 3 because coverage goes beyond 1-2 concrete actions with the two Auto CDC modes and dual-language/dual-mode support.

4 / 5

Completeness

Explicitly answers what ('Develop Lakeflow Spark Declarative Pipelines... on Databricks') and when ('Use when building batch or streaming data pipelines with Python or SQL, including Auto CDC from event streams or periodic complete snapshots') with concrete trigger phrases, plus timing guidance ('Invoke BEFORE starting implementation'). This matches the anchor for clear, explicit what-and-when with concrete triggers; it is neither missing the when (3) nor only weakly specific (4).

5 / 5

Trigger Term Quality

Good natural-term coverage: 'batch or streaming data pipelines', 'Python or SQL', 'event streams', 'snapshots', and especially the former product name 'Delta Live Tables', a synonym real users still say. Not 5 because common variations like 'ETL', 'Auto Loader', 'medallion', or the 'DLT' abbreviation itself are absent; not 3 because the included synonyms go well beyond a bare domain mention.

4 / 5

Distinctiveness Conflict Risk

A named product niche (Lakeflow/DLT on Databricks) with qualifiers makes it mostly distinct with minor overlap risk against closely related skills (e.g., the databricks-core parent) or generic pipeline tooling requests. Not 5 because the trigger phrase 'building batch or streaming data pipelines' is broad enough to invite overlap with non-Databricks pipeline requests; not 3 because the product naming and Auto CDC specificity keep it clearly distinguishable.

4 / 5

Total

17

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

15

/

16

Passed

Repository

databricks/databricks-agent-skills

Path

skills/databricks-pipelines/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.