databricks-spark-declarative-pipelines

Creates, configures, and updates Databricks Lakeflow Spark Declarative Pipelines (SDP/LDP) using serverless compute. Handles data ingestion with streaming tables, materialized views, CDC, SCD Type 2, and Auto Loader ingestion patterns. Use when building data pipelines, working with Delta Live Tables, ingesting streaming data, implementing change data capture, or when the user mentions SDP, LDP, DLT, Lakeflow pipelines, streaming tables, or bronze/silver/gold medallion architectures.

Quality

92%

Does it follow best practices?

Impact

—

No eval scenarios have been run

Securityby

Passed

No known issues

Quality

Discovery

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

This is an excellent skill description that hits all the marks. It provides specific concrete actions, comprehensive trigger terms covering both acronyms and full names, an explicit 'Use when...' clause, and a highly distinctive technology niche. The description is well-structured, uses third person voice correctly, and balances conciseness with thoroughness.

Dimension	Reasoning	Score
Specificity	Lists multiple specific concrete actions: creates/configures/updates pipelines, handles data ingestion with streaming tables, materialized views, CDC, SCD Type 2, and Auto Loader ingestion patterns. Very detailed and actionable.	3 / 3
Completeness	Clearly answers both 'what' (creates/configures/updates Databricks Lakeflow pipelines with specific patterns) and 'when' (explicit 'Use when...' clause listing multiple trigger scenarios and terms).	3 / 3
Trigger Term Quality	Excellent coverage of natural terms users would say: 'data pipelines', 'Delta Live Tables', 'streaming data', 'change data capture', 'SDP', 'LDP', 'DLT', 'Lakeflow pipelines', 'streaming tables', 'bronze/silver/gold medallion architectures'. Covers both acronyms and full names.	3 / 3
Distinctiveness Conflict Risk	Highly distinctive with a clear niche in Databricks Lakeflow/DLT pipelines. The specific technology stack (Databricks, Lakeflow, SDP/LDP, Delta Live Tables) and domain-specific terms (medallion architecture, SCD Type 2, Auto Loader) make it very unlikely to conflict with other skills.	3 / 3
	Total	12 / 12 Passed

Implementation

85%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

This is a well-structured, highly actionable skill for a complex domain (Databricks SDP pipelines). Its greatest strengths are the clear workflow routing, executable code examples, comprehensive validation steps with feedback loops, and excellent progressive disclosure to reference files. The main weakness is moderate verbosity—some sections are redundant (task routing appears in multiple forms) and guidance like 'preserve key dimensions' could be trimmed, though the domain-specific content largely justifies its length.

Suggestions

Consolidate the 'Required Checklist' table and 'Task-Based Routing' section to eliminate duplication of the same reference file links appearing in both places.

Trim the 'Gold Layer: Preserve Key Dimensions' section—this is general data modeling advice that Claude can infer; reduce to 1-2 sentences or move to a reference file.

Dimension	Reasoning	Score
Conciseness	The skill is quite long (~300+ lines) with some redundancy—workflow routing is explained multiple times, the task-based routing tables duplicate the checklist table, and sections like 'Gold Layer: Preserve Key Dimensions' include guidance Claude could infer. However, most content is domain-specific (SDP syntax, legacy API mappings, platform constraints) that Claude genuinely needs, so it's not egregiously verbose.	2 / 3
Actionability	The skill provides executable SQL and Python code examples, specific CLI commands with arguments, concrete JSON config examples, and precise tool invocations (e.g., `get_table_stats_and_schema` with parameters). The decision tables for language selection and workflow choice are concrete and copy-paste ready.	3 / 3
Workflow Clarity	The skill has a clear three-option workflow selection (Standalone/Existing Bundle/MCP), a required checklist before writing code, and an explicit three-step post-run validation process with feedback loops (trace upstream, fix, re-upload, re-run). The validation section explicitly states 'you MUST validate' and includes debugging steps for common failure modes.	3 / 3
Progressive Disclosure	The skill is well-structured as an overview with clear one-level-deep references to language-specific guides (sql/1-syntax-basics.md, python/2-ingestion.md, etc.) and general guides (1-project-initialization.md, 2-mcp-approach.md). Navigation is organized by task and language with well-signaled links. The main SKILL.md provides enough context to route correctly without requiring reading all references upfront.	3 / 3
	Total	11 / 12 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 11 / 11 Passed

Validation for skill structure

No warnings or errors.

Repository: databricks-solutions/ai-dev-kit
Commit: 93cb4e3

Reviewed: 1 day ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.