CtrlK
BlogDocsLog inGet started
Tessl Logo

databricks-spark-structured-streaming

Comprehensive guide to Spark Structured Streaming for production workloads. Use when building streaming pipelines, working with Kafka ingestion, implementing Real-Time Mode (RTM), configuring triggers (processingTime, availableNow), handling stateful operations with watermarks, optimizing checkpoints, performing stream-stream or stream-static joins, writing to multiple sinks, or tuning streaming cost and performance.

76

Quality

96%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

92%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is concise, actionable, and well-organized with a usable checklist, but its progressive-disclosure design is undercut because none of the referenced detail files are present in the bundle. Resolving the missing references would make the skill complete.

Suggestions

Add the referenced bundle files (kafka-streaming.md, stream-stream-joins.md, stream-static-joins.md, multi-sink-writes.md, merge-operations.md, checkpoint-best-practices.md, stateful-operations.md, trigger-and-cost-optimization.md, streaming-best-practices.md) under a references/ directory so the navigation links resolve.

If any reference is not yet authored, mark it clearly or remove the link rather than leaving dangling references that lead to dead ends.

Consider defining the JSON schema used in the Quick Start example (or noting it as user-supplied) so the code is fully copy-paste ready.

DimensionReasoningScore

Conciseness

The body is lean and assumes Claude's competence: it does not explain what Spark or Kafka is, and navigation tables plus a short Quick Start earn their tokens with no padding.

3 / 3

Actionability

The Quick Start provides a fully-structured executable readStream/writeStream example with concrete options, and the Production Checklist gives specific actionable items; only the user-supplied JSON schema is left as an expected placeholder.

3 / 3

Workflow Clarity

As a navigation/overview skill the single quick-start action is unambiguous, and the Production Checklist supplies explicit validation checkpoints for a complex process; no destructive/batch feedback loop is required here.

3 / 3

Progressive Disclosure

References are well-signaled and one level deep in clean tables, but the bundle contains no references/, scripts/, or assets/ directories, so the nine referenced .md files (e.g. kafka-streaming.md, stateful-operations.md) do not exist and the disclosure does not actually deliver.

2 / 3

Total

11

/

12

Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is strong across all dimensions: it enumerates specific capabilities, provides natural trigger terms, includes an explicit 'Use when' clause, and occupies a clear niche. Third-person/imperative voice ('Use when ...') matches the accepted good examples, so no voice penalty applies.

DimensionReasoningScore

Specificity

Lists many concrete actions such as 'building streaming pipelines', 'working with Kafka ingestion', 'handling stateful operations with watermarks', and 'performing stream-stream or stream-static joins', matching the multiple-specific-actions anchor.

3 / 3

Completeness

Clearly answers what ('Comprehensive guide to Spark Structured Streaming for production workloads') and when via an explicit 'Use when ...' clause enumerating triggers, matching the both-what-and-when anchor.

3 / 3

Trigger Term Quality

Natural terms a Spark/Databricks user would actually say are well covered, including 'Kafka ingestion', 'Real-Time Mode (RTM)', 'watermarks', 'checkpoints', and 'processingTime/availableNow', matching the good-coverage anchor.

3 / 3

Distinctiveness Conflict Risk

The niche is clear and specific to Spark Structured Streaming with distinct triggers (RTM, stream-stream joins, watermark handling) unlikely to fire for unrelated skills.

3 / 3

Total

12

/

12

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

relative_links

Relative link issues: 9 missing

Warning

Total

15

/

16

Passed

Repository
databricks-solutions/ai-dev-kit
Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.