CtrlK
BlogDocsLog inGet started
Tessl Logo

spark-python-data-source

Build custom Python data sources for Apache Spark using the PySpark DataSource API — batch and streaming readers/writers for external systems. Use this skill whenever someone wants to connect Spark to an external system (database, API, message queue, custom protocol), build a Spark connector or plugin in Python, implement a DataSourceReader or DataSourceWriter, pull data from or push data to a system via Spark, or work with the PySpark DataSource API in any way. Even if they just say "read from X in Spark" or "write DataFrame to Y" and there's no native connector, this skill applies.

70

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

72%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-organized, mostly lean skill body with excellent progressive disclosure and concrete examples. The main weakness is workflow clarity: batch/write operations to external systems would benefit from an explicit validation checkpoint in the primary build flow.

Suggestions

Add an explicit validate-checkpoint to the main workflow (e.g., after implementing the writer, run `uv run pytest` against the registered source and only proceed once tests pass) so the build flow satisfies the batch/destructive validation requirement.

Consolidate the trailing References section with the inline links earlier in the body to remove the duplicated reference listing and tighten the token budget.

Inline a minimal complete reader or writer skeleton (beyond the test snippet) so the core actionability stands on its own without requiring the external template.

DimensionReasoningScore

Conciseness

Mostly lean and efficient — it explicitly avoids repeating general Python best practices and assumes Spark/Python competence; the trailing References section re-lists links already introduced inline, which is mild redundancy keeping it just below a 5.

4 / 5

Actionability

Provides concrete, executable guidance (uv init/add commands, a copy-paste test fixture, a full skeleton in implementation-template.md); the inline test example is runnable but the core build steps rely on the external skeleton for completeness rather than standing alone.

4 / 5

Workflow Clarity

Sections convey the build sequence (setup, implement, test) but the main flow lacks an explicit validation checkpoint before a write to an external system; because this is a batch/write skill, the missing validate-then-proceed loop caps workflow_clarity at 3 per the rubric.

3 / 5

Progressive Disclosure

Clear overview with well-signaled one-level-deep references — each of the 8 reference files is real, linked with 'read when' guidance, and content is appropriately split so SKILL.md stays an index.

5 / 5

Total

16

/

20

Passed

Description

100%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, well-constructed description that clearly states capabilities and provides explicit, natural-language trigger guidance. It avoids vagueness and buzzwords while remaining concise enough not to be padded.

DimensionReasoningScore

Specificity

Lists multiple concrete actions — batch and streaming readers/writers, connecting Spark to external systems, building connectors/plugins, implementing DataSourceReader/Writer, pulling/pushing data — giving comprehensive coverage of what the skill does.

5 / 5

Completeness

Explicitly answers both what ('Build custom Python data sources for Apache Spark using the PySpark DataSource API') and when ('Use this skill whenever someone wants to connect Spark to an external system...') with concrete trigger phrases.

5 / 5

Trigger Term Quality

Rich natural trigger phrasing ('connect Spark to an external system (database, API, message queue, custom protocol)', 'read from X in Spark', 'write DataFrame to Y', 'no native connector') covers the synonyms and phrasings users would actually say.

5 / 5

Distinctiveness Conflict Risk

Targets a clear niche (PySpark DataSource API, custom Python connectors) with distinct triggers and minimal overlap with other skills; third-person voice is correctly used throughout.

5 / 5

Total

20

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository
databricks/databricks-agent-skills
Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.