spark-job-creator

Spark Job Creator - Auto-activating skill for Data Pipelines. Triggers on: spark job creator, spark job creator Part of the Data Pipelines skill category.

0.98x

Quality

Does it follow best practices?

Impact

82%

0.98x

Average score across 3 eval scenarios

Securityby

Passed

No known issues

Optimize this skill with Tessl

npx tessl skill review --optimize ./planned-skills/generated/11-data-pipelines/spark-job-creator/SKILL.md

Evaluation results

77%

-2%

Retail Sales ETL Pipeline

Production-ready ETL Spark job

Criteria

Without context

With context

SparkSession initialization

100%

Externalized configuration

100%

No collect() on full dataset

100%

Parquet output

100%

Output partitioning

100%

Error handling

37%

Logging present

Schema definition

100%

Aggregation correctness

100%

SparkSession stop

100%

Data quality check

12%

Step-by-step structure

100%

Without context: $0.5313 · 2m 34s · 30 turns · 30 in / 7,820 out tokens

With context: $0.5869 · 2m 40s · 34 turns · 289 in / 7,219 out tokens

84%

Real-Time IoT Sensor Anomaly Detector

Spark Structured Streaming pipeline

Criteria

Without context

With context

readStream usage

100%

Explicit schema

100%

Externalized thresholds

100%

Externalized paths

100%

Checkpoint location

100%

writeStream with trigger

100%

Output mode specified

100%

Anomaly filter logic

100%

SparkSession initialization

100%

Design notes present

100%

Logging present

Error handling

Step-by-step structure

100%

Without context: $0.2605 · 1m 12s · 17 turns · 66 in / 3,845 out tokens

With context: $0.5363 · 1m 56s · 31 turns · 96 in / 6,990 out tokens

85%

-1%

Automated Data Warehouse Refresh Pipeline

Airflow-orchestrated Spark workflow

Criteria

Without context

With context

Airflow DAG defined

100%

Three sequential tasks

100%

SparkSubmitOperator or BashOperator for Spark

100%

Externalized config

100%

Date parameter usage

100%

Parquet for staging

100%

Aggregation in transform job

100%

Data quality checks present

100%

SparkSession per job

100%

Logging present

Error handling

Step-by-step job structure

100%

83%

No collect() on full dataset

100%

Without context: $0.7076 · 2m 53s · 28 turns · 70 in / 13,234 out tokens

With context: $0.8026 · 3m 14s · 34 turns · 66 in / 13,059 out tokens

Repository: jeremylongshore/claude-code-plugins-plus-skills
Commit: 0c08951

Evaluated: 17 days ago
Agent: Claude Code
Model: Claude Sonnet 4.6

Table of Contents

Retail Sales ETL Pipeline Real-Time IoT Sensor Anomaly Detector Automated Data Warehouse Refresh Pipeline

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.