CtrlK
BlogDocsLog inGet started
Tessl Logo

spark-optimization

Optimize Apache Spark jobs with partitioning, caching, shuffle optimization, and memory tuning. Use when improving Spark performance, debugging slow jobs, or scaling data processing pipelines.

64

Quality

76%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./plugins/AI-Agents-Safe-Coding-Skills-claude/skills/spark-optimization/SKILL.md

The canonical home for this skill is spark-optimization in administrakt0r/AI-Agents-Safe-Coding-Skills

SKILL.md
Quality
Evals
Security

Quality

Content

64%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

This skill provides highly actionable, executable Spark optimization patterns with good coverage of key topics (partitioning, joins, caching, memory, shuffles, formats, monitoring). However, it is overly long and monolithic for a SKILL.md, with some redundancy between sections and explanations of concepts Claude already knows. It lacks a diagnostic workflow with validation checkpoints and has a broken reference to a non-existent bundle file.

Suggestions

Add a diagnostic workflow section with explicit steps: 'Check Spark UI → identify bottleneck type → apply specific pattern → verify improvement via metrics', including validation checkpoints.

Move the detailed pattern implementations into a separate file (e.g., `resources/patterns.md`) and keep SKILL.md as a concise overview with pattern summaries and navigation links.

Remove explanations of concepts Claude already knows, such as the Spark execution model diagram, storage level descriptions, and inline comments like '# Spark pushes this down'.

Either create the referenced `resources/implementation-playbook.md` bundle file or remove the broken reference.

DimensionReasoningScore

Conciseness

The skill contains substantial useful content but is verbose for a skill file (~300+ lines). The execution model diagram, storage levels explanation, and some inline comments explain concepts Claude already knows. The configuration cheat sheet partially duplicates configs already shown in patterns. The Do's/Don'ts section restates points already demonstrated in the patterns.

3 / 5

Actionability

Excellent actionability with fully executable Python code examples covering partitioning, joins, caching, memory tuning, shuffle optimization, data formats, and monitoring. Functions like `salt_join`, `check_partition_skew`, and `calculate_partitions` are copy-paste ready with clear parameters and comments.

5 / 5

Workflow Clarity

The skill presents individual optimization patterns clearly but lacks a cohesive workflow for diagnosing and fixing Spark performance issues. There are no validation checkpoints or feedback loops — e.g., no 'run this, check Spark UI for X, if Y then apply pattern Z' sequence. For a skill involving potentially destructive operations like repartitioning production data, the absence of verification steps is notable.

3 / 5

Progressive Disclosure

The skill references `resources/implementation-playbook.md` but no bundle files exist to support it, making this a broken reference. The content is largely monolithic — the 7 detailed patterns, configuration cheat sheet, and best practices could benefit from being split into separate files with the SKILL.md serving as an overview with navigation links.

3 / 5

Total

14

/

20

Passed

Description

87%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

This is a strong skill description that clearly identifies its domain (Apache Spark optimization), lists specific techniques, and includes an explicit 'Use when' clause with natural trigger scenarios. Minor improvements could include additional trigger terms covering common user phrasings like 'OOM', 'data skew', or 'PySpark' to improve discoverability.

DimensionReasoningScore

Specificity

Lists several specific actions: partitioning, caching, shuffle optimization, and memory tuning. These are concrete Spark optimization techniques, though it could be more comprehensive by mentioning specific actions like adjusting partition counts, broadcast joins, or analyzing execution plans.

4 / 5

Completeness

Clearly answers both 'what' (optimize Spark jobs with partitioning, caching, shuffle optimization, memory tuning) and 'when' (improving Spark performance, debugging slow jobs, scaling data processing pipelines) with an explicit 'Use when' clause containing concrete trigger phrases.

5 / 5

Trigger Term Quality

Includes good natural keywords like 'Spark', 'partitioning', 'caching', 'shuffle', 'memory tuning', 'performance', 'slow jobs', and 'data processing pipelines'. Missing some common variations like 'SparkSQL', 'PySpark', 'out of memory errors', 'OOM', 'skew', or 'stage failures' that users might naturally say.

4 / 5

Distinctiveness Conflict Risk

Clearly scoped to Apache Spark optimization specifically, with distinct triggers like 'Spark', 'shuffle optimization', 'partitioning', and 'memory tuning' that are unlikely to conflict with other skills. The niche is well-defined.

5 / 5

Total

18

/

20

Passed

Validation

90%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation10 / 11 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

10

/

11

Passed

Repository
administrakt0r/AI-Agents-Safe-Coding-Skills
Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.