Name: spark-optimization
Rating: 65.60000000000001 (1 reviews)
Author: administrakt0r

spark-optimization

Optimize Apache Spark jobs with partitioning, caching, shuffle optimization, and memory tuning. Use when improving Spark performance, debugging slow jobs, or scaling data processing pipelines.

Quality

78%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./plugins/AI-Agents-Safe-Coding-Skills-claude/skills/spark-optimization/SKILL.md

The canonical home for this skill is spark-optimization in administrakt0r/AI-Agents-Safe-Coding-Skills

Quality

Content

65%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

Highly actionable code catalog with strong, executable Spark optimization patterns, but it is weakened by inlined conceptual explanation, a missing validation workflow for destructive writes, and a broken reference to a non-existent resources/implementation-playbook.md file.

Suggestions

Add an explicit validate→fix→retry feedback loop for destructive/batch write operations (e.g., verify output row counts or schema before .mode('overwrite') writes to S3) to lift workflow_clarity above the cap.

Create the referenced resources/implementation-playbook.md file (or remove the broken reference) and move the bulkier pattern details and config cheat sheet into one-level-deep reference files so SKILL.md stays a lean overview.

Trim concepts Claude already knows — the execution-model diagram, 'storage levels explained' list, and memory breakdown prose — to improve token efficiency.

Dimension	Reasoning	Score
Conciseness	Mostly efficient code-reference material, but it re-explains concepts Claude already knows (the Driver→Job→Stage→Task execution diagram, the full 'storage levels explained' list, and the 8GB memory breakdown), which could be trimmed.	3 / 5
Actionability	Fully executable, copy-paste-ready PySpark code with concrete config values across all seven patterns and a production cheat sheet, covering the common Spark optimization cases.	5 / 5
Workflow Clarity	The 'Instructions' section is generic ('Apply relevant best practices and validate outcomes') with no sequenced checkpoints, and the destructive/batch write operations (repeated .mode('overwrite') to S3) lack validate→fix→retry feedback loops, triggering the workflow_clarity cap at 3.	3 / 5
Progressive Disclosure	Section structure is clear, but ~420 lines of patterns and a full config cheat sheet are inlined into SKILL.md rather than split into reference files, and the one external reference — 'open resources/implementation-playbook.md' — points to a file that does not exist.	3 / 5
	Total	14 / 20 Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, well-structured description that clearly states both capabilities and explicit 'Use when' triggers tied to natural user phrasing. The only minor gap is trigger-term breadth — a few synonyms or alternate phrasings would push trigger_term_quality to the top anchor.

Dimension	Reasoning	Score
Specificity	Names four concrete optimization actions — 'partitioning, caching, shuffle optimization, and memory tuning' — giving comprehensive coverage of the domain, matching the multiple-specific-actions anchor.	5 / 5
Completeness	It explicitly states what it does ('Optimize Apache Spark jobs with...') and when to use it ('Use when improving Spark performance, debugging slow jobs, or scaling...') with concrete trigger phrases, matching the top anchor.	5 / 5
Trigger Term Quality	Natural phrases a user would say are present ('improving Spark performance', 'debugging slow jobs', 'scaling data processing pipelines'), but it lacks synonyms and variant phrasings, stopping short of the comprehensive-coverage anchor.	4 / 5
Distinctiveness Conflict Risk	Apache Spark optimization is a clear, narrow niche with distinct triggers and minimal overlap risk with other skills.	5 / 5
	Total	19 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: administrakt0r/AI-Agents-Safe-Coding-Skills
Path: plugins/AI-Agents-Safe-Coding-Skills-claude/skills/spark-optimization/SKILL.md
Commit: 57c135b

Reviewed: 9 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.