CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

spark-optimization

Optimize Apache Spark jobs with partitioning, caching, shuffle optimization, and memory tuning. Use when improving Spark performance, debugging slow jobs, or scaling data processing pipelines.

Quality

82%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

65%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable, with executable code and specific configuration values throughout, but it is a monolithic reference dump rather than an overview with progressive disclosure: known-concept explanations and duplicated configs add tokens, no diagnostic workflow sequences the patterns, and its only external reference points to a nonexistent file. Restructuring into reference files and adding a validation-driven tuning workflow would raise the weakest dimensions.

Suggestions

Fix or remove the dangling reference to `resources/implementation-playbook.md` — the file and its directory do not exist in the bundle.

Move the seven detailed patterns and the configuration cheat sheet into separate reference files (e.g., references/patterns.md) and keep SKILL.md as a concise overview with clearly signaled one-level-deep links.

Add a sequenced diagnostic workflow with validation checkpoints, e.g., check Spark UI / run check_partition_skew → apply the matching pattern (salting, broadcast join, repartition) → re-measure to verify improvement, replacing the generic 'Instructions' section.

DimensionReasoningScore

Conciseness

The bulk is dense, executable code that earns its place, but several sections pad it: "Core Concepts" explains the Spark execution model and a key-factors table Claude already knows, the "Instructions" section is generic filler ("Clarify goals, constraints, and required inputs"), and the "Configuration Cheat Sheet" repeats configs already shown in Pattern 4 and Quick Start. This fits anchor 3 (mostly efficient but includes some unnecessary explanation or could be tightened) rather than 2, since the unnecessary parts are discrete sections, not pervasive padding.

3 / 5

Actionability

Quick Start is copy-paste ready, and every pattern provides executable PySpark code with specific config values (e.g., skew join settings, storage levels, broadcast threshold), plus helper functions like salt_join and check_partition_skew. Common cases are covered concretely, matching the fully executable top anchor.

5 / 5

Workflow Clarity

There is no sequenced diagnostic workflow: the "Instructions" section is vague ("Apply relevant best practices and validate outcomes"), and Pattern 7 provides skew-check code but never sequences it into a 'detect skew → apply salting → verify improvement' loop. This lands at anchor 3 (elements present but checkpoints/sequence implicit) rather than 4, because validation steps exist only as isolated code snippets with no explicit ordering or feedback loop.

3 / 5

Progressive Disclosure

Section structure is good (per-pattern headers, cheat sheet, resources), but the ~420-line body inlines seven full patterns that belong in reference files, and the single reference — "open `resources/implementation-playbook.md`" — is buried in the Instructions list, not a clearly signaled link, and the file does not exist in the bundle (no resources/ directory is present). This matches anchor 3 (some structure, references present but not clearly signaled, content that should be separate is inline); the dangling reference is the main reason it does not reach 4.

3 / 5

Total

14

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is a strong example: third-person, concise, and specific, with explicit 'what' and 'when' clauses and natural trigger phrases. Only minor keyword variations (e.g., 'tuning', 'PySpark', 'data skew') are missing from the trigger list.

Suggestions

Add common trigger variations such as 'tuning', 'PySpark', or 'data skew' to the 'Use when' clause to broaden natural keyword coverage.

DimensionReasoningScore

Specificity

"Optimize Apache Spark jobs with partitioning, caching, shuffle optimization, and memory tuning" lists four specific concrete techniques (partitioning, caching, shuffle optimization, memory tuning), matching the anchor for comprehensive coverage of concrete actions. It does not fall to 4 because there are no notable gaps in coverage of the domain's core actions.

5 / 5

Completeness

It explicitly answers 'what' ("Optimize Apache Spark jobs with partitioning, caching, shuffle optimization, and memory tuning") and 'when' ("Use when improving Spark performance, debugging slow jobs, or scaling data processing pipelines") with concrete trigger phrases, exactly matching the top anchor and the good overall example pattern. Not 4, because the 'when' clause is explicit and specific rather than improvable.

5 / 5

Trigger Term Quality

"improving Spark performance, debugging slow jobs, or scaling data processing pipelines" covers natural phrases users would say, but common variations like "tuning", "PySpark", "slow query", or "data skew" are missing. It exceeds anchor 3 (only some relevant keywords) but is not the comprehensive synonym/extension coverage of anchor 5.

4 / 5

Distinctiveness Conflict Risk

"Apache Spark jobs" carves out a clear niche with distinct triggers (Spark performance, slow Spark jobs, data pipelines at Spark scale); it would not plausibly trigger for non-Spark data-processing or general performance skills. Minimal conflict risk matches the top anchor.

5 / 5

Total

19

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

15

/

16

Passed

Repository

administrakt0r/AI-Agents-Safe-Coding-Skills

Path

skills/spark-optimization/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.