CtrlK
BlogDocsLog inGet started
Tessl Logo

spark-optimization

Optimize Apache Spark jobs with partitioning, caching, shuffle optimization, and memory tuning. Use when improving Spark performance, debugging slow jobs, or scaling data processing pipelines.

84

1.28x
Quality

Does it follow best practices?

Impact

77%

1.28x

Average score across 3 eval scenarios

SecuritybySnyk

Passed

No known issues

SKILL.md
Quality
Evals
Security

Quality

Content

65%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable with executable, copy-paste-ready code, but it is a monolithic ~410-line document that repeats the description and would benefit from splitting detailed patterns into reference files. It lacks explicit validation checkpoints in its workflows.

Suggestions

Split the seven detailed code patterns and the configuration cheat sheet into separate reference files (e.g., PATTERNS.md, CONFIG.md) and link them one level deep from the SKILL.md overview to improve progressive disclosure.

Remove the redundant introductory paragraph and 'When to Use This Skill' list since they restate the frontmatter description, tightening conciseness.

Add explicit validation checkpoints (e.g., verify partition sizes, check the Spark UI for spills/skew after applying a pattern) to the pattern sequences so multi-step operations have feedback loops.

DimensionReasoningScore

Conciseness

The body is mostly efficient executable code, but the introductory paragraph and 'When to Use This Skill' list restate the description, and some prose could be trimmed; not fully lean, so it sits below the score-3 anchor.

2 / 3

Actionability

Extensive copy-paste-ready PySpark code across all seven patterns — SparkSession configs, broadcast/bucket/salt joins, StorageLevel caching, memory tuning, and monitoring helpers — matches the fully-executable score-3 anchor.

3 / 3

Workflow Clarity

The content is organized as a pattern catalog with implicit read-transform-write ordering, but there is no explicit multi-step workflow with validation checkpoints or feedback loops, so it does not reach the score-3 anchor.

2 / 3

Progressive Disclosure

No bundle files exist and the ~410-line body keeps all seven detailed patterns and the config cheat sheet inline with no external references; sections are well-organized but content that should be split remains in one file.

2 / 3

Total

9

/

12

Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is strong: it states concrete capabilities, uses natural trigger phrasing, and explicitly pairs 'what' with 'when'. It is distinguishable from other skills and uses correct third-person voice.

DimensionReasoningScore

Specificity

Names the Spark domain and lists multiple concrete capabilities — 'partitioning, caching, shuffle optimization, and memory tuning' — matching the multiple-specific-actions anchor rather than the partial score-2 anchor.

3 / 3

Completeness

Explicitly answers both what ('Optimize Apache Spark jobs with partitioning, caching, shuffle optimization, and memory tuning') and when via a 'Use when...' clause with multiple triggers, satisfying the score-3 anchor.

3 / 3

Trigger Term Quality

Natural user phrasings like 'improving Spark performance', 'debugging slow jobs', and 'scaling data processing pipelines' give good coverage of terms a user would actually say, not just technical jargon.

3 / 3

Distinctiveness Conflict Risk

Apache Spark optimization is a clear niche with distinct triggers unlikely to overlap with unrelated skills; no first/second-person voice present to penalize.

3 / 3

Total

12

/

12

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository
Dicklesworthstone/pi_agent_rust
Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.