spark-engineer
Use when writing Spark jobs, debugging performance issues, or configuring cluster settings for Apache Spark applications, distributed data processing pipelines, or big data workloads. Invoke to write DataFrame transformations, optimize Spark SQL queries, implement RDD pipelines, tune shuffle operations, configure executor memory, process .parquet files, handle data partitioning, or build structured streaming analytics.
94
100%
Does it follow best practices?
Impact
89%
1.05xAverage score across 6 eval scenarios
Passed
No known issues
Quality
Discovery
100%Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.
This is an excellent skill description that comprehensively covers the Apache Spark domain. It uses third person voice correctly, provides explicit 'Use when' and 'Invoke to' clauses, lists numerous specific actions, and includes a rich set of natural trigger terms that users would actually say when needing Spark help.
| Dimension | Reasoning | Score |
|---|---|---|
Specificity | Lists multiple specific concrete actions: 'write DataFrame transformations', 'optimize Spark SQL queries', 'implement RDD pipelines', 'tune shuffle operations', 'configure executor memory', 'process .parquet files', 'handle data partitioning', 'build structured streaming analytics'. | 3 / 3 |
Completeness | Clearly answers both what (DataFrame transformations, Spark SQL optimization, RDD pipelines, etc.) and when ('Use when writing Spark jobs, debugging performance issues, or configuring cluster settings') with explicit trigger guidance at the start. | 3 / 3 |
Trigger Term Quality | Excellent coverage of natural terms users would say: 'Spark jobs', 'performance issues', 'cluster settings', 'Apache Spark', 'distributed data processing', 'big data', 'DataFrame', 'Spark SQL', 'RDD', 'shuffle', 'executor memory', '.parquet files', 'partitioning', 'structured streaming'. | 3 / 3 |
Distinctiveness Conflict Risk | Very clear niche focused on Apache Spark ecosystem with distinct triggers like 'Spark', 'RDD', 'DataFrame', '.parquet', 'executor memory', 'shuffle operations' that are unlikely to conflict with general data processing or other database skills. | 3 / 3 |
Total | 12 / 12 Passed |
Implementation
100%Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
This is an exemplary skill file that demonstrates best practices across all dimensions. It provides immediately actionable code examples, clear workflow with validation checkpoints, efficient use of tokens without explaining concepts Claude already knows, and well-organized progressive disclosure to reference materials. The constraints section with MUST DO/MUST NOT DO provides clear guardrails for safe operation.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The content is lean and efficient, assuming Claude's competence with Spark concepts. No unnecessary explanations of what Spark is or how distributed computing works—it jumps straight to actionable patterns and code. | 3 / 3 |
Actionability | Provides fully executable PySpark code examples including a complete mini-pipeline, broadcast join, skew handling with salting, and caching patterns. All examples are copy-paste ready with proper imports and realistic configurations. | 3 / 3 |
Workflow Clarity | The core workflow has clear sequencing with explicit validation checkpoints (step 5 includes checking Spark UI for shuffle spill, verifying partition counts, and a feedback loop to return to step 4 if issues detected). The caching example also includes materialization verification. | 3 / 3 |
Progressive Disclosure | Excellent structure with a clear reference table pointing to one-level-deep topic-specific files (spark-sql-dataframes.md, performance-tuning.md, etc.) with explicit 'Load When' guidance. The main skill provides quick-start content while deferring detailed guidance appropriately. | 3 / 3 |
Total | 12 / 12 Passed |
Validation
100%Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.
Validation — 11 / 11 Passed
Validation for skill structure
No warnings or errors.
- Repository
- jeffallan/claude-skills
- Commit
5b76101
- Reviewed
Table of Contents
Is this your skill?
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.