Content
57%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
Rich, concrete, and largely executable Spark optimization reference material, weakened by the absence of any diagnostic workflow with validation checkpoints, no progressive disclosure into reference files, and some re-explanation of concepts Claude already knows.
Suggestions
Add a sequenced diagnostic workflow (inspect Spark UI / stage metrics → identify bottleneck: skew, spill, shuffle → apply the matching pattern → re-measure and verify improvement) so the patterns are applied in a validated order rather than presented as a catalog.
Split the configuration cheat sheet and the monitoring/debugging helpers into reference files (e.g., references/config.md, references/monitoring.py) and link them one level deep, keeping SKILL.md as a lean overview.
Fix the invalid `spark.sql.shuffle.partitions = "auto"` setting (it is an integer config; use AQE's advisoryPartitionSizeInBytes instead) and trim the execution-model diagram and factors table, which restate knowledge Claude already has.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | Largely code-forward and efficient, but spends tokens on concepts Claude already knows: the Driver→Job→Stage→Task execution-model diagram, a Key Performance Factors table that restates the later patterns, and commented storage-level glosses. | 3 / 5 |
Actionability | Nearly all code is copy-paste-ready with concrete config values, but `spark.conf.set("spark.sql.shuffle.partitions", "auto")` is an invalid value for an integer config and would throw at runtime, and the memory monitor uses private `_jsc` APIs — minor executable gaps. | 4 / 5 |
Workflow Clarity | The body is a pattern catalog with no sequenced tuning workflow — no 'diagnose via Spark UI → identify skew/spill → apply pattern → verify improvement' ordering, and no validation checkpoints despite batch/overwrite write operations. | 3 / 5 |
Progressive Disclosure | A single ~420-line SKILL.md with no bundle files and no offloading; the configuration cheat sheet and monitoring/debug scripts clearly belong in separate reference files, though internal section structure is well organized. | 3 / 5 |
Total | 13 / 20 Passed |