Optimize your skills and tiles: review SKILL.md quality, generate eval scenarios, run evals, compare across models, diagnose gaps, and re-run until scores improve.
91
91%
Does it follow best practices?
Impact
92%
1.10xAverage score across 25 eval scenarios
Passed
No known issues
The engineering team at Fieldstone has built an invoice-processor tile containing several specialized skills for handling different parts of their invoice automation pipeline. The tile is new — no evals have been run against it yet. Before investing hours of compute time in scored evals, the tech lead wants a clear plan for how to kick off evaluation in the right order.
The team has heard that running the wrong type of eval first can waste time (e.g., running scored evals against scenarios that route to the wrong skill). They want a concrete plan — ideally a shell script — that:
The tile is already present on disk at ./invoice-processor/.
Produce a file called eval-kickoff-plan.sh (a shell script) and a short eval-notes.md explaining the reasoning behind the eval ordering strategy chosen.
The script should:
The notes file should explain:
evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10
scenario-11
scenario-12
scenario-13
scenario-14
scenario-15
scenario-16
scenario-17
scenario-18
scenario-19
scenario-20
scenario-21
scenario-22
scenario-23
scenario-24
scenario-25
skills
compare-skill-model-performance
optimize-skill-instructions
references
optimize-skill-performance
optimize-skill-performance-and-instructions