CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/skill-optimizer

Optimize your skills and tiles: review SKILL.md quality, generate eval scenarios, run evals, compare across models, diagnose gaps, and re-run until scores improve.

91

1.10x
Quality

91%

Does it follow best practices?

Impact

92%

1.10x

Average score across 25 eval scenarios

SecuritybySnyk

Passed

No known issues

Overview
Quality
Evals
Security
Files

task.mdevals/scenario-6/

Eval Kickoff Plan for invoice-processor Tile

Problem Description

The engineering team at Fieldstone has built an invoice-processor tile containing several specialized skills for handling different parts of their invoice automation pipeline. The tile is new — no evals have been run against it yet. Before investing hours of compute time in scored evals, the tech lead wants a clear plan for how to kick off evaluation in the right order.

The team has heard that running the wrong type of eval first can waste time (e.g., running scored evals against scenarios that route to the wrong skill). They want a concrete plan — ideally a shell script — that:

  1. Detects what kind of tile they're working with
  2. Chooses the right eval strategy based on what's found
  3. Describes what to do after the first round of results comes in

The tile is already present on disk at ./invoice-processor/.

Output Specification

Produce a file called eval-kickoff-plan.sh (a shell script) and a short eval-notes.md explaining the reasoning behind the eval ordering strategy chosen.

The script should:

  • Detect the number of skills in the tile
  • Run the appropriate eval type first given the tile structure and the absence of existing results
  • Show what commands to run after the first results are available

The notes file should explain:

  • Why the chosen eval type was run first
  • What to do after those results come back
  • Under what circumstances you would skip the first eval type entirely

evals

README.md

tile.json