Analyze eval results, diagnose low-scoring criteria, fix tile content, and re-run evals — the full improvement loop automated
Does it follow best practices?
Evaluation — 100%
↑ 1.02xAgent success when using this tile
Validation for skill structure
{
"context": "Tests whether the agent applies minimal, targeted fixes to the tile — adding explicit retry timing rather than vague guidance, matching the rubric's exact language, and leaving every unrelated working section untouched without restructuring or bloating.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Explicit retry intervals",
"description": "The updated SKILL.md specifies concrete delay values for retries (e.g. '1s, 2s, 4s' or specific millisecond values) rather than vague phrases like 'increasing delays' or 'wait between retries'",
"max_score": 20
},
{
"name": "Rubric language used",
"description": "The added retry content uses 'exponential backoff' (the exact term from eval_criteria.json) rather than paraphrases like 'progressive delay' or 'back-off strategy'",
"max_score": 12
},
{
"name": "HMAC section unchanged",
"description": "The HMAC-SHA256 signature validation instruction in the Setup section is preserved verbatim or functionally identical",
"max_score": 9
},
{
"name": "TLS section unchanged",
"description": "The TLS 1.2 configuration line in the Setup section is not modified or removed",
"max_score": 7
},
{
"name": "Observability section unchanged",
"description": "The Observability section (structured log entry with event type, source IP, duration, outcome) is not modified",
"max_score": 8
},
{
"name": "Processing section unchanged",
"description": "The four numbered steps in the Processing section are still present in the same order in the updated tile",
"max_score": 7
},
{
"name": "Retry section only changed",
"description": "Only the Reliability section (or equivalent retry-specific section) was substantively modified; Setup and Processing sections are structurally intact",
"max_score": 12
},
{
"name": "Concise addition",
"description": "The Reliability section (or equivalent) does not exceed 100 words in the updated tile",
"max_score": 10
},
{
"name": "Max retry count preserved",
"description": "The 3-attempt maximum retry limit is still present in the updated tile (not removed or changed to a different number)",
"max_score": 8
},
{
"name": "Fast acknowledgement preserved",
"description": "The instruction to acknowledge receipt with a 200 response quickly (before heavy processing) is still present",
"max_score": 7
}
]
}