CtrlK
BlogDocsLog inGet started
Tessl Logo

experiments/eval-improve

Analyze eval results, diagnose low-scoring criteria, fix tile content, and re-run evals — the full improvement loop automated

Does it follow best practices?

Evaluation100%

1.02x

Agent success when using this tile

Validation for skill structure

Overview
Skills
Evals
Files

rubric.jsonevals/scenario-2/

{
  "context": "Tests whether the agent applies minimal, targeted fixes to the tile — adding explicit retry timing rather than vague guidance, matching the rubric's exact language, and leaving every unrelated working section untouched without restructuring or bloating.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Explicit retry intervals",
      "description": "The updated SKILL.md specifies concrete delay values for retries (e.g. '1s, 2s, 4s' or specific millisecond values) rather than vague phrases like 'increasing delays' or 'wait between retries'",
      "max_score": 20
    },
    {
      "name": "Rubric language used",
      "description": "The added retry content uses 'exponential backoff' (the exact term from eval_criteria.json) rather than paraphrases like 'progressive delay' or 'back-off strategy'",
      "max_score": 12
    },
    {
      "name": "HMAC section unchanged",
      "description": "The HMAC-SHA256 signature validation instruction in the Setup section is preserved verbatim or functionally identical",
      "max_score": 9
    },
    {
      "name": "TLS section unchanged",
      "description": "The TLS 1.2 configuration line in the Setup section is not modified or removed",
      "max_score": 7
    },
    {
      "name": "Observability section unchanged",
      "description": "The Observability section (structured log entry with event type, source IP, duration, outcome) is not modified",
      "max_score": 8
    },
    {
      "name": "Processing section unchanged",
      "description": "The four numbered steps in the Processing section are still present in the same order in the updated tile",
      "max_score": 7
    },
    {
      "name": "Retry section only changed",
      "description": "Only the Reliability section (or equivalent retry-specific section) was substantively modified; Setup and Processing sections are structurally intact",
      "max_score": 12
    },
    {
      "name": "Concise addition",
      "description": "The Reliability section (or equivalent) does not exceed 100 words in the updated tile",
      "max_score": 10
    },
    {
      "name": "Max retry count preserved",
      "description": "The 3-attempt maximum retry limit is still present in the updated tile (not removed or changed to a different number)",
      "max_score": 8
    },
    {
      "name": "Fast acknowledgement preserved",
      "description": "The instruction to acknowledge receipt with a 200 response quickly (before heavy processing) is still present",
      "max_score": 7
    }
  ]
}

Install with Tessl CLI

npx tessl i experiments/eval-improve@0.4.0

evals

README.md

tile.json