CtrlK
BlogDocsLog inGet started
Tessl Logo

jbaruch/koog

Koog 1.0 idioms, gotchas, and scaffolding skills for Kotlin agents on the JVM

87

1.85x
Quality

88%

Does it follow best practices?

Impact

87%

1.85x

Average score across 45 eval scenarios

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-27/

{
  "context": "Negative case — tests whether the agent recognizes the developer's workflow has a known compile-time topology (three fixed phases, LLM only picks tools within each phase) and routes them to the graph DSL instead of building a planner. A planner here would be over-engineering: extra LLM round-trips for planning decisions the developer already made, with no autonomy benefit.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Names the topology as the disqualifying signal",
      "description": "Identifies in prose that the developer's workflow has a fixed phase ordering known at design time — not that the LLM picks tools within phases. Surfaces the test from the Koog tile: the planner is for runtime-decided sequencing, not for tool selection within fixed steps",
      "max_score": 25
    },
    {
      "name": "Redirects to the graph DSL",
      "description": "Recommends authoring a graph strategy (or invokes the appropriate sibling skill) instead of installing the planner module. Names the strategy DSL specifically — not a vague \"different approach\"",
      "max_score": 30
    },
    {
      "name": "Does not install agents-planner",
      "description": "Does NOT add ai.koog:agents-planner to build.gradle.kts and does NOT produce Planners.llmBased / Planners.goap factory calls. Installing the wrong module on the developer's say-so without surfacing the mismatch is the failure mode this scenario tests",
      "max_score": 20
    },
    {
      "name": "Acknowledges the developer's framing without capitulating",
      "description": "Engages with the developer's stated reason (\"letting the agent plan its work sounded right\") and explains the distinction between LLM-picks-tools (graph) and LLM-picks-step-sequence (planner). Does not just refuse without explanation",
      "max_score": 15
    },
    {
      "name": "Mentions the cost penalty of an unwarranted planner",
      "description": "Notes that a planner adds an LLM round-trip per planning decision — for a fixed three-phase workflow that's wasted cost with no autonomy benefit. The developer needs this to internalize the recommendation",
      "max_score": 10
    }
  ]
}

evals

README.md

tile.json