CtrlK
BlogDocsLog inGet started
Tessl Logo

jbaruch/koog

Koog 1.0 idioms, gotchas, and scaffolding skills for Kotlin agents on the JVM

88

1.95x
Quality

88%

Does it follow best practices?

Impact

88%

1.95x

Average score across 43 eval scenarios

SecuritybySnyk

Passed

No known issues

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-1/

{
  "context": "Tests whether the agent wires RAG as a tool (the LLM decides when to retrieve, matching the developer's stated preference) rather than as a prompt augmenter (which would augment every input — explicitly rejected).",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Exposes retrieval as a Tool, not as an augmenter",
      "description": "Wraps the vector store search in a tool that the LLM can invoke — typically a @Tool-annotated ToolSet method. Does NOT install a UserPromptAugmenter or SystemPromptAugmenter that would inject retrieval into every input — the developer explicitly rejected that",
      "max_score": 30
    },
    {
      "name": "Builds an embedder from a provider",
      "description": "Constructs an embedder backed by a provider (e.g., OpenAIEmbedder against an embedding model). Does not skip embedder construction or use a placeholder TODO",
      "max_score": 15
    },
    {
      "name": "Builds a vector store and indexes documents",
      "description": "Constructs a VectorStore (e.g., InMemoryVectorStore) and indexes the developer's existing List<Document>, calling store.add(...) for each. Does not require a pre-existing index",
      "max_score": 20
    },
    {
      "name": "Adds the RAG and embeddings dependencies",
      "description": "Adds ai.koog:rag-base, ai.koog:rag-vector, ai.koog:embeddings-base, and ai.koog:embeddings-llm at 1.0.0+. Does not skip embeddings-llm (the embedder needs it)",
      "max_score": 15
    },
    {
      "name": "Registers the search tool with the agent's ToolRegistry",
      "description": "Adds the search tool to a ToolRegistry that's passed to AIAgent(...) as toolRegistry. Does not mutate the registry after construction",
      "max_score": 10
    },
    {
      "name": "Search tool returns formatted results, not raw vector hits",
      "description": "The search tool returns text (e.g., concatenated top-K results with source paths) — not raw VectorStore.SearchResult objects that the LLM wouldn't understand. Formatting can be terse but must produce LLM-readable output",
      "max_score": 10
    }
  ]
}

evals

scenario-1

criteria.json

task.md

README.md

tile.json