Use when the user asks about a Calibre ebook library or book collection: find books, search metadata or full text in EPUB/AZW3 files, or locate book paths via Calibre databases.
98
98%
Does it follow best practices?
Impact
99%
1.80xAverage score across 4 eval scenarios
Passed
No known issues
{
"context": "This scenario checks that the agent handles no-match responses honestly and does not hallucinate a nearest result.",
"type": "weighted_checklist",
"checklist": [
{
"name": "no-match search executed",
"description": "Performs a library content search for the exact requested quote in `./sample-library` text content (not filename/path heuristics).",
"max_score": 35
},
{
"name": "no results reported clearly",
"description": "Explicitly reports the exact quote query and that zero matches were returned.",
"max_score": 35
},
{
"name": "no fabrication",
"description": "Does not invent a title, author, book_id, or excerpt when no results are returned.",
"max_score": 20
},
{
"name": "brief method note",
"description": "Explains which library lookup path and query were used to verify the outcome.",
"max_score": 10
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4
references
sample-library
Arthur C. Clarke
Warp Drive Studies (993)
Bertrand Russell
The Problems of Philosophy (4)
The Problems of Philosophy (997)
Jules Verne
Leviathan Under the Pacific (998)
Twenty Thousand Leagues under the Sea (1)
Twenty Thousand Leagues under the Sea (999)
Karl Marx
The Communist Manifesto (6)
The Communist Manifesto (995)
The Prince of the Workers (994)
scripts