Use when the user asks about a Calibre ebook library or book collection: find books, search metadata or full text in EPUB/AZW3 files, or locate book paths via Calibre databases.
98
98%
Does it follow best practices?
Impact
99%
1.80xAverage score across 4 eval scenarios
Passed
No known issues
{
"context": "This scenario checks that the agent first locates the real bundled Calibre library, then answers from searchable library content rather than metadata clues in file names.",
"type": "weighted_checklist",
"checklist": [
{
"name": "library discovered correctly",
"description": "Verifies and reports the real bundled Calibre library path or database files before searching, instead of assuming or inventing a substitute library.",
"max_score": 20
},
{
"name": "correct book identified",
"description": "Reports the exact title 'The Problems of Philosophy', author 'Bertrand Russell', and book_id 4 from the library.",
"max_score": 30
},
{
"name": "content-derived discovery",
"description": "Uses content or metadata lookup against the real bundled library data (not filename inference) and identifies the book based on the sense-data/G.E. Moore clue.",
"max_score": 25
},
{
"name": "evidence snippet",
"description": "Includes a short quote/snippet from the library lookup showing both relevance to the provided clue and source of truth.",
"max_score": 15
},
{
"name": "process explanation",
"description": "Briefly explains the lookup sequence used (library discovery + query + validation) and stays consistent with the returned evidence.",
"max_score": 10
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4
references
sample-library
Arthur C. Clarke
Warp Drive Studies (993)
Bertrand Russell
The Problems of Philosophy (4)
The Problems of Philosophy (997)
Jules Verne
Leviathan Under the Pacific (998)
Twenty Thousand Leagues under the Sea (1)
Twenty Thousand Leagues under the Sea (999)
Karl Marx
The Communist Manifesto (6)
The Communist Manifesto (995)
The Prince of the Workers (994)
scripts