Use when the user asks about a Calibre ebook library or book collection: find books, search metadata or full text in EPUB/AZW3 files, or locate book paths via Calibre databases.
98
98%
Does it follow best practices?
Impact
99%
1.80xAverage score across 4 eval scenarios
Passed
No known issues
{
"context": "This scenario checks that the agent can use Calibre metadata to locate the correct book record and resolve a real file path, instead of inventing or guessing from directory names.",
"type": "weighted_checklist",
"checklist": [
{
"name": "correct book identified",
"description": "Reports the exact title 'The Problems of Philosophy', author 'Bertrand Russell', and book_id 4 from the library.",
"max_score": 30
},
{
"name": "metadata-first discovery",
"description": "Uses library metadata lookup to find the book before resolving the file path, rather than guessing from folder structure.",
"max_score": 25
},
{
"name": "real epub path resolved",
"description": "Returns a real resolved EPUB path from the Calibre library, or honestly reports available formats if EPUB is unavailable.",
"max_score": 30
},
{
"name": "method explanation",
"description": "Briefly explains the lookup sequence used (metadata lookup then path resolution) and stays consistent with the returned evidence.",
"max_score": 15
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4
references
sample-library
Arthur C. Clarke
Warp Drive Studies (993)
Bertrand Russell
The Problems of Philosophy (4)
The Problems of Philosophy (997)
Jules Verne
Leviathan Under the Pacific (998)
Twenty Thousand Leagues under the Sea (1)
Twenty Thousand Leagues under the Sea (999)
Karl Marx
The Communist Manifesto (6)
The Communist Manifesto (995)
The Prince of the Workers (994)
scripts