Use when the user asks about a Calibre ebook library or book collection: find books, search metadata or full text in EPUB/AZW3 files, or locate book paths via Calibre databases.
98
98%
Does it follow best practices?
Impact
99%
1.80xAverage score across 4 eval scenarios
Passed
No known issues
{
"context": "This scenario tests path-resistance: the library includes plausible-looking directories and files that should be ignored unless supported by searchable text evidence.",
"type": "weighted_checklist",
"checklist": [
{
"name": "correct book identified",
"description": "Reports the exact title 'The Communist Manifesto', author(s) 'Karl Marx, Friedrich Engels', and book_id 6.",
"max_score": 35
},
{
"name": "uses library content search",
"description": "Uses a library content search path that proves the match from text (not from filenames/folder names).",
"max_score": 30
},
{
"name": "supports claim with snippet",
"description": "Includes evidence snippet containing both terms (`all that is solid melts into air` and `all that is holy is profaned`), showing the source is real content from the library.",
"max_score": 20
},
{
"name": "notes method",
"description": "Briefly notes the discovery method and states why the chosen match was selected from content evidence.",
"max_score": 15
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4
references
sample-library
Arthur C. Clarke
Warp Drive Studies (993)
Bertrand Russell
The Problems of Philosophy (4)
The Problems of Philosophy (997)
Jules Verne
Leviathan Under the Pacific (998)
Twenty Thousand Leagues under the Sea (1)
Twenty Thousand Leagues under the Sea (999)
Karl Marx
The Communist Manifesto (6)
The Communist Manifesto (995)
The Prince of the Workers (994)
scripts