Use when the user asks about a Calibre ebook library or book collection: find books, search metadata or full text in EPUB/AZW3 files, or locate book paths via Calibre databases.
100
Quality
100%
Does it follow best practices?
Impact
100%
1.35xAverage score across 3 eval scenarios
{
"context": "This criteria evaluates whether the agent can identify the philosophy-themed book in the fixture library and use the ebook-library skill to resolve a concrete file path for it.",
"type": "weighted_checklist",
"checklist": [
{
"name": "correct title and author",
"description": "Correctly identifies the relevant book as 'The Problems of Philosophy' by Bertrand Russell.",
"max_score": 30
},
{
"name": "uses appropriate discovery flow",
"description": "Uses metadata search, content search, or both in a sensible order to confirm the strategy-themed book rather than guessing from prior knowledge.",
"max_score": 20
},
{
"name": "uses resolve_book.py for final path",
"description": "Uses resolve_book.py to obtain a concrete filesystem path, preferably requesting EPUB if available, instead of constructing the path manually.",
"max_score": 35
},
{
"name": "returns a concrete path and method note",
"description": "Returns a concrete resolved path plus a brief explanation of which script or scripts were used.",
"max_score": 15
}
]
}