Use when the user asks about a Calibre ebook library or book collection: find books, search metadata or full text in EPUB/AZW3 files, or locate book paths via Calibre databases.
100
Quality
100%
Does it follow best practices?
Impact
100%
1.35xAverage score across 3 eval scenarios
{
"context": "This criteria evaluates whether the agent uses the ebook-library skill's metadata lookup flow correctly on a small Calibre fixture library, and whether it identifies the Jules Verne title accurately without guessing.",
"type": "weighted_checklist",
"checklist": [
{
"name": "correct title and author",
"description": "Correctly identifies the book as 'Twenty Thousand Leagues under the Sea' by Jules Verne.",
"max_score": 40
},
{
"name": "reports a book id",
"description": "Returns a concrete book_id obtained from the fixture library rather than omitting it or inventing one.",
"max_score": 20
},
{
"name": "uses metadata-first workflow",
"description": "Uses an appropriate metadata lookup flow such as find_books.py or list_books.py, instead of starting with a broad global content search.",
"max_score": 25
},
{
"name": "brief method explanation",
"description": "Briefly explains which script or command was used to find the answer.",
"max_score": 15
}
]
}