A ready-to-use web spider that works with proxies, asynchrony, rate limit, configurable request pools, jQuery, and HTTP/2 support.
94
{
"context": "This evaluation assesses the engineer's ability to use the crawler package's priority queue and concurrency management features. The focus is on proper instantiation of the Crawler class with appropriate options, correct usage of the add() method with priority configuration, proper callback handling with the done() function, and detecting completion using the drain event.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Crawler instantiation",
"description": "Creates a new Crawler instance using 'new Crawler()' or 'Crawler()' constructor",
"max_score": 10
},
{
"name": "maxConnections option",
"description": "Configures the crawler with maxConnections option set to the value passed to the function",
"max_score": 15
},
{
"name": "Priority option usage",
"description": "Uses the add() or queue() method to queue requests with the priority option specified for each source based on the source's priority value",
"max_score": 25
},
{
"name": "Callback implementation",
"description": "Provides a callback function in either constructor options or per-request add() options to handle responses",
"max_score": 10
},
{
"name": "Cheerio extraction",
"description": "Uses the $ (Cheerio) object from the response (res.$) to extract article titles using jQuery-like selectors (e.g., $('h2.article-title'))",
"max_score": 20
},
{
"name": "done() invocation",
"description": "Calls the done() function in the callback to signal completion and release the queue slot for the next request",
"max_score": 10
},
{
"name": "Drain event handling",
"description": "Uses the 'drain' event listener (crawler.on('drain', ...)) to detect when all crawling tasks are complete and resolve/return results",
"max_score": 10
}
]
}Install with Tessl CLI
npx tessl i tessl/npm-crawlerevals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10