A ready-to-use web spider that works with proxies, asynchrony, rate limit, configurable request pools, jQuery, and HTTP/2 support.
94
{
"context": "This evaluation assesses how well the engineer uses the crawler package's user agent rotation capabilities to solve the web scraping problem. The focus is on proper configuration of user agents and correct usage of the crawler's API for handling multiple requests and completion callbacks.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Crawler instantiation",
"description": "Creates a Crawler instance using the correct constructor pattern (e.g., `new Crawler()` or `const crawler = new Crawler(options)`)",
"max_score": 15
},
{
"name": "userAgents configuration",
"description": "Configures the `userAgents` option in the crawler constructor or per-request options with an array of user agent strings to enable automatic rotation",
"max_score": 25
},
{
"name": "Request queuing",
"description": "Uses the crawler's `queue()` or `add()` method to queue multiple URL requests for scraping",
"max_score": 15
},
{
"name": "Callback implementation",
"description": "Implements a `callback` function in the crawler options that receives `(error, res, done)` parameters to process each response",
"max_score": 15
},
{
"name": "Done function call",
"description": "Calls the `done()` function within each callback to release the queue slot and allow the crawler to continue processing",
"max_score": 10
},
{
"name": "Completion handling",
"description": "Uses the crawler's `drain` event listener (e.g., `crawler.on('drain', callback)`) or tracks completion manually to detect when all URLs have been scraped",
"max_score": 15
},
{
"name": "Result aggregation",
"description": "Collects and aggregates results from all scraped pages to pass to the completion callback, ensuring all requested data is captured",
"max_score": 5
}
]
}Install with Tessl CLI
npx tessl i tessl/npm-crawlerevals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10