A ready-to-use web spider that works with proxies, asynchrony, rate limit, configurable request pools, jQuery, and HTTP/2 support.
94
{
"context": "This criteria evaluates how well the engineer uses the crawler package's cookie jar integration with tough-cookie to maintain session state across multiple web scraping requests. The focus is on proper instantiation and configuration of cookie jars, and their integration with the crawler instance.",
"type": "weighted_checklist",
"checklist": [
{
"name": "CookieJar instantiation",
"description": "Creates a CookieJar instance from the tough-cookie library using 'new CookieJar()' or 'new tough.CookieJar()'",
"max_score": 25
},
{
"name": "Crawler instantiation",
"description": "Creates a Crawler instance from the crawler package using 'new Crawler()' or similar constructor",
"max_score": 15
},
{
"name": "CookieJar configuration",
"description": "Passes the CookieJar instance to the crawler via the 'cookieJar' option in the Crawler constructor or in individual request options",
"max_score": 35
},
{
"name": "Request queueing",
"description": "Uses the crawler's add() or queue() method to add multiple URLs to be scraped with shared cookie state",
"max_score": 15
},
{
"name": "Callback handling",
"description": "Implements proper callback function that accepts (error, res, done) parameters and calls done() to release the queue",
"max_score": 10
}
]
}Install with Tessl CLI
npx tessl i tessl/npm-crawlerevals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10