A ready-to-use web spider that works with proxies, asynchrony, rate limit, configurable request pools, jQuery, and HTTP/2 support.
94
{
"context": "This evaluation assesses how well the engineer uses the crawler package to handle different Content-Type responses. The focus is on leveraging the crawler's automatic content-type detection, jQuery integration for HTML parsing, JSON parsing capability, and binary data handling.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Crawler instantiation",
"description": "Creates a Crawler instance with appropriate configuration (e.g., maxConnections, callback handling)",
"max_score": 15
},
{
"name": "HTML parsing with jQuery",
"description": "Uses the automatic jQuery/Cheerio integration (response.$) to extract data from HTML responses, such as parsing the page title using $('title').text()",
"max_score": 20
},
{
"name": "Charset handling",
"description": "Leverages crawler's automatic charset detection for HTML content, either by using default behavior or explicitly setting jQuery: true to enable automatic parsing",
"max_score": 10
},
{
"name": "JSON content processing",
"description": "Correctly handles JSON responses by either setting jQuery: false or using the body property directly when crawler detects JSON Content-Type",
"max_score": 20
},
{
"name": "Binary data handling",
"description": "Properly configures crawler for binary content using jQuery: false and encoding: null to preserve raw binary data as Buffer",
"max_score": 20
},
{
"name": "Queue-based URL processing",
"description": "Uses crawler.queue() or crawler.add() method to process multiple URLs, allowing the crawler to manage concurrent requests",
"max_score": 10
},
{
"name": "Callback flow control",
"description": "Properly implements callback pattern with done() function to signal completion and allow the crawler to process the next item in queue",
"max_score": 5
}
]
}Install with Tessl CLI
npx tessl i tessl/npm-crawlerevals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10