CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/npm-crawler

A ready-to-use web spider that works with proxies, asynchrony, rate limit, configurable request pools, jQuery, and HTTP/2 support.

94

1.17x
Overview
Eval results
Files

rubric.jsonevals/scenario-7/

{
  "context": "This evaluation assesses how well the engineer utilizes the crawler package's charset detection and encoding conversion capabilities to scrape international content with various character encodings. The focus is entirely on proper usage of crawler's encoding-related features.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Crawler instantiation",
      "description": "Creates a Crawler instance using 'new Crawler()' or 'Crawler()' with appropriate configuration",
      "max_score": 10
    },
    {
      "name": "forceUTF8 option",
      "description": "Uses the 'forceUTF8' option set to true to enable automatic charset detection and UTF-8 conversion",
      "max_score": 25
    },
    {
      "name": "jQuery parsing",
      "description": "Leverages the automatic jQuery/Cheerio integration (response.$) to parse HTML and extract data from elements",
      "max_score": 15
    },
    {
      "name": "Callback structure",
      "description": "Implements the callback function with the correct signature: callback(error, response, done) and calls done() to release the queue",
      "max_score": 15
    },
    {
      "name": "Queue management",
      "description": "Uses crawler.add() or crawler.queue() to add URLs to the crawler's request queue",
      "max_score": 10
    },
    {
      "name": "Encoding information",
      "description": "Accesses encoding information from the response object (e.g., response.charset or response.options.encoding) to capture detected encoding",
      "max_score": 15
    },
    {
      "name": "drain event",
      "description": "Uses the 'drain' event listener to detect when all scraping tasks are complete and invoke the onComplete callback",
      "max_score": 10
    }
  ]
}

Install with Tessl CLI

npx tessl i tessl/npm-crawler

tile.json