CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/npm-crawler

A ready-to-use web spider that works with proxies, asynchrony, rate limit, configurable request pools, jQuery, and HTTP/2 support.

94

1.17x
Overview
Eval results
Files

rubric.jsonevals/scenario-5/

{
  "context": "This evaluation assesses how well the engineer uses the crawler package to handle different Content-Type responses. The focus is on leveraging the crawler's automatic content-type detection, jQuery integration for HTML parsing, JSON parsing capability, and binary data handling.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Crawler instantiation",
      "description": "Creates a Crawler instance with appropriate configuration (e.g., maxConnections, callback handling)",
      "max_score": 15
    },
    {
      "name": "HTML parsing with jQuery",
      "description": "Uses the automatic jQuery/Cheerio integration (response.$) to extract data from HTML responses, such as parsing the page title using $('title').text()",
      "max_score": 20
    },
    {
      "name": "Charset handling",
      "description": "Leverages crawler's automatic charset detection for HTML content, either by using default behavior or explicitly setting jQuery: true to enable automatic parsing",
      "max_score": 10
    },
    {
      "name": "JSON content processing",
      "description": "Correctly handles JSON responses by either setting jQuery: false or using the body property directly when crawler detects JSON Content-Type",
      "max_score": 20
    },
    {
      "name": "Binary data handling",
      "description": "Properly configures crawler for binary content using jQuery: false and encoding: null to preserve raw binary data as Buffer",
      "max_score": 20
    },
    {
      "name": "Queue-based URL processing",
      "description": "Uses crawler.queue() or crawler.add() method to process multiple URLs, allowing the crawler to manage concurrent requests",
      "max_score": 10
    },
    {
      "name": "Callback flow control",
      "description": "Properly implements callback pattern with done() function to signal completion and allow the crawler to process the next item in queue",
      "max_score": 5
    }
  ]
}

Install with Tessl CLI

npx tessl i tessl/npm-crawler

tile.json