Library to work against complex domain names, subdomains and URIs
82
Build a domain analyzer that can correctly identify and classify domains hosted on cloud platforms versus regular ICANN domains.
Cloud platforms like AWS S3, GitHub Pages, and Heroku provide hosting services using subdomains of their own domains (e.g., bucket.s3.amazonaws.com, username.github.io). For security and analytics purposes, it's important to correctly identify the registrable domain for such URLs versus regular website domains.
Your task is to build a tool that can analyze a list of URLs and produce a report showing:
Your program should read URLs from a file called urls.txt (one URL per line) and analyze each URL.
Generate a JSON report file called domain-report.json with the following structure:
{
"icann_domains": [
{
"url": "original url",
"domain": "registrable domain",
"publicSuffix": "public suffix",
"subdomain": "subdomain if any"
}
],
"private_domains": [
{
"url": "original url",
"domain": "registrable domain on cloud platform",
"publicSuffix": "private suffix",
"subdomain": "subdomain if any"
}
],
"summary": {
"total": 0,
"icann_count": 0,
"private_count": 0
}
}For cloud platform URLs, the tool should identify the actual registrable domain including the private suffix (e.g., for my-bucket.s3.amazonaws.com, the domain should be my-bucket.s3.amazonaws.com not just amazonaws.com).
Analyzing https://www.google.com/search produces an ICANN domain entry with domain google.com, publicSuffix com, subdomain www @test
Analyzing https://my-app.herokuapp.com/page produces a private domain entry with domain my-app.herokuapp.com, publicSuffix herokuapp.com, subdomain empty @test
Analyzing https://docs.example.co.uk/guide produces an ICANN domain entry with domain example.co.uk, publicSuffix co.uk, subdomain docs @test
Analyzing https://mybucket.s3.amazonaws.com/file.txt produces a private domain entry with domain mybucket.s3.amazonaws.com, publicSuffix s3.amazonaws.com, subdomain empty @test
Create the following files:
src/analyzer.ts - Main analyzer implementationtest/analyzer.test.ts - Test suiteurls.txt - Sample input file (create with at least 5 diverse URLs)@generates
Provides URL parsing and domain extraction capabilities.
Install with Tessl CLI
npx tessl i tessl/npm-tldtsdocs
evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10