or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

cli-interface.mderror-handling.mdindex.mdsimple-api.mdsitemap-index.mdsitemap-parsing.mdsitemap-streams.mdvalidation-utilities.mdxml-validation.md
tile.json

tessl/npm-sitemap

Sitemap-generating library and CLI tool for creating XML sitemaps that comply with the sitemaps.org protocol

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
npmpkg:npm/sitemap@8.0.x

To install, run

npx @tessl/cli install tessl/npm-sitemap@8.0.0

index.mddocs/

Sitemap

Sitemap is a comprehensive TypeScript library and CLI tool for creating XML sitemaps that comply with the sitemaps.org protocol. It provides streaming APIs for handling large numbers of URLs, supports advanced sitemap features including images, videos, news articles, and alternate language links, and offers both programmatic and command-line interfaces for sitemap creation, parsing, validation, and updating.

Package Information

  • Package Name: sitemap
  • Package Type: npm
  • Language: TypeScript
  • Installation: npm install sitemap
  • Requirements: Node.js 14.0.0+, NPM 6.0.0+

Core Imports

import { 
  SitemapStream, 
  SitemapAndIndexStream, 
  streamToPromise,
  parseSitemap,
  simpleSitemapAndIndex
} from "sitemap";

For CommonJS:

const { 
  SitemapStream, 
  SitemapAndIndexStream, 
  streamToPromise,
  parseSitemap,
  simpleSitemapAndIndex
} = require("sitemap");

Basic Usage

import { SitemapStream, streamToPromise } from "sitemap";
import { createWriteStream } from "fs";

// Create a sitemap stream
const sitemap = new SitemapStream({ hostname: "https://example.com" });

// Add URLs to the sitemap
sitemap.write({ url: "/page-1", changefreq: "daily", priority: 0.3 });
sitemap.write({ url: "/page-2", changefreq: "weekly", priority: 0.7 });

// Generate the XML and write to file
sitemap.end();
streamToPromise(sitemap).then((data) => {
  console.log(data.toString());
});

// Or pipe directly to a file
const writeStream = createWriteStream("./sitemap.xml");
sitemap.pipe(writeStream);

Architecture

The sitemap library is built around several key components:

  • Streaming Architecture: Built on Node.js Transform streams for memory-efficient processing of large sitemaps
  • Validation System: Comprehensive validation of sitemap items according to sitemaps.org specifications
  • Multi-format Support: Handles various sitemap extensions (images, videos, news, alternate languages)
  • Parsing Capabilities: Can parse existing sitemaps back into JavaScript objects
  • CLI Tools: Command-line interface for sitemap generation and validation
  • Type Safety: Full TypeScript definitions with strict typing for all API components

Capabilities

Core Sitemap Generation

Primary streaming interfaces for generating XML sitemaps with full control over the output format and validation.

class SitemapStream extends Transform {
  constructor(opts?: SitemapStreamOptions);
  write(item: SitemapItemLoose): boolean;
}

interface SitemapStreamOptions {
  hostname?: string;
  level?: ErrorLevel;
  lastmodDateOnly?: boolean;
  xmlns?: NSArgs;
  xslUrl?: string;
  errorHandler?: ErrorHandler;
}

function streamToPromise(stream: Readable): Promise<Buffer>;

Core Sitemap Generation

Sitemap Index Generation

Advanced streaming capabilities for creating sitemap indices and managing multiple sitemap files for large websites.

class SitemapIndexStream extends Transform {
  constructor(opts?: SitemapIndexStreamOptions);
  write(item: IndexItem | string): boolean;
}

class SitemapAndIndexStream extends SitemapIndexStream {
  constructor(opts: SitemapAndIndexStreamOptions);
}

interface SitemapAndIndexStreamOptions extends SitemapIndexStreamOptions {
  limit?: number;
  getSitemapStream: (i: number) => [IndexItem | string, SitemapStream, WriteStream];
}

Sitemap Index Generation

Sitemap Parsing

Functionality for parsing existing XML sitemaps back into JavaScript objects for analysis and manipulation.

function parseSitemap(xml: Readable): Promise<SitemapItem[]>;
function parseSitemapIndex(xml: Readable): Promise<IndexItem[]>;

class XMLToSitemapItemStream extends Transform {
  constructor(opts?: XMLToSitemapItemStreamOptions);
}

Sitemap Parsing

Simple API

High-level convenience functions for quickly generating sitemaps and indices with minimal configuration.

function simpleSitemapAndIndex(options: {
  hostname: string;
  sitemapHostname?: string;
  sourceData: SitemapItemLoose[] | string | Readable | string[];
  destinationDir: string;
  publicBasePath?: string;
  limit?: number;
  gzip?: boolean;
}): Promise<void>;

Simple API

Validation and Utilities

Validation functions and utility methods for working with sitemap data, URL normalization, and error handling.

function validateSMIOptions(
  conf: SitemapItem,
  level?: ErrorLevel,
  errorHandler?: ErrorHandler
): SitemapItem;

function normalizeURL(
  elem: string | SitemapItemLoose,
  hostname?: string,
  lastmodDateOnly?: boolean
): SitemapItem;

enum ErrorLevel {
  SILENT = 'silent',
  WARN = 'warn',
  THROW = 'throw'
}

Validation and Utilities

XML Validation

External validation capabilities using xmllint for ensuring generated sitemaps comply with XML schemas.

function xmlLint(xml: string | Readable): Promise<void>;

XML Validation

Error Handling

Comprehensive error classes for validation, configuration, and data format issues.

enum ErrorLevel {
  SILENT = 'silent',
  WARN = 'warn', 
  THROW = 'throw'
}

type ErrorHandler = (error: Error, level: ErrorLevel) => void;

class NoURLError extends Error { }
class PriorityInvalidError extends Error { }
class ChangeFreqInvalidError extends Error { }
class InvalidVideoFormat extends Error { }
// Plus 18+ additional error classes for specific validation cases

Error Handling

CLI Interface

Command-line interface for sitemap generation, validation, and parsing.

# Generate sitemap from URL list  
npx sitemap < urls.txt > sitemap.xml

# Validate existing sitemap
npx sitemap --validate sitemap.xml

# Parse sitemap to JSON
npx sitemap --parse sitemap.xml

CLI Interface

Core Types

interface SitemapItem {
  url: string;
  lastmod?: string;
  changefreq?: EnumChangefreq;
  priority?: number;
  fullPrecisionPriority?: boolean;
  img: Img[];
  video: VideoItem[];
  links: LinkItem[];
  news?: NewsItem;
  expires?: string;
  androidLink?: string;
  ampLink?: string;
}

interface SitemapItemLoose {
  url: string;
  lastmod?: string;
  changefreq?: EnumChangefreq;
  priority?: number;
  fullPrecisionPriority?: boolean;
  img?: string | Img | (string | Img)[];
  video?: VideoItemLoose | VideoItemLoose[];
  links?: LinkItem[];
  news?: NewsItem;
  expires?: string;
  androidLink?: string;
  ampLink?: string;
  lastmodfile?: string | Buffer | URL;
  lastmodISO?: string;
  lastmodrealtime?: boolean;
}

interface IndexItem {
  url: string;
  lastmod?: string;
}

enum EnumChangefreq {
  DAILY = 'daily',
  MONTHLY = 'monthly',
  ALWAYS = 'always',
  HOURLY = 'hourly',
  WEEKLY = 'weekly',
  YEARLY = 'yearly',
  NEVER = 'never'
}

enum ErrorLevel {
  SILENT = 'silent',
  WARN = 'warn',
  THROW = 'throw'
}

type ErrorHandler = (error: Error, level: ErrorLevel) => void;

interface Img {
  /** The URL of the image */
  url: string;
  /** The caption of the image */
  caption?: string;
  /** The title of the image */
  title?: string;
  /** The geographic location of the image */
  geoLocation?: string;
  /** A URL to the license of the image */
  license?: string;
}

interface VideoItem {
  /** A URL pointing to the video thumbnail image file */
  thumbnail_loc: string;
  /** The title of the video */
  title: string;
  /** A description of the video (max 2048 characters) */
  description: string;
  /** Tags describing the video */
  tag: string[];
  /** A URL pointing to the actual video media file */
  content_loc?: string;
  /** A URL pointing to a player for the video */
  player_loc?: string;
  /** Query param for auto playback */
  'player_loc:autoplay'?: string;
  /** Whether search engines can embed the video */
  'player_loc:allow_embed'?: EnumYesNo;
  /** The length of the video in seconds */
  duration?: number;
  /** The date after which the video will no longer be available */
  expiration_date?: string;
  /** The number of times the video has been viewed */
  view_count?: number;
  /** The date the video was first published */
  publication_date?: string;
  /** A short description of the video category */
  category?: string;
  /** Whether to show or hide video in search results from specific countries */
  restriction?: string;
  /** Whether the countries in restriction are allowed or denied */
  'restriction:relationship'?: EnumAllowDeny;
  /** The video uploader's name */
  uploader?: string;
  /** URL with additional information about the uploader */
  'uploader:info'?: string;
  /** Gallery location URL */
  gallery_loc?: string;
  /** Title for gallery location */
  'gallery_loc:title'?: string;
  /** The price to download or view the video */
  price?: string;
  /** Specifies the resolution of the purchased version */
  'price:resolution'?: Resolution;
  /** Specifies the currency in ISO4217 format */
  'price:currency'?: string;
  /** Specifies the purchase option */
  'price:type'?: PriceType;
  /** Whether to show or hide video in search results on specified platforms */
  platform?: string;
  /** Platform relationship (allow/deny) */
  'platform:relationship'?: EnumAllowDeny;
  /** Video ID */
  id?: string;
  /** The rating of the video (0-5) */
  rating?: number;
  /** Whether the video is appropriate for family viewing */
  family_friendly?: EnumYesNo;
  /** Whether a subscription is required to view the video */
  requires_subscription?: EnumYesNo;
  /** Whether the video is a live stream */
  live?: EnumYesNo;
}

interface VideoItemLoose {
  /** A URL pointing to the video thumbnail image file */
  thumbnail_loc: string;
  /** The title of the video */
  title: string;
  /** A description of the video (max 2048 characters) */
  description: string;
  /** Tags describing the video (loose format - string or array) */
  tag?: string | string[];
  /** A URL pointing to the actual video media file */
  content_loc?: string;
  /** A URL pointing to a player for the video */
  player_loc?: string;
  /** Query param for auto playback */
  'player_loc:autoplay'?: string;
  /** Whether search engines can embed the video */
  'player_loc:allow_embed'?: EnumYesNo;
  /** The length of the video in seconds */
  duration?: number;
  /** The date after which the video will no longer be available */
  expiration_date?: string;
  /** The number of times the video has been viewed */
  view_count?: number;
  /** The date the video was first published */
  publication_date?: string;
  /** A short description of the video category */
  category?: string;
  /** Whether to show or hide video in search results from specific countries */
  restriction?: string;
  /** Whether the countries in restriction are allowed or denied */
  'restriction:relationship'?: EnumAllowDeny;
  /** The video uploader's name */
  uploader?: string;
  /** URL with additional information about the uploader */
  'uploader:info'?: string;
  /** Gallery location URL */
  gallery_loc?: string;
  /** Title for gallery location */
  'gallery_loc:title'?: string;
  /** The price to download or view the video */
  price?: string;
  /** Specifies the resolution of the purchased version */
  'price:resolution'?: Resolution;
  /** Specifies the currency in ISO4217 format */
  'price:currency'?: string;
  /** Specifies the purchase option */
  'price:type'?: PriceType;
  /** Whether to show or hide video in search results on specified platforms */
  platform?: string;
  /** Platform relationship (allow/deny) */
  'platform:relationship'?: EnumAllowDeny;
  /** Video ID */
  id?: string;
  /** The rating of the video (loose format - string or number) */
  rating?: string | number;
  /** Whether the video is appropriate for family viewing (loose format) */
  family_friendly?: EnumYesNo | boolean;
  /** Whether a subscription is required to view the video (loose format) */
  requires_subscription?: EnumYesNo | boolean;
  /** Whether the video is a live stream (loose format) */
  live?: EnumYesNo | boolean;
}

interface NewsItem {
  /** Access level for the news article */
  access?: 'Registration' | 'Subscription';
  /** Publication information */
  publication: {
    /** Name of the publication */
    name: string;
    /** Language of the publication (ISO 639 code) */
    language: string;
  };
  /** Genres of the news article */
  genres?: string;
  /** Article publication date in W3C format */
  publication_date: string;
  /** The title of the news article */
  title: string;
  /** Keywords describing the article */
  keywords?: string;
  /** Stock tickers mentioned in the article */
  stock_tickers?: string;
}

interface LinkItem {
  /** Language code */
  lang: string;
  /** Hreflang attribute value */
  hreflang?: string;
  /** URL of the alternate version */
  url: string;
}

enum EnumYesNo {
  YES = 'YES',
  NO = 'NO',
  Yes = 'Yes',
  No = 'No',
  yes = 'yes',
  no = 'no'
}

enum EnumAllowDeny {
  ALLOW = 'allow',
  DENY = 'deny'
}

type PriceType = 'rent' | 'purchase' | 'RENT' | 'PURCHASE';
type Resolution = 'HD' | 'hd' | 'sd' | 'SD';