0
# Sitemap
1
2
Sitemap is a comprehensive TypeScript library and CLI tool for creating XML sitemaps that comply with the sitemaps.org protocol. It provides streaming APIs for handling large numbers of URLs, supports advanced sitemap features including images, videos, news articles, and alternate language links, and offers both programmatic and command-line interfaces for sitemap creation, parsing, validation, and updating.
3
4
## Package Information
5
6
- **Package Name**: sitemap
7
- **Package Type**: npm
8
- **Language**: TypeScript
9
- **Installation**: `npm install sitemap`
10
- **Requirements**: Node.js 14.0.0+, NPM 6.0.0+
11
12
## Core Imports
13
14
```typescript
15
import {
16
SitemapStream,
17
SitemapAndIndexStream,
18
streamToPromise,
19
parseSitemap,
20
simpleSitemapAndIndex
21
} from "sitemap";
22
```
23
24
For CommonJS:
25
26
```javascript
27
const {
28
SitemapStream,
29
SitemapAndIndexStream,
30
streamToPromise,
31
parseSitemap,
32
simpleSitemapAndIndex
33
} = require("sitemap");
34
```
35
36
## Basic Usage
37
38
```typescript
39
import { SitemapStream, streamToPromise } from "sitemap";
40
import { createWriteStream } from "fs";
41
42
// Create a sitemap stream
43
const sitemap = new SitemapStream({ hostname: "https://example.com" });
44
45
// Add URLs to the sitemap
46
sitemap.write({ url: "/page-1", changefreq: "daily", priority: 0.3 });
47
sitemap.write({ url: "/page-2", changefreq: "weekly", priority: 0.7 });
48
49
// Generate the XML and write to file
50
sitemap.end();
51
streamToPromise(sitemap).then((data) => {
52
console.log(data.toString());
53
});
54
55
// Or pipe directly to a file
56
const writeStream = createWriteStream("./sitemap.xml");
57
sitemap.pipe(writeStream);
58
```
59
60
## Architecture
61
62
The sitemap library is built around several key components:
63
64
- **Streaming Architecture**: Built on Node.js Transform streams for memory-efficient processing of large sitemaps
65
- **Validation System**: Comprehensive validation of sitemap items according to sitemaps.org specifications
66
- **Multi-format Support**: Handles various sitemap extensions (images, videos, news, alternate languages)
67
- **Parsing Capabilities**: Can parse existing sitemaps back into JavaScript objects
68
- **CLI Tools**: Command-line interface for sitemap generation and validation
69
- **Type Safety**: Full TypeScript definitions with strict typing for all API components
70
71
## Capabilities
72
73
### Core Sitemap Generation
74
75
Primary streaming interfaces for generating XML sitemaps with full control over the output format and validation.
76
77
```typescript { .api }
78
class SitemapStream extends Transform {
79
constructor(opts?: SitemapStreamOptions);
80
write(item: SitemapItemLoose): boolean;
81
}
82
83
interface SitemapStreamOptions {
84
hostname?: string;
85
level?: ErrorLevel;
86
lastmodDateOnly?: boolean;
87
xmlns?: NSArgs;
88
xslUrl?: string;
89
errorHandler?: ErrorHandler;
90
}
91
92
function streamToPromise(stream: Readable): Promise<Buffer>;
93
```
94
95
[Core Sitemap Generation](./sitemap-streams.md)
96
97
### Sitemap Index Generation
98
99
Advanced streaming capabilities for creating sitemap indices and managing multiple sitemap files for large websites.
100
101
```typescript { .api }
102
class SitemapIndexStream extends Transform {
103
constructor(opts?: SitemapIndexStreamOptions);
104
write(item: IndexItem | string): boolean;
105
}
106
107
class SitemapAndIndexStream extends SitemapIndexStream {
108
constructor(opts: SitemapAndIndexStreamOptions);
109
}
110
111
interface SitemapAndIndexStreamOptions extends SitemapIndexStreamOptions {
112
limit?: number;
113
getSitemapStream: (i: number) => [IndexItem | string, SitemapStream, WriteStream];
114
}
115
```
116
117
[Sitemap Index Generation](./sitemap-index.md)
118
119
### Sitemap Parsing
120
121
Functionality for parsing existing XML sitemaps back into JavaScript objects for analysis and manipulation.
122
123
```typescript { .api }
124
function parseSitemap(xml: Readable): Promise<SitemapItem[]>;
125
function parseSitemapIndex(xml: Readable): Promise<IndexItem[]>;
126
127
class XMLToSitemapItemStream extends Transform {
128
constructor(opts?: XMLToSitemapItemStreamOptions);
129
}
130
```
131
132
[Sitemap Parsing](./sitemap-parsing.md)
133
134
### Simple API
135
136
High-level convenience functions for quickly generating sitemaps and indices with minimal configuration.
137
138
```typescript { .api }
139
function simpleSitemapAndIndex(options: {
140
hostname: string;
141
sitemapHostname?: string;
142
sourceData: SitemapItemLoose[] | string | Readable | string[];
143
destinationDir: string;
144
publicBasePath?: string;
145
limit?: number;
146
gzip?: boolean;
147
}): Promise<void>;
148
```
149
150
[Simple API](./simple-api.md)
151
152
### Validation and Utilities
153
154
Validation functions and utility methods for working with sitemap data, URL normalization, and error handling.
155
156
```typescript { .api }
157
function validateSMIOptions(
158
conf: SitemapItem,
159
level?: ErrorLevel,
160
errorHandler?: ErrorHandler
161
): SitemapItem;
162
163
function normalizeURL(
164
elem: string | SitemapItemLoose,
165
hostname?: string,
166
lastmodDateOnly?: boolean
167
): SitemapItem;
168
169
enum ErrorLevel {
170
SILENT = 'silent',
171
WARN = 'warn',
172
THROW = 'throw'
173
}
174
```
175
176
[Validation and Utilities](./validation-utilities.md)
177
178
### XML Validation
179
180
External validation capabilities using xmllint for ensuring generated sitemaps comply with XML schemas.
181
182
```typescript { .api }
183
function xmlLint(xml: string | Readable): Promise<void>;
184
```
185
186
[XML Validation](./xml-validation.md)
187
188
### Error Handling
189
190
Comprehensive error classes for validation, configuration, and data format issues.
191
192
```typescript { .api }
193
enum ErrorLevel {
194
SILENT = 'silent',
195
WARN = 'warn',
196
THROW = 'throw'
197
}
198
199
type ErrorHandler = (error: Error, level: ErrorLevel) => void;
200
201
class NoURLError extends Error { }
202
class PriorityInvalidError extends Error { }
203
class ChangeFreqInvalidError extends Error { }
204
class InvalidVideoFormat extends Error { }
205
// Plus 18+ additional error classes for specific validation cases
206
```
207
208
[Error Handling](./error-handling.md)
209
210
### CLI Interface
211
212
Command-line interface for sitemap generation, validation, and parsing.
213
214
```bash
215
# Generate sitemap from URL list
216
npx sitemap < urls.txt > sitemap.xml
217
218
# Validate existing sitemap
219
npx sitemap --validate sitemap.xml
220
221
# Parse sitemap to JSON
222
npx sitemap --parse sitemap.xml
223
```
224
225
[CLI Interface](./cli-interface.md)
226
227
## Core Types
228
229
```typescript { .api }
230
interface SitemapItem {
231
url: string;
232
lastmod?: string;
233
changefreq?: EnumChangefreq;
234
priority?: number;
235
fullPrecisionPriority?: boolean;
236
img: Img[];
237
video: VideoItem[];
238
links: LinkItem[];
239
news?: NewsItem;
240
expires?: string;
241
androidLink?: string;
242
ampLink?: string;
243
}
244
245
interface SitemapItemLoose {
246
url: string;
247
lastmod?: string;
248
changefreq?: EnumChangefreq;
249
priority?: number;
250
fullPrecisionPriority?: boolean;
251
img?: string | Img | (string | Img)[];
252
video?: VideoItemLoose | VideoItemLoose[];
253
links?: LinkItem[];
254
news?: NewsItem;
255
expires?: string;
256
androidLink?: string;
257
ampLink?: string;
258
lastmodfile?: string | Buffer | URL;
259
lastmodISO?: string;
260
lastmodrealtime?: boolean;
261
}
262
263
interface IndexItem {
264
url: string;
265
lastmod?: string;
266
}
267
268
enum EnumChangefreq {
269
DAILY = 'daily',
270
MONTHLY = 'monthly',
271
ALWAYS = 'always',
272
HOURLY = 'hourly',
273
WEEKLY = 'weekly',
274
YEARLY = 'yearly',
275
NEVER = 'never'
276
}
277
278
enum ErrorLevel {
279
SILENT = 'silent',
280
WARN = 'warn',
281
THROW = 'throw'
282
}
283
284
type ErrorHandler = (error: Error, level: ErrorLevel) => void;
285
286
interface Img {
287
/** The URL of the image */
288
url: string;
289
/** The caption of the image */
290
caption?: string;
291
/** The title of the image */
292
title?: string;
293
/** The geographic location of the image */
294
geoLocation?: string;
295
/** A URL to the license of the image */
296
license?: string;
297
}
298
299
interface VideoItem {
300
/** A URL pointing to the video thumbnail image file */
301
thumbnail_loc: string;
302
/** The title of the video */
303
title: string;
304
/** A description of the video (max 2048 characters) */
305
description: string;
306
/** Tags describing the video */
307
tag: string[];
308
/** A URL pointing to the actual video media file */
309
content_loc?: string;
310
/** A URL pointing to a player for the video */
311
player_loc?: string;
312
/** Query param for auto playback */
313
'player_loc:autoplay'?: string;
314
/** Whether search engines can embed the video */
315
'player_loc:allow_embed'?: EnumYesNo;
316
/** The length of the video in seconds */
317
duration?: number;
318
/** The date after which the video will no longer be available */
319
expiration_date?: string;
320
/** The number of times the video has been viewed */
321
view_count?: number;
322
/** The date the video was first published */
323
publication_date?: string;
324
/** A short description of the video category */
325
category?: string;
326
/** Whether to show or hide video in search results from specific countries */
327
restriction?: string;
328
/** Whether the countries in restriction are allowed or denied */
329
'restriction:relationship'?: EnumAllowDeny;
330
/** The video uploader's name */
331
uploader?: string;
332
/** URL with additional information about the uploader */
333
'uploader:info'?: string;
334
/** Gallery location URL */
335
gallery_loc?: string;
336
/** Title for gallery location */
337
'gallery_loc:title'?: string;
338
/** The price to download or view the video */
339
price?: string;
340
/** Specifies the resolution of the purchased version */
341
'price:resolution'?: Resolution;
342
/** Specifies the currency in ISO4217 format */
343
'price:currency'?: string;
344
/** Specifies the purchase option */
345
'price:type'?: PriceType;
346
/** Whether to show or hide video in search results on specified platforms */
347
platform?: string;
348
/** Platform relationship (allow/deny) */
349
'platform:relationship'?: EnumAllowDeny;
350
/** Video ID */
351
id?: string;
352
/** The rating of the video (0-5) */
353
rating?: number;
354
/** Whether the video is appropriate for family viewing */
355
family_friendly?: EnumYesNo;
356
/** Whether a subscription is required to view the video */
357
requires_subscription?: EnumYesNo;
358
/** Whether the video is a live stream */
359
live?: EnumYesNo;
360
}
361
362
interface VideoItemLoose {
363
/** A URL pointing to the video thumbnail image file */
364
thumbnail_loc: string;
365
/** The title of the video */
366
title: string;
367
/** A description of the video (max 2048 characters) */
368
description: string;
369
/** Tags describing the video (loose format - string or array) */
370
tag?: string | string[];
371
/** A URL pointing to the actual video media file */
372
content_loc?: string;
373
/** A URL pointing to a player for the video */
374
player_loc?: string;
375
/** Query param for auto playback */
376
'player_loc:autoplay'?: string;
377
/** Whether search engines can embed the video */
378
'player_loc:allow_embed'?: EnumYesNo;
379
/** The length of the video in seconds */
380
duration?: number;
381
/** The date after which the video will no longer be available */
382
expiration_date?: string;
383
/** The number of times the video has been viewed */
384
view_count?: number;
385
/** The date the video was first published */
386
publication_date?: string;
387
/** A short description of the video category */
388
category?: string;
389
/** Whether to show or hide video in search results from specific countries */
390
restriction?: string;
391
/** Whether the countries in restriction are allowed or denied */
392
'restriction:relationship'?: EnumAllowDeny;
393
/** The video uploader's name */
394
uploader?: string;
395
/** URL with additional information about the uploader */
396
'uploader:info'?: string;
397
/** Gallery location URL */
398
gallery_loc?: string;
399
/** Title for gallery location */
400
'gallery_loc:title'?: string;
401
/** The price to download or view the video */
402
price?: string;
403
/** Specifies the resolution of the purchased version */
404
'price:resolution'?: Resolution;
405
/** Specifies the currency in ISO4217 format */
406
'price:currency'?: string;
407
/** Specifies the purchase option */
408
'price:type'?: PriceType;
409
/** Whether to show or hide video in search results on specified platforms */
410
platform?: string;
411
/** Platform relationship (allow/deny) */
412
'platform:relationship'?: EnumAllowDeny;
413
/** Video ID */
414
id?: string;
415
/** The rating of the video (loose format - string or number) */
416
rating?: string | number;
417
/** Whether the video is appropriate for family viewing (loose format) */
418
family_friendly?: EnumYesNo | boolean;
419
/** Whether a subscription is required to view the video (loose format) */
420
requires_subscription?: EnumYesNo | boolean;
421
/** Whether the video is a live stream (loose format) */
422
live?: EnumYesNo | boolean;
423
}
424
425
interface NewsItem {
426
/** Access level for the news article */
427
access?: 'Registration' | 'Subscription';
428
/** Publication information */
429
publication: {
430
/** Name of the publication */
431
name: string;
432
/** Language of the publication (ISO 639 code) */
433
language: string;
434
};
435
/** Genres of the news article */
436
genres?: string;
437
/** Article publication date in W3C format */
438
publication_date: string;
439
/** The title of the news article */
440
title: string;
441
/** Keywords describing the article */
442
keywords?: string;
443
/** Stock tickers mentioned in the article */
444
stock_tickers?: string;
445
}
446
447
interface LinkItem {
448
/** Language code */
449
lang: string;
450
/** Hreflang attribute value */
451
hreflang?: string;
452
/** URL of the alternate version */
453
url: string;
454
}
455
456
enum EnumYesNo {
457
YES = 'YES',
458
NO = 'NO',
459
Yes = 'Yes',
460
No = 'No',
461
yes = 'yes',
462
no = 'no'
463
}
464
465
enum EnumAllowDeny {
466
ALLOW = 'allow',
467
DENY = 'deny'
468
}
469
470
type PriceType = 'rent' | 'purchase' | 'RENT' | 'PURCHASE';
471
type Resolution = 'HD' | 'hd' | 'sd' | 'SD';
472
```