0
# Sitemap Index Generation
1

2
Advanced streaming capabilities for creating sitemap indices and managing multiple sitemap files for large websites. These classes handle the complexity of splitting large URL sets across multiple sitemaps and creating proper index files.
3

4
## Capabilities
5

6
### SitemapIndexStream
7

8
Transform stream for generating sitemap index XML that references multiple sitemap files.
9

10
```typescript { .api }
11
/**
12
 * Transform stream that takes IndexItems or sitemap URL strings 
13
 * and outputs sitemap index XML
14
 * ⚠️ Must be read (piped) before 'finish' event will be emitted
15
 */
16
class SitemapIndexStream extends Transform {
17
  constructor(opts?: SitemapIndexStreamOptions);
18
  
19
  /** Whether to output only date portion of lastmod */
20
  lastmodDateOnly: boolean;
21
  
22
  /** Error handling level */
23
  level: ErrorLevel;
24
  
25
  /** XSL stylesheet URL */
26
  xslUrl?: string;
27
  
28
  /** Whether XML header has been output */
29
  private hasHeadOutput: boolean;
30
}
31

32
interface SitemapIndexStreamOptions extends TransformOptions {
33
  /** Whether to output the lastmod date only (no time) */
34
  lastmodDateOnly?: boolean;
35
  
36
  /** How to handle errors in passed in urls */
37
  level?: ErrorLevel;
38
  
39
  /** URL to an XSL stylesheet to include in the XML */
40
  xslUrl?: string;
41
}
42
```
43

44
**Usage Examples:**
45

46
```typescript
47
import { SitemapIndexStream } from "sitemap";
48
import { createWriteStream } from "fs";
49

50
// Create sitemap index
51
const sitemapIndex = new SitemapIndexStream({
52
  lastmodDateOnly: false
53
});
54

55
// Add sitemap references
56
sitemapIndex.write({
57
  url: "https://example.com/sitemap-1.xml",
58
  lastmod: "2023-01-01T00:00:00.000Z"
59
});
60

61
sitemapIndex.write("https://example.com/sitemap-2.xml");
62

63
// Output to file
64
sitemapIndex.pipe(createWriteStream("sitemap-index.xml"));
65
sitemapIndex.end();
66
```
67

68
### SitemapAndIndexStream
69

70
Advanced stream that automatically creates multiple sitemap files and generates an index when URL limits are reached.
71

72
```typescript { .api }
73
/**
74
 * Transform stream that takes sitemap items, writes them to sitemap files,
75
 * adds the sitemap files to a sitemap index, and creates new sitemap files
76
 * when the count limit is reached
77
 * ⚠️ Must be read (piped) before 'finish' event will be emitted
78
 */
79
class SitemapAndIndexStream extends SitemapIndexStream {
80
  constructor(opts: SitemapAndIndexStreamOptions);
81
  
82
  /** Number of items written to current sitemap */
83
  private itemsWritten: number;
84
  
85
  /** Function to get new sitemap streams */
86
  private getSitemapStream: getSitemapStreamFunc;
87
  
88
  /** Current sitemap being written to */
89
  private currentSitemap?: SitemapStream;
90
  
91
  /** Maximum items per sitemap file */
92
  private limit: number;
93
  
94
  /** Current sitemap write stream */
95
  private currentSitemapPipeline?: WriteStream;
96
}
97

98
interface SitemapAndIndexStreamOptions extends SitemapIndexStreamOptions {
99
  /** Max number of items in each sitemap XML file (1-50,000) */
100
  limit?: number;
101
  
102
  /** Callback that creates a new sitemap stream for a given sitemap index */
103
  getSitemapStream: getSitemapStreamFunc;
104
}
105

106
type getSitemapStreamFunc = (
107
  i: number
108
) => [IndexItem | string, SitemapStream, WriteStream];
109
```
110

111
**Usage Examples:**
112

113
```typescript
114
import { SitemapAndIndexStream, SitemapStream } from "sitemap";
115
import { createWriteStream } from "fs";
116
import { createGzip } from "zlib";
117

118
const sitemapAndIndex = new SitemapAndIndexStream({
119
  limit: 50000,
120
  getSitemapStream: (i) => {
121
    const sitemapStream = new SitemapStream({
122
      hostname: "https://example.com"
123
    });
124
    
125
    const path = `./sitemap-${i}.xml`;
126
    const writeStream = createWriteStream(path);
127
    
128
    return [
129
      `https://example.com/sitemap-${i}.xml`,
130
      sitemapStream,
131
      sitemapStream.pipe(writeStream)
132
    ];
133
  }
134
});
135

136
// Input large number of URLs
137
const urls = Array.from({ length: 100000 }, (_, i) => ({
138
  url: `/page-${i}`,
139
  changefreq: "weekly" as const,
140
  priority: 0.5
141
}));
142

143
// Stream will automatically create multiple sitemaps
144
const Readable = require("stream").Readable;
145
const urlStream = Readable.from(urls);
146

147
urlStream.pipe(sitemapAndIndex);
148
sitemapAndIndex.pipe(createWriteStream("sitemap-index.xml"));
149
```
150

151
### Gzip Support Example
152

153
```typescript
154
import { SitemapAndIndexStream, SitemapStream } from "sitemap";
155
import { createWriteStream } from "fs";
156
import { createGzip } from "zlib";
157

158
const sitemapAndIndex = new SitemapAndIndexStream({
159
  limit: 45000,
160
  getSitemapStream: (i) => {
161
    const sitemapStream = new SitemapStream({
162
      hostname: "https://example.com"
163
    });
164
    
165
    const path = `./sitemap-${i}.xml.gz`;
166
    const writeStream = sitemapStream
167
      .pipe(createGzip())
168
      .pipe(createWriteStream(path));
169
    
170
    return [
171
      `https://example.com/sitemap-${i}.xml.gz`,
172
      sitemapStream,
173
      writeStream
174
    ];
175
  }
176
});
177
```
178

179
## Index Item Types
180

181
```typescript { .api }
182
interface IndexItem {
183
  /** URL of the sitemap file */
184
  url: string;
185
  
186
  /** Last modification date of the sitemap file */
187
  lastmod?: string;
188
}
189

190
enum IndexTagNames {
191
  sitemap = 'sitemap',
192
  sitemapindex = 'sitemapindex',
193
  loc = 'loc',
194
  lastmod = 'lastmod'
195
}
196
```
197

198
## Best Practices
199

200
### Sitemap Limits
201

202
- Keep individual sitemaps under 50,000 URLs
203
- Keep sitemap files under 50MB (10MB recommended for faster processing)  
204
- Use `SitemapAndIndexStream` for large sites with many URLs
205
- Consider gzip compression to reduce file sizes
206

207
### Error Handling
208

209
```typescript
210
import { SitemapAndIndexStream } from "sitemap";
211

212
const sitemapAndIndex = new SitemapAndIndexStream({
213
  limit: 50000,
214
  level: ErrorLevel.WARN, // Log warnings but continue processing
215
  getSitemapStream: (i) => {
216
    // ... stream creation logic
217
  }
218
});
219

220
// Handle stream errors
221
sitemapAndIndex.on('error', (error) => {
222
  console.error('Sitemap generation error:', error);
223
});
224
```
225

226
## XML Output Structure
227

228
SitemapIndexStream generates XML with the following structure:
229

230
```xml
231
<?xml version="1.0" encoding="UTF-8"?>
232
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
233
  <sitemap>
234
    <loc>https://example.com/sitemap-0.xml</loc>
235
    <lastmod>2023-01-01T00:00:00.000Z</lastmod>
236
  </sitemap>
237
  <sitemap>
238
    <loc>https://example.com/sitemap-1.xml</loc>
239
    <lastmod>2023-01-01T12:00:00.000Z</lastmod>
240
  </sitemap>
241
</sitemapindex>
242
```