0
# Klaw
1

2
Klaw is a Node.js file system walker that provides a Readable stream interface for recursively traversing directories and their contents. It enables developers to walk through file systems asynchronously, emitting objects containing file paths and fs.Stats for each discovered item (files, directories, symlinks).
3

4
## Package Information
5

6
- **Package Name**: klaw
7
- **Package Type**: npm
8
- **Language**: JavaScript
9
- **Installation**: `npm install klaw`
10
- **Minimum Node.js**: 14.14.0
11

12
## Core Imports
13

14
```javascript
15
const klaw = require('klaw');
16
```
17

18
For ES modules:
19

20
```javascript
21
import klaw from 'klaw';
22
```
23

24
## Basic Usage
25

26
```javascript
27
const klaw = require('klaw');
28

29
// Simple traversal collecting all paths
30
const items = [];
31
klaw('/some/directory')
32
  .on('data', item => {
33
    items.push(item.path);
34
    console.log(item.path, item.stats.isDirectory() ? 'DIR' : 'FILE');
35
  })
36
  .on('end', () => console.log('Done!', items.length, 'items'));
37

38
// Using readable stream pattern
39
klaw('/some/directory')
40
  .on('readable', function () {
41
    let item;
42
    while ((item = this.read())) {
43
      // Process each item
44
      console.log(item.path);
45
    }
46
  });
47

48
// Modern async iteration
49
for await (const item of klaw('/some/directory')) {
50
  console.log(item.path, item.stats.size);
51
}
52
```
53

54
## Architecture
55

56
Klaw is built around a simple, focused architecture:
57

58
- **Walker Class**: Internal Readable stream that implements the recursive traversal logic
59
- **Factory Function**: Main exported function that creates Walker instances with configuration
60
- **Stream Interface**: Full Node.js Readable stream compatibility with object mode
61
- **Async Processing**: Non-blocking file system operations using Node.js fs callbacks
62
- **Configurable Strategy**: Pluggable traversal order, filtering, and depth control
63

64
## Capabilities
65

66
### File System Walking
67

68
Core functionality for recursively traversing directory structures and emitting file system items.
69

70
```javascript { .api }
71
/**
72
 * Creates a file system walker that recursively traverses directories
73
 * @param {string|URL} directory - Root directory to walk (string path or file URL)
74
 * @param {WalkerOptions} [options] - Configuration options for traversal behavior
75
 * @returns {Walker} Readable stream in object mode emitting file system items
76
 */
77
function klaw(directory, options);
78

79
interface WalkerOptions {
80
  /** Method for processing directory contents: 'shift' (breadth-first) or 'pop' (depth-first) */
81
  queueMethod?: 'shift' | 'pop';
82
  /** Function to sort directory contents before processing */
83
  pathSorter?: (a: string, b: string) => number;
84
  /** Function to filter which paths to include in traversal */
85
  filter?: (path: string) => boolean;
86
  /** Maximum recursion depth (-1 for unlimited) */
87
  depthLimit?: number;
88
  /** Whether to follow symlinks (false) or treat as items (true) */
89
  preserveSymlinks?: boolean;
90
  /** Custom file system implementation (defaults to Node.js fs) */
91
  fs?: FileSystemInterface;
92
  /** Additional Readable stream options (objectMode is always true) */
93
  [key: string]: any;
94
}
95

96
interface FileSystemInterface {
97
  stat: (path: string, callback: (err: Error | null, stats: fs.Stats) => void) => void;
98
  lstat: (path: string, callback: (err: Error | null, stats: fs.Stats) => void) => void;
99
  readdir: (path: string, callback: (err: Error | null, files: string[]) => void) => void;
100
}
101

102
interface WalkerItem {
103
  /** Full path to the file or directory */
104
  path: string;
105
  /** Node.js fs.Stats object containing file system metadata */
106
  stats: fs.Stats;
107
}
108
```
109

110
**Stream Events:**
111

112
- **`data`**: Emitted for each file system item with WalkerItem object
113
- **`end`**: Emitted when traversal is complete
114
- **`error`**: Emitted on file system errors, receives (error, item) parameters
115
- **`readable`**: Standard Readable stream event for pull-based consumption
116

117
**Usage Examples:**
118

119
```javascript
120
const klaw = require('klaw');
121
const path = require('path');
122

123
// Basic traversal with data events
124
klaw('/home/user/documents')
125
  .on('data', item => {
126
    if (item.stats.isFile()) {
127
      console.log('File:', item.path, 'Size:', item.stats.size);
128
    }
129
  })
130
  .on('error', (err, item) => {
131
    console.error('Error processing:', item.path, err.message);
132
  })
133
  .on('end', () => console.log('Traversal complete'));
134

135
// Filtering hidden files and directories
136
const filterFunc = item => {
137
  const basename = path.basename(item);
138
  return basename === '.' || basename[0] !== '.';
139
};
140

141
klaw('/some/directory', { filter: filterFunc })
142
  .on('data', item => {
143
    // Only non-hidden items reach here
144
    console.log(item.path);
145
  });
146

147
// Depth-limited traversal
148
klaw('/deep/directory', { depthLimit: 2 })
149
  .on('data', item => {
150
    console.log(item.path);
151
  });
152

153
// Custom sorting (alphabetical)
154
klaw('/directory', { 
155
  pathSorter: (a, b) => a.localeCompare(b) 
156
})
157
  .on('data', item => {
158
    console.log(item.path);
159
  });
160

161
// File URL support
162
const { pathToFileURL } = require('url');
163
klaw(pathToFileURL('/some/directory'))
164
  .on('data', item => {
165
    console.log(item.path);
166
  });
167
```
168

169
### Stream Processing Integration
170

171
Since klaw returns a standard Node.js Readable stream, it integrates seamlessly with stream processing libraries.
172

173
**Example with through2 for advanced filtering:**
174

175
```javascript
176
const klaw = require('klaw');
177
const through2 = require('through2');
178
const path = require('path');
179

180
// Filter to only include .js files
181
const jsFilesOnly = through2.obj(function (item, enc, next) {
182
  if (!item.stats.isDirectory() && path.extname(item.path) === '.js') {
183
    this.push(item);
184
  }
185
  next();
186
});
187

188
klaw('/project/src')
189
  .pipe(jsFilesOnly)
190
  .on('data', item => {
191
    console.log('JavaScript file:', item.path);
192
  });
193

194
// Aggregate file sizes by extension
195
let totalSizes = {};
196
const aggregateByExtension = through2.obj(function (item, enc, next) {
197
  if (item.stats.isFile()) {
198
    const ext = path.extname(item.path) || 'no-extension';
199
    totalSizes[ext] = (totalSizes[ext] || 0) + item.stats.size;
200
  }
201
  this.push(item);
202
  next();
203
});
204

205
klaw('/project')
206
  .pipe(aggregateByExtension)
207
  .on('end', () => {
208
    console.log('File sizes by extension:', totalSizes);
209
  });
210
```
211

212
### Error Handling
213

214
Klaw provides comprehensive error handling for file system operations.
215

216
```javascript { .api }
217
// Error event signature
218
walker.on('error', (error: Error, item: WalkerItem) => void);
219
```
220

221
**Error Handling Examples:**
222

223
```javascript
224
const klaw = require('klaw');
225

226
klaw('/some/directory')
227
  .on('data', item => {
228
    console.log(item.path);
229
  })
230
  .on('error', (err, item) => {
231
    // Handle individual file/directory errors
232
    console.error(`Error accessing ${item.path}:`, err.message);
233
    
234
    // Common error types:
235
    // - ENOENT: File or directory doesn't exist
236
    // - EACCES: Permission denied
237
    // - ENOTDIR: Expected directory but found file
238
  })
239
  .on('end', () => {
240
    console.log('Traversal completed despite errors');
241
  });
242

243
// Propagating errors through streams
244
const through2 = require('through2');
245

246
const transform = through2.obj(function (item, enc, next) {
247
  // Process items
248
  this.push(item);
249
  next();
250
});
251

252
klaw('/directory')
253
  .on('error', err => transform.emit('error', err)) // Forward errors
254
  .pipe(transform)
255
  .on('error', err => {
256
    console.error('Stream error:', err.message);
257
  });
258
```
259

260
### Advanced Configuration
261

262
Klaw supports advanced configuration options for customizing traversal behavior.
263

264
**Queue Method Configuration:**
265

266
```javascript
267
// Breadth-first traversal (default)
268
klaw('/directory', { queueMethod: 'shift' })
269
  .on('data', item => console.log(item.path));
270

271
// Depth-first traversal  
272
klaw('/directory', { queueMethod: 'pop' })
273
  .on('data', item => console.log(item.path));
274
```
275

276
**Symlink Handling:**
277

278
```javascript
279
// Default: follow symlinks and traverse target
280
klaw('/directory', { preserveSymlinks: false })
281
  .on('data', item => {
282
    // item.path shows target path for symlinks
283
    console.log(item.path);
284
  });
285

286
// Treat symlinks as items themselves
287
klaw('/directory', { preserveSymlinks: true })
288
  .on('data', item => {
289
    if (item.stats.isSymbolicLink()) {
290
      console.log('Symlink:', item.path);
291
    }
292
  });
293
```
294

295
**Custom File System:**
296

297
```javascript
298
// Using mock-fs for testing
299
const mockFs = require('mock-fs');
300

301
klaw('/virtual/directory', { fs: mockFs })
302
  .on('data', item => {
303
    console.log('Virtual file:', item.path);
304
  });
305
```
306

307
## Types
308

309
```javascript { .api }
310
/**
311
 * Walker class extends Node.js Readable stream
312
 * Internal class, not directly exported
313
 */
314
class Walker extends Readable {
315
  constructor(directory: string | URL, options?: WalkerOptions);
316
  
317
  /** Standard Readable stream methods */
318
  read(size?: number): WalkerItem | null;
319
  pipe<T extends NodeJS.WritableStream>(destination: T, options?: object): T;
320
  
321
  /** Event emitter methods for stream events */
322
  on(event: 'data', listener: (item: WalkerItem) => void): this;
323
  on(event: 'end', listener: () => void): this;
324
  on(event: 'error', listener: (error: Error, item: WalkerItem) => void): this;
325
  on(event: 'readable', listener: () => void): this;
326
}
327
```