0
# Directory Walking
1

2
Comprehensive directory tree traversal with multiple interfaces including async/await, streaming, and synchronous options. Provides advanced filtering, symlink following, and performance optimization for large directory structures.
3

4
## Capabilities
5

6
### Asynchronous Walking (Array-Based)
7

8
Walk directory trees asynchronously, collecting all results into arrays.
9

10
```typescript { .api }
11
/**
12
 * Walk directory tree asynchronously, returning all entries as array
13
 * @param entry - Starting directory (defaults to current working directory)
14
 * @param opts - Walking options
15
 * @returns Promise resolving to array of all found entries
16
 */
17
walk(): Promise<PathBase[]>;
18
walk(opts: WalkOptionsWithFileTypesTrue | WalkOptionsWithFileTypesUnset): Promise<PathBase[]>;
19
walk(opts: WalkOptionsWithFileTypesFalse): Promise<string[]>;
20
walk(opts: WalkOptions): Promise<string[] | PathBase[]>;
21
walk(entry: string | PathBase): Promise<PathBase[]>;
22
walk(entry: string | PathBase, opts: WalkOptionsWithFileTypesTrue | WalkOptionsWithFileTypesUnset): Promise<PathBase[]>;
23
walk(entry: string | PathBase, opts: WalkOptionsWithFileTypesFalse): Promise<string[]>;
24
walk(entry: string | PathBase, opts: WalkOptions): Promise<PathBase[] | string[]>;
25
```
26

27
**Usage Examples:**
28

29
```typescript
30
import { PathScurry } from "path-scurry";
31

32
const pw = new PathScurry();
33

34
// Basic tree walk
35
const allEntries = await pw.walk();
36
console.log(`Found ${allEntries.length} total entries`);
37

38
// Walk specific directory
39
const srcEntries = await pw.walk("./src");
40
const jsFiles = srcEntries.filter(entry => 
41
  entry.isFile() && entry.name.endsWith(".js")
42
);
43

44
// Get paths as strings
45
const pathStrings = await pw.walk("./dist", { withFileTypes: false });
46
console.log("All paths:", pathStrings);
47

48
// Walk with filtering
49
const textFiles = await pw.walk("./docs", {
50
  filter: (entry) => entry.isFile() && /\.(md|txt)$/.test(entry.name)
51
});
52

53
// Follow symbolic links
54
const allWithSymlinks = await pw.walk({
55
  follow: true,
56
  filter: (entry) => !entry.name.startsWith(".")
57
});
58
```
59

60
### Synchronous Walking
61

62
Walk directory trees synchronously with identical API to async version.
63

64
```typescript { .api }
65
/**
66
 * Walk directory tree synchronously, returning all entries as array
67
 * @param entry - Starting directory (defaults to current working directory)
68
 * @param opts - Walking options
69
 * @returns Array of all found entries
70
 */
71
walkSync(): PathBase[];
72
walkSync(opts: WalkOptionsWithFileTypesTrue | WalkOptionsWithFileTypesUnset): PathBase[];
73
walkSync(opts: WalkOptionsWithFileTypesFalse): string[];
74
walkSync(opts: WalkOptions): string[] | PathBase[];
75
walkSync(entry: string | PathBase): PathBase[];
76
walkSync(entry: string | PathBase, opts: WalkOptionsWithFileTypesUnset | WalkOptionsWithFileTypesTrue): PathBase[];
77
walkSync(entry: string | PathBase, opts: WalkOptionsWithFileTypesFalse): string[];
78
walkSync(entry: string | PathBase, opts: WalkOptions): PathBase[] | string[];
79
```
80

81
**Usage Examples:**
82

83
```typescript
84
const pw = new PathScurry("/project");
85

86
// Synchronous walk
87
const entries = pw.walkSync("src");
88
const totalSize = entries
89
  .filter(e => e.isFile())
90
  .reduce((sum, e) => sum + (e.size || 0), 0);
91

92
console.log(`Total size: ${totalSize} bytes`);
93

94
// Process build directory
95
const buildFiles = pw.walkSync("./dist", { withFileTypes: false });
96
const htmlFiles = buildFiles.filter(path => path.endsWith(".html"));
97
console.log(`Found ${htmlFiles.length} HTML files`);
98
```
99

100
### Streaming Interface
101

102
Stream entries as they're discovered for memory-efficient processing of large directory trees.
103

104
```typescript { .api }
105
/**
106
 * Create stream that emits entries as they're discovered
107
 * @param entry - Starting directory (defaults to current working directory)
108
 * @param opts - Walking options
109
 * @returns Minipass stream emitting Path objects or strings
110
 */
111
stream(): Minipass<PathBase>;
112
stream(opts: WalkOptionsWithFileTypesTrue | WalkOptionsWithFileTypesUnset): Minipass<PathBase>;
113
stream(opts: WalkOptionsWithFileTypesFalse): Minipass<string>;
114
stream(opts: WalkOptions): Minipass<string | PathBase>;
115
stream(entry: string | PathBase): Minipass<PathBase>;
116
stream(entry: string | PathBase, opts: WalkOptionsWithFileTypesUnset | WalkOptionsWithFileTypesTrue): Minipass<PathBase>;
117
stream(entry: string | PathBase, opts: WalkOptionsWithFileTypesFalse): Minipass<string>;
118
stream(entry: string | PathBase, opts: WalkOptions): Minipass<string> | Minipass<PathBase>;
119

120
/**
121
 * Synchronous version of stream interface
122
 */
123
streamSync(): Minipass<PathBase>;
124
streamSync(opts: WalkOptionsWithFileTypesTrue | WalkOptionsWithFileTypesUnset): Minipass<PathBase>;
125
streamSync(opts: WalkOptionsWithFileTypesFalse): Minipass<string>;
126
streamSync(opts: WalkOptions): Minipass<string | PathBase>;
127
```
128

129
**Usage Examples:**
130

131
```typescript
132
const pw = new PathScurry();
133

134
// Stream processing
135
const stream = pw.stream("./large-directory");
136
let fileCount = 0;
137

138
stream.on("data", (entry) => {
139
  if (entry.isFile()) {
140
    fileCount++;
141
    if (fileCount % 1000 === 0) {
142
      console.log(`Processed ${fileCount} files so far...`);
143
    }
144
  }
145
});
146

147
stream.on("end", () => {
148
  console.log(`Total files processed: ${fileCount}`);
149
});
150

151
// Pipeline example with filtering
152
const filteredStream = pw.stream({
153
  filter: (entry) => entry.isFile() && entry.name.endsWith(".log")
154
});
155

156
filteredStream.on("data", (logFile) => {
157
  console.log(`Found log file: ${logFile.fullpath()}`);
158
});
159

160
// Synchronous stream (completes in single tick if fully consumed)
161
const syncStream = pw.streamSync("./config");
162
const configFiles: string[] = [];
163

164
syncStream.on("data", (entry) => {
165
  if (entry.isFile()) {
166
    configFiles.push(entry.fullpath());
167
  }
168
});
169

170
syncStream.on("end", () => {
171
  console.log("Config files:", configFiles);
172
});
173
```
174

175
### Async Iterator Interface
176

177
Use async iterators for `for await` loops and functional programming patterns.
178

179
```typescript { .api }
180
/**
181
 * Create async iterator for directory walking
182
 * @param entry - Starting directory (defaults to current working directory)
183
 * @param options - Walking options
184
 * @returns AsyncGenerator yielding Path objects or strings
185
 */
186
iterate(): AsyncGenerator<PathBase, void, void>;
187
iterate(opts: WalkOptionsWithFileTypesTrue | WalkOptionsWithFileTypesUnset): AsyncGenerator<PathBase, void, void>;
188
iterate(opts: WalkOptionsWithFileTypesFalse): AsyncGenerator<string, void, void>;
189
iterate(opts: WalkOptions): AsyncGenerator<string | PathBase, void, void>;
190
iterate(entry: string | PathBase): AsyncGenerator<PathBase, void, void>;
191
iterate(entry: string | PathBase, opts: WalkOptionsWithFileTypesTrue | WalkOptionsWithFileTypesUnset): AsyncGenerator<PathBase, void, void>;
192
iterate(entry: string | PathBase, opts: WalkOptionsWithFileTypesFalse): AsyncGenerator<string, void, void>;
193
iterate(entry: string | PathBase, opts: WalkOptions): AsyncGenerator<PathBase | string, void, void>;
194

195
/**
196
 * Default async iterator (alias for iterate())
197
 */
198
[Symbol.asyncIterator](): AsyncGenerator<PathBase, void, void>;
199
```
200

201
**Usage Examples:**
202

203
```typescript
204
const pw = new PathScurry();
205

206
// For-await-of loop
207
for await (const entry of pw) {
208
  if (entry.isFile() && entry.name.endsWith(".ts")) {
209
    console.log(`TypeScript file: ${entry.relative()}`);
210
  }
211
}
212

213
// Explicit iteration
214
const iterator = pw.iterate("./src", { withFileTypes: false });
215
for await (const path of iterator) {
216
  if (path.includes("test")) {
217
    console.log(`Test file: ${path}`);
218
  }
219
}
220

221
// Functional processing
222
const results = [];
223
for await (const entry of pw.iterate({ filter: e => e.isFile() })) {
224
  if (entry.size && entry.size > 1024 * 1024) { // Files > 1MB
225
    results.push({
226
      path: entry.fullpath(),
227
      size: entry.size,
228
      modified: entry.mtime
229
    });
230
  }
231
}
232
```
233

234
### Synchronous Iterator Interface
235

236
Use synchronous iterators for `for of` loops.
237

238
```typescript { .api }
239
/**
240
 * Create synchronous iterator for directory walking
241
 * @param entry - Starting directory (defaults to current working directory)
242
 * @param opts - Walking options
243
 * @returns Generator yielding Path objects or strings
244
 */
245
iterateSync(): Generator<PathBase, void, void>;
246
iterateSync(opts: WalkOptionsWithFileTypesTrue | WalkOptionsWithFileTypesUnset): Generator<PathBase, void, void>;
247
iterateSync(opts: WalkOptionsWithFileTypesFalse): Generator<string, void, void>;
248
iterateSync(opts: WalkOptions): Generator<string | PathBase, void, void>;
249
iterateSync(entry: string | PathBase): Generator<PathBase, void, void>;
250
iterateSync(entry: string | PathBase, opts: WalkOptionsWithFileTypesTrue | WalkOptionsWithFileTypesUnset): Generator<PathBase, void, void>;
251
iterateSync(entry: string | PathBase, opts: WalkOptionsWithFileTypesFalse): Generator<string, void, void>;
252
iterateSync(entry: string | PathBase, opts: WalkOptions): Generator<PathBase | string, void, void>;
253

254
/**
255
 * Default sync iterator (alias for iterateSync())
256
 */
257
[Symbol.iterator](): Generator<PathBase, void, void>;
258
```
259

260
**Usage Examples:**
261

262
```typescript
263
const pw = new PathScurry();
264

265
// Synchronous for-of loop
266
for (const entry of pw) {
267
  if (entry.isDirectory() && entry.name.startsWith(".")) {
268
    console.log(`Hidden directory: ${entry.name}`);
269
  }
270
}
271

272
// Process specific directory
273
const packages = [];
274
for (const entry of pw.iterateSync("./node_modules")) {
275
  if (entry.isDirectory() && entry.parent?.name === "node_modules") {
276
    packages.push(entry.name);
277
  }
278
}
279
console.log(`Found ${packages.length} packages`);
280
```
281

282
### Walking Options
283

284
Configure walking behavior with comprehensive options.
285

286
```typescript { .api }
287
interface WalkOptions {
288
  /**
289
   * Return Path objects (true) or path strings (false)
290
   * @default true
291
   */
292
  withFileTypes?: boolean;
293

294
  /**
295
   * Follow symbolic links to directories
296
   * @default false
297
   */
298
  follow?: boolean;
299

300
  /**
301
   * Filter function to include/exclude entries from results
302
   * Does not prevent directory traversal
303
   */
304
  filter?: (entry: PathBase) => boolean;
305

306
  /**
307
   * Filter function to control which directories are traversed
308
   * Does not affect result inclusion
309
   */
310
  walkFilter?: (entry: PathBase) => boolean;
311
}
312
```
313

314
**Advanced Usage Examples:**
315

316
```typescript
317
const pw = new PathScurry();
318

319
// Complex filtering
320
const results = await pw.walk({
321
  // Only include source files in results
322
  filter: (entry) => {
323
    if (entry.isFile()) {
324
      return /\.(js|ts|jsx|tsx)$/.test(entry.name);
325
    }
326
    return true; // Include directories in results too
327
  },
328
  
329
  // Don't traverse hidden or node_modules directories
330
  walkFilter: (entry) => {
331
    if (entry.isDirectory()) {
332
      const name = entry.name;
333
      return !name.startsWith(".") && name !== "node_modules";
334
    }
335
    return true;
336
  },
337
  
338
  // Follow symlinks but be careful of cycles
339
  follow: true,
340
  
341
  // Return as Path objects for rich metadata
342
  withFileTypes: true
343
});
344

345
// Find large files while avoiding certain directories
346
const largeFiles = [];
347
for await (const entry of pw.iterate({
348
  filter: (entry) => entry.isFile() && (entry.size || 0) > 10 * 1024 * 1024,
349
  walkFilter: (entry) => entry.name !== "node_modules" && !entry.name.startsWith(".")
350
})) {
351
  largeFiles.push({
352
    path: entry.fullpath(),
353
    size: entry.size,
354
    sizeInMB: Math.round((entry.size || 0) / (1024 * 1024))
355
  });
356
}
357

358
console.log("Large files (>10MB):", largeFiles);
359
```
360

361
### Performance Considerations
362

363
- **Array-based methods** (`walk`, `walkSync`): Memory-intensive but convenient for small/medium trees
364
- **Streaming methods** (`stream`, `streamSync`): Memory-efficient, ideal for large trees
365
- **Iterator methods** (`iterate`, `iterateSync`): Good balance, supports functional patterns
366
- **Caching**: Warm cache provides 10-80x performance improvement
367
- **Symlink following**: Adds overhead due to additional `readlink` calls
368
- **Large directories**: Use streaming or iterator interfaces to avoid memory issues