CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/npm-walkdir

Directory traversal library that walks file system trees with multiple interface options including callback, event emitter, synchronous, and promise-based patterns.

Pending
Overview
Eval results
Files

options-configuration.mddocs/

Configuration Options

Comprehensive options for controlling walkdir traversal behavior, performance optimization, and output formatting.

Capabilities

WalkOptions Interface

Complete configuration interface for all walkdir operations.

/**
 * Configuration options for walkdir operations
 */
interface WalkOptions {
  /** Follow symbolic links during traversal (default: false) */
  follow_symlinks?: boolean;
  
  /** Only traverse one level deep, don't recurse into subdirectories */
  no_recurse?: boolean;
  
  /** Maximum traversal depth (emits 'maxdepth' event when reached) */
  max_depth?: number;
  
  /** Track inodes to prevent infinite loops (default: true) */
  track_inodes?: boolean;
  
  /** Make operation synchronous (equivalent to calling walkdir.sync) */
  sync?: boolean;
  
  /** Return {path: stat} object instead of path array */
  return_object?: boolean;
  
  /** Don't build internal result collection (memory optimization) */
  no_return?: boolean;
  
  /** Filter function for controlling which files/directories are processed */
  filter?: (directory: string, files: string[]) => string[] | Promise<string[]>;
  
  /** Custom fs implementation (e.g., graceful-fs) */
  fs?: any;
  
  /** Use lstat vs stat for link detection (default: true) */
  find_links?: boolean;
}

Symlink Handling

Control how symbolic links are processed during traversal.

/**
 * Symlink handling options
 * @param {boolean} follow_symlinks - Whether to follow symbolic links (default: false)
 * @param {boolean} find_links - Whether to detect symbolic links (default: true)
 */

Usage Examples:

const walkdir = require('walkdir');

// Follow symbolic links (be careful of loops)
const paths = await walkdir.async('./project', {
  follow_symlinks: true,
  max_depth: 10  // Prevent infinite loops
});

// Detect but don't follow symlinks (default behavior)
const emitter = walkdir('./project', {
  follow_symlinks: false,  // explicit default
  find_links: true         // will emit 'link' events
});

emitter.on('link', (path, stat) => {
  console.log('Found symlink:', path);
});

// Ignore symlinks entirely
const paths2 = walkdir.sync('./project', {
  find_links: false  // Use stat() instead of lstat(), won't detect links
});

Depth Control

Options for controlling how deep the traversal goes.

/**
 * Depth control options
 * @param {boolean} no_recurse - Only go one level deep
 * @param {number} max_depth - Maximum depth to traverse
 */

Usage Examples:

// Shallow traversal - only immediate children
const topLevel = walkdir.sync('./project', {
  no_recurse: true
});

// Limited depth traversal
const limitedPaths = await walkdir.async('./project', {
  max_depth: 3
});

// Handle depth limit reached
const emitter = walkdir('./deep-project', { max_depth: 5 });
emitter.on('maxdepth', (path, stat, depth) => {
  console.log('Stopped at max depth:', depth, 'at path:', path);
});

// Combine with filtering for precise control
const controlledPaths = walkdir.sync('./project', {
  max_depth: 4,
  filter: (dir, files) => {
    // Skip deep node_modules
    if (dir.includes('node_modules')) return [];
    return files;
  }
});

Performance Optimization

Options for optimizing memory usage and processing speed.

/**
 * Performance optimization options
 * @param {boolean} no_return - Don't build internal result arrays/objects
 * @param {boolean} track_inodes - Track inodes to prevent processing duplicates
 * @param {object} fs - Custom fs implementation for better performance
 */

Memory Optimization Examples:

// Process huge directories without memory buildup
let totalSize = 0;
let fileCount = 0;

walkdir.sync('/var/log', { 
  no_return: true  // Don't store paths in memory
}, (path, stat) => {
  if (stat.isFile()) {
    fileCount++;
    totalSize += stat.size;
  }
});

console.log(`${fileCount} files, ${totalSize} bytes total`);

// Disable inode tracking for filesystems without unique inodes
const windowsPaths = walkdir.sync('C:\\Projects', {
  track_inodes: false  // Better for Windows/FAT32 filesystems
});

// Use graceful-fs for better performance and reliability
const gracefulFs = require('graceful-fs');
const paths = await walkdir.async('./project', {
  fs: gracefulFs  // Handles EMFILE/ENFILE errors better
});

Output Format Control

Options for controlling the format of returned results.

/**
 * Output format options
 * @param {boolean} return_object - Return {path: stat} instead of path array
 * @param {boolean} no_return - Return nothing (for callback-only processing)
 */

Format Examples:

// Default: array of paths
const pathArray = walkdir.sync('./project');
console.log('Paths:', pathArray);

// Object format: {path: stat}
const pathStats = walkdir.sync('./project', {
  return_object: true
});

for (const [path, stat] of Object.entries(pathStats)) {
  console.log(path, ':', stat.isDirectory() ? 'DIR' : 'FILE', stat.size);
}

// No return value, callback-only processing
walkdir.sync('./project', { no_return: true }, (path, stat) => {
  // Process immediately, no memory storage
  if (stat.isFile() && path.endsWith('.js')) {
    analyzeJavaScriptFile(path);
  }
});

// Async versions support the same formats
const asyncStats = await walkdir.async('./project', {
  return_object: true
});

Advanced Filtering

Sophisticated filtering capabilities including async filters.

/**
 * Filter function signature
 * @param {string} directory - Current directory being read
 * @param {string[]} files - Array of filenames in the directory
 * @returns {string[]|Promise<string[]>} - Filtered array of filenames to process
 */
type FilterFunction = (directory: string, files: string[]) => string[] | Promise<string[]>;

Filtering Examples:

// Simple synchronous filter
const jsFiles = walkdir.sync('./src', {
  filter: (dir, files) => {
    return files.filter(f => f.endsWith('.js') || f.endsWith('.ts'));
  }
});

// Exclude directories and files
const filtered = walkdir.sync('./project', {
  filter: (dir, files) => {
    // Skip node_modules and .git directories entirely
    if (dir.includes('node_modules') || dir.includes('.git')) {
      return [];
    }
    
    // Only include source files
    return files.filter(f => {
      return f.endsWith('.js') || f.endsWith('.ts') || f.endsWith('.json');
    });
  }
});

// Async filter with file system checks
const validFiles = await walkdir.async('./uploads', {
  filter: async (dir, files) => {
    const results = [];
    
    for (const file of files) {
      const fullPath = path.join(dir, file);
      try {
        const stat = await fs.promises.stat(fullPath);
        // Only include files smaller than 10MB
        if (stat.isFile() && stat.size < 10 * 1024 * 1024) {
          results.push(file);
        }
      } catch (error) {
        // Skip files we can't stat
      }
    }
    
    return results;
  }
});

// Pattern-based filtering
const matchedFiles = walkdir.sync('./docs', {
  filter: (dir, files) => {
    return files.filter(f => {
      // Include markdown files and directories
      return f.endsWith('.md') || !f.includes('.');
    });
  }
});

Custom File System Implementation

Use alternative fs implementations for enhanced functionality.

/**
 * Custom fs implementation option
 * @param {object} fs - Object with fs methods: stat, lstat, readdir, readlink (and sync versions)
 */

Custom FS Examples:

const gracefulFs = require('graceful-fs');
const mockFs = require('mock-fs');

// Use graceful-fs for better error handling
const robustPaths = await walkdir.async('./project', {
  fs: gracefulFs  // Handles EMFILE errors, retries failed operations
});

// Use mock-fs for testing
mockFs({
  '/fake-dir': {
    'file1.txt': 'content',
    'subdir': {
      'file2.js': 'console.log("test");'
    }
  }
});

const mockPaths = walkdir.sync('/fake-dir', {
  fs: mockFs  // Uses mocked filesystem
});

mockFs.restore();

// Custom fs wrapper with logging
const loggingFs = {
  ...require('fs'),
  readdir: (path, callback) => {
    console.log('Reading directory:', path);
    return require('fs').readdir(path, callback);
  },
  readdirSync: (path) => {
    console.log('Reading directory sync:', path);
    return require('fs').readdirSync(path);
  }
};

const loggedPaths = walkdir.sync('./project', {
  fs: loggingFs  // Logs all directory reads
});

Option Combinations

Common combinations of options for specific use cases.

// High-performance large directory scan
const bigScan = await walkdir.async('/huge/directory', {
  no_return: true,      // Don't store results in memory
  track_inodes: false,  // Skip inode tracking overhead
  fs: gracefulFs        // Better error handling
}, (path, stat) => {
  // Process each path immediately
  processPath(path, stat);
});

// Safe symlink following with depth protection
const safePaths = await walkdir.async('./project', {
  follow_symlinks: true,
  max_depth: 8,
  track_inodes: true    // Prevent loops
});

// Fast shallow scan with filtering
const quickScan = walkdir.sync('./src', {
  no_recurse: true,     // Only top level
  return_object: true,  // Get stat info
  filter: (dir, files) => files.filter(f => !f.startsWith('.'))
});

Install with Tessl CLI

npx tessl i tessl/npm-walkdir

docs

async-operations.md

core-walking.md

events-control.md

index.md

options-configuration.md

sync-operations.md

tile.json