or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

index.md
tile.json

tessl/npm-watchpack

Wrapper library for directory and file watching with three-level architecture and optimized resource usage

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
npmpkg:npm/watchpack@2.4.x

To install, run

npx @tessl/cli install tessl/npm-watchpack@2.4.0

index.mddocs/

Watchpack

Watchpack is a wrapper library for directory and file watching that implements a three-level architecture to ensure only a single watcher exists per directory. It provides an event-driven API for monitoring file and directory changes with features including aggregated event handling, polling fallback for network paths, symlink handling options, and flexible ignore patterns. The library is designed for maximum efficiency through reference-counting, supports watching files that don't yet exist, and provides comprehensive time information tracking for all monitored resources.

Package Information

  • Package Name: watchpack
  • Package Type: npm
  • Language: JavaScript
  • Installation: npm install watchpack

Core Imports

const Watchpack = require("watchpack");

For ES modules:

import Watchpack from "watchpack";

Basic Usage

const Watchpack = require("watchpack");

// Create a new watcher instance
const wp = new Watchpack({
  aggregateTimeout: 1000, // Fire aggregated event after 1000ms of no changes
  poll: false, // Use native watching (set to true for network paths)
  followSymlinks: false, // Don't follow symlinks
  ignored: "**/.git" // Ignore git directories
});

// Set up event listeners
wp.on("change", function(filePath, mtime, explanation) {
  console.log("File changed:", filePath);
  console.log("Modified time:", mtime);
  console.log("Detection method:", explanation);
});

wp.on("remove", function(filePath, explanation) {
  console.log("File removed:", filePath);
  console.log("Detection method:", explanation);
});

wp.on("aggregated", function(changes, removals) {
  console.log("Changed files:", Array.from(changes));
  console.log("Removed files:", Array.from(removals));
});

// Start watching
wp.watch({
  files: ["/path/to/file.js", "/path/to/another.js"],
  directories: ["/path/to/watch"],
  missing: ["/path/that/might/be/created"],
  startTime: Date.now() - 10000 // Start watching from 10 seconds ago
});

// Later: pause, resume, or close
wp.pause(); // Stop emitting events but keep watchers
wp.close(); // Stop watching completely

Architecture

Watchpack implements a three-level architecture:

  • High-level API: The Watchpack class provides the user-facing interface
  • DirectoryWatchers: Managed by WatcherManager, ensures only one watcher per directory
  • Real Watchers: Created by DirectoryWatcher, handles actual file system events
  • Reference Counting: Automatic cleanup when watchers are no longer needed
  • Optimization: Uses reducePlan to stay within system watcher limits

Key design principles:

  • Files are never watched directly to keep watcher count low
  • Watching can start in the past for consistency
  • Symlinks are watched as symlinks, not followed by default
  • Aggregated events reduce noise from rapid file changes

Capabilities

Main Watchpack Class

The primary interface for file and directory watching with event aggregation and resource optimization.

/**
 * Main watcher class providing high-level file/directory watching API
 * @param {WatchpackOptions} options - Configuration options
 */
class Watchpack extends EventEmitter {
  constructor(options);
}

/**
 * Configuration options for Watchpack constructor
 */
interface WatchpackOptions {
  /** Timeout in ms for aggregated events (default: 200) */
  aggregateTimeout?: number;
  /** Enable polling: true/false or polling interval in ms */
  poll?: boolean | number;
  /** Follow symlinks when watching (default: false) */
  followSymlinks?: boolean;
  /** Patterns/functions to ignore files/directories */
  ignored?: string | string[] | RegExp | ((path: string) => boolean);
}

Watching Operations

Core methods for starting, controlling, and stopping file system monitoring.

/**
 * Start watching specified files and directories
 * @param {WatchOptions} options - What to watch
 * @returns {void}
 */
watch(options);

/**
 * Alternative signature for backward compatibility
 * @param {string[]} files - Files to watch
 * @param {string[]} directories - Directories to watch  
 * @param {number} [startTime] - Optional start time
 * @returns {void}
 */
watch(files, directories, startTime);

/**
 * Watch options interface
 */
interface WatchOptions {
  /** Files to watch for content and existence changes */
  files?: Iterable<string>;
  /** Directories to watch for content changes (recursive) */
  directories?: Iterable<string>;
  /** Files/directories expected not to exist initially */
  missing?: Iterable<string>;
  /** Timestamp to start watching from */
  startTime?: number;
}

/**
 * Stop emitting events and close all watchers
 * @returns {void}
 */
close();

/**
 * Stop emitting events but keep watchers open for reuse
 * @returns {void}
 */
pause();

Time Information Access

Methods for accessing file modification times and metadata collected during watching.

/**
 * Get change times for all known files (deprecated)
 * @returns {Object<string, number>} Object with file paths as keys, timestamps as values
 */
getTimes();

/**
 * Get comprehensive time info entries for files and directories
 * @returns {Map<string, TimeInfoEntry>} Map with paths and time info
 */
getTimeInfoEntries();

/**
 * Collect time info entries into provided maps
 * @param {Map<string, TimeInfoEntry>} fileTimestamps - Map for file time info
 * @param {Map<string, TimeInfoEntry>} directoryTimestamps - Map for directory time info
 * @returns {void}
 */
collectTimeInfoEntries(fileTimestamps, directoryTimestamps);

/**
 * Time information entry structure
 */
interface TimeInfoEntry {
  /** Safe time when all changes happened before this point */
  safeTime: number;
  /** File modification time (files only) */
  timestamp?: number;
  /** Timing accuracy information */
  accuracy?: number;
}

Aggregated Event Access

Methods for accessing batched file change information, useful when watching is paused.

/**
 * Get current aggregated changes and removals, clearing internal state
 * @returns {AggregatedResult} Current changes and removals
 */
getAggregated();

/**
 * Result structure for aggregated changes
 */
interface AggregatedResult {
  /** Set of all changed file paths */
  changes: Set<string>;
  /** Set of all removed file paths */
  removals: Set<string>;
}

Events

File system events emitted during watching operations.

/**
 * Emitted when a file changes
 * @event Watchpack#change
 * @param {string} filePath - Path of changed file
 * @param {number} mtime - Last modified time in milliseconds since Unix epoch
 * @param {string} explanation - How change was detected (e.g., "watch", "poll", "initial scan")
 */
on('change', (filePath, mtime, explanation) => void);

/**
 * Emitted when a file is removed  
 * @event Watchpack#remove
 * @param {string} filePath - Path of removed file
 * @param {string} explanation - How removal was detected (e.g., "watch", "poll")
 */
on('remove', (filePath, explanation) => void);

/**
 * Emitted after aggregateTimeout with batched changes
 * @event Watchpack#aggregated
 * @param {Set<string>} changes - Set of all changed file paths
 * @param {Set<string>} removals - Set of all removed file paths
 */
on('aggregated', (changes, removals) => void);

Configuration Options

Aggregate Timeout

Controls when the aggregated event fires after file changes stop:

const wp = new Watchpack({
  aggregateTimeout: 1000 // Wait 1000ms after last change
});

Polling Mode

For network paths or when native watching fails:

const wp = new Watchpack({
  poll: true, // Use default polling interval
  // or
  poll: 5000 // Poll every 5 seconds
});

Can also be controlled via WATCHPACK_POLLING environment variable.

Symlink Handling

Control how symlinks are handled:

const wp = new Watchpack({
  followSymlinks: true // Follow symlinks and watch both symlink and target
});

Ignore Patterns

Flexible ignore patterns to exclude files and directories:

const wp = new Watchpack({
  ignored: "**/.git", // Glob pattern
  // or
  ignored: ["**/node_modules", "**/.git"], // Multiple patterns
  // or  
  ignored: /\\.tmp$/, // Regular expression
  // or
  ignored: (path) => path.includes('temp') // Custom function
});

Watch Types

Files

Files are watched for content changes and existence. If a file doesn't exist initially, a remove event is emitted:

wp.watch({
  files: ["/path/to/file.js", "/path/to/config.json"]
});

Directories

Directories are watched recursively for any changes to contained files and subdirectories:

wp.watch({
  directories: ["/path/to/src", "/path/to/assets"]
});

Missing Files

Files or directories expected not to exist initially. No remove event is emitted if they don't exist:

wp.watch({
  missing: ["/path/that/might/be/created", "/future/directory"]
});

Advanced Usage

Starting Time

Watch for changes that occurred after a specific time:

wp.watch({
  files: ["file.js"],
  startTime: Date.now() - 10000 // Changes in last 10 seconds
});

Pausing and Resuming

Pause watching while keeping watcher instances for efficiency:

wp.pause(); // Stop events but keep watchers

// Get changes that occurred while paused
const { changes, removals } = wp.getAggregated();

// Resume with new watch targets
wp.watch({ files: ["newfile.js"] });

Time Information

Access detailed timing information for build tools:

const timeInfo = wp.getTimeInfoEntries();
for (const [filePath, info] of timeInfo) {
  console.log(`${filePath}: safeTime=${info.safeTime}, mtime=${info.timestamp}`);
}

Error Handling

Watchpack handles common file system errors gracefully:

Common Error Types

  • ENOENT (File not found): Triggers a remove event when a previously watched file is deleted
  • EPERM (Permission denied): Error is logged internally and watching continues for other files
  • EMFILE/ENFILE (Too many open files): Automatically optimizes watcher usage via reducePlan algorithm
  • Network path errors: Automatically falls back to polling mode when native watching fails

Error Handling Examples

const wp = new Watchpack({
  // Enable polling for network paths that might fail with native watching
  poll: 1000,
  // Ignore permission denied errors in certain directories
  ignored: path => {
    try {
      fs.accessSync(path, fs.constants.R_OK);
      return false;
    } catch (err) {
      return err.code === 'EPERM';
    }
  }
});

wp.on('change', (filePath, mtime, explanation) => {
  // explanation provides context about how change was detected
  if (explanation.includes('error')) {
    console.warn(`Change detected with error context: ${explanation}`);
  }
});

Constructor Errors

The Watchpack constructor may throw errors for invalid options:

try {
  const wp = new Watchpack({
    ignored: 123 // Invalid type - should be string, array, RegExp, or function
  });
} catch (err) {
  console.error('Invalid option for ignored:', err.message);
}

Performance Considerations

  • Watcher Limits: Automatically optimizes to stay within system limits (configurable via WATCHPACK_WATCHER_LIMIT)
  • Reference Counting: Shares watchers between multiple watch targets
  • Batch Operations: Use during initial setup for better performance
  • Recursive Watching: Uses efficient recursive watchers on supported platforms (macOS, Windows)

Environment Variables

Watchpack behavior can be controlled through several environment variables:

WATCHPACK_POLLING

Controls polling mode when native file watching is unavailable or unreliable.

  • Values: "true" or polling interval in milliseconds (e.g., "1000")
  • Default: undefined (use native watching)
  • Example: WATCHPACK_POLLING=5000 node app.js (poll every 5 seconds)

WATCHPACK_WATCHER_LIMIT

Sets the maximum number of watchers that can be created before optimization kicks in.

  • Values: Positive integer
  • Default: 10000 on Linux, 20 on macOS (due to system limitations)
  • Example: WATCHPACK_WATCHER_LIMIT=5000 node build.js

WATCHPACK_RECURSIVE_WATCHER_LOGGING

Enables detailed logging for recursive watcher operations, useful for debugging.

  • Values: "true" to enable, any other value or unset to disable
  • Default: undefined (disabled)
  • Example: WATCHPACK_RECURSIVE_WATCHER_LOGGING=true npm run dev

Usage Example

# Force polling mode with 2-second intervals and debug logging
WATCHPACK_POLLING=2000 WATCHPACK_RECURSIVE_WATCHER_LOGGING=true node build.js

# Limit watchers for resource-constrained environments
WATCHPACK_WATCHER_LIMIT=100 node server.js