CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/npm-batch

Simple async batch with concurrency control and progress reporting.

—

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Pending

The risk profile of this skill

Overview
Eval results
Files

index.mddocs/

Batch

Batch is a simple async batch processing library for Node.js that enables developers to execute multiple asynchronous functions with configurable concurrency control and real-time progress reporting. It provides an intuitive API for batching async operations, tracking completion percentages, monitoring execution times, and handling errors gracefully.

Package Information

  • Package Name: batch
  • Package Type: npm
  • Language: JavaScript
  • Installation: npm install batch

Core Imports

const Batch = require('batch');

For ES modules (if using build tools):

import Batch from 'batch';

Basic Usage

const Batch = require('batch');

// Create a new batch
const batch = new Batch();

// Configure concurrency (optional)
batch.concurrency(4);

// Add functions to the batch
['user1', 'user2', 'user3'].forEach(userId => {
  batch.push(function(done) {
    // Simulate async operation
    setTimeout(() => {
      done(null, { id: userId, name: `User ${userId}` });
    }, Math.random() * 1000);
  });
});

// Track progress (optional)
batch.on('progress', function(e) {
  console.log(`Progress: ${e.percent}% (${e.complete}/${e.total})`);
});

// Execute the batch
batch.end(function(err, results) {
  if (err) {
    console.error('Batch failed:', err);
  } else {
    console.log('All operations completed:', results);
  }
});

Architecture

Batch is built around a single class that extends EventEmitter, providing:

  • Queue Management: Functions are queued and executed with controlled concurrency
  • Event-Driven Progress: Real-time progress events with detailed execution metrics
  • Error Handling: Configurable error strategies (fail-fast or collect-all)
  • Result Ordering: Results maintain the order functions were added, not execution order
  • Cross-Platform: Works in both Node.js and browser environments

Capabilities

Batch Constructor

Creates a new Batch instance for managing async operations.

/**
 * Create a new Batch instance
 * @param {...Function} functions - Optional functions to add immediately
 * @returns {Batch} New Batch instance
 */
function Batch(...functions);

The constructor can be called with or without new and accepts optional functions to add immediately:

// All equivalent ways to create a batch
const batch1 = new Batch();
const batch2 = Batch();
const batch3 = new Batch(fn1, fn2, fn3); // Pre-populate with functions

Concurrency Control

Controls the maximum number of functions executing simultaneously.

/**
 * Set concurrency level
 * @param {number} n - Maximum number of concurrent operations
 * @returns {Batch} This instance for chaining
 */
concurrency(n);

Usage Example:

const batch = new Batch();
batch.concurrency(3); // Max 3 operations at once
// Default is Infinity (no limit)

Function Queuing

Adds functions to the batch queue for execution.

/**
 * Add a function to the batch queue
 * @param {Function} fn - Async function that accepts a callback
 * @returns {Batch} This instance for chaining
 */
push(fn);

Functions must follow the Node.js callback pattern:

batch.push(function(callback) {
  // Perform async operation
  someAsyncOperation((err, result) => {
    callback(err, result); // Call callback with (error, result)
  });
});

Error Handling Configuration

Controls how the batch handles errors from individual functions.

/**
 * Configure error handling behavior
 * @param {boolean} throws - If true, stop on first error; if false, collect all errors
 * @returns {Batch} This instance for chaining
 */
throws(throws);

Usage Examples:

// Default behavior: stop on first error
const batch1 = new Batch();
batch1.throws(true); // Default

// Collect all errors and results
const batch2 = new Batch();
batch2.throws(false);

Batch Execution

Executes all queued functions and handles results.

/**
 * Execute all queued functions
 * @param {Function} callback - Completion callback
 * @returns {Batch} This instance
 */
end(callback);

The callback signature depends on the error handling mode:

  • When throws(true) (default): callback(err, results)
    • err: First error encountered, or null if all succeeded
    • results: Array of results in order functions were added
  • When throws(false): callback(errors, results)
    • errors: Array of errors/nulls matching function positions
    • results: Array of results/undefined matching function positions

Usage Examples:

// Fail-fast mode (default)
batch.end(function(err, results) {
  if (err) {
    console.error('First error:', err);
  } else {
    console.log('All results:', results);
  }
});

// Collect-all mode  
batch.throws(false);
batch.end(function(errors, results) {
  errors.forEach((err, index) => {
    if (err) console.error(`Function ${index} failed:`, err);
  });
  console.log('All results:', results);
});

Progress Monitoring

The batch emits progress events as functions complete.

/**
 * Progress event data
 */
interface ProgressEvent {
  index: number;        // Zero-based index of completed function
  value: any;          // Return value from the function
  error: Error | null; // Error from the function, if any
  pending: number;     // Number of functions still pending
  total: number;       // Total number of functions in batch
  complete: number;    // Number of completed functions
  percent: number;     // Completion percentage (0-100, integer)
  start: Date;         // Start time of this function
  end: Date;           // End time of this function  
  duration: number;    // Execution duration in milliseconds
}

Usage Example:

batch.on('progress', function(e) {
  console.log(`Function ${e.index} completed in ${e.duration}ms`);
  console.log(`Progress: ${e.percent}% (${e.complete}/${e.total})`);
  
  if (e.error) {
    console.warn(`Function ${e.index} had error:`, e.error);
  } else {
    console.log(`Function ${e.index} result:`, e.value);
  }
});

EventEmitter Methods

Batch inherits from EventEmitter, providing standard event methods:

/**
 * EventEmitter methods available on Batch instances
 */
on(event, listener);          // Add event listener
emit(event, ...args);         // Emit event
removeListener(event, listener); // Remove specific listener
removeAllListeners(event);    // Remove all listeners for event
once(event, listener);        // Add one-time listener
listenerCount(event);         // Count listeners for event

Error Handling

Batch provides two error handling strategies:

Fail-Fast (Default)

const batch = new Batch();
batch.throws(true); // Default behavior

batch.push(fn1);
batch.push(fn2); // If this fails, execution stops
batch.push(fn3); // This won't execute if fn2 fails

batch.end((err, results) => {
  // err is the first error encountered
  // results contains results from successful functions only
});

Collect-All Errors

const batch = new Batch();
batch.throws(false);

batch.push(fn1);
batch.push(fn2); // Even if this fails, fn3 still executes
batch.push(fn3);

batch.end((errors, results) => {
  // errors[0] = null or error from fn1
  // errors[1] = null or error from fn2
  // errors[2] = null or error from fn3
  // results[0] = result from fn1 or undefined
  // results[1] = result from fn2 or undefined  
  // results[2] = result from fn3 or undefined
});

Browser Compatibility

Batch works in browsers with a build system. The package.json includes browser field mappings:

{
  "browser": {
    "emitter": "events"
  }
}

When bundled for browsers, it uses a component-based EventEmitter instead of Node.js's built-in events module.

Common Patterns

API Rate Limiting

const batch = new Batch();
batch.concurrency(2); // Limit to 2 concurrent API calls

apiEndpoints.forEach(url => {
  batch.push(callback => {
    fetch(url)
      .then(response => response.json())
      .then(data => callback(null, data))
      .catch(err => callback(err));
  });
});

File Processing

const batch = new Batch();
batch.concurrency(5);

files.forEach(file => {
  batch.push(callback => {
    fs.readFile(file, 'utf8', (err, content) => {
      if (err) return callback(err);
      // Process content
      const processed = processContent(content);
      callback(null, { file, processed });
    });
  });
});

Database Operations

const batch = new Batch();
batch.concurrency(10);

userIds.forEach(id => {
  batch.push(callback => {
    db.users.findById(id, callback);
  });
});

batch.end((err, users) => {
  if (err) {
    console.error('Database batch failed:', err);
  } else {
    console.log(`Loaded ${users.length} users`);
  }
});

docs

index.md

tile.json