tessl/npm-jest-worker

Module for executing heavy tasks under forked processes in parallel, by providing a Promise based interface, minimum overhead, and bound workers.

—

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Securityby

Pending

The risk profile of this skill

Overview

Eval results

Files

Worker Communication

Name: tessl/npm-jest-worker
Author: tessl

Message passing system for communication between parent process and worker processes/threads. Jest Worker provides structured messaging capabilities including custom message handling and bidirectional communication patterns.

Capabilities

Parent-to-Worker Messaging

The main communication from parent to workers happens automatically through method calls, but understanding the underlying message structure helps with debugging and advanced usage.

// Message types sent from parent to workers
type ChildMessage =
  | ChildMessageInitialize
  | ChildMessageCall  
  | ChildMessageEnd
  | ChildMessageMemUsage
  | ChildMessageCallSetup;

type ChildMessageInitialize = [
  type: typeof CHILD_MESSAGE_INITIALIZE,
  isProcessed: boolean,
  fileName: string,
  setupArgs: Array<unknown>,
  workerId: string | undefined
];

type ChildMessageCall = [
  type: typeof CHILD_MESSAGE_CALL,
  isProcessed: boolean,
  methodName: string,
  args: Array<unknown>
];

type ChildMessageEnd = [
  type: typeof CHILD_MESSAGE_END,
  isProcessed: boolean
];

type ChildMessageMemUsage = [type: typeof CHILD_MESSAGE_MEM_USAGE];

type ChildMessageCallSetup = [type: typeof CHILD_MESSAGE_CALL_SETUP];

// Message constants
const CHILD_MESSAGE_INITIALIZE = 0;
const CHILD_MESSAGE_CALL = 1;
const CHILD_MESSAGE_END = 2;
const CHILD_MESSAGE_MEM_USAGE = 3;
const CHILD_MESSAGE_CALL_SETUP = 4;

Worker-to-Parent Messaging

Workers send responses and status updates back to the parent process through structured message types.

// Message types sent from workers to parent
type ParentMessage =
  | ParentMessageOk
  | ParentMessageError
  | ParentMessageCustom
  | ParentMessageMemUsage;

type ParentMessageOk = [
  type: typeof PARENT_MESSAGE_OK, 
  result: unknown
];

type ParentMessageError = [
  type: PARENT_MESSAGE_ERROR,
  constructorName: string,
  message: string,
  stack: string,
  extra: unknown
];

type ParentMessageCustom = [
  type: typeof PARENT_MESSAGE_CUSTOM,
  result: unknown
];

type ParentMessageMemUsage = [
  type: typeof PARENT_MESSAGE_MEM_USAGE,
  usedMemory: number
];

type PARENT_MESSAGE_ERROR =
  | typeof PARENT_MESSAGE_CLIENT_ERROR
  | typeof PARENT_MESSAGE_SETUP_ERROR;

// Message constants  
const PARENT_MESSAGE_OK = 0;
const PARENT_MESSAGE_CLIENT_ERROR = 1;
const PARENT_MESSAGE_SETUP_ERROR = 2;
const PARENT_MESSAGE_CUSTOM = 3;
const PARENT_MESSAGE_MEM_USAGE = 4;

Custom Message Handling

Workers can send custom messages to the parent process using the messageParent function, and the parent can listen for these messages.

/**
 * Send custom messages from worker to parent process
 * @param message - Any serializable data to send
 * @param parentProcess - Process to send to (defaults to global process)
 */
function messageParent(
  message: unknown,
  parentProcess?: NodeJS.Process
): void;

Usage in Worker Module:

// worker.js
const { messageParent } = require("jest-worker");

exports.longRunningTask = async function(data) {
  // Send progress updates
  messageParent({ type: "progress", percent: 0 });
  
  for (let i = 0; i < 100; i++) {
    // Do work...
    await processChunk(data, i);
    
    // Send progress update
    messageParent({ 
      type: "progress", 
      percent: ((i + 1) / 100) * 100,
      message: `Processed chunk ${i + 1}` 
    });
  }
  
  return { completed: true };
};

exports.logStatus = function(message) {
  messageParent({ 
    type: "log", 
    level: "info", 
    message,
    timestamp: new Date().toISOString()
  });
  return "logged";
};

Promise with Custom Messages

Method calls on workers return enhanced promises that support custom message listeners.

/**
 * Enhanced promise that supports custom message handling
 */
interface PromiseWithCustomMessage<T> extends Promise<T> {
  /**
   * Register listener for custom messages from worker
   * @param listener - Function to handle custom messages
   * @returns Function to unregister the listener
   */
  UNSTABLE_onCustomMessage?: (listener: OnCustomMessage) => () => void;
}

/**
 * Handler for custom messages from workers
 * @param message - Message data sent from worker
 */
type OnCustomMessage = (message: Array<unknown> | unknown) => void;

Usage in Parent Process:

import { Worker } from "jest-worker";

const worker = new Worker(require.resolve("./worker"));

// Listen for custom messages during task execution
const promise = worker.longRunningTask(largeDataSet);

if (promise.UNSTABLE_onCustomMessage) {
  const unsubscribe = promise.UNSTABLE_onCustomMessage((message) => {
    if (message.type === "progress") {
      console.log(`Progress: ${message.percent}% - ${message.message}`);
    } else if (message.type === "log") {
      console.log(`[${message.level}] ${message.message}`);
    }
  });
  
  // Clean up listener when done
  promise.finally(() => unsubscribe());
}

const result = await promise;
console.log("Task completed:", result);

Communication Event Handlers

Lower-level event handlers for worker communication lifecycle.

/**
 * Called when a worker task starts processing
 * @param worker - The worker instance handling the task
 */
type OnStart = (worker: WorkerInterface) => void;

/**
 * Called when a worker task completes or fails
 * @param err - Error if task failed, null if successful
 * @param result - Task result if successful
 */
type OnEnd = (err: Error | null, result: unknown) => void;

/**
 * Worker callback function for sending messages
 * @param workerId - ID of target worker
 * @param request - Message to send
 * @param onStart - Handler for task start
 * @param onEnd - Handler for task completion
 * @param onCustomMessage - Handler for custom messages
 */
type WorkerCallback = (
  workerId: number,
  request: ChildMessage,
  onStart: OnStart,
  onEnd: OnEnd,
  onCustomMessage: OnCustomMessage
) => void;

Advanced Communication Patterns

Progress Reporting

Implement progress reporting for long-running tasks:

// worker.js - Progress reporting worker
const { messageParent } = require("jest-worker");

exports.processLargeDataset = async function(dataset) {
  const total = dataset.length;
  const results = [];
  
  for (let i = 0; i < total; i++) {
    const item = dataset[i];
    
    // Process item
    const result = await processItem(item);
    results.push(result);
    
    // Report progress every 10 items or on last item
    if (i % 10 === 0 || i === total - 1) {
      messageParent({
        type: "progress",
        completed: i + 1,
        total,
        percent: Math.round(((i + 1) / total) * 100)
      });
    }
  }
  
  return { results, total: results.length };
};

// parent.js - Progress listener
const worker = new Worker("./progress-worker.js");

const promise = worker.processLargeDataset(bigDataset);

promise.UNSTABLE_onCustomMessage?.((message) => {
  if (message.type === "progress") {
    console.log(`Processing: ${message.completed}/${message.total} (${message.percent}%)`);
  }
});

const results = await promise;

Logging and Debugging

Send structured log messages from workers:

// worker.js - Logging worker
const { messageParent } = require("jest-worker");

function log(level, message, data = {}) {
  messageParent({
    type: "log",
    level,
    message,
    data,
    timestamp: new Date().toISOString(),
    workerId: process.env.JEST_WORKER_ID
  });
}

exports.complexTask = async function(input) {
  log("info", "Starting complex task", { inputSize: input.length });
  
  try {
    const preprocessed = await preprocess(input);
    log("debug", "Preprocessing completed", { outputSize: preprocessed.length });
    
    const result = await mainProcess(preprocessed);
    log("info", "Task completed successfully");
    
    return result;
  } catch (error) {
    log("error", "Task failed", { error: error.message, stack: error.stack });
    throw error;
  }
};

State Synchronization

Synchronize state between parent and workers:

// worker.js - State synchronization
const { messageParent } = require("jest-worker");

let workerState = { processed: 0, cache: new Map() };

exports.processWithState = function(data) {
  // Update local state
  workerState.processed++;
  workerState.cache.set(data.id, data.result);
  
  // Sync state to parent
  messageParent({
    type: "state-update",
    workerId: process.env.JEST_WORKER_ID,
    state: {
      processed: workerState.processed,
      cacheSize: workerState.cache.size
    }
  });
  
  return { success: true, processed: workerState.processed };
};

exports.getState = function() {
  return workerState;
};

Error Handling in Communication

Worker Error Messages

Workers automatically send error messages for unhandled exceptions:

// Error message structure
type ParentMessageError = [
  type: PARENT_MESSAGE_ERROR,
  constructorName: string, // Error constructor name (e.g., "TypeError")
  message: string,         // Error message
  stack: string,          // Stack trace
  extra: unknown          // Additional error data
];

Custom Error Handling

Handle communication errors in custom messages:

// worker.js - Error handling
const { messageParent } = require("jest-worker");

exports.taskWithCustomErrors = function(data) {
  try {
    const result = processData(data);
    
    // Send success with custom metadata
    messageParent({
      type: "task-complete",
      success: true,
      metadata: { processingTime: Date.now() - startTime }
    });
    
    return result;
  } catch (error) {
    // Send custom error message
    messageParent({
      type: "task-error",
      success: false,
      error: {
        name: error.constructor.name,
        message: error.message,
        code: error.code || "UNKNOWN_ERROR",
        recoverable: isRecoverableError(error)
      }
    });
    
    throw error; // Still throw for normal error handling
  }
};

Communication Best Practices

Message Serialization

Jest Worker handles serialization automatically, but be aware of limitations:

// Supported data types
messageParent({
  string: "text",
  number: 42,
  boolean: true,
  array: [1, 2, 3],
  object: { key: "value" },
  null: null,
  undefined: undefined // Becomes null in serialization
});

// Problematic data types (handled automatically with fallbacks)
messageParent({
  function: () => {}, // Will be serialized using structured cloning
  symbol: Symbol("test"), // Will be handled with fallback serialization
  circular: circularObject // Will be handled with structured cloning
});

Performance Considerations

Custom messages add overhead - use sparingly for frequent operations
Batch multiple updates into single messages when possible
Consider message size for large data transfers
Use worker binding to reduce cross-worker communication

docs

index.md

task-queues.md

worker-communication.md

worker-management.md

tile.json

tessl/npm-jest-worker