0
# Worker Management
1

2
Core worker farm functionality for creating, managing, and terminating pools of worker processes or threads. The Worker class provides the main interface for parallel task execution with comprehensive lifecycle management.
3

4
## Capabilities
5

6
### Worker Class
7

8
Main class for creating and managing worker farms. Automatically exposes methods from the worker module as promise-returning methods on the Worker instance.
9

10
```typescript { .api }
11
/**
12
 * Creates a new worker farm for parallel task execution
13
 * @param workerPath - Absolute path or URL to the worker module
14
 * @param options - Configuration options for the worker farm
15
 */
16
class Worker {
17
  constructor(workerPath: string | URL, options?: WorkerFarmOptions);
18
  
19
  /** Returns readable stream of all workers' stdout */
20
  getStdout(): NodeJS.ReadableStream;
21
  
22
  /** Returns readable stream of all workers' stderr */
23
  getStderr(): NodeJS.ReadableStream;
24
  
25
  /** Starts all workers and calls their setup functions */
26
  start(): Promise<void>;
27
  
28
  /** Terminates all workers gracefully */
29
  end(): Promise<PoolExitResult>;
30
}
31
```
32

33
**Usage Examples:**
34

35
```typescript
36
import { Worker } from "jest-worker";
37

38
// Basic worker creation
39
const worker = new Worker(require.resolve("./my-worker"));
40

41
// Worker with custom configuration
42
const configuredWorker = new Worker("./worker.js", {
43
  numWorkers: 8,
44
  enableWorkerThreads: true,
45
  maxRetries: 5,
46
  idleMemoryLimit: 0.1, // 10% of system memory
47
  computeWorkerKey: (method, filename) => filename // Bind by filename
48
});
49

50
// Using the worker
51
const result = await worker.processFile("data.txt");
52
await worker.end();
53
```
54

55
### Worker Configuration Options
56

57
Comprehensive configuration interface for customizing worker behavior, performance, and resource management.
58

59
```typescript { .api }
60
import type { ForkOptions } from 'child_process';
61
import type { ResourceLimits } from 'worker_threads';
62

63
interface WorkerFarmOptions {
64
  /** Function to compute worker binding key for task caching */
65
  computeWorkerKey?: (method: string, ...args: Array<unknown>) => string | null;
66
  
67
  /** Use worker_threads instead of child_process.fork */
68
  enableWorkerThreads?: boolean;
69
  
70
  /** Explicitly specify which methods to expose from worker module */
71
  exposedMethods?: ReadonlyArray<string>;
72
  
73
  /** Options passed to child_process.fork */
74
  forkOptions?: ForkOptions;
75
  
76
  /** Maximum number of retries for failed tasks */
77
  maxRetries?: number;
78
  
79
  /** Number of worker processes/threads to spawn */
80
  numWorkers?: number;
81
  
82
  /** Resource limits for worker_threads */
83
  resourceLimits?: ResourceLimits;
84
  
85
  /** Arguments passed to worker setup function */
86
  setupArgs?: Array<unknown>;
87
  
88
  /** Custom task queue implementation */
89
  taskQueue?: TaskQueue;
90
  
91
  /** Custom worker pool implementation */
92
  WorkerPool?: new (workerPath: string, options?: WorkerPoolOptions) => WorkerPoolInterface;
93
  
94
  /** Strategy for assigning tasks to idle workers */
95
  workerSchedulingPolicy?: WorkerSchedulingPolicy;
96
  
97
  /** Memory limit for worker recycling */
98
  idleMemoryLimit?: number;
99
}
100
```
101

102
**Configuration Examples:**
103

104
```typescript
105
// Memory-conscious configuration
106
const memoryWorker = new Worker("./processor.js", {
107
  idleMemoryLimit: 0.05, // Restart workers using >5% system memory
108
  numWorkers: 2,
109
  maxRetries: 1
110
});
111

112
// High-performance configuration
113
const fastWorker = new Worker("./fast-processor.js", {
114
  enableWorkerThreads: true,
115
  numWorkers: 16,
116
  workerSchedulingPolicy: "round-robin",
117
  resourceLimits: {
118
    maxOldGenerationSizeMb: 100,
119
    maxYoungGenerationSizeMb: 50
120
  }
121
});
122

123
// Bound worker for caching
124
const cachingWorker = new Worker("./file-processor.js", {
125
  computeWorkerKey: (method, filepath) => {
126
    // Same files always go to same worker for caching
127
    return method === "processFile" ? filepath : null;
128
  },
129
  numWorkers: 4
130
});
131
```
132

133
### Worker Pool Interface
134

135
Low-level interface for worker pool implementations, allowing custom worker management strategies.
136

137
```typescript { .api }
138
interface WorkerPoolInterface {
139
  /** Get combined stderr stream from all workers */
140
  getStderr(): NodeJS.ReadableStream;
141
  
142
  /** Get combined stdout stream from all workers */
143
  getStdout(): NodeJS.ReadableStream;
144
  
145
  /** Get array of all worker instances */
146
  getWorkers(): Array<WorkerInterface>;
147
  
148
  /** Create a new worker with specified options */
149
  createWorker(options: WorkerOptions): WorkerInterface;
150
  
151
  /** Send message to specific worker */
152
  send: WorkerCallback;
153
  
154
  /** Start all workers in the pool */
155
  start(): Promise<void>;
156
  
157
  /** Terminate all workers in the pool */
158
  end(): Promise<PoolExitResult>;
159
}
160

161
interface WorkerPoolOptions {
162
  setupArgs: Array<unknown>;
163
  forkOptions: ForkOptions;
164
  resourceLimits: ResourceLimits;
165
  maxRetries: number;
166
  numWorkers: number;
167
  enableWorkerThreads: boolean;
168
  idleMemoryLimit?: number;
169
}
170
```
171

172
### Individual Worker Interface
173

174
Interface for individual worker instances within a worker pool, providing fine-grained worker control and monitoring.
175

176
```typescript { .api }
177
interface WorkerInterface {
178
  /** Current state of the worker */
179
  readonly state: WorkerStates;
180
  
181
  /** Send a message to this specific worker */
182
  send(
183
    request: ChildMessage,
184
    onProcessStart: OnStart,
185
    onProcessEnd: OnEnd,
186
    onCustomMessage: OnCustomMessage
187
  ): void;
188
  
189
  /** Wait for worker to exit gracefully */
190
  waitForExit(): Promise<void>;
191
  
192
  /** Force worker termination */
193
  forceExit(): void;
194
  
195
  /** Get unique worker identifier */
196
  getWorkerId(): number;
197
  
198
  /** Get worker's stderr stream */
199
  getStderr(): NodeJS.ReadableStream | null;
200
  
201
  /** Get worker's stdout stream */
202
  getStdout(): NodeJS.ReadableStream | null;
203
  
204
  /** Get system-level worker identifier (PID, thread ID, etc.) */
205
  getWorkerSystemId(): number;
206
  
207
  /** Get current memory usage of worker */
208
  getMemoryUsage(): Promise<number | null>;
209
  
210
  /** Check if worker process/thread is running */
211
  isWorkerRunning(): boolean;
212
  
213
  /** Wait for worker to be ready to handle requests */
214
  waitForWorkerReady(): Promise<void>;
215
}
216

217
enum WorkerStates {
218
  STARTING = 'starting',
219
  OK = 'ok',
220
  OUT_OF_MEMORY = 'oom',
221
  RESTARTING = 'restarting',
222
  SHUTTING_DOWN = 'shutting-down',
223
  SHUT_DOWN = 'shut-down'
224
}
225
```
226

227
### Exit Results
228

229
Information about worker farm termination, indicating whether forced termination was necessary.
230

231
```typescript { .api }
232
interface PoolExitResult {
233
  /** True if at least one worker required force termination */
234
  forceExited: boolean;
235
}
236
```
237

238
**Usage Example:**
239

240
```typescript
241
const worker = new Worker("./long-running-worker.js");
242

243
// Do work...
244
await worker.processLongTask();
245

246
// Clean shutdown
247
const result = await worker.end();
248

249
if (result.forceExited) {
250
  console.warn("Some workers had to be force-terminated");
251
  console.warn("Check for memory leaks or hanging operations");
252
}
253
```
254

255
## Worker Module Requirements
256

257
Worker modules loaded by jest-worker should follow these patterns:
258

259
### Required Module Structure
260

261
```javascript
262
// my-worker.js
263

264
// Export functions that will be exposed as worker methods
265
exports.processData = function(data) {
266
  // Synchronous or asynchronous work
267
  return { result: data.processed };
268
};
269

270
exports.heavyComputation = async function(input) {
271
  // Async work is fully supported
272
  const result = await someAsyncOperation(input);
273
  return result;
274
};
275

276
// Optional lifecycle hooks
277
exports.setup = function(...setupArgs) {
278
  // Called once when worker starts
279
  // setupArgs come from WorkerFarmOptions.setupArgs
280
  console.log("Worker started with args:", setupArgs);
281
};
282

283
exports.teardown = function() {
284
  // Called when worker is shutting down
285
  console.log("Worker shutting down");
286
};
287
```
288

289
### Environment Variables
290

291
Workers have access to special environment variables:
292

293
- `process.env.JEST_WORKER_ID`: Unique worker identifier (string starting from '1')
294

295
```javascript
296
// In worker module
297
exports.getWorkerId = function() {
298
  return process.env.JEST_WORKER_ID;
299
};
300
```
301

302
### Method Discovery
303

304
By default, jest-worker automatically discovers exportable methods. You can control this with the `exposedMethods` option:
305

306
```typescript
307
const worker = new Worker("./worker.js", {
308
  exposedMethods: ["processData", "compute"] // Only expose these methods
309
});
310
```