0
# Jest Worker
1

2
Jest Worker is a robust parallel processing library that enables executing heavy tasks across multiple child processes or worker threads. It provides a Promise-based interface with minimal overhead, worker binding capabilities for caching optimization, and comprehensive worker lifecycle management. Designed for maximum reusability, it's ideal for testing frameworks, build tools, and any application requiring distributed task execution.
3

4
## Package Information
5

6
- **Package Name**: jest-worker
7
- **Package Type**: npm
8
- **Language**: TypeScript
9
- **Installation**: `npm install jest-worker`
10

11
## Core Imports
12

13
```typescript
14
import { Worker } from "jest-worker";
15
```
16

17
For CommonJS:
18

19
```javascript
20
const { Worker } = require("jest-worker");
21
```
22

23
Additional imports:
24

25
```typescript
26
import { 
27
  Worker, 
28
  PriorityQueue, 
29
  FifoQueue, 
30
  messageParent,
31
  type JestWorkerFarm,
32
  type WorkerFarmOptions,
33
  type WorkerPoolInterface,
34
  type WorkerPoolOptions,
35
  type PromiseWithCustomMessage,
36
  type TaskQueue
37
} from "jest-worker";
38
```
39

40
## Basic Usage
41

42
```typescript
43
import { Worker } from "jest-worker";
44

45
async function main() {
46
  // Create a worker farm pointing to your worker module
47
  const worker = new Worker(require.resolve("./my-worker"), {
48
    numWorkers: 4,
49
    enableWorkerThreads: false
50
  });
51

52
  // Call worker methods - they return promises
53
  const result = await worker.processData({ input: "data" });
54
  
55
  // Terminate the worker farm
56
  await worker.end();
57
}
58
```
59

60
Worker module (`my-worker.js`):
61

62
```javascript
63
exports.processData = function(data) {
64
  // Heavy computation here
65
  return { processed: data.input.toUpperCase() };
66
};
67

68
// Optional setup/teardown hooks
69
exports.setup = function() {
70
  console.log("Worker initialized");
71
};
72

73
exports.teardown = function() {
74
  console.log("Worker shutting down");
75
};
76
```
77

78
## Architecture
79

80
Jest Worker is built around several key components:
81

82
- **Worker Class**: Main interface for creating and managing worker farms
83
- **Farm Management**: Internal task distribution and worker coordination
84
- **Worker Pool**: Abstraction layer supporting both child processes and worker threads
85
- **Task Queues**: Pluggable queue implementations (FIFO, Priority) for task scheduling
86
- **Message System**: Structured communication between parent and worker processes
87
- **Bound Workers**: Optional worker binding based on task parameters for caching optimization
88

89
## Capabilities
90

91
### Worker Farm Management
92

93
Core worker farm functionality for creating, managing, and terminating pools of worker processes or threads.
94

95
```typescript { .api }
96
import type { ForkOptions } from 'child_process';
97
import type { ResourceLimits } from 'worker_threads';
98

99
class Worker {
100
  constructor(workerPath: string | URL, options?: WorkerFarmOptions);
101
  
102
  getStdout(): NodeJS.ReadableStream;
103
  getStderr(): NodeJS.ReadableStream;
104
  start(): Promise<void>;
105
  end(): Promise<PoolExitResult>;
106
}
107

108
interface WorkerFarmOptions {
109
  computeWorkerKey?: (method: string, ...args: Array<unknown>) => string | null;
110
  enableWorkerThreads?: boolean;
111
  exposedMethods?: ReadonlyArray<string>;
112
  forkOptions?: ForkOptions;
113
  maxRetries?: number;
114
  numWorkers?: number;
115
  resourceLimits?: ResourceLimits;
116
  setupArgs?: Array<unknown>;
117
  taskQueue?: TaskQueue;
118
  WorkerPool?: new (workerPath: string, options?: WorkerPoolOptions) => WorkerPoolInterface;
119
  workerSchedulingPolicy?: WorkerSchedulingPolicy;
120
  idleMemoryLimit?: number;
121
}
122

123
interface PoolExitResult {
124
  forceExited: boolean;
125
}
126
```
127

128
[Worker Management](./worker-management.md)
129

130
### Task Queue Systems
131

132
Configurable task scheduling systems that control how method calls are distributed and prioritized across workers.
133

134
```typescript { .api }
135
interface TaskQueue {
136
  enqueue(task: QueueChildMessage, workerId?: number): void;
137
  dequeue(workerId: number): QueueChildMessage | null;
138
}
139

140
interface QueueChildMessage {
141
  request: ChildMessageCall;
142
  onStart: OnStart;
143
  onEnd: OnEnd;
144
  onCustomMessage: OnCustomMessage;
145
}
146

147
type ChildMessageCall = [
148
  type: typeof CHILD_MESSAGE_CALL,
149
  isProcessed: boolean,
150
  methodName: string,
151
  args: Array<unknown>
152
];
153

154
const CHILD_MESSAGE_CALL = 1;
155

156
type OnStart = (worker: WorkerInterface) => void;
157
type OnEnd = (err: Error | null, result: unknown) => void;
158
type OnCustomMessage = (message: Array<unknown> | unknown) => void;
159

160
class FifoQueue implements TaskQueue {
161
  enqueue(task: QueueChildMessage, workerId?: number): void;
162
  dequeue(workerId: number): QueueChildMessage | null;
163
}
164

165
class PriorityQueue implements TaskQueue {
166
  // See Task Queues documentation for detailed constructor and method signatures
167
  enqueue(task: QueueChildMessage, workerId?: number): void;
168
  dequeue(workerId: number): QueueChildMessage | null;
169
}
170
```
171

172
[Task Queues](./task-queues.md)
173

174
### Worker Communication
175

176
Message passing system for communication between parent process and worker processes/threads, including custom message handling.
177

178
```typescript { .api }
179
function messageParent(
180
  message: unknown,
181
  parentProcess?: NodeJS.Process
182
): void;
183

184
interface PromiseWithCustomMessage<T> extends Promise<T> {
185
  UNSTABLE_onCustomMessage?: (listener: OnCustomMessage) => () => void;
186
}
187

188
type OnCustomMessage = (message: Array<unknown> | unknown) => void;
189
```
190

191
[Worker Communication](./worker-communication.md)
192

193
## Core Types
194

195
```typescript { .api }
196
type JestWorkerFarm<T extends Record<string, unknown>> = Worker & WorkerModule<T>;
197

198
type WorkerModule<T> = {
199
  [K in keyof T as Extract<
200
    ExcludeReservedKeys<K>,
201
    MethodLikeKeys<T>
202
  >]: T[K] extends FunctionLike ? Promisify<T[K]> : never;
203
};
204

205
type ReservedKeys = 'end' | 'getStderr' | 'getStdout' | 'setup' | 'teardown';
206
type ExcludeReservedKeys<K> = Exclude<K, ReservedKeys>;
207

208
type FunctionLike = (...args: any) => unknown;
209

210
type MethodLikeKeys<T> = {
211
  [K in keyof T]: T[K] extends FunctionLike ? K : never;
212
}[keyof T];
213

214
type Promisify<T extends FunctionLike> =
215
  ReturnType<T> extends Promise<infer R>
216
    ? (...args: Parameters<T>) => Promise<R>
217
    : (...args: Parameters<T>) => Promise<ReturnType<T>>;
218

219
type WorkerSchedulingPolicy = 'round-robin' | 'in-order';
220

221
enum WorkerStates {
222
  STARTING = 'starting',
223
  OK = 'ok',
224
  OUT_OF_MEMORY = 'oom',
225
  RESTARTING = 'restarting',
226
  SHUTTING_DOWN = 'shutting-down',
227
  SHUT_DOWN = 'shut-down'
228
}
229

230
enum WorkerEvents {
231
  STATE_CHANGE = 'state-change'
232
}
233
```