0
# thread-loader
1

2
thread-loader is a webpack loader that runs subsequent loaders in a worker pool to improve build performance for expensive operations. It creates separate Node.js processes (workers) to execute loaders in parallel, providing significant performance benefits for computationally intensive tasks like Babel transpilation, TypeScript compilation, or CSS preprocessing.
3

4
## Package Information
5

6
- **Package Name**: thread-loader
7
- **Package Type**: npm
8
- **Language**: JavaScript
9
- **Installation**: `npm install --save-dev thread-loader`
10

11
## Core Imports
12

13
```javascript
14
const threadLoader = require('thread-loader');
15
```
16

17
The package exports CommonJS modules with the following functions:
18

19
```javascript
20
// Access warmup function directly
21
threadLoader.warmup(options, requires);
22

23
// The pitch function is used internally by webpack - do NOT call directly  
24
const { pitch, warmup } = threadLoader;
25
```
26

27
## Basic Usage
28

29
### As a Webpack Loader
30

31
```javascript
32
// webpack.config.js
33
module.exports = {
34
  module: {
35
    rules: [
36
      {
37
        test: /\.js$/,
38
        include: path.resolve('src'),
39
        use: [
40
          'thread-loader',
41
          // your expensive loader (e.g babel-loader)
42
          'babel-loader',
43
        ],
44
      },
45
    ],
46
  },
47
};
48
```
49

50
### With Options
51

52
```javascript
53
// webpack.config.js
54
module.exports = {
55
  module: {
56
    rules: [
57
      {
58
        test: /\.js$/,
59
        use: [
60
          {
61
            loader: 'thread-loader',
62
            options: {
63
              workers: 2,
64
              workerParallelJobs: 50,
65
              poolTimeout: 2000,
66
              name: 'js-pool',
67
            },
68
          },
69
          'babel-loader',
70
        ],
71
      },
72
    ],
73
  },
74
};
75
```
76

77
### Pre-warming Workers
78

79
```javascript
80
const threadLoader = require('thread-loader');
81

82
threadLoader.warmup(
83
  {
84
    // pool options, like passed to loader options
85
    // must match loader options to boot the correct pool
86
    workers: 2,
87
    workerParallelJobs: 50,
88
  },
89
  [
90
    // modules to load
91
    'babel-loader',
92
    'babel-preset-es2015',
93
    'sass-loader',
94
  ]
95
);
96
```
97

98
## Architecture
99

100
thread-loader is built around several key components:
101

102
- **Worker Pool Management**: Creates and manages separate Node.js processes for parallel loader execution
103
- **Job Distribution**: Intelligently distributes loader jobs across available workers
104
- **Inter-Process Communication**: JSON-based message protocol for communication between main process and workers
105
- **Webpack Integration**: Seamless integration with webpack's loader API and context
106
- **Performance Optimization**: Configurable worker lifecycle, job parallelism, and pre-warming capabilities
107

108
## Capabilities
109

110
### Worker Pool Pre-warming
111

112
Pre-warm worker pools to reduce initial startup delays by pre-booting workers and loading specified modules into the Node.js module cache.
113

114
```javascript { .api }
115
/**
116
 * Pre-warm worker pool to reduce startup delays by starting workers and loading modules
117
 * @param options - Pool configuration options (must match loader options used in webpack config)
118
 * @param requires - Array of module names/paths to pre-load in worker processes 
119
 * @returns void
120
 */
121
function warmup(options: WorkerPoolOptions, requires: string[]): void;
122
```
123

124
**Usage Examples:**
125

126
```javascript
127
const threadLoader = require('thread-loader');
128

129
// Pre-warm with basic options
130
threadLoader.warmup({}, ['babel-loader']);
131

132
// Pre-warm with custom pool configuration
133
threadLoader.warmup(
134
  {
135
    workers: 4,
136
    workerParallelJobs: 20,
137
    poolTimeout: Infinity, // Keep workers alive for watching builds
138
    name: 'babel-pool',
139
  },
140
  [
141
    'babel-loader',
142
    'babel-preset-env',
143
    '@babel/preset-react',
144
  ]
145
);
146

147
// Pre-warm multiple different pools
148
threadLoader.warmup({ name: 'js-pool' }, ['babel-loader']);
149
threadLoader.warmup({ name: 'css-pool' }, ['sass-loader', 'postcss-loader']);
150
```
151

152
### Webpack Loader Integration
153

154
The main loader function that intercepts webpack loader execution and delegates to worker pool. This is automatically called by webpack when the loader is used in a configuration.
155

156
```javascript { .api }
157
/**
158
 * Webpack loader pitch function that executes subsequent loaders in worker pool
159
 * This function is automatically called by webpack's loader system
160
 * @param this - Webpack loader context with resource info and callback functions
161
 * @returns void - Uses async callback pattern
162
 */
163
function pitch(): void;
164
```
165

166
**Note**: The `pitch` function is called automatically by webpack and should not be invoked directly. It's the main entry point when thread-loader is used in a webpack configuration.
167

168
## Configuration Options
169

170
### WorkerPoolOptions
171

172
Configuration options that can be passed to the thread-loader and warmup function.
173

174
```javascript { .api }
175
interface WorkerPoolOptions {
176
  /** Number of spawned workers. Defaults to (number of cpus - 1) or fallback to 1 */
177
  workers?: number;
178
  
179
  /** Number of jobs a worker processes in parallel. Defaults to 20 */
180
  workerParallelJobs?: number;
181
  
182
  /** Additional node.js arguments for worker processes */
183
  workerNodeArgs?: string[];
184
  
185
  /** Allow respawning a dead worker pool. Defaults to false */
186
  poolRespawn?: boolean;
187
  
188
  /** Timeout for killing idle worker processes in milliseconds. Defaults to 500 */
189
  poolTimeout?: number;
190
  
191
  /** Number of jobs the pool distributes to workers. Defaults to 200 */
192
  poolParallelJobs?: number;
193
  
194
  /** Name of the pool for creating different pools with identical options */
195
  name?: string;
196
}
197
```
198

199
### Configuration Details
200

201
- **workers**: Controls the number of parallel worker processes. Each worker is a separate Node.js process with ~600ms overhead
202
- **workerParallelJobs**: How many jobs each individual worker can process simultaneously
203
- **workerNodeArgs**: Pass Node.js flags like `--max-old-space-size=1024` to worker processes
204
- **poolRespawn**: Whether to restart the entire pool if a worker dies (slows compilation, should be false in development)
205
- **poolTimeout**: Time to wait before killing idle workers. Set to `Infinity` for watching builds to keep workers alive
206
- **poolParallelJobs**: Total job distribution capacity across all workers
207
- **name**: Create separate pools with different names even if other options are identical
208

209
## Usage Guidelines
210

211
### When to Use thread-loader
212

213
**Use thread-loader for:**
214
- Expensive transformations (Babel, TypeScript, Sass)
215
- CPU-intensive operations that can be parallelized
216
- Large codebases where build time is a bottleneck
217

218
**Don't use thread-loader for:**
219
- Simple, fast loaders (file-loader, url-loader)
220
- Small projects where overhead exceeds benefits
221
- Loaders that need webpack-specific APIs not available in workers
222

223
### Performance Considerations
224

225
- Each worker has ~600ms startup overhead
226
- Inter-process communication adds latency
227
- Benefits increase with more CPU cores and expensive operations
228
- Pre-warming reduces cold start penalties
229

230
### Worker Limitations
231

232
Loaders running in workers have limitations:
233
- Cannot emit files
234
- Cannot use custom loader API (via plugins)
235
- Cannot access webpack options
236
- Limited access to webpack's full context
237

238
## Error Handling
239

240
Errors from worker processes are enhanced with stack trace information from both the main process and worker process, providing clear debugging information.
241

242
```javascript { .api }
243
/**
244
 * Enhanced error class for worker process errors with combined stack traces
245
 */
246
class WorkerError extends Error {
247
  /**
248
   * Create a new WorkerError with enhanced stack trace information
249
   * @param err - Original error object or message from worker
250
   * @param workerId - ID of the worker process that generated the error
251
   */
252
  constructor(err: ErrorLike | string, workerId: number);
253
  
254
  /** Error name, inherited from original error */
255
  name: string;
256
  
257
  /** Error message, inherited from original error */
258
  message: string;
259
  
260
  /** Combined stack trace from main process and worker process */
261
  stack: string;
262
}
263

264
interface ErrorLike {
265
  name?: string;
266
  message: string;
267
  stack?: string;
268
}
269
```
270

271
Worker errors include:
272
- Combined stack traces from main and worker processes with clear separation
273
- Worker ID in the stack trace header for debugging multi-worker issues  
274
- Original error name, message and details preserved
275
- Thread Loader identification in stack trace