0
# Stream Transform
1
2
Stream Transform is a Node.js stream transformation framework that extends the native `stream.Transform` API to provide object transformations with multiple consumption patterns. It offers stream-based, callback-based, and synchronous APIs for scalable data processing.
3
4
## Package Information
5
6
- **Package Name**: stream-transform
7
- **Package Type**: npm
8
- **Language**: JavaScript/TypeScript
9
- **Installation**: `npm install stream-transform`
10
11
## Core Imports
12
13
ESM (ES Modules):
14
15
```javascript
16
import { transform, Transformer } from "stream-transform";
17
import { transform as syncTransform } from "stream-transform/sync";
18
19
// Default export (same as named export)
20
import transform from "stream-transform";
21
```
22
23
CommonJS:
24
25
```javascript
26
const { transform, Transformer } = require("stream-transform");
27
const { transform: syncTransform } = require("stream-transform/sync");
28
```
29
30
Browser ESM:
31
32
```javascript
33
import { transform, Transformer } from "stream-transform/browser/esm";
34
import { transform as syncTransform } from "stream-transform/browser/esm/sync";
35
```
36
37
## Basic Usage
38
39
```javascript
40
import { transform } from "stream-transform";
41
import { transform as syncTransform } from "stream-transform/sync";
42
43
// Stream API - for scalable processing
44
const transformer = transform((record) => {
45
record.push(record.shift()); // Move first element to end
46
return record;
47
});
48
49
// Callback API - for convenience with small datasets
50
transform([
51
["a", "b", "c"],
52
["1", "2", "3"]
53
], (record) => record.join("|"), (err, results) => {
54
console.log(results); // ["a|b|c", "1|2|3"]
55
});
56
57
// Sync API - for synchronous processing
58
const results = syncTransform([
59
["a", "b", "c"],
60
["1", "2", "3"]
61
], (record) => record.join("|"));
62
console.log(results); // ["a|b|c", "1|2|3"]
63
```
64
65
## Architecture
66
67
Stream Transform is built around several key components:
68
69
- **Transform Stream**: Extends Node.js `stream.Transform` for scalable data processing
70
- **Handler Functions**: User-defined transformation logic supporting sync, async, and promise-based patterns
71
- **Execution Modes**: Stream, callback, and synchronous APIs for different use cases
72
- **Parallel Processing**: Configurable concurrency for asynchronous transformations
73
- **State Management**: Built-in tracking of transformation progress and statistics
74
- **Multi-format Support**: Available as ESM, CommonJS, UMD, IIFE, and browser builds
75
76
## Capabilities
77
78
### Stream API
79
80
Core streaming transformation functionality for scalable data processing. Ideal for large datasets and real-time processing pipelines.
81
82
```javascript { .api }
83
function transform<T, U>(handler: Handler<T, U>, callback?: Callback): Transformer;
84
function transform<T, U>(options: Options, handler: Handler<T, U>, callback?: Callback): Transformer;
85
86
class Transformer extends stream.Transform {
87
constructor(options: Options);
88
readonly options: Options;
89
readonly state: State;
90
}
91
```
92
93
[Stream API](./stream-api.md)
94
95
### Callback API
96
97
Convenient callback-based API for processing data arrays with automatic result collection. Perfect for small to medium datasets where convenience is prioritized.
98
99
```javascript { .api }
100
function transform<T, U>(
101
records: Array<T>,
102
handler: Handler<T, U>,
103
callback?: Callback
104
): Transformer;
105
function transform<T, U>(
106
records: Array<T>,
107
options: Options,
108
handler: Handler<T, U>,
109
callback?: Callback
110
): Transformer;
111
```
112
113
[Callback API](./callback-api.md)
114
115
### Synchronous API
116
117
Synchronous data transformation for immediate processing of data arrays. Best for small datasets where blocking execution is acceptable.
118
119
```javascript { .api }
120
function transform<T, U>(records: Array<T>, handler: Handler<T, U>): Array<U>;
121
function transform<T, U>(
122
records: Array<T>,
123
options: Options,
124
handler: Handler<T, U>
125
): Array<U>;
126
```
127
128
[Synchronous API](./sync-api.md)
129
130
## Types
131
132
### Core Types
133
134
```javascript { .api }
135
// Main API Handler - callback is required for async handlers
136
type Handler<T = any, U = any> = (
137
record: T,
138
callback: HandlerCallback,
139
params?: any
140
) => U;
141
142
type HandlerCallback<T = any> = (err?: null | Error, record?: T) => void;
143
144
type Callback = (err?: null | Error, output?: any[]) => void;
145
146
interface Options extends stream.TransformOptions {
147
/** Auto-consume stream when no consumer present */
148
consume?: boolean;
149
/** Number of parallel transformation callbacks (default: 100) */
150
parallel?: number;
151
/** User-defined parameters passed to handler */
152
params?: any;
153
}
154
155
interface State {
156
/** Number of completed transformations */
157
finished: number;
158
/** Number of currently running transformations */
159
running: number;
160
/** Number of started transformations */
161
started: number;
162
/** Whether the stream is currently paused due to backpressure */
163
paused: boolean;
164
}
165
```
166
167
### Handler Execution Patterns
168
169
Stream Transform supports multiple handler execution patterns detected by function signature:
170
171
- **Synchronous** (1 param + optional params): `(record) => result` or `(record, params) => result`
172
- **Asynchronous** (2 params + optional params): `(record, callback) => void` or `(record, callback, params) => void`
173
- **Promise-based** (returns Promise): `(record) => Promise<result>` or `(record, params) => Promise<result>`
174
175
The library automatically detects the handler type based on function length and return value characteristics.