0
# Stream Transform
1

2
Stream Transform is a Node.js stream transformation framework that extends the native `stream.Transform` API to provide object transformations with multiple consumption patterns. It offers stream-based, callback-based, and synchronous APIs for scalable data processing.
3

4
## Package Information
5

6
- **Package Name**: stream-transform
7
- **Package Type**: npm  
8
- **Language**: JavaScript/TypeScript
9
- **Installation**: `npm install stream-transform`
10

11
## Core Imports
12

13
ESM (ES Modules):
14

15
```javascript
16
import { transform, Transformer } from "stream-transform";
17
import { transform as syncTransform } from "stream-transform/sync";
18

19
// Default export (same as named export)
20
import transform from "stream-transform";
21
```
22

23
CommonJS:
24

25
```javascript
26
const { transform, Transformer } = require("stream-transform");
27
const { transform: syncTransform } = require("stream-transform/sync");
28
```
29

30
Browser ESM:
31

32
```javascript
33
import { transform, Transformer } from "stream-transform/browser/esm";
34
import { transform as syncTransform } from "stream-transform/browser/esm/sync";
35
```
36

37
## Basic Usage
38

39
```javascript
40
import { transform } from "stream-transform";
41
import { transform as syncTransform } from "stream-transform/sync";
42

43
// Stream API - for scalable processing
44
const transformer = transform((record) => {
45
  record.push(record.shift()); // Move first element to end
46
  return record;
47
});
48

49
// Callback API - for convenience with small datasets
50
transform([
51
  ["a", "b", "c"],
52
  ["1", "2", "3"]
53
], (record) => record.join("|"), (err, results) => {
54
  console.log(results); // ["a|b|c", "1|2|3"]
55
});
56

57
// Sync API - for synchronous processing
58
const results = syncTransform([
59
  ["a", "b", "c"],
60
  ["1", "2", "3"]
61
], (record) => record.join("|"));
62
console.log(results); // ["a|b|c", "1|2|3"]
63
```
64

65
## Architecture
66

67
Stream Transform is built around several key components:
68

69
- **Transform Stream**: Extends Node.js `stream.Transform` for scalable data processing
70
- **Handler Functions**: User-defined transformation logic supporting sync, async, and promise-based patterns  
71
- **Execution Modes**: Stream, callback, and synchronous APIs for different use cases
72
- **Parallel Processing**: Configurable concurrency for asynchronous transformations
73
- **State Management**: Built-in tracking of transformation progress and statistics
74
- **Multi-format Support**: Available as ESM, CommonJS, UMD, IIFE, and browser builds
75

76
## Capabilities
77

78
### Stream API
79

80
Core streaming transformation functionality for scalable data processing. Ideal for large datasets and real-time processing pipelines.
81

82
```javascript { .api }
83
function transform<T, U>(handler: Handler<T, U>, callback?: Callback): Transformer;
84
function transform<T, U>(options: Options, handler: Handler<T, U>, callback?: Callback): Transformer;
85

86
class Transformer extends stream.Transform {
87
  constructor(options: Options);
88
  readonly options: Options;
89
  readonly state: State;
90
}
91
```
92

93
[Stream API](./stream-api.md)
94

95
### Callback API
96

97
Convenient callback-based API for processing data arrays with automatic result collection. Perfect for small to medium datasets where convenience is prioritized.
98

99
```javascript { .api }
100
function transform<T, U>(
101
  records: Array<T>, 
102
  handler: Handler<T, U>, 
103
  callback?: Callback
104
): Transformer;
105
function transform<T, U>(
106
  records: Array<T>, 
107
  options: Options, 
108
  handler: Handler<T, U>, 
109
  callback?: Callback
110
): Transformer;
111
```
112

113
[Callback API](./callback-api.md)
114

115
### Synchronous API
116

117
Synchronous data transformation for immediate processing of data arrays. Best for small datasets where blocking execution is acceptable.
118

119
```javascript { .api }
120
function transform<T, U>(records: Array<T>, handler: Handler<T, U>): Array<U>;
121
function transform<T, U>(
122
  records: Array<T>, 
123
  options: Options, 
124
  handler: Handler<T, U>
125
): Array<U>;
126
```
127

128
[Synchronous API](./sync-api.md)
129

130
## Types
131

132
### Core Types
133

134
```javascript { .api }
135
// Main API Handler - callback is required for async handlers
136
type Handler<T = any, U = any> = (
137
  record: T,
138
  callback: HandlerCallback,
139
  params?: any
140
) => U;
141

142
type HandlerCallback<T = any> = (err?: null | Error, record?: T) => void;
143

144
type Callback = (err?: null | Error, output?: any[]) => void;
145

146
interface Options extends stream.TransformOptions {
147
  /** Auto-consume stream when no consumer present */
148
  consume?: boolean;
149
  /** Number of parallel transformation callbacks (default: 100) */
150
  parallel?: number;
151
  /** User-defined parameters passed to handler */
152
  params?: any;
153
}
154

155
interface State {
156
  /** Number of completed transformations */
157
  finished: number;
158
  /** Number of currently running transformations */
159
  running: number;
160
  /** Number of started transformations */
161
  started: number;
162
  /** Whether the stream is currently paused due to backpressure */
163
  paused: boolean;
164
}
165
```
166

167
### Handler Execution Patterns
168

169
Stream Transform supports multiple handler execution patterns detected by function signature:
170

171
- **Synchronous** (1 param + optional params): `(record) => result` or `(record, params) => result`
172
- **Asynchronous** (2 params + optional params): `(record, callback) => void` or `(record, callback, params) => void`
173
- **Promise-based** (returns Promise): `(record) => Promise<result>` or `(record, params) => Promise<result>`
174

175
The library automatically detects the handler type based on function length and return value characteristics.