0
# Stream Buffers
1

2
Stream Buffers provides buffer-backed Readable and Writable Stream implementations for Node.js that store data in memory using internal Buffer objects. It offers WritableStreamBuffer for accumulating written data and ReadableStreamBuffer for pumping out buffered data, designed for testing, debugging, and utility scenarios requiring in-memory stream buffering.
3

4
## Package Information
5

6
- **Package Name**: stream-buffers
7
- **Package Type**: npm
8
- **Language**: JavaScript
9
- **Installation**: `npm install stream-buffers`
10

11
## Core Imports
12

13
```javascript
14
const streamBuffers = require('stream-buffers');
15
```
16

17
You can also destructure the main classes:
18

19
```javascript
20
const { WritableStreamBuffer, ReadableStreamBuffer } = require('stream-buffers');
21
```
22

23
For TypeScript or when you need the stream module types:
24

25
```javascript
26
const stream = require('stream');
27
const streamBuffers = require('stream-buffers');
28
```
29

30
## Basic Usage
31

32
```javascript
33
const streamBuffers = require('stream-buffers');
34

35
// Create a writable stream buffer for collecting output
36
const myWritableStreamBuffer = new streamBuffers.WritableStreamBuffer({
37
  initialSize: (100 * 1024),   // start at 100 kilobytes
38
  incrementAmount: (10 * 1024) // grow by 10 kilobytes each time buffer overflows
39
});
40

41
// Write different types of data
42
myWritableStreamBuffer.write('Hello World!\n');
43
myWritableStreamBuffer.write(Buffer.from([1, 2, 3, 4]));
44
myWritableStreamBuffer.write('More text data', 'utf8');
45

46
// End the stream (optional - doesn't delete contents)
47
myWritableStreamBuffer.end();
48

49
// Get contents (note: this consumes the data)
50
const contents = myWritableStreamBuffer.getContents(); // Returns Buffer or false
51
const contentsAsString = myWritableStreamBuffer.getContentsAsString('utf8'); // Returns string or false
52

53
// Create a readable stream buffer for providing input data
54
const myReadableStreamBuffer = new streamBuffers.ReadableStreamBuffer({
55
  frequency: 10,   // pump data every 10 milliseconds
56
  chunkSize: 2048  // in 2KB chunks
57
});
58

59
// Add data to be streamed out
60
myReadableStreamBuffer.put('First line of data\n');
61
myReadableStreamBuffer.put(Buffer.from('Binary data here'));
62
myReadableStreamBuffer.put('Final line\n');
63

64
// Listen for data using standard stream events
65
myReadableStreamBuffer.on('data', function(chunk) {
66
  console.log('Received chunk:', chunk.toString());
67
});
68

69
myReadableStreamBuffer.on('end', function() {
70
  console.log('Stream finished');
71
});
72

73
// Signal that no more data will be added
74
myReadableStreamBuffer.stop();
75
```
76

77
## Architecture
78

79
Stream Buffers implements the standard Node.js Stream interfaces while providing additional buffer management capabilities:
80

81
- **WritableStreamBuffer**: Extends `stream.Writable` to accumulate data in a resizable Buffer
82
- **ReadableStreamBuffer**: Extends `stream.Readable` to pump out data from an internal Buffer at configurable intervals
83
- **Dynamic Resizing**: Both stream types automatically grow their internal buffers when needed
84
- **Memory-based**: All data is stored in memory using Node.js Buffer objects
85

86
## Capabilities
87

88
### Configuration Constants
89

90
Default configuration values for stream buffer initialization.
91

92
```javascript { .api }
93
// Default initial buffer size (8192 bytes)
94
streamBuffers.DEFAULT_INITIAL_SIZE
95

96
// Default buffer growth increment (8192 bytes)
97
streamBuffers.DEFAULT_INCREMENT_AMOUNT
98

99
// Default ReadableStreamBuffer frequency (1 millisecond)
100
streamBuffers.DEFAULT_FREQUENCY
101

102
// Default ReadableStreamBuffer chunk size (1024 bytes)
103
streamBuffers.DEFAULT_CHUNK_SIZE
104
```
105

106
### WritableStreamBuffer
107

108
Accumulates written data in a dynamically-resizing buffer with configurable initial size and growth increments.
109

110
```javascript { .api }
111
/**
112
 * Creates a new WritableStreamBuffer instance
113
 * @param {Object} [opts] - Configuration options (optional)
114
 * @param {number} [opts.initialSize=8192] - Initial buffer size in bytes
115
 * @param {number} [opts.incrementAmount=8192] - Buffer growth increment in bytes
116
 */
117
new streamBuffers.WritableStreamBuffer(opts)
118

119
class WritableStreamBuffer extends stream.Writable {
120
  // Inherits all standard stream.Writable methods and events
121
  /**
122
   * Get current buffer data size in bytes
123
   * @returns {number} Size of data currently stored in buffer
124
   */
125
  size();
126

127
  /**
128
   * Get current buffer capacity in bytes
129
   * @returns {number} Maximum size buffer can hold before next resize
130
   */
131
  maxSize();
132

133
  /**
134
   * Retrieve buffer contents as Buffer, optionally limiting length
135
   * Note: This method consumes the retrieved data from the buffer
136
   * @param {number} [length] - Maximum number of bytes to retrieve
137
   * @returns {Buffer|false} Buffer containing data, or false if buffer is empty
138
   */
139
  getContents(length);
140

141
  /**
142
   * Retrieve buffer contents as string with optional encoding and length
143
   * Note: This method consumes the retrieved data from the buffer
144
   * Care should be taken with multi-byte characters as buffer boundaries may split them
145
   * @param {string} [encoding='utf8'] - String encoding to use
146
   * @param {number} [length] - Maximum number of bytes to retrieve
147
   * @returns {string|false} String containing data, or false if buffer is empty
148
   */
149
  getContentsAsString(encoding, length);
150

151
  /**
152
   * Standard writable stream method - writes data to buffer
153
   * @param {Buffer|string} chunk - Data to write
154
   * @param {string} [encoding] - String encoding if chunk is string
155
   * @param {Function} [callback] - Completion callback
156
   * @returns {boolean} True if buffer can accept more data
157
   */
158
  write(chunk, encoding, callback);
159
}
160
```
161

162
**Usage Examples:**
163

164
```javascript
165
const myBuffer = new streamBuffers.WritableStreamBuffer();
166

167
// Write different types of data
168
myBuffer.write('Hello World!');
169
myBuffer.write(Buffer.from([1, 2, 3, 4]));
170
myBuffer.write('More text', 'utf8');
171

172
// Check buffer status
173
console.log('Current size:', myBuffer.size());
174
console.log('Max capacity:', myBuffer.maxSize()); 
175

176
// Retrieve all contents (this consumes the data from buffer)
177
const allData = myBuffer.getContents();
178
if (allData) {
179
  console.log('Retrieved:', allData.length, 'bytes');
180
}
181

182
// Retrieve as string (this also consumes the data from buffer)
183
const allText = myBuffer.getContentsAsString('utf8');
184
if (allText) {
185
  console.log('Text content:', allText);
186
}
187

188
// Retrieve partial contents (consumes specified amount from buffer)
189
const first10Bytes = myBuffer.getContents(10);
190
const remaining = myBuffer.getContents(); // Gets whatever is left
191
```
192

193
### ReadableStreamBuffer
194

195
Pumps out buffered data in configurable chunks at specified frequencies, with data inserted programmatically.
196

197
```javascript { .api }
198
/**
199
 * Creates a new ReadableStreamBuffer instance
200
 * @param {Object} [opts] - Configuration options (optional)
201
 * @param {number} [opts.frequency=1] - Frequency in milliseconds for data pumping
202
 * @param {number} [opts.chunkSize=1024] - Size of chunks to pump out in bytes
203
 * @param {number} [opts.initialSize=8192] - Initial buffer size in bytes
204
 * @param {number} [opts.incrementAmount=8192] - Buffer growth increment in bytes
205
 */
206
new streamBuffers.ReadableStreamBuffer(opts)
207

208
class ReadableStreamBuffer extends stream.Readable {
209
  // Inherits all standard stream.Readable methods and events
210
  /**
211
   * Add data to internal buffer to be pumped out
212
   * @param {Buffer|string} data - Data to add (Buffer or string)
213
   * @param {string} [encoding='utf8'] - String encoding if data is string
214
   * @throws {Error} If called on stopped stream
215
   */
216
  put(data, encoding);
217

218
  /**
219
   * Stop the stream and prepare for end event emission
220
   * @throws {Error} If called on already stopped stream
221
   */
222
  stop();
223

224
  /**
225
   * Get current buffer data size in bytes
226
   * @returns {number} Size of data currently stored in buffer
227
   */
228
  size();
229

230
  /**
231
   * Get current buffer capacity in bytes
232
   * @returns {number} Maximum size buffer can hold before next resize
233
   */
234
  maxSize();
235
}
236
```
237

238
**Usage Examples:**
239

240
```javascript
241
// Create readable stream with custom timing
242
const myStream = new streamBuffers.ReadableStreamBuffer({
243
  frequency: 50,   // pump data every 50ms
244
  chunkSize: 512   // in 512-byte chunks
245
});
246

247
// Add data to be streamed out
248
myStream.put('First chunk of data');
249
myStream.put(Buffer.from('Binary data'));
250
myStream.put('Final chunk');
251

252
// Listen for data using streams1 style
253
myStream.on('data', function(chunk) {
254
  console.log('Received:', chunk.toString());
255
});
256

257
// Or streams2+ style
258
myStream.on('readable', function() {
259
  let chunk;
260
  while ((chunk = myStream.read()) !== null) {
261
    console.log('Read:', chunk.toString());
262
  }
263
});
264

265
// Handle end of stream
266
myStream.on('end', function() {
267
  console.log('Stream ended');
268
});
269

270
// Stop streaming when done adding data
271
myStream.stop();
272
```
273

274
## Error Handling
275

276
The library throws specific errors in these cases:
277

278
- **ReadableStreamBuffer.stop()**: Throws Error with message "stop() called on already stopped ReadableStreamBuffer" if called on already stopped stream
279
- **ReadableStreamBuffer.put()**: Throws Error with message "Tried to write data to a stopped ReadableStreamBuffer" if called on stopped stream
280

281
**Buffer Management:**
282
- Both stream types handle buffer overflow gracefully by automatically resizing their internal buffers
283
- Buffer growth is calculated dynamically based on the increment amount to accommodate incoming data
284
- WritableStreamBuffer and ReadableStreamBuffer both use the same buffer expansion algorithm
285

286
## Types
287

288
### Configuration Options
289

290
```javascript { .api }
291
// WritableStreamBuffer configuration options
292
// All properties are optional
293
{
294
  initialSize: number,      // Initial buffer size in bytes (default: 8192)
295
  incrementAmount: number   // Buffer growth increment in bytes (default: 8192)
296
}
297

298
// ReadableStreamBuffer configuration options
299
// All properties are optional
300
{
301
  frequency: number,        // Data pumping frequency in milliseconds (default: 1)
302
  chunkSize: number,        // Chunk size in bytes (default: 1024)
303
  initialSize: number,      // Initial buffer size in bytes (default: 8192)
304
  incrementAmount: number   // Buffer growth increment in bytes (default: 8192)
305
}
306
```