or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

index.md
tile.json

tessl/npm-stream-buffers

Buffer-backed Readable and Writable Stream implementations for Node.js

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
npmpkg:npm/stream-buffers@3.0.x

To install, run

npx @tessl/cli install tessl/npm-stream-buffers@3.0.0

index.mddocs/

Stream Buffers

Stream Buffers provides buffer-backed Readable and Writable Stream implementations for Node.js that store data in memory using internal Buffer objects. It offers WritableStreamBuffer for accumulating written data and ReadableStreamBuffer for pumping out buffered data, designed for testing, debugging, and utility scenarios requiring in-memory stream buffering.

Package Information

  • Package Name: stream-buffers
  • Package Type: npm
  • Language: JavaScript
  • Installation: npm install stream-buffers

Core Imports

const streamBuffers = require('stream-buffers');

You can also destructure the main classes:

const { WritableStreamBuffer, ReadableStreamBuffer } = require('stream-buffers');

For TypeScript or when you need the stream module types:

const stream = require('stream');
const streamBuffers = require('stream-buffers');

Basic Usage

const streamBuffers = require('stream-buffers');

// Create a writable stream buffer for collecting output
const myWritableStreamBuffer = new streamBuffers.WritableStreamBuffer({
  initialSize: (100 * 1024),   // start at 100 kilobytes
  incrementAmount: (10 * 1024) // grow by 10 kilobytes each time buffer overflows
});

// Write different types of data
myWritableStreamBuffer.write('Hello World!\n');
myWritableStreamBuffer.write(Buffer.from([1, 2, 3, 4]));
myWritableStreamBuffer.write('More text data', 'utf8');

// End the stream (optional - doesn't delete contents)
myWritableStreamBuffer.end();

// Get contents (note: this consumes the data)
const contents = myWritableStreamBuffer.getContents(); // Returns Buffer or false
const contentsAsString = myWritableStreamBuffer.getContentsAsString('utf8'); // Returns string or false

// Create a readable stream buffer for providing input data
const myReadableStreamBuffer = new streamBuffers.ReadableStreamBuffer({
  frequency: 10,   // pump data every 10 milliseconds
  chunkSize: 2048  // in 2KB chunks
});

// Add data to be streamed out
myReadableStreamBuffer.put('First line of data\n');
myReadableStreamBuffer.put(Buffer.from('Binary data here'));
myReadableStreamBuffer.put('Final line\n');

// Listen for data using standard stream events
myReadableStreamBuffer.on('data', function(chunk) {
  console.log('Received chunk:', chunk.toString());
});

myReadableStreamBuffer.on('end', function() {
  console.log('Stream finished');
});

// Signal that no more data will be added
myReadableStreamBuffer.stop();

Architecture

Stream Buffers implements the standard Node.js Stream interfaces while providing additional buffer management capabilities:

  • WritableStreamBuffer: Extends stream.Writable to accumulate data in a resizable Buffer
  • ReadableStreamBuffer: Extends stream.Readable to pump out data from an internal Buffer at configurable intervals
  • Dynamic Resizing: Both stream types automatically grow their internal buffers when needed
  • Memory-based: All data is stored in memory using Node.js Buffer objects

Capabilities

Configuration Constants

Default configuration values for stream buffer initialization.

// Default initial buffer size (8192 bytes)
streamBuffers.DEFAULT_INITIAL_SIZE

// Default buffer growth increment (8192 bytes)
streamBuffers.DEFAULT_INCREMENT_AMOUNT

// Default ReadableStreamBuffer frequency (1 millisecond)
streamBuffers.DEFAULT_FREQUENCY

// Default ReadableStreamBuffer chunk size (1024 bytes)
streamBuffers.DEFAULT_CHUNK_SIZE

WritableStreamBuffer

Accumulates written data in a dynamically-resizing buffer with configurable initial size and growth increments.

/**
 * Creates a new WritableStreamBuffer instance
 * @param {Object} [opts] - Configuration options (optional)
 * @param {number} [opts.initialSize=8192] - Initial buffer size in bytes
 * @param {number} [opts.incrementAmount=8192] - Buffer growth increment in bytes
 */
new streamBuffers.WritableStreamBuffer(opts)

class WritableStreamBuffer extends stream.Writable {
  // Inherits all standard stream.Writable methods and events
  /**
   * Get current buffer data size in bytes
   * @returns {number} Size of data currently stored in buffer
   */
  size();

  /**
   * Get current buffer capacity in bytes
   * @returns {number} Maximum size buffer can hold before next resize
   */
  maxSize();

  /**
   * Retrieve buffer contents as Buffer, optionally limiting length
   * Note: This method consumes the retrieved data from the buffer
   * @param {number} [length] - Maximum number of bytes to retrieve
   * @returns {Buffer|false} Buffer containing data, or false if buffer is empty
   */
  getContents(length);

  /**
   * Retrieve buffer contents as string with optional encoding and length
   * Note: This method consumes the retrieved data from the buffer
   * Care should be taken with multi-byte characters as buffer boundaries may split them
   * @param {string} [encoding='utf8'] - String encoding to use
   * @param {number} [length] - Maximum number of bytes to retrieve
   * @returns {string|false} String containing data, or false if buffer is empty
   */
  getContentsAsString(encoding, length);

  /**
   * Standard writable stream method - writes data to buffer
   * @param {Buffer|string} chunk - Data to write
   * @param {string} [encoding] - String encoding if chunk is string
   * @param {Function} [callback] - Completion callback
   * @returns {boolean} True if buffer can accept more data
   */
  write(chunk, encoding, callback);
}

Usage Examples:

const myBuffer = new streamBuffers.WritableStreamBuffer();

// Write different types of data
myBuffer.write('Hello World!');
myBuffer.write(Buffer.from([1, 2, 3, 4]));
myBuffer.write('More text', 'utf8');

// Check buffer status
console.log('Current size:', myBuffer.size());
console.log('Max capacity:', myBuffer.maxSize()); 

// Retrieve all contents (this consumes the data from buffer)
const allData = myBuffer.getContents();
if (allData) {
  console.log('Retrieved:', allData.length, 'bytes');
}

// Retrieve as string (this also consumes the data from buffer)
const allText = myBuffer.getContentsAsString('utf8');
if (allText) {
  console.log('Text content:', allText);
}

// Retrieve partial contents (consumes specified amount from buffer)
const first10Bytes = myBuffer.getContents(10);
const remaining = myBuffer.getContents(); // Gets whatever is left

ReadableStreamBuffer

Pumps out buffered data in configurable chunks at specified frequencies, with data inserted programmatically.

/**
 * Creates a new ReadableStreamBuffer instance
 * @param {Object} [opts] - Configuration options (optional)
 * @param {number} [opts.frequency=1] - Frequency in milliseconds for data pumping
 * @param {number} [opts.chunkSize=1024] - Size of chunks to pump out in bytes
 * @param {number} [opts.initialSize=8192] - Initial buffer size in bytes
 * @param {number} [opts.incrementAmount=8192] - Buffer growth increment in bytes
 */
new streamBuffers.ReadableStreamBuffer(opts)

class ReadableStreamBuffer extends stream.Readable {
  // Inherits all standard stream.Readable methods and events
  /**
   * Add data to internal buffer to be pumped out
   * @param {Buffer|string} data - Data to add (Buffer or string)
   * @param {string} [encoding='utf8'] - String encoding if data is string
   * @throws {Error} If called on stopped stream
   */
  put(data, encoding);

  /**
   * Stop the stream and prepare for end event emission
   * @throws {Error} If called on already stopped stream
   */
  stop();

  /**
   * Get current buffer data size in bytes
   * @returns {number} Size of data currently stored in buffer
   */
  size();

  /**
   * Get current buffer capacity in bytes
   * @returns {number} Maximum size buffer can hold before next resize
   */
  maxSize();
}

Usage Examples:

// Create readable stream with custom timing
const myStream = new streamBuffers.ReadableStreamBuffer({
  frequency: 50,   // pump data every 50ms
  chunkSize: 512   // in 512-byte chunks
});

// Add data to be streamed out
myStream.put('First chunk of data');
myStream.put(Buffer.from('Binary data'));
myStream.put('Final chunk');

// Listen for data using streams1 style
myStream.on('data', function(chunk) {
  console.log('Received:', chunk.toString());
});

// Or streams2+ style
myStream.on('readable', function() {
  let chunk;
  while ((chunk = myStream.read()) !== null) {
    console.log('Read:', chunk.toString());
  }
});

// Handle end of stream
myStream.on('end', function() {
  console.log('Stream ended');
});

// Stop streaming when done adding data
myStream.stop();

Error Handling

The library throws specific errors in these cases:

  • ReadableStreamBuffer.stop(): Throws Error with message "stop() called on already stopped ReadableStreamBuffer" if called on already stopped stream
  • ReadableStreamBuffer.put(): Throws Error with message "Tried to write data to a stopped ReadableStreamBuffer" if called on stopped stream

Buffer Management:

  • Both stream types handle buffer overflow gracefully by automatically resizing their internal buffers
  • Buffer growth is calculated dynamically based on the increment amount to accommodate incoming data
  • WritableStreamBuffer and ReadableStreamBuffer both use the same buffer expansion algorithm

Types

Configuration Options

// WritableStreamBuffer configuration options
// All properties are optional
{
  initialSize: number,      // Initial buffer size in bytes (default: 8192)
  incrementAmount: number   // Buffer growth increment in bytes (default: 8192)
}

// ReadableStreamBuffer configuration options
// All properties are optional
{
  frequency: number,        // Data pumping frequency in milliseconds (default: 1)
  chunkSize: number,        // Chunk size in bytes (default: 1024)
  initialSize: number,      // Initial buffer size in bytes (default: 8192)
  incrementAmount: number   // Buffer growth increment in bytes (default: 8192)
}