or run

npx @tessl/cli init

Version

Tile

Overview

Evals

Files

docs

index.md packet-operations.md payload-operations.md streaming-operations.md

tile.json

Streaming Operations

Transform stream interfaces for processing packet data in streaming scenarios, using WebSocket-inspired framing protocol for efficient binary data transmission.

Capabilities

Packet Encoder Stream

Creates a TransformStream that encodes packet objects into binary frames with length-prefixed headers, suitable for streaming protocols like WebTransport.

/**
 * Creates a TransformStream for encoding packets to binary format
 * @returns TransformStream that accepts Packet objects and outputs binary frames
 */
function createPacketEncoderStream(): TransformStream;

Usage Examples:

import { createPacketEncoderStream, Packet } from "engine.io-parser";

// Basic stream usage
const encoderStream = createPacketEncoderStream();
const writer = encoderStream.writable.getWriter();
const reader = encoderStream.readable.getReader();

// Encode a text packet
writer.write({ type: "message", data: "hello" });

// Read the encoded result (header + payload)
const header = await reader.read(); // Uint8Array containing length info
const payload = await reader.read(); // Uint8Array containing encoded packet

// Encode binary data
const binaryPacket: Packet = {
  type: "message",
  data: new Uint8Array([1, 2, 3, 4])
};
writer.write(binaryPacket);

// Binary packets produce frames with binary flag set in header
const binaryHeader = await reader.read(); // Header with binary flag (0x80)
const binaryPayload = await reader.read(); // Original binary data

Packet Decoder Stream

Creates a TransformStream that decodes binary frames back into packet objects, with payload size limits and error handling.

/**
 * Creates a TransformStream for decoding binary frames to packets
 * @param maxPayload - Maximum allowed payload size in bytes
 * @param binaryType - How to handle binary data ("nodebuffer", "arraybuffer", "blob")
 * @returns TransformStream that accepts binary frames and outputs Packet objects
 */
function createPacketDecoderStream(
  maxPayload: number,
  binaryType: BinaryType
): TransformStream;

Usage Examples:

import { createPacketDecoderStream } from "engine.io-parser";

// Create decoder with 1MB limit
const decoderStream = createPacketDecoderStream(1_000_000, "arraybuffer");
const writer = decoderStream.writable.getWriter();
const reader = decoderStream.readable.getReader();

// Decode a text packet frame
writer.write(new Uint8Array([5])); // Length header: 5 bytes
writer.write(new Uint8Array([52, 104, 101, 108, 108, 111])); // "4hello"

const textPacket = await reader.read();
console.log(textPacket.value); // { type: "message", data: "hello" }

// Decode a binary packet frame
writer.write(new Uint8Array([131])); // Length header: 3 bytes + binary flag (0x80)
writer.write(new Uint8Array([1, 2, 3])); // Binary data

const binaryPacket = await reader.read();
console.log(binaryPacket.value); // { type: "message", data: ArrayBuffer([1, 2, 3]) }

// Handle payload size limit errors
writer.write(new Uint8Array([255])); // Payload too large

const errorPacket = await reader.read();
console.log(errorPacket.value); // { type: "error", data: "parser error" }

Frame Format

The streaming protocol uses a WebSocket-inspired framing format:

Header Structure

Small payloads (< 126 bytes):

[1 byte: length + flags]

Medium payloads (126-65535 bytes):

[1 byte: 126 + flags][2 bytes: length]

Large payloads (65536+ bytes):

[1 byte: 127 + flags][8 bytes: length]

Header Flags

Bit 7 (0x80): Binary flag
- 0: Payload is text data
- 1: Payload is binary data
Bits 0-6: Length information
- 0-125: Direct length value
- 126: Length follows in next 2 bytes (uint16)
- 127: Length follows in next 8 bytes (uint64)

Frame Examples

// Text packet "4hello" (5 bytes)
// Header: [5] (no binary flag)
// Payload: [52, 104, 101, 108, 108, 111] ("4hello")

// Binary packet with 3 bytes
// Header: [131] (3 bytes + binary flag 0x80 = 128 + 3 = 131)  
// Payload: [1, 2, 3] (raw binary data)

// Medium payload (12345 bytes)
// Header: [126, 48, 57] (126 = extended length, 12345 = 0x3039)
// Payload: [packet data...]

// Large binary payload (123456789 bytes)  
// Header: [255, 0, 0, 0, 0, 7, 91, 205, 21] (127 + 0x80, then uint64)
// Payload: [packet data...]

State Machine

The decoder stream uses a state machine for parsing frames:

enum State {
  READ_HEADER,              // Reading initial header byte
  READ_EXTENDED_LENGTH_16,  // Reading 2-byte extended length
  READ_EXTENDED_LENGTH_64,  // Reading 8-byte extended length  
  READ_PAYLOAD             // Reading payload data
}

State Transitions

READ_HEADER: Determines payload size and binary flag
- Length < 126: → READ_PAYLOAD
- Length = 126: → READ_EXTENDED_LENGTH_16
- Length = 127: → READ_EXTENDED_LENGTH_64
READ_EXTENDED_LENGTH_16: Reads 2-byte length → READ_PAYLOAD
READ_EXTENDED_LENGTH_64: Reads 8-byte length → READ_PAYLOAD
READ_PAYLOAD: Reads payload data → READ_HEADER (next frame)

Error Conditions

Encoder Stream Errors

Invalid packet objects (missing required fields)
Memory allocation failures for large payloads

Decoder Stream Errors

Payload size exceeds maxPayload limit
Invalid length encoding (zero-length payloads)
Length values exceeding JavaScript's MAX_SAFE_INTEGER
Incomplete frame data (stream closed mid-frame)

All errors result in error packets being emitted:

const ERROR_PACKET: Packet = { type: "error", data: "parser error" };

Performance Characteristics

Memory Usage

Encoder: Minimal buffering, frames written immediately
Decoder: Accumulates chunks until complete frame is available
Large payloads are processed efficiently without full duplication

Throughput

Binary payloads avoid encoding overhead (base64, etc.)
Framing adds minimal overhead (1-9 bytes per packet)
Suitable for high-throughput streaming scenarios

Browser Compatibility

Requires TransformStream support (modern browsers)
Graceful degradation when TransformStream is unavailable
TextDecoder is lazily initialized for compatibility

Types

interface Packet {
  type: PacketType;
  options?: { compress: boolean };
  data?: RawData;
}

type BinaryType = "nodebuffer" | "arraybuffer" | "blob";

type PacketType = 
  | "open" | "close" | "ping" | "pong" 
  | "message" | "upgrade" | "noop" | "error";

type RawData = any; // string | Buffer | ArrayBuffer | ArrayBufferView | Blob

Version

Tile

Files