CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/npm-linebyline

Simple streaming readline module for Node.js that enables memory-efficient line-by-line file reading

—

Pending
Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Pending

The risk profile of this skill

Overview
Eval results
Files

index.mddocs/

LineByLine

LineByLine is a simple streaming readline module for Node.js that enables memory-efficient line-by-line file reading. It creates a readable stream from a file and emits line events for each line of text, making it ideal for processing large files without loading them entirely into memory.

Package Information

  • Package Name: linebyline
  • Package Type: npm
  • Language: JavaScript
  • Installation: npm install linebyline

Core Imports

const readLine = require('linebyline');

Basic Usage

const readLine = require('linebyline');

// Read a file line by line
const rl = readLine('./somefile.txt');

rl.on('line', function(line, lineCount, byteCount) {
  console.log('Line ' + lineCount + ': ' + line);
})
.on('error', function(e) {
  console.error('Error reading file:', e);
})
.on('end', function() {
  console.log('Finished reading file');
});

Architecture

LineByLine is built around a simple streaming architecture:

  • EventEmitter Pattern: The main class extends EventEmitter to provide event-based line processing
  • Stream Integration: Works with both file paths and existing readable streams
  • Memory Efficiency: Uses a configurable buffer size to read files without loading them entirely into memory
  • Line Detection: Processes data byte-by-byte to detect line endings (LF and CR)
  • Error Handling: Comprehensive error propagation through the event system

Capabilities

Constructor Function

Creates a new ReadLine instance for reading files line by line.

/**
 * Creates a new ReadLine instance for reading files line by line
 * @param {string|Stream} file - File path string or readable stream object
 * @param {Object} [opts] - Configuration options
 * @param {number} [opts.maxLineLength=4096] - Maximum line length in bytes
 * @param {boolean} [opts.retainBuffer=false] - If true, emits raw Buffer objects instead of strings
 * @returns {ReadLine} ReadLine instance (EventEmitter)
 */
function readLine(file, opts);

Usage Examples:

// Basic usage with file path
const rl = readLine('./data.txt');

// With options
const rl = readLine('./large-file.txt', {
  maxLineLength: 8192,  // 8K buffer
  retainBuffer: false   // Convert to strings
});

// With existing stream
const fs = require('fs');
const stream = fs.createReadStream('./file.txt');
const rl = readLine(stream);

// Can be called with or without 'new'
const rl1 = readLine('./file.txt');
const rl2 = new readLine('./file.txt');

Events

The ReadLine instance emits several events during file processing.

Line Event

Emitted for each line read from the file.

/**
 * Emitted for each line read from the file
 * @param {string|Buffer} line - Line content (string by default, Buffer if retainBuffer=true)
 * @param {number} lineCount - Current line number (1-based)
 * @param {number} byteCount - Total bytes processed so far
 */
rl.on('line', function(line, lineCount, byteCount) {
  // Process line
});

Usage Examples:

// Standard string processing
rl.on('line', function(line, lineCount, byteCount) {
  console.log(`Line ${lineCount} (${byteCount} bytes): ${line}`);
});

// Custom encoding with retainBuffer
const iconv = require('iconv-lite');
const rl = readLine('./file-in-win1251.txt', { retainBuffer: true });

rl.on('line', function(buffer, lineCount, byteCount) {
  const line = iconv.decode(buffer, 'win1251');
  console.log(`Decoded line ${lineCount}: ${line}`);
});

Open Event

Emitted when the underlying file stream opens.

/**
 * Emitted when the underlying file stream opens
 * @param {number} fd - File descriptor
 */
rl.on('open', function(fd) {
  // File opened successfully
});

Error Event

Emitted when an error occurs during reading or processing.

/**
 * Emitted when an error occurs during reading or processing
 * @param {Error} error - Error object
 */
rl.on('error', function(error) {
  // Handle error
});

Usage Examples:

rl.on('error', function(err) {
  if (err.code === 'ENOENT') {
    console.error('File not found:', err.path);
  } else {
    console.error('Reading error:', err.message);
  }
});

End Event

Emitted when the file has been completely read.

/**
 * Emitted when the file has been completely read
 */
rl.on('end', function() {
  // File reading completed
});

Close Event

Emitted when the underlying stream closes.

/**
 * Emitted when the underlying stream closes
 */
rl.on('close', function() {
  // Stream closed
});

Configuration Options

maxLineLength

Controls the maximum buffer size for a single line.

/**
 * Maximum buffer size for a single line
 * @type {number}
 * @default 4096
 */
opts.maxLineLength

Lines longer than this limit will not be read properly and may cause data loss.

retainBuffer

Controls whether to emit raw Buffer objects instead of converted strings.

/**
 * When true, the 'line' event receives raw Buffer objects instead of strings
 * @type {boolean} 
 * @default false
 */
opts.retainBuffer

Useful for custom encoding scenarios where you need to decode the buffer manually.

Usage Examples:

// High precision for large lines
const rl = readLine('./big-data.txt', {
  maxLineLength: 16384  // 16K buffer
});

// Custom encoding handling
const rl = readLine('./encoded-file.txt', {
  retainBuffer: true
});

rl.on('line', function(buffer, lineCount, byteCount) {
  // Custom decoding logic here
  const line = buffer.toString('utf16le');
  console.log(line);
});

Types

/**
 * ReadLine class - extends EventEmitter
 */
class ReadLine extends EventEmitter {
  constructor(file, opts);
  
  /** The underlying readable stream */
  input: ReadableStream;
}

/**
 * Configuration options for ReadLine constructor
 */
interface ReadLineOptions {
  /** Maximum line length in bytes (default: 4096) */
  maxLineLength?: number;
  /** If true, emits raw Buffer objects instead of strings (default: false) */
  retainBuffer?: boolean;
}

Error Handling

LineByLine provides comprehensive error handling through the event system:

  • File System Errors: File not found, permission errors, etc. are emitted as 'error' events
  • Processing Errors: Errors during line emission are caught and emitted as 'error' events
  • Stream Errors: Underlying stream errors are propagated through the 'error' event
  • Buffer Overflow: Lines exceeding maxLineLength may cause data loss but don't emit errors

Best Practices:

const rl = readLine('./file.txt');

// Always handle errors
rl.on('error', function(err) {
  console.error('Error:', err.message);
  // Handle error appropriately
});

// Ensure proper cleanup
rl.on('end', function() {
  console.log('Processing completed');
});

rl.on('close', function() {
  console.log('Stream closed');
});

Line Ending Support

LineByLine handles different line ending formats:

  • LF (Line Feed - ASCII 10): Standard Unix/Linux line endings - triggers line emission
  • CR (Carriage Return - ASCII 13): Classic Mac line endings - ignored but processed
  • CRLF: Windows line endings work correctly (CR is ignored, LF triggers emission)
  • EOF: Final line is emitted even without trailing line ending

Performance Characteristics

  • Memory Efficient: Streaming implementation processes files without loading them entirely into memory
  • Configurable Buffer: Default 4K line buffer, adjustable via maxLineLength option
  • Byte-by-byte Processing: Processes data one byte at a time for accurate line detection
  • Event-driven: Non-blocking, event-based processing suitable for large files
  • Lazy Evaluation: Lines are processed as they are read from the stream

docs

index.md

tile.json