CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/npm-iconv-lite

Convert character encodings in pure javascript.

Pending
Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Pending

The risk profile of this skill

Overview
Eval results
Files

low-level.mddocs/

Low-level API

Direct access to encoder and decoder instances for advanced use cases, custom stream implementations, and performance-critical applications.

Capabilities

Get Encoder

Returns a low-level encoder instance for the specified encoding that can be used to encode data incrementally.

/**
 * Get encoder instance for incremental encoding
 * @param encoding - Target encoding name
 * @param options - Encoding options
 * @returns Encoder instance with write() and end() methods
 */
function getEncoder(encoding, options);

Usage Examples:

const iconv = require('iconv-lite');

// Basic encoder usage
const encoder = iconv.getEncoder('utf16le');
const chunk1 = encoder.write('Hello ');
const chunk2 = encoder.write('World');
const finalChunk = encoder.end();

const result = Buffer.concat([chunk1, chunk2, finalChunk || Buffer.alloc(0)]);

// Encoder with BOM
const encoderWithBOM = iconv.getEncoder('utf16le', { addBOM: true });
const encoded = encoderWithBOM.write('Text with BOM');
const final = encoderWithBOM.end();

Get Decoder

Returns a low-level decoder instance for the specified encoding that can be used to decode data incrementally.

/**
 * Get decoder instance for incremental decoding
 * @param encoding - Source encoding name
 * @param options - Decoding options
 * @returns Decoder instance with write() and end() methods
 */
function getDecoder(encoding, options);

Usage Examples:

const iconv = require('iconv-lite');

// Basic decoder usage
const decoder = iconv.getDecoder('gbk');
const str1 = decoder.write(buffer1);
const str2 = decoder.write(buffer2);
const finalStr = decoder.end();

const result = str1 + str2 + (finalStr || '');

// Decoder with BOM preservation
const decoder = iconv.getDecoder('utf16le', { stripBOM: false });
const decoded = decoder.write(bufferWithBOM);

Get Codec

Returns the internal codec object for the specified encoding, providing access to the underlying encoder and decoder constructors.

/**
 * Get internal codec for encoding
 * @param encoding - Encoding name
 * @returns Codec object with encoder and decoder constructors
 */
function getCodec(encoding);

Usage Examples:

const iconv = require('iconv-lite');

// Get codec information
const codec = iconv.getCodec('utf8');
console.log(codec.encodingName); // Canonical encoding name
console.log(codec.bomAware);     // Boolean indicating BOM support

// Create encoder/decoder instances directly
const encoder = new codec.encoder({}, codec);
const decoder = new codec.decoder({}, codec);

// Check codec properties
if (codec.bomAware) {
  console.log('This encoding supports BOM handling');
}

Low-level Interfaces

Encoder Interface

Low-level encoder instances returned by getEncoder():

/**
 * Encoder stream interface
 */
const EncoderStream = {
  /**
   * Encode a string chunk
   * @param str - String to encode
   * @returns Encoded buffer chunk
   */
  write(str),
  
  /**
   * Finalize encoding and get any remaining bytes
   * @returns Final buffer chunk or undefined
   */
  end()
};

Decoder Interface

Low-level decoder instances returned by getDecoder():

/**
 * Decoder stream interface
 */
const DecoderStream = {
  /**
   * Decode a buffer chunk
   * @param buf - Buffer to decode
   * @returns Decoded string chunk
   */
  write(buf),
  
  /**
   * Finalize decoding and get any remaining characters
   * @returns Final string chunk or undefined
   */
  end()
};

Codec Interface

Internal codec objects returned by getCodec():

/**
 * Codec interface
 */
const Codec = {
  /** Canonical encoding name */
  encodingName,
  /** Whether this encoding supports BOM handling */
  bomAware,
  /** Encoder constructor */
  encoder,
  /** Decoder constructor */
  decoder
};

Internal Properties

These internal properties are exposed for advanced use cases but should be used with caution:

Codec Data Cache

/**
 * Internal cache of loaded codec instances
 * Object with null prototype containing codec instances keyed by encoding name
 */
const _codecDataCache;

Encoding Canonicalization

/**
 * Normalize encoding name to canonical form
 * @param encoding - Raw encoding name
 * @returns Canonicalized encoding name (lowercase, alphanumeric only)
 */
function _canonicalizeEncoding(encoding);

Usage Examples:

const iconv = require('iconv-lite');

// Encoding name normalization
console.log(iconv._canonicalizeEncoding('UTF-8'));      // 'utf8'
console.log(iconv._canonicalizeEncoding('iso-8859-1')); // 'iso88591'
console.log(iconv._canonicalizeEncoding('Windows-1252:2000')); // 'windows1252'

Encodings Registry

/**
 * Registry of all available encodings and aliases
 * Lazy-loaded on first access, initially null
 */
const encodings;

BOM Handling Classes

Internal classes used for Byte Order Mark handling:

BOM Character Constant

/** BOM (Byte Order Mark) character constant */
const BOMChar; // "\uFEFF"

PrependBOM Wrapper

/**
 * Encoder wrapper that prepends BOM to output
 * @param encoder - Base encoder instance
 * @param options - Options (unused)
 */
class PrependBOMWrapper {
  constructor(encoder, options);
  write(str);
  end();
}

StripBOM Wrapper

/**
 * Decoder wrapper that strips BOM from output
 * @param decoder - Base decoder instance
 * @param options - Options with optional stripBOM callback
 */
class StripBOMWrapper {
  constructor(decoder, options);
  write(buf);
  end();
}

Performance Considerations

  • Incremental Processing: Use low-level API for processing data in chunks to reduce memory usage
  • Codec Caching: Codecs are cached after first use; repeated calls to getCodec() are fast
  • BOM Handling: BOM wrappers add minimal overhead and are applied automatically when needed
  • Encoding Validation: encodingExists() is faster than try/catch around getCodec()

Advanced Usage Patterns

const iconv = require('iconv-lite');

// Custom stream implementation
class CustomEncodingStream extends Transform {
  constructor(encoding, options) {
    super(options);
    this.encoder = iconv.getEncoder(encoding, options);
  }
  
  _transform(chunk, encoding, callback) {
    try {
      const result = this.encoder.write(chunk.toString());
      callback(null, result);
    } catch (err) {
      callback(err);
    }
  }
  
  _flush(callback) {
    try {
      const final = this.encoder.end();
      callback(null, final);
    } catch (err) {
      callback(err);
    }
  }
}

docs

core-encoding.md

index.md

low-level.md

streaming.md

tile.json