or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

index.md
tile.json

tessl/npm-cjs-module-lexer

High-performance CommonJS module lexer for detecting named exports and reexports from JavaScript source code

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
npmpkg:npm/cjs-module-lexer@2.1.x

To install, run

npx @tessl/cli install tessl/npm-cjs-module-lexer@2.1.0

index.mddocs/

CJS Module Lexer

CJS Module Lexer is a high-performance JavaScript CommonJS module syntax lexer that detects named exports and reexports from source code without full parsing. It provides both JavaScript and WebAssembly implementations optimized for speed, processing JavaScript at ~90ms per MB cold and ~15ms per MB warm.

Package Information

  • Package Name: cjs-module-lexer
  • Package Type: npm
  • Language: JavaScript/TypeScript
  • Installation: npm install cjs-module-lexer

Core Imports

const { parse, init, initSync } = require('cjs-module-lexer');

For ES modules (with WebAssembly support):

import { parse, init, initSync } from 'cjs-module-lexer';

Note: The package provides different builds for different module systems:

  • CommonJS: Uses the main lexer.js file
  • ES Modules: Uses the WebAssembly-enabled dist/lexer.mjs build

Basic Usage

const { parse } = require('cjs-module-lexer');

// Parse CommonJS source code
const { exports, reexports } = parse(`
  // Named exports detection
  module.exports.hello = 'world';
  exports.foo = 'bar';
  Object.defineProperty(exports, 'baz', { value: 'qux' });
  
  // Reexports detection
  module.exports = require('./other-module.js');
  
  // Literal exports assignments
  module.exports = { api: true, version: '1.0' };
`);

console.log(exports);   // ['hello', 'foo', 'baz', 'api', 'version']
console.log(reexports); // ['./other-module.js']

For WebAssembly version with better performance:

import { parse, init } from 'cjs-module-lexer';

// Initialize WebAssembly (required for ESM usage)
await init();

const { exports, reexports } = parse(sourceCode);

Architecture

CJS Module Lexer is built around a streaming lexical analysis approach:

  • Token Parser: Parses JavaScript tokens (comments, strings, templates, blocks, punctuators) without building a full AST
  • Pattern Matching: Uses specific pattern recognition for CommonJS export syntax variations
  • Export Detection: Identifies exports.name, exports['name'], Object.defineProperty(exports, ...) patterns
  • Transpiler Support: Recognizes common transpiler patterns from Babel, TypeScript, and bundlers
  • Dual Implementation: JavaScript fallback and WebAssembly build for optimal performance

Capabilities

Source Code Parsing

Analyzes CommonJS source code to extract export information using lexical analysis.

/**
 * Parse CommonJS source code to detect exports and reexports
 * @param source - JavaScript source code to analyze
 * @param name - Optional module name for error reporting and stack traces (defaults to '@')
 * @returns Object containing detected exports and reexports arrays
 */
function parse(source: string, name?: string): Exports;

interface Exports {
  /** Array of detected named export identifiers */
  exports: string[];
  /** Array of detected reexport module specifiers */
  reexports: string[];
}

Usage Examples:

const { parse } = require('cjs-module-lexer');

// Basic named exports
const result1 = parse(`
  exports.myFunction = function() {};
  exports.myConstant = 42;
`);
// result1.exports: ['myFunction', 'myConstant']

// Object.defineProperty patterns
const result2 = parse(`
  Object.defineProperty(exports, 'computed', {
    enumerable: true,
    get: function() { return someValue; }
  });
`);
// result2.exports: ['computed']

// Module reexports
const result3 = parse(`
  if (condition) {
    module.exports = require('./dependency');
  }
`);
// result3.reexports: ['./dependency']

// Literal object assignments
const result4 = parse(`
  module.exports = {
    method1: function() {},
    method2: async function() {},
    constant: 'value'
  };
`);
// result4.exports: ['method1', 'method2', 'constant']

WebAssembly Initialization

Initializes the WebAssembly version for better performance (ESM usage only).

/**
 * Asynchronously initialize WebAssembly compilation
 * Required when using ESM imports, no-op for CommonJS
 * @returns Promise that resolves when WASM is ready
 */
function init(): Promise<void>;

/**
 * Synchronously initialize WebAssembly compilation
 * Alternative to init() for blocking compilation
 * @returns void - compilation happens synchronously
 */
function initSync(): void;

Usage Examples:

// Async initialization (recommended) - uses dist/lexer.mjs
import { parse, init } from 'cjs-module-lexer';
await init();
const result = parse(sourceCode);

// Sync initialization (blocking) - uses dist/lexer.mjs
import { parse, initSync } from 'cjs-module-lexer';
initSync();
const result = parse(sourceCode);

// CommonJS usage (no WASM, no initialization needed) - uses lexer.js
const { parse } = require('cjs-module-lexer');
const result = parse(sourceCode);

Export Detection Patterns

The lexer detects these CommonJS export patterns:

Named Exports

  • exports.name = value - Dot notation assignment
  • exports['name'] = value - Bracket notation assignment
  • module.exports.name = value - Module exports dot notation
  • Object.defineProperty(exports, 'name', { value: ... }) - Property descriptor
  • Object.defineProperty(exports, 'name', { get: function() { return ...; } }) - Getter functions

Object Assignments

  • module.exports = { name: value, ... } - Object literal assignment
  • module.exports = { name, ... } - Shorthand property syntax
  • module.exports = { 'name': value, ... } - String property keys
  • module.exports = { ...require('module') } - Spread syntax with requires

Reexport Patterns

  • module.exports = require('module') - Direct reexport assignment
  • module.exports = { ...require('a'), ...require('b') } - Multiple spread reexports

Transpiler Support

  • Babel: Object.keys(_external).forEach(function (key) { ... }) patterns
  • TypeScript: __export(require("external")) and __exportStar patterns
  • Bundlers: _interopRequireWildcard patterns from various build tools

Error Handling

/**
 * Custom error thrown for ESM syntax in CommonJS modules
 * Extends the standard Error interface with additional properties
 */
interface ESMSyntaxError extends Error {
  /** Error code identifying this as an ESM syntax error */
  code: 'ERR_LEXER_ESM_SYNTAX';
  /** Character position in source where error occurred */
  loc: number;
  /** Standard error message describing the issue */
  message: string;
  /** Standard error name property */
  name: string;
}

Error Examples:

try {
  parse('import { foo } from "bar";'); // ESM syntax in CJS
} catch (error) {
  console.log(error.code); // 'ERR_LEXER_ESM_SYNTAX'
  console.log(error.loc);  // Character position
  console.log(error.message); // 'Unexpected import statement in CJS module.'
}

try {
  parse('export const foo = "bar";'); // ESM export in CJS
} catch (error) {
  console.log(error.message); // 'Unexpected export statement in CJS module.'
}

Performance Characteristics

  • JavaScript Implementation: ~90ms per MB cold, ~15ms per MB warm
  • WebAssembly Implementation: ~1.5x faster, no cold start penalty
  • Memory Efficient: Streaming lexer without full AST construction
  • Node.js Optimized: Used internally by Node.js core for ESM/CJS interoperability

Environment Support:

  • Node.js 10+
  • All browsers with WebAssembly support
  • Both CommonJS and ES module environments