CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/npm-nearley

Simple, fast, powerful parser toolkit for JavaScript using the Earley parsing algorithm

Pending
Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Pending

The risk profile of this skill

Overview
Eval results
Files

grammar-management.mddocs/

Grammar Management

Grammar representation and compilation system for working with context-free grammars and production rules.

Capabilities

Grammar Class

Represents a compiled grammar with rules and metadata.

/**
 * Create a new grammar from rules and start symbol
 * @param rules - Array of Rule instances
 * @param start - Start symbol name (defaults to first rule's name)
 */
class Grammar {
  constructor(rules: Rule[], start?: string);
}

/**
 * Grammar instance properties
 */
interface Grammar {
  /** Array of all grammar rules */
  rules: Rule[];
  /** Start symbol name */
  start: string;
  /** Rules indexed by name for fast lookup */
  byName: {[name: string]: Rule[]};
  /** Associated lexer (if any) */
  lexer?: object;
}

Usage Examples:

const nearley = require("nearley");

// Create rules
const rules = [
  new nearley.Rule("expr", ["number"], id => id[0]),
  new nearley.Rule("expr", ["expr", "+", "expr"], data => data[0] + data[2]),
  new nearley.Rule("number", [/[0-9]+/], data => parseInt(data[0]))
];

// Create grammar
const grammar = new nearley.Grammar(rules, "expr");
console.log("Start symbol:", grammar.start);
console.log("Total rules:", grammar.rules.length);

Grammar From Compiled

Create grammar from compiled grammar objects (generated by nearleyc).

/**
 * Create Grammar from compiled grammar object
 * @param rules - Compiled grammar object with ParserRules and metadata
 * @param start - Optional start symbol override
 * @returns Grammar instance ready for parsing
 */
static Grammar.fromCompiled(rules: object, start?: string): Grammar;

Usage Examples:

const nearley = require("nearley");

// Load compiled grammar (generated with nearleyc)
const compiledGrammar = require("./my-grammar.js");

// Create grammar instance
const grammar = nearley.Grammar.fromCompiled(compiledGrammar);

// Grammar is ready to use with parser
const parser = new nearley.Parser(grammar);

// Access grammar properties
console.log("Grammar start:", grammar.start);
console.log("Grammar rules:", grammar.rules.length);
console.log("Has lexer:", !!grammar.lexer);

Rule Class

Represents individual production rules in the grammar.

/**
 * Create a new grammar rule
 * @param name - Left-hand side symbol name
 * @param symbols - Right-hand side symbols array
 * @param postprocess - Optional postprocessing function
 */
class Rule {
  constructor(name: string, symbols: any[], postprocess?: Function);
}

/**
 * Rule instance properties
 */
interface Rule {
  /** Unique rule identifier */
  id: number;
  /** Left-hand side symbol name */
  name: string;
  /** Right-hand side symbols */
  symbols: any[];
  /** Postprocessing function */
  postprocess?: Function;
}

/**
 * Static rule counter
 */
static Rule.highestId: number;

Usage Examples:

const nearley = require("nearley");

// Simple terminal rule
const numberRule = new nearley.Rule(
  "number",
  [/[0-9]+/],
  data => parseInt(data[0])
);

// Rule with multiple symbols
const addRule = new nearley.Rule(
  "expr",
  ["expr", "+", "expr"],
  data => data[0] + data[2]
);

// Rule with no postprocessor
const parenRule = new nearley.Rule(
  "expr", 
  ["(", "expr", ")"],
  data => data[1]  // Extract middle element
);

console.log("Rule name:", numberRule.name);
console.log("Rule symbols:", numberRule.symbols);
console.log("Rule ID:", numberRule.id);

Rule String Representation

/**
 * Convert rule to string representation
 * @param withCursorAt - Optional position to show cursor (for debugging)
 * @returns String representation like "expr → number + expr"
 */
toString(withCursorAt?: number): string;

Usage Examples:

const rule = new nearley.Rule("expr", ["number", "+", "expr"]);

// Basic string representation
console.log(rule.toString());
// Output: "expr → number + expr"

// With cursor for debugging (shows parse position)
console.log(rule.toString(1));  
// Output: "expr → number ● + expr"

console.log(rule.toString(3));
// Output: "expr → number + expr ●"

Working with Compiled Grammars

When working with grammars compiled by nearleyc, the structure includes additional metadata.

/**
 * Compiled grammar object structure (generated by nearleyc)
 */
interface CompiledGrammar {
  /** Parser rules array */
  ParserRules: object[];
  /** Start symbol name */
  ParserStart: string;
  /** Associated lexer */
  Lexer?: object;
}

Usage Examples:

// Example of manually creating a grammar equivalent to compiled output
const manualGrammar = {
  ParserRules: [
    {
      name: "expr",
      symbols: ["number"],
      postprocess: id => id[0]
    },
    {
      name: "expr", 
      symbols: ["expr", {literal: "+"}, "expr"],
      postprocess: data => data[0] + data[2]
    }
  ],
  ParserStart: "expr"
};

const grammar = nearley.Grammar.fromCompiled(manualGrammar);

// Grammar symbols can be:
// - Strings (non-terminals): "expr", "number" 
// - Objects with literal: {literal: "+"}
// - RegExp objects: /[0-9]+/
// - Objects with test function: {test: (char) => char === 'x'}
// - Objects with type (for lexer tokens): {type: "NUMBER"}

Grammar Inspection

/**
 * Access grammar rules by name
 */
interface Grammar {
  byName: {[name: string]: Rule[]};
}

Usage Examples:

const grammar = nearley.Grammar.fromCompiled(compiledGrammar);

// Find all rules for a specific symbol
const exprRules = grammar.byName["expr"];
console.log(`Found ${exprRules.length} rules for 'expr'`);

exprRules.forEach((rule, i) => {
  console.log(`Rule ${i + 1}: ${rule.toString()}`);
});

// Check if symbol exists in grammar
if (grammar.byName["statement"]) {
  console.log("Grammar includes 'statement' rules");
}

// List all non-terminal symbols
const symbols = Object.keys(grammar.byName);
console.log("Grammar symbols:", symbols);

Integration with Lexers

/**
 * Grammar with lexer integration
 */
interface GrammarWithLexer extends Grammar {
  lexer: object;
}

Usage Examples:

const moo = require("moo");

// Create a lexer
const lexer = moo.compile({
  number: /[0-9]+/,
  plus: "+",
  ws: {match: /\s+/, lineBreaks: true}
});

// In compiled grammar, lexer is attached
const grammarWithLexer = nearley.Grammar.fromCompiled({
  ParserRules: [/* rules using lexer tokens */],
  ParserStart: "expr", 
  Lexer: lexer
});

console.log("Grammar has lexer:", !!grammarWithLexer.lexer);

// Parser will use the attached lexer
const parser = new nearley.Parser(grammarWithLexer);

docs

cli-tools.md

core-parsing.md

grammar-management.md

index.md

stream-processing.md

text-generation.md

tile.json