CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/npm-nearley

Simple, fast, powerful parser toolkit for JavaScript using the Earley parsing algorithm

Pending
Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Pending

The risk profile of this skill

Overview
Eval results
Files

cli-tools.mddocs/

CLI Tools

Command-line tools for grammar compilation, testing, railroad diagram generation, and text generation.

Capabilities

nearleyc - Grammar Compiler

Compile .ne grammar files into JavaScript modules ready for use with nearley parsers.

nearleyc [options] <file.ne>

Options:
  -o, --out [filename.js]     File to output to (defaults to stdout)
  -e, --export [name]         Variable to set parser to (default: "grammar")
  -q, --quiet                 Suppress linter warnings
  --nojs                      Do not compile postprocessors
  -v, --version               Display version number
  -h, --help                  Display help information

Usage Examples:

# Compile grammar to stdout
nearleyc grammar.ne

# Compile grammar to file
nearleyc -o grammar.js grammar.ne

# Compile with custom export name
nearleyc -e myGrammar -o parser.js grammar.ne

# Compile without JavaScript postprocessors
nearleyc --nojs -o parser.js grammar.ne

# Suppress linter warnings
nearleyc -q -o parser.js grammar.ne

Programmatic Usage:

// The nearleyc tool uses these internal modules:
const nearley = require("nearley");
const Compile = require("nearley/lib/compile");
const generate = require("nearley/lib/generate");
const lint = require("nearley/lib/lint");

// Parse .ne file and compile
const fs = require("fs");
const grammarSource = fs.readFileSync("grammar.ne", "utf8");

// Parse grammar with nearley's bootstrapped parser
const parserGrammar = nearley.Grammar.fromCompiled(
  require("nearley/lib/nearley-language-bootstrapped")
);
const parser = new nearley.Parser(parserGrammar);
parser.feed(grammarSource);

// Compile the parsed grammar
const compiled = Compile(parser.results[0], {
  version: "2.20.1"
});

// Generate JavaScript code
const jsCode = generate(compiled, "grammar");
console.log(jsCode);

nearley-test - Grammar Tester

Test compiled grammars with input strings and inspect parse results.

nearley-test [options] <file.js>

Options:
  -i, --input [string]        Input string to parse (stdin if not provided)
  -s, --start [symbol]        Start symbol (uses parser default if not provided)
  -o, --out [filename]        Output file (defaults to stdout)
  -q, --quiet                 Output parse results only (hide Earley table)
  -v, --version               Display version number
  -h, --help                  Display help information

Usage Examples:

# Test with input string
nearley-test -i "2 + 3 * 4" arithmetic.js

# Test with stdin input
echo "hello world" | nearley-test language.js

# Test with custom start symbol
nearley-test -s expression -i "x + y" math.js

# Quiet mode (results only)
nearley-test -q -i "test input" grammar.js

# Save output to file
nearley-test -i "data" -o results.txt parser.js

Example Output:

Table length: 15
Number of parses: 1
Parse Charts
Chart: 0
0: {expression → ● term}, from: 0
1: {term → ● factor}, from: 0
...

Parse results: 
[
  {
    type: 'expression',
    left: { type: 'number', value: 2 },
    operator: '+',
    right: {
      type: 'expression', 
      left: { type: 'number', value: 3 },
      operator: '*',
      right: { type: 'number', value: 4 }
    }
  }
]

nearley-railroad - Diagram Generator

Generate railroad diagrams (syntax diagrams) from .ne grammar files.

nearley-railroad [options] <file.ne>

Options:
  -o, --out [filename.svg]    Output file (defaults to stdout)  
  -v, --version               Display version number
  -h, --help                  Display help information

Usage Examples:

# Generate diagram to stdout (HTML format)
nearley-railroad grammar.ne

# Generate diagram to HTML file
nearley-railroad -o diagram.html grammar.ne

# View diagram directly in browser
nearley-railroad grammar.ne > diagram.html && open diagram.html

Example Grammar for Railroad Diagrams:

# arithmetic.ne
expr -> expr "+" term | term
term -> term "*" factor | factor  
factor -> "(" expr ")" | number
number -> [0-9]:+

Generated Diagram Structure:

The output is a complete HTML document with embedded CSS and SVG railroad diagrams showing the visual structure of each grammar rule.

nearley-unparse - Text Generator

Generate random text from compiled grammars for testing and example generation.

nearley-unparse [options] <file.js>

Options:
  -s, --start [name]          Start symbol (uses parser default if not provided)
  -n, --count [n]             Number of samples to generate (default: 1)
  -d, --depth [n]             Depth bound for generation (default: -1 for unbounded)
  -o, --out [filename]        Output file (defaults to stdout)
  -v, --version               Display version number
  -h, --help                  Display help information

Usage Examples:

# Generate one random sample
nearley-unparse grammar.js

# Generate 10 random samples
nearley-unparse -n 10 grammar.js

# Generate with depth limit
nearley-unparse -d 5 -n 5 grammar.js

# Generate with custom start symbol
nearley-unparse -s expression -n 3 math-grammar.js

# Save output to file
nearley-unparse -n 100 -o samples.txt grammar.js

Example Output:

$ nearley-unparse -n 5 arithmetic.js
42
17 + 23
(8 * 3) + 15
91 + (7 * 2)
((4 + 6) * 5) + 12

Integration with npm Scripts

Common patterns for integrating nearley CLI tools into build workflows.

Usage Examples:

{
  "scripts": {
    "build:grammar": "nearleyc -o lib/parser.js grammar/language.ne",
    "test:parser": "nearley-test -i 'test input' lib/parser.js",
    "diagram": "nearley-railroad -o docs/syntax.html grammar/language.ne",
    "examples": "nearley-unparse -n 20 -o examples/samples.txt lib/parser.js",
    "validate": "nearleyc -q grammar/language.ne > /dev/null",
    "watch:grammar": "chokidar 'grammar/*.ne' -c 'npm run build:grammar'"
  },
  "devDependencies": {
    "nearley": "^2.20.1",
    "chokidar-cli": "^3.0.0"
  }
}

Makefile Integration:

# Build grammar
lib/parser.js: grammar/language.ne
	nearleyc -o $@ $<

# Generate examples
examples/samples.txt: lib/parser.js
	nearley-unparse -n 50 -o $@ $<

# Generate documentation
docs/syntax.html: grammar/language.ne
	nearley-railroad -o $@ $<

# Test grammar
test: lib/parser.js
	nearley-test -i "example input" $<

# Validate grammar syntax
validate: grammar/language.ne
	nearleyc -q $< > /dev/null

.PHONY: test validate

Error Handling and Debugging

Common error scenarios and debugging techniques with CLI tools.

Usage Examples:

# Check grammar syntax
nearleyc -q grammar.ne
# Exit code 0 = success, non-zero = syntax error

# Debug parser issues with verbose output
nearley-test -i "problematic input" parser.js
# Shows full parse table and state transitions

# Generate simple examples for debugging
nearley-unparse -d 2 -n 10 parser.js
# Shallow examples are easier to debug

# Validate compiled output
node -e "console.log(require('./parser.js'))"
# Check if compiled grammar loads correctly

Common Error Messages:

# Grammar compilation errors
nearleyc: SyntaxError: Unexpected token at line 5 col 10
nearleyc: Error: Cannot find built-in "nonexistent.ne"

# Parser test errors  
nearley-test: Error: No valid parse found
nearley-test: SyntaxError: Unexpected character 'x' at line 1 col 5

# Unparse errors
nearley-unparse: Error: Cannot match rule: undefined_symbol

Advanced CLI Usage

Complex workflows combining multiple CLI tools.

Usage Examples:

#!/bin/bash
# Complete grammar development workflow

echo "Building grammar..."
nearleyc -o parser.js grammar.ne || exit 1

echo "Validating with test cases..."
while IFS= read -r test_case; do
  echo "Testing: $test_case"
  nearley-test -q -i "$test_case" parser.js || echo "FAILED: $test_case"
done < test_cases.txt

echo "Generating examples..."
nearley-unparse -n 20 -o examples.txt parser.js

echo "Creating documentation..."
nearley-railroad -o syntax-diagram.html grammar.ne

echo "Grammar development complete!"

Tool Configuration

Environment variables and configuration options for CLI tools.

Usage Examples:

# Set custom nearley installation path
export NEARLEY_PATH=/usr/local/lib/node_modules/nearley

# Configure output formatting
export NEARLEY_QUIET=1  # Suppress warnings by default

# Batch processing with configuration
find grammars/ -name "*.ne" -exec nearleyc -o compiled/{}.js {} \;

# Test multiple grammars
for grammar in compiled/*.js; do
  echo "Testing $grammar..."
  nearley-test -q -i "test" "$grammar" || echo "Failed: $grammar"
done

docs

cli-tools.md

core-parsing.md

grammar-management.md

index.md

stream-processing.md

text-generation.md

tile.json