Lezer Elixir

Lezer Elixir provides an Elixir grammar implementation for the Lezer parser system, which is used in CodeMirror editors for syntax highlighting and parsing. Based directly on tree-sitter-elixir, it offers specialized syntax tree nodes optimized for Lezer's performance requirements while providing comprehensive support for Elixir's unique syntax features.

Package Information

• Package Name: lezer-elixir
• Package Type: npm
• Language: JavaScript/TypeScript
• Installation: npm install lezer-elixir

Core Imports

import { parser } from "lezer-elixir";

For CommonJS:

const { parser } = require("lezer-elixir");

Note: The package exports only the main parser instance. Syntax highlighting is built into the parser and accessed via CodeMirror's language integration.

Basic Usage

import { parser } from "lezer-elixir";
import { EditorView, basicSetup } from "codemirror";
import { EditorState } from "@codemirror/state";
import { LRLanguage } from "@codemirror/language";
import { styleTags, tags as t } from "@lezer/highlight";
import { syntaxHighlighting, defaultHighlightStyle } from "@codemirror/language";

// Create Elixir language instance with highlighting
const elixir = LRLanguage.define({
  parser: parser,
  languageData: {
    commentTokens: { line: "#" },
    indentOnInput: /^\s*end\s*$/
  }
});

// Create CodeMirror editor with Elixir support
const elixirCode = `
def fibonacci(n) when n <= 1, do: n
def fibonacci(n), do: fibonacci(n - 1) + fibonacci(n - 2)

IO.puts fibonacci(10)
`;

const view = new EditorView({
  state: EditorState.create({
    doc: elixirCode,
    extensions: [
      basicSetup,
      elixir,
      syntaxHighlighting(defaultHighlightStyle)
    ]
  }),
  parent: document.body
});

Architecture

Lezer Elixir is built around several key components:

• Parser: Core LR parser instance for parsing Elixir syntax trees
• Grammar Definition: Generated from elixir.grammar file using @lezer/generator
• Tokenizers: Specialized external tokenizers for complex Elixir syntax elements
• Highlighting: Built-in syntax highlighting configuration using @lezer/highlight
• Language Support: Integrates with CodeMirror's language system for syntax highlighting

The grammar provides a superset of tree-sitter-elixir's syntax tree with additional specialized nodes for Lezer's performance requirements.

Internal Tokenizers

The parser includes several external tokenizers for handling complex Elixir syntax:

• Quoted Content Tokenizers: Handle various string and sigil delimiters (parentheses, brackets, braces, etc.)
• Sigil Tokenizers: Process sigil syntax with modifiers
• Identifier Tokenizers: Distinguish between regular, underscored, and special identifiers
• Operator Tokenizers: Handle context-sensitive operators like @ for attributes
• Newline Tokenizers: Manage whitespace-sensitive parsing

These tokenizers are automatically used by the parser and are not directly exposed to users.

Capabilities

Parser

The main Lezer parser instance for parsing Elixir code into syntax trees with built-in syntax highlighting support.

/**
 * The main Lezer parser instance for parsing Elixir code
 * Generated from elixir.grammar using @lezer/generator
 * Includes built-in syntax highlighting configuration and specialized tokenizers
 */
export const parser: LRParser;

Types

/**
 * Lezer LR parser instance with Elixir grammar and highlighting
 * Imported from @lezer/lr
 */
interface LRParser {
  /** Parse input text into a syntax tree */
  parse(input: Input | string): Tree;
  /** Start incremental parsing for large documents */
  startParse(
    input: Input,
    fragments?: readonly TreeFragment[],
    ranges?: readonly {from: number, to: number}[]
  ): PartialParse;
  /** Configure parser with additional options */
  configure(config: ParserConfig): LRParser;
  /** Check if parser has recovery information for error handling */
  get hasWrappers(): boolean;
  /** Get the parser's name for debugging */
  get name(): string;
  /** Create a Language instance for CodeMirror integration */
  get language(): Language;
}

/**
 * Input interface for parser
 */
interface Input {
  /** Get character code at position */
  get(pos: number): number;
  /** Read characters into string from range */
  read(from: number, to: number): string;
  /** Length of the input text */
  get length(): number;
  /** Get line information at position */
  lineAfter(pos: number): string;
  /** Cursor for efficient navigation */
  cursor(pos?: number): Cursor;
}

/**
 * Parser configuration options
 */
interface ParserConfig {
  /** Custom node properties for syntax highlighting and behavior */
  props?: readonly NodeProp<any>[];
  /** Override top-level rule */
  top?: string;
  /** Add wrapper parsers for mixed-language support */
  wrap?: ParseWrapper;
  /** Strict mode for error reporting */
  strict?: boolean;
}

/**
 * Syntax tree representation
 */
interface Tree {
  /** Total length of the parsed text */
  get length(): number;
  /** String representation for debugging */
  toString(): string;
  /** Iterate over tree nodes */
  iterate(config: IterateConfig): TreeCursor;
  /** Get cursor at specific position */
  cursorAt(pos: number, side?: -1 | 0 | 1): TreeCursor;
  /** Resolve position to node */
  resolve(pos: number, side?: -1 | 0 | 1): SyntaxNode;
}

Specialized Syntax Tree Nodes

The grammar includes specialized nodes beyond the standard tree-sitter-elixir implementation:

Identifier Specializations

• SpecialIdentifier: __MODULE__ and similar special identifiers
• UnderscoredIdentifier: Variables starting with underscore (_x)

Call Specializations

• FunctionDefinitionCall: Function definitions (def fun(x), do: 1)
• KernelCall: Kernel built-ins (if, case, etc.)

Operator Specializations

• AtOperator: Module attributes (@x)
• DocOperator: Documentation attributes (@doc "...")
• CaptureOperator: Function captures (&fun/1)
• CaptureOperand: Capture references (&1)
• WhenOperator: Guard clauses (left when right)
• PipeOperator: Pipe operations (left |> right)

Sigil Specializations

• StringSigil: String sigils (~s"string", ~S"string")

Named Operator Groups

• Operator: Standard operators (+, -, *, etc.)
• WordOperator: Word-based operators (and, or, when)

Error Handling

The parser handles malformed Elixir code gracefully, producing partial syntax trees that can still be used for highlighting and basic analysis. Invalid syntax is marked with error nodes in the syntax tree.

Common error scenarios:

• Unterminated strings or sigils
• Mismatched delimiters
• Invalid operator usage
• Incomplete function definitions

Integration Notes

• Designed specifically for CodeMirror 6+ integration
• Requires @lezer/lr and @lezer/highlight as dependencies
• Grammar is generated during build process from src/elixir.grammar using @lezer/generator
• Supports both ES modules and CommonJS imports
• Full TypeScript type definitions included in distribution
• Works with standard CodeMirror highlighting extensions
• Built-in highlighting configuration optimized for Elixir syntax

Dependencies

Runtime Dependencies:

• @lezer/lr - Core LR parser functionality
• @lezer/highlight - Syntax highlighting system

Build Dependencies:

• @lezer/generator - Grammar compilation (development only)

CodeMirror Integration

To use with CodeMirror, you'll typically want to combine the parser with CodeMirror's language support:

import { parser } from "lezer-elixir";
import { LRLanguage, LanguageSupport } from "@codemirror/language";
import { styleTags, tags as t } from "@lezer/highlight";

// Create language definition
const elixirLanguage = LRLanguage.define({
  parser: parser,
  languageData: {
    commentTokens: { line: "#" },
    indentOnInput: /^\\s*(end|catch|rescue|after|else)\\s*$/,
    closeBrackets: { brackets: ["(", "[", "{", "\"", "'"] }
  }
});

// Create full language support
export function elixir() {
  return new LanguageSupport(elixirLanguage);
}