Architecture

Marked uses a two-phase architecture that separates tokenization from rendering.

Two-Phase Architecture

1. Lexing Phase

The Lexer tokenizes markdown text into a structured token tree:

• Recognizes block-level elements (headings, lists, paragraphs)
• Queues inline elements for processing
• Stores link reference definitions

2. Parsing Phase

The Parser walks the token tree and uses a Renderer to generate output:

• Processes tokens sequentially
• Calls renderer methods for each token type
• Supports custom renderers for different output formats

Processing Pipeline

Markdown Source
    ↓
[Preprocess Hook]
    ↓
Lexer (Tokenization)
    ↓
[ProcessAllTokens Hook]
    ↓
[WalkTokens Extension]
    ↓
Parser + Renderer
    ↓
[Postprocess Hook]
    ↓
HTML Output

Benefits

• Token manipulation: Inspect and modify tokens between lexing and parsing
• Custom renderers: Output formats other than HTML (plain text, LaTeX, etc.)
• Multi-pass processing: Process tokens multiple times with different configurations
• Extensions: Add functionality at either the lexing or parsing phase

Related Documentation

• Lexer Reference
• Parser & Renderer Reference
• Extension System
• Hooks System