0
# ANTLR4
1

2
ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files. It transforms formal grammars into parsers and lexers for multiple target languages including Java, C#, Python, JavaScript, Go, C++, and Swift.
3

4
## Package Information
5

6
- **Package Name**: ANTLR4
7
- **Package Type**: Maven
8
- **Language**: Java
9
- **Installation**: 
10
  ```xml
11
  <dependency>
12
    <groupId>org.antlr</groupId>
13
    <artifactId>antlr4</artifactId>
14
    <version>4.7.2</version>
15
  </dependency>
16
  ```
17
- **Runtime Dependency**:
18
  ```xml
19
  <dependency>
20
    <groupId>org.antlr</groupId>
21
    <artifactId>antlr4-runtime</artifactId>
22
    <version>4.7.2</version>
23
  </dependency>
24
  ```
25

26
## Core Imports
27

28
Tool API (grammar compilation):
29
```java
30
import org.antlr.v4.Tool;
31
import org.antlr.v4.tool.Grammar;
32
import org.antlr.v4.tool.LexerGrammar;
33
```
34

35
Runtime API (parsing):
36
```java
37
import org.antlr.v4.runtime.*;
38
import org.antlr.v4.runtime.tree.*;
39
```
40

41
Generated parser usage:
42
```java
43
// Your generated classes
44
import MyLexer;
45
import MyParser;
46
```
47

48
## Basic Usage
49

50
### Grammar Compilation
51
```java
52
import org.antlr.v4.Tool;
53

54
// Command line compilation
55
String[] args = {"-visitor", "-listener", "MyGrammar.g4"};
56
Tool antlr = new Tool(args);
57
antlr.processGrammarsOnCommandLine();
58

59
// Programmatic compilation
60
Tool tool = new Tool();
61
tool.outputDirectory = "generated/";  // Set output directory
62
tool.gen_visitor = true;            // Enable visitor generation
63
Grammar g = tool.loadGrammar("MyGrammar.g4");
64
tool.process(g, true);              // Process with code generation
65
```
66

67
### Runtime Parsing
68
```java
69
import org.antlr.v4.runtime.*;
70
import org.antlr.v4.runtime.tree.*;
71

72
// Create input stream
73
CharStream input = CharStreams.fromFileName("input.txt");
74

75
// Create lexer
76
MyLexer lexer = new MyLexer(input);
77
CommonTokenStream tokens = new CommonTokenStream(lexer);
78

79
// Create parser
80
MyParser parser = new MyParser(tokens);
81
ParseTree tree = parser.startRule();
82

83
// Walk tree with visitor
84
MyBaseVisitor<String> visitor = new MyBaseVisitor<String>() {
85
    @Override
86
    public String visitSomeRule(MyParser.SomeRuleContext ctx) {
87
        return "Processing: " + ctx.getText();
88
    }
89
};
90
String result = visitor.visit(tree);
91

92
// Or use listener
93
ParseTreeWalker walker = new ParseTreeWalker();
94
MyBaseListener listener = new MyBaseListener() {
95
    @Override
96
    public void enterSomeRule(MyParser.SomeRuleContext ctx) {
97
        System.out.println("Entering: " + ctx.getText());
98
    }
99
};
100
walker.walk(listener, tree);
101
```
102

103
## Architecture
104

105
ANTLR4 consists of several key components:
106

107
- **Tool System**: Grammar compilation and code generation (`org.antlr.v4.Tool`)
108
- **Runtime System**: Parser and lexer execution framework (`org.antlr.v4.runtime`)
109
- **Parse Tree API**: Tree construction and traversal utilities (`org.antlr.v4.runtime.tree`)
110
- **ATN System**: Augmented Transition Network for parsing decisions (`org.antlr.v4.runtime.atn`)
111
- **Error Handling**: Comprehensive error recovery and reporting (`org.antlr.v4.runtime.*ErrorStrategy`)
112
- **Multi-Language Support**: Code generation for 8 target languages
113

114
## Capabilities
115

116
### Grammar Compilation Tool
117

118
Core functionality for compiling ANTLR4 grammars into parser and lexer code for various target languages.
119

120
```java { .api }
121
public class Tool {
122
    public static void main(String[] args);
123
    public Tool();
124
    public Tool(String[] args);
125
    public void processGrammarsOnCommandLine();
126
    public Grammar loadGrammar(String fileName);
127
    public void process(Grammar g, boolean gencode);
128
    public Grammar createGrammar(GrammarRootAST ast);
129
    public GrammarRootAST parseGrammar(String fileName);
130
    public void processNonCombinedGrammar(Grammar g, boolean gencode);
131
    public boolean checkForRuleIssues(Grammar g);
132
    public void generateATNs(Grammar g);
133
    public int getNumErrors();
134
    public void addListener(ANTLRToolListener tl);
135
    public void info(String msg);
136
    public void error(ANTLRMessage msg);
137
    public void warning(ANTLRMessage msg);
138
    
139
    // Configuration fields (direct access)
140
    public String outputDirectory;
141
    public String libDirectory;
142
    public boolean gen_visitor;
143
    public boolean gen_listener;
144
    public ErrorManager errMgr;
145
    public String packageName;
146
    public String language;
147
}
148

149
public class Grammar {
150
    public String name;
151
    public String fileName;
152
    public Map<String, Rule> rules;
153
}
154
```
155

156
[Tool API](./tool-api.md)
157

158
### Runtime Parser and Lexer Framework
159

160
Base classes and interfaces for generated parsers and lexers, providing the core parsing infrastructure.
161

162
```java { .api }
163
public abstract class Parser extends Recognizer<Token, ParserATNSimulator> {
164
    public void reset();
165
    public Token match(int ttype) throws RecognitionException;
166
    public Token consume();
167
    public Token matchWildcard() throws RecognitionException;
168
    public void addParseListener(ParseTreeListener listener);
169
    public void removeParseListener(ParseTreeListener listener);
170
    public List<ParseTreeListener> getParseListeners();
171
    public void removeParseListeners();
172
    public TokenStream getTokenStream();
173
    public void setTokenStream(TokenStream input);
174
    public ParserRuleContext getContext();
175
    public void setBuildParseTree(boolean buildParseTrees);
176
    public boolean getBuildParseTree();
177
    public IntervalSet getExpectedTokens();
178
    public Token getCurrentToken();
179
    public void setTrace(boolean trace);
180
    public void enterRule(ParserRuleContext localctx, int state, int ruleIndex);
181
    public void exitRule();
182
    public ANTLRErrorStrategy getErrorHandler();
183
    public void setErrorHandler(ANTLRErrorStrategy handler);
184
    public int getNumberOfSyntaxErrors();
185
    public void notifyErrorListeners(String msg);
186
    public TerminalNode createTerminalNode(ParserRuleContext parent, Token t);
187
    public ErrorNode createErrorNode(ParserRuleContext parent, Token t);
188
    public void enterOuterAlt(ParserRuleContext localctx, int altNum);
189
    public int getPrecedence();
190
    public boolean isExpectedToken(int symbol);
191
    public IntervalSet getExpectedTokensWithinCurrentRule();
192
    public int getRuleIndex(String ruleName);
193
    public List<String> getRuleInvocationStack();
194
}
195

196
public abstract class Lexer extends Recognizer<Integer, LexerATNSimulator> {
197
    public Token nextToken();
198
    public void skip();
199
    public void more();
200
    public void mode(int m);
201
}
202

203
public interface Token {
204
    /** End-of-file token type */
205
    int EOF = -1;
206
    
207
    /** Invalid token type */
208
    int INVALID_TYPE = 0;
209
    
210
    /** Default channel for tokens */
211
    int DEFAULT_CHANNEL = 0;
212
    
213
    /** Hidden channel for whitespace/comments */
214
    int HIDDEN_CHANNEL = 1;
215
    
216
    /** Epsilon token used during lookahead */
217
    int EPSILON = -2;
218
    
219
    /** Minimum constant value for user-defined token types */
220
    int MIN_USER_TOKEN_TYPE = 1;
221
    
222
    /** Minimum constant value for user-defined token channels */
223
    int MIN_USER_CHANNEL_VALUE = 2;
224
    
225
    int getType();
226
    String getText();
227
    int getLine();
228
    int getCharPositionInLine();
229
    int getChannel();
230
    int getTokenIndex();
231
    int getStartIndex();
232
    int getStopIndex();
233
    TokenSource getTokenSource();
234
    CharStream getInputStream();
235
}
236
```
237

238
[Runtime API](./runtime-api.md)
239

240
### Parse Tree Construction and Traversal
241

242
Complete API for building, navigating, and manipulating parse trees with visitor and listener patterns.
243

244
```java { .api }
245
public interface ParseTree extends SyntaxTree {
246
    ParseTree getParent();
247
    ParseTree getChild(int i);
248
    int getChildCount();
249
    <T> T accept(ParseTreeVisitor<? extends T> visitor);
250
}
251

252
public interface ParseTreeVisitor<T> {
253
    T visit(ParseTree tree);
254
    T visitChildren(RuleNode node);
255
    T visitTerminal(TerminalNode node);
256
    T visitErrorNode(ErrorNode node);
257
}
258

259
public interface ParseTreeListener {
260
    void enterEveryRule(ParserRuleContext ctx);
261
    void exitEveryRule(ParserRuleContext ctx);
262
    void visitTerminal(TerminalNode node);
263
    void visitErrorNode(ErrorNode node);
264
}
265

266
public class ParseTreeWalker {
267
    public static void walk(ParseTreeListener listener, ParseTree t);
268
}
269
```
270

271
[Parse Tree API](./parse-tree-api.md)
272

273
### Error Handling and Recovery
274

275
Comprehensive error handling system with customizable recovery strategies and detailed error reporting.
276

277
```java { .api }
278
public interface ANTLRErrorListener {
279
    void syntaxError(Recognizer<?,?> recognizer, Object offendingSymbol, 
280
                    int line, int charPositionInLine, String msg, 
281
                    RecognitionException e);
282
}
283

284
public interface ANTLRErrorStrategy {
285
    void reset(Parser recognizer);
286
    Token recoverInline(Parser recognizer) throws RecognitionException;
287
    void recover(Parser recognizer, RecognitionException e) throws RecognitionException;
288
}
289

290
public class DefaultErrorStrategy implements ANTLRErrorStrategy;
291
public class BailErrorStrategy extends DefaultErrorStrategy;
292
```
293

294
[Error Handling](./error-handling.md)
295

296
### Stream and Token Management
297

298
Input stream management and token processing infrastructure for feeding data to parsers and lexers.
299

300
```java { .api }
301
public class CharStreams {
302
    public static CharStream fromFileName(String fileName) throws IOException;
303
    public static CharStream fromString(String s);
304
    public static CharStream fromReader(Reader r) throws IOException;
305
}
306

307
public class CommonTokenStream extends BufferedTokenStream {
308
    public CommonTokenStream(TokenSource tokenSource);
309
    public List<Token> getTokens();
310
    public List<Token> getTokens(int start, int stop);
311
}
312

313
public class CommonToken implements WritableToken {
314
    public CommonToken(int type, String text);
315
    public String getText();
316
    public int getType();
317
    public int getLine();
318
    public int getCharPositionInLine();
319
}
320
```
321

322
[Runtime API](./runtime-api.md)
323

324
### Utility Classes and Helpers
325

326
Supporting utilities for tree manipulation, data structures, and testing infrastructure.
327

328
```java { .api }
329
public class Trees {
330
    public static String toStringTree(ParseTree t);
331
    public static List<ParseTree> getChildren(ParseTree t);
332
    public static List<ParseTree> getAncestors(ParseTree t);
333
    public static Collection<ParseTree> findAllTokenNodes(ParseTree t, int ttype);
334
    public static Collection<ParseTree> findAllRuleNodes(ParseTree t, int ruleIndex);
335
}
336

337
public class ParseTreeProperty<V> {
338
    public V get(ParseTree node);
339
    public void put(ParseTree node, V value);
340
    public V removeFrom(ParseTree node);
341
}
342

343
public class TestRig {
344
    public static void main(String[] args) throws Exception;
345
}
346
```
347

348
[Utilities](./utilities.md)
349

350
## Types
351

352
### Core Interfaces
353

354
```java { .api }
355
public interface CharStream extends IntStream {
356
    String getText(Interval interval);
357
    String toString();
358
}
359

360
public interface TokenStream extends IntStream {
361
    Token get(int index);
362
    TokenSource getTokenSource();
363
    String getText();
364
    String getText(Interval interval);
365
}
366

367
public interface IntStream {
368
    void consume();
369
    int LA(int i);
370
    int mark();
371
    void release(int marker);
372
    int index();
373
    void seek(int index);
374
    int size();
375
    String getSourceName();
376
}
377
```
378

379
### Context Classes
380

381
```java { .api }
382
public class ParserRuleContext extends RuleContext implements RuleNode {
383
    public ParserRuleContext parent;
384
    public int invokingState;
385
    public List<ParseTree> children;
386
    
387
    public void addChild(TerminalNode t);
388
    public void addChild(RuleContext ruleInvocation);
389
    public <T> T accept(ParseTreeVisitor<? extends T> visitor);
390
}
391

392
public class RuleContext {
393
    public RuleContext parent;
394
    public int invokingState;
395
    
396
    public int depth();
397
    public boolean isEmpty();
398
    public Interval getSourceInterval();
399
    public RuleContext getPayload();
400
}
401
```