or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

Files

docs

command-line.mdcustom-components.mdfilter-system.mdformatter-management.mdhigh-level-api.mdindex.mdlexer-management.mdstyle-management.md

high-level-api.mddocs/

0

# High-Level API

1

2

The high-level API provides the most convenient interface for syntax highlighting tasks. These three functions form the core of Pygments and can handle most common highlighting needs.

3

4

## Capabilities

5

6

### Lexical Analysis

7

8

Tokenizes source code using a lexer and returns an iterable of tokens.

9

10

```python { .api }

11

def lex(code: str, lexer) -> Iterator[tuple[TokenType, str]]:

12

"""

13

Lex code with the lexer and return an iterable of tokens.

14

15

Parameters:

16

- code: Source code string to tokenize

17

- lexer: Lexer instance (must be instantiated, not a class)

18

19

Returns:

20

Iterator of (token_type, value) tuples

21

22

Raises:

23

TypeError: If lexer is a class instead of an instance

24

"""

25

```

26

27

Usage example:

28

29

```python

30

from pygments import lex

31

from pygments.lexers import PythonLexer

32

33

code = "print('Hello, World!')"

34

lexer = PythonLexer()

35

tokens = list(lex(code, lexer))

36

# tokens: [(Token.Name.Builtin, 'print'), (Token.Punctuation, '('), ...]

37

```

38

39

### Token Formatting

40

41

Formats a token stream using a formatter and optionally writes to a file.

42

43

```python { .api }

44

def format(tokens: Iterator[tuple[TokenType, str]], formatter, outfile=None) -> str:

45

"""

46

Format tokens with the formatter.

47

48

Parameters:

49

- tokens: Iterable of (token_type, value) tuples

50

- formatter: Formatter instance (must be instantiated, not a class)

51

- outfile: Optional file object with write() method

52

53

Returns:

54

Formatted string if outfile is None, otherwise None (writes to outfile)

55

56

Raises:

57

TypeError: If formatter is a class instead of an instance

58

"""

59

```

60

61

Usage example:

62

63

```python

64

from pygments import lex, format

65

from pygments.lexers import PythonLexer

66

from pygments.formatters import HtmlFormatter

67

68

code = "print('Hello, World!')"

69

tokens = lex(code, PythonLexer())

70

result = format(tokens, HtmlFormatter())

71

# result: '<div class="highlight"><pre><span></span><span class="nb">print</span>...'

72

```

73

74

### Complete Highlighting

75

76

Combines lexing and formatting in a single function call.

77

78

```python { .api }

79

def highlight(code: str, lexer, formatter, outfile=None) -> str:

80

"""

81

High-level highlighting function combining lex and format.

82

83

Parameters:

84

- code: Source code string to highlight

85

- lexer: Lexer instance for tokenization

86

- formatter: Formatter instance for output generation

87

- outfile: Optional file object with write() method

88

89

Returns:

90

Formatted string if outfile is None, otherwise None (writes to outfile)

91

"""

92

```

93

94

Usage example:

95

96

```python

97

from pygments import highlight

98

from pygments.lexers import PythonLexer

99

from pygments.formatters import HtmlFormatter

100

101

code = '''

102

def fibonacci(n):

103

if n <= 1:

104

return n

105

return fibonacci(n-1) + fibonacci(n-2)

106

'''

107

108

result = highlight(code, PythonLexer(), HtmlFormatter())

109

print(result) # HTML with syntax highlighting

110

```

111

112

### File Output

113

114

All three functions support writing directly to files:

115

116

```python

117

# Write to file

118

with open('output.html', 'w') as f:

119

highlight(code, PythonLexer(), HtmlFormatter(), outfile=f)

120

121

# Or collect as string

122

html_output = highlight(code, PythonLexer(), HtmlFormatter())

123

```

124

125

## Error Handling

126

127

Common errors when using the high-level API:

128

129

- **TypeError**: Passing lexer or formatter classes instead of instances

130

- **Encoding errors**: When working with binary data or mixed encodings

131

132

```python

133

# Incorrect - will raise TypeError

134

from pygments.lexers import PythonLexer

135

from pygments.formatters import HtmlFormatter

136

result = highlight(code, PythonLexer, HtmlFormatter) # Classes, not instances

137

138

# Correct - instantiate the classes

139

result = highlight(code, PythonLexer(), HtmlFormatter()) # Instances

140

```