or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

command-line.mdcustom-components.mdfilter-system.mdformatter-management.mdhigh-level-api.mdindex.mdlexer-management.mdstyle-management.md
tile.json

tessl/pypi-pygments

A syntax highlighting package that supports over 500 programming languages and text formats with extensive output format options

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/pygments@2.19.x

To install, run

npx @tessl/cli install tessl/pypi-pygments@2.19.0

index.mddocs/

Pygments

A comprehensive syntax highlighting package that supports over 500 programming languages and text formats. Pygments is a generic syntax highlighter designed for use in code hosting platforms, forums, wikis, and other applications requiring source code prettification.

Package Information

  • Package Name: Pygments
  • Language: Python
  • Installation: pip install Pygments
  • Python Version: >=3.8

Core Imports

import pygments

Common for highlighting code:

from pygments import highlight
from pygments.lexers import get_lexer_by_name
from pygments.formatters import HtmlFormatter

Or using the high-level API:

from pygments import lex, format, highlight

Basic Usage

from pygments import highlight
from pygments.lexers import PythonLexer
from pygments.formatters import HtmlFormatter

# Highlight Python code to HTML
code = '''
def hello_world():
    print("Hello, World!")
    return True
'''

# Basic highlighting
result = highlight(code, PythonLexer(), HtmlFormatter())
print(result)

# Using lexer lookup by name
from pygments.lexers import get_lexer_by_name
from pygments.formatters import get_formatter_by_name

lexer = get_lexer_by_name('python')
formatter = get_formatter_by_name('html')
result = highlight(code, lexer, formatter)

Architecture

Pygments follows a three-stage pipeline architecture:

  • Lexers: Tokenize source code into semantic tokens (keywords, strings, comments, etc.)
  • Formatters: Convert token streams into various output formats (HTML, LaTeX, RTF, SVG, terminal, etc.)
  • Styles: Define color schemes and formatting for different token types
  • Filters: Post-process token streams for special effects (highlighting names, visible whitespace, etc.)

The modular design allows mixing any lexer with any formatter and style, providing extensive customization while maintaining clean separation of concerns.

Capabilities

High-Level API

Core highlighting functions that provide the most convenient interface for syntax highlighting tasks.

def lex(code: str, lexer) -> Iterator[tuple[TokenType, str]]: ...
def format(tokens: Iterator[tuple[TokenType, str]], formatter, outfile=None) -> str: ...
def highlight(code: str, lexer, formatter, outfile=None) -> str: ...

High-Level API

Lexer Management

Functions for discovering, loading, and working with syntax lexers for different programming languages and text formats.

def get_lexer_by_name(_alias: str, **options): ...
def get_lexer_for_filename(_fn: str, code=None, **options): ...
def guess_lexer(_text: str, **options): ...
def get_all_lexers(plugins: bool = True) -> Iterator[tuple[str, list[str], list[str], list[str]]]: ...

Lexer Management

Formatter Management

Functions for working with output formatters that convert highlighted tokens into various formats.

def get_formatter_by_name(_alias: str, **options): ...
def get_formatter_for_filename(fn: str, **options): ...
def get_all_formatters() -> Iterator[type]: ...

Formatter Management

Style Management

Functions for working with color schemes and visual styles for highlighted code.

def get_style_by_name(name: str): ...
def get_all_styles() -> Iterator[str]: ...
def find_plugin_styles() -> Iterator[type]: ...

Style Management

Filter System

Token stream filters for post-processing highlighted code with special effects and transformations.

def get_filter_by_name(filtername: str, **options): ...
def get_all_filters() -> Iterator[str]: ...

Filter System

Custom Lexers and Formatters

Base classes and utilities for creating custom lexers and formatters.

class Lexer: ...
class RegexLexer(Lexer): ...
class Formatter: ...
class Style: ...

Custom Components

Command Line Interface

The pygmentize command-line tool for syntax highlighting from the terminal.

pygmentize [options] [file]
pygmentize -l <lexer> -f <formatter> [options] [file]
pygmentize -g [options] [file]  # guess lexer

Command Line Interface

Plugin System

Plugin loading and discovery system for external lexers, formatters, and styles.

def find_plugin_lexers() -> Iterator[type]: ...
def find_plugin_formatters() -> Iterator[type]: ...
def find_plugin_styles() -> Iterator[type]: ...  
def find_plugin_filters() -> Iterator[type]: ...

Modeline Parsing

Editor modeline parsing for automatic lexer detection.

def get_filetype_from_buffer(buf: bytes, max_chars: int = 65536) -> str: ...

Token Types

Core token type system used throughout Pygments for semantic categorization of code elements.

class _TokenType(tuple):
    def split(self) -> list[_TokenType]: ...
    def __contains__(self, val) -> bool: ...

# Root token type
Token: _TokenType

# Common token types
Text: _TokenType
Whitespace: _TokenType  
Error: _TokenType
Other: _TokenType
Keyword: _TokenType
Name: _TokenType
Literal: _TokenType
String: _TokenType
Number: _TokenType
Punctuation: _TokenType
Operator: _TokenType
Comment: _TokenType
Generic: _TokenType

Utility Functions

Core utility functions for text processing, option handling, and encoding detection.

def string_to_tokentype(s: str) -> _TokenType: ...
def is_token_subtype(ttype: _TokenType, other: _TokenType) -> bool: ...
def get_bool_opt(options: dict, optname: str, default=None) -> bool: ...
def get_int_opt(options: dict, optname: str, default=None) -> int: ...
def get_list_opt(options: dict, optname: str, default=None) -> list: ...
def docstring_headline(obj) -> str: ...
def make_analysator(f): ...
def shebang_matches(text: str, regex) -> bool: ...
def doctype_matches(text: str, regex) -> bool: ...
def html_doctype_matches(text: str) -> bool: ...
def looks_like_xml(text: str) -> bool: ...
def surrogatepair(c: int) -> tuple[int, int]: ...
def format_lines(var_name: str, seq, raw: bool = False, indent_level: int = 0) -> str: ...
def duplicates_removed(it, already_seen=()) -> Iterator: ...

Exception Classes

class ClassNotFound(ValueError):
    """Raised when lookup functions can't find a matching class."""

class OptionError(Exception):
    """Raised by option processing functions for invalid options."""