0
# MyST-Parser
1

2
An extended CommonMark compliant parser with bridges to docutils and Sphinx. MyST-Parser provides a rich and extensible flavor of Markdown designed for technical documentation and publishing, offering comprehensive Markdown parsing that extends CommonMark with additional syntax for technical writing, including support for Sphinx directives, roles, and cross-references.
3

4
## Package Information
5

6
- **Package Name**: myst-parser
7
- **Language**: Python
8
- **Installation**: `pip install myst-parser`
9

10
## Core Imports
11

12
```python
13
import myst_parser
14
```
15

16
For Sphinx extension usage:
17

18
```python
19
# Add to Sphinx conf.py extensions list
20
extensions = ['myst_parser']
21
```
22

23
For programmatic parsing:
24

25
```python
26
from myst_parser.config.main import MdParserConfig
27
from myst_parser.parsers.docutils_ import Parser
28
from myst_parser.parsers.sphinx_ import MystParser
29
```
30

31
## Basic Usage
32

33
### Sphinx Extension Usage
34

35
```python
36
# In Sphinx conf.py
37
extensions = ['myst_parser']
38

39
# Configure MyST parser options
40
myst_enable_extensions = [
41
    "deflist",
42
    "tasklist",
43
    "substitution",
44
    "colon_fence",
45
]
46

47
myst_heading_anchors = 3
48
myst_footnote_transition = True
49
```
50

51
### Programmatic Parsing with Docutils
52

53
```python
54
from myst_parser.parsers.docutils_ import Parser
55
from docutils.core import publish_doctree
56

57
# Create parser instance
58
parser = Parser()
59

60
# Parse MyST markdown to docutils document tree
61
source = """
62
# Example MyST Document
63

64
This is a MyST markdown document with {emphasis}`substitution syntax`
65
and support for Sphinx directives.
66

67
:::{note}
68
This is a note directive.
69
:::
70
"""
71

72
document = publish_doctree(source, parser=parser)
73
print(document.pformat())
74
```
75

76
### Configuration-Driven Parsing
77

78
```python
79
from myst_parser.config.main import MdParserConfig
80
from myst_parser.parsers.mdit import create_md_parser
81
from myst_parser.mdit_to_docutils.base import DocutilsRenderer, make_document
82

83
# Create custom configuration
84
config = MdParserConfig(
85
    enable_extensions={"tasklist", "deflist", "substitution"},
86
    heading_anchors=2,
87
    footnote_sort=True,
88
    commonmark_only=False
89
)
90

91
# Create document and renderer
92
document = make_document("example.md", parser_cls=None)
93
renderer = DocutilsRenderer(document)
94

95
# Create configured markdown parser
96
md_parser = create_md_parser(config, renderer)
97

98
# Parse markdown content
99
result = md_parser.render("""
100
# Document Title
101

102
- [x] Completed task
103
- [ ] Pending task
104

105
Term
106
: Definition of the term
107
""")
108
```
109

110
## Architecture
111

112
MyST-Parser operates through several interconnected components:
113

114
- **Configuration System**: `MdParserConfig` provides comprehensive control over parsing behavior, syntax extensions, and output formatting
115
- **Parser Layer**: Separate implementations for Sphinx (`MystParser`) and docutils (`Parser`) environments
116
- **Markdown Processing**: Built on markdown-it-py with MyST-specific extensions and token processing
117
- **Rendering Engine**: Converts markdown-it tokens to docutils AST nodes via `DocutilsRenderer` and `SphinxRenderer`
118
- **Sphinx Integration**: Full extension with directives, roles, reference resolution, and configuration management
119
- **CLI Tools**: Command-line utilities for various output formats and analysis
120

121
This architecture enables MyST-Parser to serve as both a standalone Markdown processor and a comprehensive Sphinx extension for technical documentation.
122

123
## Capabilities
124

125
### Configuration Management
126

127
Comprehensive configuration system with `MdParserConfig` dataclass providing control over parsing behavior, syntax extensions, cross-references, and output formatting. Includes validation, file-level overrides, and YAML frontmatter support.
128

129
```python { .api }
130
class MdParserConfig:
131
    commonmark_only: bool = False
132
    enable_extensions: set[str] = field(default_factory=set)
133
    heading_anchors: int = 0
134
    footnote_sort: bool = False
135
    ref_domains: Iterable[str] | None = None
136
    # ... 20+ additional configuration options
137

138
def merge_file_level(config: MdParserConfig, topmatter: dict, warning: Callable) -> MdParserConfig: ...
139
def read_topmatter(text: str) -> dict: ...
140
```
141

142
[Configuration](./configuration.md)
143

144
### Parsing System
145

146
Dual parser implementations for Sphinx and docutils environments, with factory functions for creating configured markdown-it-py parsers and comprehensive directive/option parsing support.
147

148
```python { .api }
149
class MystParser:
150
    supported: tuple[str, ...] = ("md", "markdown", "myst")
151
    def parse(self, inputstring: str, document) -> None: ...
152

153
class Parser:
154
    def parse(self, inputstring: str, document) -> None: ...
155

156
def create_md_parser(config: MdParserConfig, renderer) -> MarkdownIt: ...
157
```
158

159
[Parsing](./parsing.md)
160

161
### Sphinx Extension
162

163
Complete Sphinx extension with setup functions, custom directives, roles, and reference resolution. Provides seamless integration with Sphinx's documentation system.
164

165
```python { .api }
166
def setup(app) -> dict: ...
167
def setup_sphinx(app, load_parser: bool = False) -> None: ...
168
def create_myst_config(app) -> MdParserConfig: ...
169

170
class FigureMarkdown(SphinxDirective): ...
171
class SubstitutionReferenceRole(SphinxRole): ...
172
class MystReferenceResolver(ReferencesResolver): ...
173
```
174

175
[Sphinx Extension](./sphinx-extension.md)
176

177
### Document Rendering
178

179
Advanced rendering system for converting markdown-it tokens to docutils AST nodes, with specialized renderers for docutils and Sphinx environments, HTML processing, and document transformations.
180

181
```python { .api }
182
class DocutilsRenderer:
183
    def __init__(self, document): ...
184
    def render(self, tokens, options, md_env, renderer=None): ...
185

186
class SphinxRenderer(DocutilsRenderer): ...
187

188
def make_document(source_path: str, parser_cls=None): ...
189
def token_line(token, default: int | None = None) -> int | None: ...
190
```
191

192
[Document Rendering](./document-rendering.md)
193

194
### CLI Tools
195

196
Command-line utilities for MyST document processing, including HTML, LaTeX, XML output generation, heading anchor extraction, and inventory management.
197

198
```python { .api }
199
def print_anchors(args=None) -> None: ...
200
def inventory_cli() -> None: ...
201

202
# CLI commands available:
203
# myst-anchors, myst-inv, myst-docutils-html, myst-docutils-html5,
204
# myst-docutils-demo, myst-docutils-latex, myst-docutils-xml,
205
# myst-docutils-pseudoxml
206
```
207

208
[CLI Tools](./cli-tools.md)
209

210
### Inventory and Warning Systems
211

212
Sphinx inventory integration for cross-references and comprehensive warning system with categorized warning types for parsing, rendering, and reference resolution issues.
213

214
```python { .api }
215
class MystWarnings(Enum):
216
    DEPRECATED = "deprecated"
217
    NOT_SUPPORTED = "not_supported"
218
    DIRECTIVE_PARSING = "directive_parsing"
219
    XREF_MISSING = "xref_missing"
220
    # ... 15+ additional warning types
221

222
def create_warning(document, message: str, subtype: MystWarnings, **kwargs) -> None: ...
223

224
def from_sphinx(inv) -> InventoryType: ...
225
def to_sphinx(inv: InventoryType): ...
226
def load(stream, base_url: str) -> InventoryType: ...
227
```
228

229
[Inventory and Warnings](./inventory-warnings.md)
230

231
## Types
232

233
Core type definitions used across the MyST-Parser API:
234

235
```python { .api }
236
class MdParserConfig:
237
    """Main configuration dataclass for MyST parser settings."""
238

239
class TopmatterReadError(Exception):
240
    """Exception raised when reading YAML frontmatter fails."""
241

242
class UrlSchemeType(TypedDict):
243
    """Configuration for external URL schemes."""
244
    allowed_schemes: tuple[str, ...]
245
    classes: list[str]
246

247
class InventoryItemType(TypedDict):
248
    """Single item in a Sphinx inventory."""
249
    name: str
250
    domain: str
251
    role: str
252
    uri: str
253
    dispname: str
254

255
class InventoryType(TypedDict):
256
    """Complete Sphinx inventory data structure."""
257
    project: str
258
    version: str
259
    items: dict[str, InventoryItemType]
260

261
ValidatorType = Callable[[object, object, object], None]
262
```