0
# Reading and Processing
1

2
Reading, parsing, and processing of Jupyter notebooks in various formats including standard .ipynb and MyST markdown notebooks. The reading system provides flexible input handling with format detection and conversion capabilities.
3

4
## Capabilities
5

6
### Standard Notebook Reading
7

8
Function for reading standard Jupyter notebook files in .ipynb format.
9

10
```python { .api }
11
def standard_nb_read(text: str) -> nbf.NotebookNode:
12
    """
13
    Read standard .ipynb notebook format.
14
    
15
    Parameters:
16
    - text: str - Raw notebook JSON content as string
17
    
18
    Returns:
19
    nbf.NotebookNode: Parsed notebook object
20
    
21
    Parses JSON notebook content and returns a structured
22
    NotebookNode object for further processing.
23
    """
24
```
25

26
### Notebook Reader Factory
27

28
Factory function for creating appropriate reader instances based on file format and configuration.
29

30
```python { .api }
31
def create_nb_reader(
32
    path: str,
33
    md_config: MdParserConfig,
34
    nb_config: NbParserConfig,
35
    content: None | str | Iterator[str]
36
) -> NbReader | None:
37
    """
38
    Create appropriate notebook reader based on format and configuration.
39
    
40
    Parameters:
41
    - path: str - Path to the input source being processed
42
    - md_config: MdParserConfig - Configuration for parsing Markdown
43
    - nb_config: NbParserConfig - Configuration for parsing Notebooks
44
    - content: None | str | Iterator[str] - Input string (optionally used to check for text-based notebooks)
45
    
46
    Returns:
47
    NbReader | None: Configured reader instance or None if format unsupported
48
    
49
    Analyzes file format and creates the appropriate reader for processing.
50
    """
51
```
52

53
### MyST Markdown Notebook Reading
54

55
Specialized reader for MyST markdown notebooks that contain notebook content in markdown format.
56

57
```python { .api }
58
def read_myst_markdown_notebook(...):
59
    """
60
    Read MyST markdown notebooks.
61
    
62
    Parameters:
63
    - content: str - MyST markdown content
64
    - path: str - File path for context
65
    - config: NbParserConfig - Reading configuration
66
    
67
    Returns:
68
    nbf.NotebookNode: Converted notebook object
69
    
70
    Parses MyST markdown format and converts to notebook structure
71
    with proper cell separation and metadata handling.
72
    """
73
```
74

75
### Notebook Reader Class
76

77
Main reader class that encapsulates notebook reading functionality and configuration.
78

79
```python { .api }
80
class NbReader:
81
    """
82
    Notebook reader with format detection and processing capabilities.
83
    
84
    Handles various notebook formats and provides unified interface
85
    for reading and processing notebook content.
86
    """
87
    
88
    def __init__(self, path: str, config: NbParserConfig):
89
        """
90
        Initialize notebook reader.
91
        
92
        Parameters:
93
        - path: str - Path to notebook file
94
        - config: NbParserConfig - Reader configuration
95
        """
96
    
97
    def read(self) -> nbf.NotebookNode:
98
        """Read and parse the notebook."""
99
        pass
100
        
101
    def detect_format(self) -> str:
102
        """Detect notebook format from file content."""
103
        pass
104
```
105

106
### Jupyter-Cache Plugin
107

108
Plugin function for integration with jupyter-cache system for cached reading.
109

110
```python { .api }
111
def myst_nb_reader_plugin():
112
    """
113
    MyST-NB reader plugin for jupyter-cache integration.
114
    
115
    Returns:
116
    Callable: Reader function compatible with jupyter-cache
117
    
118
    Provides jupyter-cache integration for cached notebook reading
119
    and processing within the caching system.
120
    """
121
```
122

123
## Supported Formats
124

125
### Standard Jupyter Notebooks (.ipynb)
126

127
Standard JSON-based Jupyter notebook format with cells, metadata, and outputs.
128

129
```json
130
{
131
  "cells": [
132
    {
133
      "cell_type": "code",
134
      "source": ["print('Hello, World!')"],
135
      "metadata": {},
136
      "outputs": []
137
    }
138
  ],
139
  "metadata": {
140
    "kernelspec": {
141
      "name": "python3",
142
      "display_name": "Python 3"
143
    }
144
  },
145
  "nbformat": 4,
146
  "nbformat_minor": 4
147
}
148
```
149

150
### MyST Markdown Detection
151

152
Function to check if input text represents a MyST markdown notebook.
153

154
```python { .api }
155
def is_myst_markdown_notebook(text: str | Iterator[str]) -> bool:
156
    """
157
    Check if the input is a MyST Markdown notebook.
158
    
159
    Parameters:
160
    - text: str | Iterator[str] - Input text to check
161
    
162
    Returns:
163
    bool: True if the input is a markdown notebook
164
    
165
    Identifies MyST markdown notebooks by checking for top-matter
166
    section containing either 'file_format: mystnb' or Jupytext
167
    format configuration.
168
    """
169
```
170

171
### MyST Notebook Reader Plugin
172

173
Plugin function for reading MyST notebooks from file URIs.
174

175
```python { .api }
176
def myst_nb_reader_plugin(uri: str) -> nbf.NotebookNode:
177
    """
178
    Read a MyST notebook from a string.
179
    
180
    Parameters:
181
    - uri: str - URI/path to the notebook file
182
    
183
    Returns:
184
    nbf.NotebookNode: Parsed notebook object
185
    
186
    Used as plugin for jupyter-cache to read MyST notebooks
187
    from file URIs with source mapping enabled.
188
    """
189
```
190

191
### Metadata Parsing Error
192

193
Exception class for MyST metadata parsing errors.
194

195
```python { .api }
196
class MystMetadataParsingError(Exception):
197
    """
198
    Error when parsing metadata from MyST formatted text.
199
    
200
    Raised when MyST markdown notebook metadata cannot be
201
    properly parsed or contains invalid structure.
202
    """
203
```
204

205
### MyST Markdown Notebooks
206

207
Markdown-based notebook format with special cell delimiters and metadata.
208

209
```markdown
210
---
211
jupyter:
212
  kernelspec:
213
    name: python3
214
    display_name: Python 3
215
---
216

217
# My Notebook
218

219
This is a markdown cell.
220

221
```{code-cell} python
222
print("Hello, World!")
223
```
224

225
Another markdown cell.
226

227
```{code-cell} python
228
:tags: [hide-output]
229
import matplotlib.pyplot as plt
230
plt.plot([1, 2, 3], [1, 4, 2])
231
plt.show()
232
```
233
```
234
235
### Custom Formats
236

237
Support for custom notebook formats through configuration:
238

239
```python
240
# In configuration
241
custom_formats = {
242
    ".Rmd": "Rmd",      # R Markdown
243
    ".qmd": "qmd",      # Quarto
244
    ".py": "py:percent" # Python with percent comments
245
}
246
```
247

248
## Usage Examples
249

250
### Basic Notebook Reading
251

252
```python
253
from myst_nb.core.read import standard_nb_read
254
import nbformat as nbf
255

256
# Read standard notebook
257
with open("notebook.ipynb", "r") as f:
258
    content = f.read()
259

260
nb = standard_nb_read(content)
261
print(f"Notebook has {len(nb.cells)} cells")
262
```
263

264
### MyST Markdown Reading
265

266
```python
267
from myst_nb.core.read import read_myst_markdown_notebook
268
from myst_nb.core.config import NbParserConfig
269

270
# Read MyST markdown notebook
271
with open("notebook.md", "r") as f:
272
    content = f.read()
273

274
config = NbParserConfig()
275
nb = read_myst_markdown_notebook(content, "notebook.md", config)
276

277
# Process cells
278
for i, cell in enumerate(nb.cells):
279
    print(f"Cell {i}: {cell.cell_type}")
280
    if cell.cell_type == "code":
281
        print(f"  Source: {cell.source[:50]}...")
282
```
283

284
### Reader Factory Usage
285

286
```python
287
from myst_nb.core.read import create_nb_reader
288
from myst_nb.core.config import NbParserConfig
289

290
config = NbParserConfig(
291
    execution_mode="auto",
292
    custom_formats={".Rmd": "Rmd"}
293
)
294

295
# Create reader for different formats
296
reader = create_nb_reader("notebook.ipynb", None, config)
297
if reader:
298
    nb = reader.read()
299
    print(f"Read notebook with {len(nb.cells)} cells")
300

301
# Handle MyST markdown
302
reader = create_nb_reader("notebook.md", "myst", config)
303
if reader:
304
    nb = reader.read()
305
```
306

307
### Format Detection
308

309
```python
310
import os
311
from myst_nb.core.read import NbReader
312

313
def process_notebook_file(filepath):
314
    """Process notebook file with automatic format detection."""
315
    reader = NbReader(filepath, config)
316
    
317
    # Detect format
318
    fmt = reader.detect_format()
319
    print(f"Detected format: {fmt}")
320
    
321
    # Read notebook
322
    nb = reader.read()
323
    
324
    # Process based on format
325
    if fmt == "ipynb":
326
        print("Processing standard Jupyter notebook")
327
    elif fmt == "myst":
328
        print("Processing MyST markdown notebook")
329
    elif fmt in config.custom_formats:
330
        print(f"Processing custom format: {fmt}")
331
    
332
    return nb
333

334
# Process various formats
335
notebooks = [
336
    "standard.ipynb",
337
    "myst_notebook.md", 
338
    "r_notebook.Rmd"
339
]
340

341
for nb_path in notebooks:
342
    if os.path.exists(nb_path):
343
        nb = process_notebook_file(nb_path)
344
```
345

346
### Batch Processing
347

348
```python
349
import glob
350
from pathlib import Path
351
from myst_nb.core.read import create_nb_reader
352

353
def read_all_notebooks(directory, config):
354
    """Read all notebooks in a directory."""
355
    notebooks = []
356
    
357
    # Find all notebook files
358
    patterns = ["*.ipynb", "*.md", "*.Rmd"]
359
    for pattern in patterns:
360
        for filepath in Path(directory).glob(pattern):
361
            reader = create_nb_reader(str(filepath), None, config)
362
            if reader:
363
                try:
364
                    nb = reader.read()
365
                    notebooks.append((filepath, nb))
366
                    print(f"Read: {filepath}")
367
                except Exception as e:
368
                    print(f"Error reading {filepath}: {e}")
369
    
370
    return notebooks
371

372
# Usage
373
config = NbParserConfig()
374
notebooks = read_all_notebooks("notebooks/", config)
375
print(f"Read {len(notebooks)} notebooks")
376
```
377

378
### Integration with Jupyter-Cache
379

380
```python
381
from myst_nb.core.read import myst_nb_reader_plugin
382
from jupyter_cache import get_cache
383

384
# Setup cache with MyST-NB reader
385
cache = get_cache(".jupyter_cache")
386

387
# Register MyST-NB reader plugin
388
reader_plugin = myst_nb_reader_plugin()
389

390
# Use with cached execution
391
cached_nb = cache.cache_notebook_file("notebook.md", reader=reader_plugin)
392
```
393

394
### Error Handling and Validation
395

396
```python
397
from myst_nb.core.read import standard_nb_read, NbReader
398
import nbformat as nbf
399

400
def safe_notebook_read(filepath):
401
    """Safely read notebook with error handling."""
402
    try:
403
        reader = NbReader(filepath, config)
404
        nb = reader.read()
405
        
406
        # Validate notebook structure
407
        nbf.validate(nb)
408
        
409
        return nb
410
    except nbf.ValidationError as e:
411
        print(f"Invalid notebook format: {e}")
412
        return None
413
    except FileNotFoundError:
414
        print(f"File not found: {filepath}")
415
        return None
416
    except Exception as e:
417
        print(f"Error reading notebook: {e}")
418
        return None
419

420
# Usage with validation
421
nb = safe_notebook_read("notebook.ipynb")
422
if nb:
423
    print(f"Successfully read valid notebook with {len(nb.cells)} cells")
424
```
425

426
The reading and processing system provides the foundation for all MyST-NB functionality, handling format detection, parsing, and conversion to enable seamless notebook integration across different input formats.