CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-myst-nb

A Jupyter Notebook Sphinx reader built on top of the MyST markdown parser.

Pending
Overview
Eval results
Files

execution.mddocs/

Execution

Multiple execution strategies for notebook cells including direct execution, cached execution via jupyter-cache, and inline variable evaluation. The execution system provides flexible notebook processing with various performance and caching options.

Capabilities

Execution Client Factory

Factory function to create appropriate execution clients based on configuration and requirements.

def create_client(
    notebook: NotebookNode,
    source: str,
    nb_config: NbParserConfig,
    logger: LoggerType,
    read_fmt: None | dict = None
) -> NotebookClientBase:
    """
    Create execution client based on configuration.
    
    Parameters:
    - notebook: NotebookNode - Notebook to execute
    - source: str - Path to or description of the input source being processed
    - nb_config: NbParserConfig - Execution configuration
    - logger: LoggerType - Logger instance for execution messages
    - read_fmt: None | dict - Format of the input source (optional)
    
    Returns:
    NotebookClientBase: Configured execution client
    """

Base Execution Client

Abstract base class defining the execution interface for all client implementations.

class NotebookClientBase:
    """
    Base class for notebook execution clients.
    
    Defines the common interface for executing notebooks
    with different strategies and backends.
    """
    
    def execute(self) -> ExecutionResult:
        """Execute the notebook and return results."""
        pass
    
    def setup_kernel(self) -> None:
        """Setup and start the kernel."""
        pass
        
    def cleanup(self) -> None:
        """Clean up resources after execution."""
        pass

Execution Result Container

TypedDict containing execution metadata and results from notebook processing.

class ExecutionResult(TypedDict):
    """
    Result of executing a notebook.
    
    Attributes:
    - mtime: float - POSIX timestamp of the execution time
    - runtime: float | None - Runtime in seconds (None if not executed)
    - method: str - Method used to execute the notebook
    - succeeded: bool - True if the notebook executed successfully
    - error: str | None - Error type if the notebook failed to execute
    - traceback: str | None - Traceback if the notebook failed
    """
    mtime: float
    runtime: float | None
    method: str
    succeeded: bool
    error: str | None
    traceback: str | None

Execution Error Handling

Exception class for execution-related errors with context and debugging information.

class ExecutionError(Exception):
    """
    Exception raised during notebook execution.
    
    Attributes:
    - cell_index: int - Index of the cell that caused the error
    - source: str - Source code of the failing cell
    - traceback: str - Full traceback information
    """
    cell_index: int
    source: str
    traceback: str

Execution Strategies

Direct Execution Client

Direct execution of notebook cells using nbclient for immediate processing.

class NotebookClientDirect(NotebookClientBase):
    """
    Direct notebook execution client.
    
    Executes notebook cells immediately using nbclient.
    Suitable for real-time execution with immediate results.
    """
    
    def __init__(self, nb: nbf.NotebookNode, config: NbParserConfig):
        """
        Initialize direct execution client.
        
        Parameters:
        - nb: NotebookNode to execute
        - config: Execution configuration
        """

Cached Execution Client

Execution using jupyter-cache for persistent caching of results across builds.

class NotebookClientCache(NotebookClientBase):
    """
    Cached notebook execution client using jupyter-cache.
    
    Provides persistent caching of execution results to avoid
    redundant computation across documentation builds.
    """
    
    def __init__(self, nb: nbf.NotebookNode, config: NbParserConfig, cache_path: str):
        """
        Initialize cached execution client.
        
        Parameters:
        - nb: NotebookNode to execute
        - config: Execution configuration  
        - cache_path: Path to jupyter-cache database
        """

Inline Execution Client

Specialized client for inline variable evaluation and expression processing.

class NotebookClientInline(NotebookClientBase):
    """
    Inline execution client for variable evaluation.
    
    Executes only inline expressions and eval blocks,
    suitable for lightweight variable substitution.
    """
    
    def evaluate_expression(self, expression: str, context: dict) -> Any:
        """
        Evaluate a single expression in the given context.
        
        Parameters:
        - expression: str - Python expression to evaluate
        - context: dict - Execution context/namespace
        
        Returns:
        Any: Evaluation result
        """

Configuration and Usage

Execution Modes

Control execution behavior through configuration:

  • "auto": Execute only if outputs are missing
  • "force": Always execute all cells
  • "cache": Use jupyter-cache for caching
  • "inline": Execute only inline evaluations
  • "off": Never execute cells

Timeout and Error Handling

from myst_nb.core.config import NbParserConfig

config = NbParserConfig(
    execution_mode="force",
    execution_timeout=120,          # 2 minute timeout per cell
    execution_allow_errors=True,    # Continue on errors
    execution_show_tb=False,        # Hide full tracebacks
    execution_in_temp=True          # Execute in temp directory
)

Usage Examples

Direct Execution

from myst_nb.core.execute import create_client, NotebookClientDirect
from myst_nb.core.config import NbParserConfig
import nbformat as nbf

# Load notebook
with open("notebook.ipynb") as f:
    nb = nbf.read(f, as_version=4)

# Configure execution
config = NbParserConfig(
    execution_mode="force",
    execution_timeout=60
)

# Create and run client
client = create_client(nb, config)
result = client.execute()

print(f"Executed in {result.execution_time:.2f}s")
print(f"Errors: {result.error_count}")

Cached Execution

from myst_nb.core.execute import NotebookClientCache
from myst_nb.core.config import NbParserConfig

config = NbParserConfig(
    execution_mode="cache",
    execution_cache_path=".jupyter_cache",
    execution_timeout=180
)

# Cached execution automatically reuses results
client = NotebookClientCache(nb, config, ".jupyter_cache")
result = client.execute()  # Uses cache if available

Inline Evaluation

from myst_nb.core.execute import NotebookClientInline

client = NotebookClientInline(nb, config)

# Evaluate inline expressions
context = {"x": 10, "y": 20}
result = client.evaluate_expression("x + y", context)
print(result)  # 30

Error Handling

from myst_nb.core.execute import ExecutionError, create_client

try:
    client = create_client(nb, config)
    result = client.execute()
except ExecutionError as e:
    print(f"Execution failed at cell {e.cell_index}")
    print(f"Source: {e.source}")
    print(f"Error: {e.traceback}")

Custom Execution Context

# Setup custom execution environment
import os
import tempfile

config = NbParserConfig(
    execution_mode="force",
    execution_in_temp=True,
    execution_timeout=300
)

# Execute in isolated environment
with tempfile.TemporaryDirectory() as temp_dir:
    os.chdir(temp_dir)
    client = create_client(nb, config)
    result = client.execute()

Batch Execution

import glob
from concurrent.futures import ThreadPoolExecutor

def execute_notebook(notebook_path):
    with open(notebook_path) as f:
        nb = nbf.read(f, as_version=4)
    
    client = create_client(nb, config)
    return client.execute()

# Execute multiple notebooks in parallel
notebook_paths = glob.glob("notebooks/*.ipynb")
with ThreadPoolExecutor(max_workers=4) as executor:
    results = list(executor.map(execute_notebook, notebook_paths))

The execution system integrates with MyST-NB's broader configuration and provides the foundation for all notebook processing, whether in Sphinx documentation builds or standalone processing pipelines.

Install with Tessl CLI

npx tessl i tessl/pypi-myst-nb

docs

cli.md

configuration.md

docutils.md

execution.md

glue.md

index.md

reading-processing.md

rendering.md

sphinx-extension.md

tile.json