CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-myst-parser

An extended CommonMark compliant parser, with bridges to docutils and Sphinx

Pending
Overview
Eval results
Files

inventory-warnings.mddocs/

Inventory and Warning Systems

Sphinx inventory integration for cross-references and comprehensive warning system with categorized warning types for parsing, rendering, and reference resolution issues.

Capabilities

Sphinx Inventory Integration

Complete integration with Sphinx's inventory system for cross-document and cross-project reference resolution, supporting loading, conversion, and manipulation of inventory data.

def from_sphinx(inv) -> InventoryType:
    """
    Convert from Sphinx inventory format to MyST format.
    
    Args:
        inv: Sphinx inventory object
        
    Returns:
        MyST-compatible inventory dictionary
    """

def to_sphinx(inv: InventoryType):
    """
    Convert MyST inventory to Sphinx inventory format.
    
    Args:
        inv: MyST inventory dictionary
        
    Returns:
        Sphinx-compatible inventory object
    """

def load(stream, base_url: str) -> InventoryType:
    """
    Load inventory from stream with base URL resolution.
    
    Args:
        stream: Input stream containing inventory data
        base_url: Base URL for resolving relative references
        
    Returns:
        Loaded inventory dictionary
        
    Raises:
        InventoryLoadError: When inventory loading fails
    """

Usage example:

from myst_parser.inventory import from_sphinx, to_sphinx, load
import urllib.request

# Load inventory from URL
with urllib.request.urlopen("https://docs.python.org/3/objects.inv") as stream:
    python_inv = load(stream, "https://docs.python.org/3/")

print(f"Loaded {len(python_inv['items'])} items from Python inventory")

# Access inventory items
for name, item in python_inv['items'].items():
    if 'dict' in name.lower():
        print(f"{item['name']}: {item['uri']}")

# Convert between formats
sphinx_inv = to_sphinx(python_inv)
myst_inv = from_sphinx(sphinx_inv)

Inventory Data Structures

Type-safe data structures for representing Sphinx inventory information with full type annotations and validation.

class InventoryItemType(TypedDict):
    """
    A single inventory item.
    
    Attributes:
        loc: The location of the item (relative if base_url not None)
        text: Implicit text to show for the item (None for no text)
    """
    loc: str
    text: str | None

class InventoryType(TypedDict):
    """
    Inventory data.
    
    Attributes:
        name: The name of the project
        version: The version of the project
        base_url: The base URL of the loc
        objects: Mapping of domain -> object type -> name -> item
    """
    name: str
    version: str
    base_url: str | None
    objects: dict[str, dict[str, dict[str, InventoryItemType]]]

Usage example:

from myst_parser.inventory import InventoryType, InventoryItemType

# Create inventory item
item: InventoryItemType = {
    "loc": "library/stdtypes.html#dict.get",
    "text": "dict.get"
}

# Create inventory
inventory: InventoryType = {
    "name": "Python",
    "version": "3.11",
    "base_url": "https://docs.python.org/3/",
    "objects": {
        "py": {
            "method": {
                "dict.get": item
            }
        }
    }
}

# Access typed data
for domain, obj_types in inventory["objects"].items():
    for obj_type, objects in obj_types.items():
        for name, item in objects.items():
            print(f"{domain}:{obj_type}:{name} -> {item['loc']}")

Warning System

Comprehensive warning system with categorized warning types for different aspects of MyST parsing, rendering, and cross-reference resolution.

class MystWarnings(Enum):
    """
    Enumeration of MyST warning types with descriptive values.
    
    Categories:
    - General: DEPRECATED, NOT_SUPPORTED, RENDER_METHOD
    - Markdown: MD_TOPMATTER, MD_DEF_DUPE, MD_HEADING_NON_CONSECUTIVE  
    - Directives: DIRECTIVE_PARSING, DIRECTIVE_OPTION, DIRECTIVE_BODY, UNKNOWN_DIRECTIVE, UNKNOWN_ROLE
    - Cross-references: XREF_AMBIGUOUS, XREF_MISSING, IREF_MISSING, IREF_AMBIGUOUS
    - Inventory: INV_LOAD, LEGACY_DOMAIN
    - Extensions: HEADING_SLUG, STRIKETHROUGH, HTML_PARSE, INVALID_ATTRIBUTE, SUBSTITUTION
    """
    
    # General warnings
    DEPRECATED = "deprecated"
    NOT_SUPPORTED = "not_supported" 
    RENDER_METHOD = "render"
    
    # Markdown parsing warnings
    MD_TOPMATTER = "topmatter"
    MD_DEF_DUPE = "duplicate_def"
    MD_HEADING_NON_CONSECUTIVE = "header"
    
    # Directive warnings
    DIRECTIVE_PARSING = "directive_parse"
    DIRECTIVE_OPTION = "directive_option"
    DIRECTIVE_BODY = "directive_body"
    UNKNOWN_DIRECTIVE = "directive_unknown"
    UNKNOWN_ROLE = "role_unknown"
    
    # Cross-reference warnings
    XREF_AMBIGUOUS = "xref_ambiguous"
    XREF_MISSING = "xref_missing"
    IREF_MISSING = "iref_missing"
    IREF_AMBIGUOUS = "iref_ambiguous"
    
    # Inventory warnings
    INV_LOAD = "inv_retrieval"
    LEGACY_DOMAIN = "domains"
    
    # Extension warnings
    HEADING_SLUG = "heading_slug"
    STRIKETHROUGH = "strikethrough"
    HTML_PARSE = "html"
    INVALID_ATTRIBUTE = "attribute"
    SUBSTITUTION = "substitution"

def create_warning(
    document,
    message: str,
    subtype: MystWarnings = MystWarnings.RENDER_METHOD,
    line: int | None = None,
    append_to: list | None = None,
    **kwargs
) -> None:
    """
    Generate warning with logging and optional collection.
    
    Args:
        document: Docutils document context
        message: Warning message text
        subtype: Warning category from MystWarnings enum
        line: Source line number (optional)
        append_to: List to append warning to (optional)
        **kwargs: Additional warning context
    """

def _is_suppressed_warning(
    type: str,
    subtype: MystWarnings,
    suppress_warnings: list[str]
) -> bool:
    """
    Check if warning should be suppressed based on configuration.
    
    Args:
        type: Warning type identifier
        subtype: Warning subtype from MystWarnings enum
        suppress_warnings: List of suppressed warning patterns
        
    Returns:
        True if warning should be suppressed
    """

Usage example:

from myst_parser.warnings_ import create_warning, MystWarnings, _is_suppressed_warning
from docutils.utils import new_document

# Create document for warning context
document = new_document('<string>')

# Generate different types of warnings
create_warning(
    document,
    "Unknown directive 'custom-block'",
    subtype=MystWarnings.UNKNOWN_DIRECTIVE,
    line=15
)

create_warning(
    document,
    "Cross-reference target 'missing-section' not found",
    subtype=MystWarnings.XREF_MISSING,
    line=23
)

create_warning(
    document,
    "Invalid YAML in frontmatter",
    subtype=MystWarnings.MD_TOPMATTER,
    line=3
)

# Check warning suppression
suppress_list = ["myst.xref_missing", "myst.unknown_directive"]

is_suppressed = _is_suppressed_warning(
    "myst",
    MystWarnings.XREF_MISSING,
    suppress_list
)
print(f"XREF_MISSING suppressed: {is_suppressed}")  # True

Warning Collection and Reporting

Advanced warning collection and reporting capabilities for batch processing and analysis of MyST documents.

class WarningCollector:
    """
    Collector for aggregating warnings during document processing.
    
    Provides functionality to collect, categorize, and report warnings
    from multiple documents or processing stages.
    """
    
    def __init__(self): ...
    
    def add_warning(
        self,
        message: str,
        subtype: MystWarnings,
        source: str | None = None,
        line: int | None = None
    ) -> None: ...
    
    def get_warnings_by_type(self, subtype: MystWarnings) -> list: ...
    
    def get_warnings_by_source(self, source: str) -> list: ...
    
    def generate_report(self, format: str = "text") -> str: ...

def collect_warnings_from_document(document) -> list[dict]:
    """
    Extract all warnings from a processed document.
    
    Args:
        document: Docutils document that may contain warnings
        
    Returns:
        List of warning dictionaries with metadata
    """

Usage example:

from myst_parser.warnings_ import WarningCollector, MystWarnings

# Create warning collector
collector = WarningCollector()

# Process multiple documents
for doc_path in ["doc1.md", "doc2.md", "doc3.md"]:
    try:
        # ... process document ...
        pass
    except Exception as e:
        collector.add_warning(
            f"Processing failed: {e}",
            MystWarnings.RENDER_METHOD,
            source=doc_path
        )

# Generate summary report
report = collector.generate_report("text")
print(report)

# Get specific warning types
xref_warnings = collector.get_warnings_by_type(MystWarnings.XREF_MISSING)
print(f"Found {len(xref_warnings)} missing cross-references")

Warning Configuration

Configuration options for controlling warning behavior in Sphinx and standalone environments.

# Sphinx configuration (conf.py)
suppress_warnings = [
    "myst.header",              # Suppress header warnings
    "myst.xref_missing",        # Suppress missing cross-reference warnings  
    "myst.unknown_directive",   # Suppress unknown directive warnings
]

# MyST-specific warning configuration
myst_suppress_warnings = [
    "md_topmatter",     # YAML frontmatter warnings
    "directive_parsing", # Directive parsing warnings
    "html_unknown_tag", # Unknown HTML tag warnings
]

# Warning level configuration
myst_warning_as_error = [
    "xref_missing",     # Treat missing refs as errors
    "unknown_directive" # Treat unknown directives as errors
]

Usage in Sphinx:

# In conf.py
extensions = ['myst_parser']

# Configure warning suppression
suppress_warnings = [
    'myst.xref_missing',
    'myst.unknown_directive'
]

# MyST-specific warning control
myst_suppress_warnings = [
    'md_topmatter',
    'directive_parsing'
]

Inventory File Formats

MyST-Parser supports multiple inventory file formats:

Standard Sphinx Format

# Binary Sphinx inventory format
# Loaded with inventory.load() from objects.inv files
inventory = load(stream, "https://docs.python.org/3/")

JSON Format

{
    "project": "Python",
    "version": "3.11",
    "items": {
        "py:class:dict": {
            "name": "dict",
            "domain": "py",
            "role": "class", 
            "uri": "library/stdtypes.html#dict",
            "dispname": "dict"
        }
    }
}

YAML Format

project: Python
version: "3.11"
items:
  py:class:dict:
    name: dict
    domain: py
    role: class
    uri: library/stdtypes.html#dict
    dispname: dict

Types

Inventory and warning system type definitions:

class InventoryItemType(TypedDict):
    """Single inventory item with required fields."""
    name: str
    domain: str
    role: str
    uri: str
    dispname: str

class InventoryType(TypedDict):
    """Complete inventory structure."""
    project: str
    version: str
    items: dict[str, InventoryItemType]

class MystWarnings(Enum):
    """Enumeration of all MyST warning categories."""

class WarningCollector:
    """Warning collection and reporting system."""

# Exception types
class InventoryLoadError(Exception):
    """Raised when inventory loading fails."""

class WarningError(Exception):
    """Raised when warnings are treated as errors."""

Install with Tessl CLI

npx tessl i tessl/pypi-myst-parser

docs

cli-tools.md

configuration.md

document-rendering.md

index.md

inventory-warnings.md

parsing.md

sphinx-extension.md

tile.json