Wurlitzer

A comprehensive solution for capturing C-level output streams (stdout/stderr) in Python applications through os.dup2 system calls. Wurlitzer provides flexible context managers for redirecting low-level output to Python streams, files, or logging objects, solving the common problem where standard Python stream redirection doesn't capture output from C extensions or system libraries.

Package Information

• Package Name: wurlitzer
• Version: 3.1.1
• Language: Python
• License: MIT
• Installation: pip install wurlitzer
• Repository: https://github.com/minrk/wurlitzer

Core Imports

import wurlitzer
from wurlitzer import pipes, sys_pipes, PIPE, STDOUT

Basic Usage

from wurlitzer import pipes

# Capture C-level output in context manager
with pipes() as (out, err):
    call_some_c_function()

stdout_content = out.read()
stderr_content = err.read()

Architecture

Wurlitzer operates at the file descriptor level using os.dup2 to redirect C-level stdout/stderr streams. It employs background threads with selectors for non-blocking pipe handling, ensuring thread safety and GIL-aware operation. The library provides multiple output destinations including StringIO objects, file handles, Python loggers, and direct system stream forwarding.

Key architectural components:

• File descriptor manipulation: Uses os.dup2 for low-level stream redirection
• Thread-based forwarding: Background threads handle pipe reading without blocking
• Cross-platform compatibility: Works on Linux, macOS with platform-specific optimizations
• Buffer management: Configurable pipe buffer sizes with automatic optimization

Capabilities

Context Manager Output Capture

Primary functionality for temporarily capturing C-level output in context managers.

def pipes(stdout=PIPE, stderr=PIPE, encoding='utf-8', bufsize=None):
    """Capture C-level stdout/stderr in a context manager.
    
    Returns:
        Tuple of (stdout, stderr) streams
    """

Output Capture

System Stream Forwarding

Forward C-level output to Python's sys.stdout/stderr, useful for Jupyter/IPython integration.

def sys_pipes(encoding='utf-8', bufsize=None):
    """Redirect C-level stdout/stderr to sys.stdout/stderr"""

System Integration

Permanent Output Redirection

Enable and disable permanent C-level output forwarding without context managers.

def sys_pipes_forever(encoding='utf-8', bufsize=None):
    """Redirect all C output to sys.stdout/err permanently"""

def stop_sys_pipes():
    """Stop permanent redirection started by sys_pipes_forever"""

Permanent Redirection

IPython/Jupyter Extension

Magic command support for seamless integration with Jupyter notebooks and IPython.

def load_ipython_extension(ip):
    """Register wurlitzer as an IPython extension"""

def unload_ipython_extension(ip):
    """Unload wurlitzer IPython extension"""

IPython Integration

Core Implementation

Low-level Wurlitzer class providing the underlying capture mechanism.

class Wurlitzer:
    """Class for Capturing Process-level FD output via dup2"""
    
    def __init__(self, stdout=None, stderr=None, encoding='utf-8', bufsize=None):
        """Initialize Wurlitzer with output destinations"""

Core Implementation

Constants and Configuration

PIPE = 3  # Indicates pipe capture should be used
STDOUT = 2  # Redirect stderr to stdout destination

These constants control output routing behavior:

• PIPE: Creates new StringIO/BytesIO objects for output capture
• STDOUT: Redirects stderr to the same destination as stdout