tessl/pypi-wurlitzer

Capture C-level stdout/stderr pipes in Python via os.dup2

Overview

Eval results

Files

IPython Integration

Name: tessl/pypi-wurlitzer
Author: tessl

Seamless integration with IPython and Jupyter notebooks through magic commands and extension loading, enabling automatic C-level output capture in interactive environments.

Capabilities

Extension Loading

def load_ipython_extension(ip):
    """Register wurlitzer as an IPython extension
    
    Args:
        ip: IPython instance
        
    Note:
        Captures all C output during execution and forwards to sys.
        Does nothing on terminal IPython.
    """

Usage with magic command:

# In IPython or Jupyter notebook
%load_ext wurlitzer

# All subsequent cell executions will capture C-level output

Extension Unloading

Disable wurlitzer extension and restore original output behavior.

def unload_ipython_extension(ip):
    """Unload wurlitzer IPython extension
    
    Args:
        ip: IPython instance
    """

Usage:

# Disable the extension
%unload_ext wurlitzer

# C-level output is no longer captured

Automatic Execution Hooks

The extension automatically hooks into IPython's execution cycle:

pre_execute: Enables C output forwarding before each cell/command
post_execute: Disables C output forwarding after each cell/command

This ensures C-level output appears in notebook cells without manual context managers.

Usage Patterns

Jupyter Notebook Setup

Enable wurlitzer in a notebook for the entire session:

# First cell - setup
%load_ext wurlitzer

# Verify it's working
import ctypes
libc = ctypes.CDLL(None)
libc.printf(b"Hello from C!\n")  # This will appear in cell output

Scientific Computing Workflows

Capture output from scientific libraries with C extensions:

# With wurlitzer extension loaded
import numpy as np
import scipy

# NumPy/SciPy C-level warnings and output appear in cells
result = np.linalg.solve(large_matrix, vector)
scipy_result = scipy.optimize.minimize(objective_function, initial_guess)

Data Analysis Notebooks

Clean output capture for data processing pipelines:

# Extension enabled at notebook start
%load_ext wurlitzer

import pandas as pd
import matplotlib.pyplot as plt
from some_c_library import process_data

# All C-level output from data processing appears inline
processed_data = process_data(raw_dataframe)
plt.plot(processed_data)  # Any C output from matplotlib backends captured

Development and Debugging

Interactive development with C extensions:

# Debug session in Jupyter
%load_ext wurlitzer

# Test C extension interactively
import my_c_extension

# C debug output, warnings, and errors appear in notebook
my_c_extension.debug_function()
my_c_extension.test_algorithm(test_data)

Environment Detection

The extension includes smart environment detection:

Jupyter Kernel Detection

Only activates in Jupyter environments, not terminal IPython:

# This code runs automatically when extension loads
if not getattr(ip, 'kernel', None):
    warnings.warn("wurlitzer extension doesn't do anything in terminal IPython")
    return

# Extension only works in Jupyter kernels

Stream Validation

Validates that sys streams are available for forwarding:

# Automatic validation during extension loading
for name in ("__stdout__", "__stderr__"):
    if getattr(sys, name) is None:
        warnings.warn(f"sys.{name} is None. Wurlitzer can't capture output without it.")
        return

Advanced Integration Patterns

Conditional Loading

Load the extension based on environment or configuration:

import os

# Load only in specific environments
if os.getenv('JUPYTER_ENABLE_WURLITZER', '').lower() == 'true':
    %load_ext wurlitzer
    print("Wurlitzer enabled for C output capture")

Notebook Configuration

Add to IPython profile for automatic loading:

# In ~/.ipython/profile_default/ipython_config.py
c.InteractiveShellApp.extensions = ['wurlitzer']

# Or in startup files
# ~/.ipython/profile_default/startup/00-wurlitzer.py
get_ipython().magic('load_ext wurlitzer')

Custom Extension Integration

Combine with other IPython extensions:

# Custom extension that includes wurlitzer
from IPython.core.magic import Magics, line_magic, magics_class
from wurlitzer import load_ipython_extension, unload_ipython_extension

@magics_class
class CustomMagics(Magics):
    
    @line_magic
    def enable_c_capture(self, line):
        """Enable C output capture"""
        load_ipython_extension(self.shell)
        print("C output capture enabled")
    
    @line_magic  
    def disable_c_capture(self, line):
        """Disable C output capture"""
        unload_ipython_extension(self.shell)
        print("C output capture disabled")

# Register the custom extension
get_ipython().register_magic_function(CustomMagics)

Widget Integration

Use with Jupyter widgets for interactive applications:

import ipywidgets as widgets
from IPython.display import display

# Extension loaded
%load_ext wurlitzer

def on_button_click(b):
    with widgets.Output():
        # C output appears in widget output area
        c_processing_function()
        display("Processing complete")

button = widgets.Button(description="Run C Function")
button.on_click(on_button_click)
output = widgets.Output()

display(button, output)

Error Handling

Extension Loading Failures

Handle cases where extension cannot be loaded:

try:
    %load_ext wurlitzer
    print("Wurlitzer extension loaded successfully")
except Exception as e:
    print(f"Failed to load wurlitzer extension: {e}")
    print("C output may not be captured in this environment")

Fallback Strategies

Provide fallbacks when extension is not available:

def setup_c_output_capture():
    try:
        get_ipython().magic('load_ext wurlitzer')
        return True
    except:
        # Fallback to manual context managers
        print("Using manual wurlitzer context managers")
        return False

# Usage
auto_capture = setup_c_output_capture()

if not auto_capture:
    # Manual capture when extension not available
    from wurlitzer import sys_pipes
    with sys_pipes():
        c_function_call()
else:
    # Extension handles capture automatically
    c_function_call()

State Management

Track extension state for complex notebooks:

import wurlitzer

def check_extension_state():
    """Check if wurlitzer extension is currently active"""
    return wurlitzer._extension_enabled

# Usage
if check_extension_state():
    print("Wurlitzer extension is active")
    # C output will be automatically captured
else:
    print("Wurlitzer extension is not active")
    # Need manual capture or load extension

Install with Tessl CLI