or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

benchmark-execution.mdcli.mddata-management.mdindex.mdutilities.md
tile.json

tessl/pypi-pyperf

Python module to run and analyze benchmarks with high precision and statistical rigor

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/pyperf@2.9.x

To install, run

npx @tessl/cli install tessl/pypi-pyperf@2.9.0

index.mddocs/

PyPerf

A comprehensive Python toolkit for writing, running, and analyzing benchmarks with high precision and statistical rigor. PyPerf provides automated benchmark execution with multiple worker processes, statistical analysis with outlier detection, and comprehensive system metadata collection for reproducible performance measurements.

Package Information

  • Package Name: pyperf
  • Language: Python
  • Installation: pip install pyperf
  • Dependencies: psutil>=5.9.0

Core Imports

import pyperf

Common usage patterns:

from pyperf import Runner, Benchmark, BenchmarkSuite

Basic Usage

import pyperf

# Create a benchmark runner
runner = pyperf.Runner()

# Simple function benchmarking
def fibonacci(n):
    if n < 2:
        return n
    return fibonacci(n-1) + fibonacci(n-2)

# Benchmark the function
benchmark = runner.bench_func('fibonacci', fibonacci, 20)
print(f"fibonacci(20): {benchmark}")

# Python code benchmarking (timeit-style)
benchmark = runner.timeit('list_creation', 
                         stmt='[i for i in range(100)]')
print(f"List creation: {benchmark}")

# Save results to file
benchmark.dump('fibonacci_results.json')

# Load and compare results later
loaded = pyperf.Benchmark.load('fibonacci_results.json')
print(f"Mean: {loaded.mean():.6f} seconds")
print(f"Standard deviation: {loaded.stdev():.6f} seconds")

Architecture

PyPerf follows a layered architecture designed for precision and statistical validity:

  • Runner: High-level interface managing worker processes and benchmark execution
  • Benchmark/BenchmarkSuite: Data containers with statistical analysis capabilities
  • Run: Individual measurement collections with metadata
  • Worker Processes: Isolated execution environments minimizing measurement noise
  • Metadata Collection: Comprehensive system state capture for reproducible results
  • Statistical Analysis: Built-in outlier detection, confidence intervals, and significance testing

This design enables PyPerf to achieve microsecond-level precision while maintaining statistical rigor through automated calibration, multi-process execution, and comprehensive metadata tracking.

Capabilities

Benchmark Execution

Core benchmarking functionality including function timing, Python code evaluation, async function benchmarking, and external command measurement. The Runner class provides the primary interface for executing benchmarks with automatic calibration and statistical validation.

class Runner:
    def __init__(self, values=None, processes=None, loops=0, min_time=0.1, 
                 metadata=None, show_name=True, program_args=None, 
                 add_cmdline_args=None, _argparser=None, warmups=1): ...
    def bench_func(self, name: str, func: callable, *args, **kwargs) -> Benchmark: ...
    def bench_time_func(self, name: str, time_func: callable, *args, **kwargs) -> Benchmark: ...
    def bench_async_func(self, name: str, func: callable, *args, **kwargs) -> Benchmark: ...
    def timeit(self, name: str, stmt=None, setup="pass", teardown="pass", 
               inner_loops=None, duplicate=None, metadata=None, globals=None) -> Benchmark: ...
    def bench_command(self, name: str, command: list) -> Benchmark: ...

Benchmark Execution

Data Management and Analysis

Comprehensive data structures for storing, analyzing, and managing benchmark results. Includes statistical analysis, metadata handling, and serialization capabilities for persistent storage and result sharing.

class Run:
    def __init__(self, values, warmups=None, metadata=None, collect_metadata=True): ...
    def get_metadata(self) -> dict: ...
    def get_loops(self) -> int: ...
    def get_total_loops(self) -> int: ...

class Benchmark:
    def __init__(self, runs): ...
    def get_name(self) -> str: ...
    def get_values(self) -> tuple: ...
    def get_nrun(self) -> int: ...
    def mean(self) -> float: ...
    def stdev(self) -> float: ...
    def median(self) -> float: ...
    def percentile(self, p: float) -> float: ...
    def add_run(self, run: Run): ...
    @staticmethod
    def load(file): ...
    def dump(self, file, compact=True, replace=False): ...

class BenchmarkSuite:
    def __init__(self, benchmarks, filename=None): ...
    def get_benchmarks(self) -> list: ...
    def get_benchmark(self, name: str) -> Benchmark: ...
    def add_benchmark(self, benchmark: Benchmark): ...

Data Management

Command-Line Interface

Comprehensive command-line tools for running benchmarks, analyzing results, and managing benchmark data. Includes specialized commands for Python code timing, external command benchmarking, result comparison, and system optimization recommendations.

# Available via: python -m pyperf <command>
# Primary commands:
# timeit      - Python code benchmarking
# command     - External command benchmarking  
# show        - Display benchmark results
# compare_to  - Compare benchmark files
# stats       - Detailed statistics
# hist        - Result histograms
# metadata    - System metadata display
# check       - Stability validation
# convert     - Data format conversion
# system      - System tuning recommendations

Command-Line Interface

Utilities and System Integration

System utilities for metadata collection, platform detection, statistical functions, and performance optimization. Includes CPU affinity management, memory tracking, and environment analysis capabilities.

def python_implementation() -> str: ...
def python_has_jit() -> bool: ...
def format_metadata(name: str, value) -> str: ...
def add_runs(filename: str, result): ...

# Version information
VERSION: tuple
__version__: str

Utilities