CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-pytest-rerunfailures

pytest plugin to re-run tests to eliminate flaky failures

Pending
Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Pending

The risk profile of this skill

Overview
Eval results
Files

pytest-rerunfailures

A pytest plugin that automatically re-runs failed tests to eliminate intermittent failures and flaky test behavior. It provides comprehensive command-line options for controlling test reruns including global rerun counts, selective rerun filtering based on exception patterns, configurable delays between reruns, and marker-based test decoration for individual test control.

Package Information

  • Package Name: pytest-rerunfailures
  • Package Type: pytest plugin
  • Language: Python
  • Installation: pip install pytest-rerunfailures
  • Requirements: Python >=3.9, pytest >=7.4 (!=8.2.2)
  • Optional: pytest-xdist >=2.3.0 for crash recovery

Core Imports

The plugin is automatically discovered by pytest when installed. No explicit imports are needed for basic usage.

For accessing plugin internals programmatically:

import pytest_rerunfailures

Basic Usage

Command Line Usage

Re-run all test failures up to 5 times:

pytest --reruns 5

Add 1-second delay between reruns:

pytest --reruns 5 --reruns-delay 1

Only rerun specific error types:

pytest --reruns 3 --only-rerun AssertionError --only-rerun ValueError

Skip reruns for specific error types:

pytest --reruns 3 --rerun-except ConnectionError

Marker Usage

Mark individual tests for reruns:

import pytest

@pytest.mark.flaky(reruns=5)
def test_sometimes_fails():
    import random
    assert random.choice([True, False])

@pytest.mark.flaky(reruns=3, reruns_delay=2)
def test_with_delay():
    # Will be retried up to 3 times with 2-second delays
    pass

@pytest.mark.flaky(reruns=2, condition="sys.platform.startswith('win32')")
def test_windows_only_reruns():
    # Only reruns on Windows
    pass

Capabilities

Command Line Options

Global configuration options for controlling test reruns across the entire test suite.

def pytest_addoption(parser):
    """
    Adds command-line options for test reruns.
    
    Options added:
    --reruns: Number of times to re-run failed tests (default: 0)
    --reruns-delay: Delay in seconds between reruns
    --only-rerun: Regex patterns for errors to rerun (repeatable)
    --rerun-except: Regex patterns for errors NOT to rerun (repeatable)
    --fail-on-flaky: Fail with exit code 7 if flaky test passes on rerun
    """

Test Execution Control

Core functions for controlling test execution and rerun behavior.

def get_reruns_count(item):
    """
    Get the number of reruns for a test item.
    Priority: marker > command line > ini file
    
    Args:
        item: pytest test item
        
    Returns:
        int: Number of reruns for this test
    """

def get_reruns_delay(item):
    """
    Get the delay between reruns for a test item.
    
    Args:
        item: pytest test item
        
    Returns:
        float: Delay in seconds between reruns
    """

def get_reruns_condition(item):
    """
    Evaluate whether reruns should happen for a test item.
    
    Args:
        item: pytest test item
        
    Returns:
        bool: Whether reruns are enabled for this test
    """

def evaluate_condition(item, mark, condition: object) -> bool:
    """
    Evaluate condition expressions for flaky marker.
    
    Args:
        item: pytest test item
        mark: pytest marker object
        condition: str or bool condition to evaluate
        
    Returns:
        bool: Result of condition evaluation
    """

Pytest Hooks

Plugin hooks that integrate with pytest's test execution lifecycle.

def pytest_configure(config):
    """
    Configure the plugin and register flaky marker.
    
    Args:
        config: pytest configuration object
    """

def pytest_runtest_protocol(item, nextitem):
    """
    Main test execution protocol with rerun logic.
    
    Args:
        item: pytest test item to execute
        nextitem: next test item in queue
        
    Returns:
        bool: True if protocol handled (reruns enabled), None for default behavior
    """

def pytest_runtest_teardown(item, nextitem):
    """
    Handle test teardown with rerun logic.
    
    Args:
        item: pytest test item
        nextitem: next test item in queue
    """

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
    """
    Create test reports with rerun tracking.
    
    Args:
        item: pytest test item
        call: pytest call info
        
    Returns:
        TestReport: Enhanced with rerun information
    """

def pytest_report_teststatus(report):
    """
    Report test status including rerun status.
    
    Args:
        report: pytest test report
        
    Returns:
        tuple: (outcome, letter, verbose_word) for rerun status
    """

def pytest_terminal_summary(terminalreporter):
    """
    Display rerun summary in terminal output.
    
    Args:
        terminalreporter: pytest terminal reporter
    """

@pytest.hookimpl(trylast=True)
def pytest_sessionfinish(session, exitstatus):
    """
    Handle session finish with flaky test detection.
    
    Args:
        session: pytest session
        exitstatus: current exit status
    """

Test State Management

Functions for managing test state during reruns, including fixture cleanup and setup state management.

def check_options(config):
    """
    Validate command-line options compatibility.
    
    Args:
        config: pytest configuration object
        
    Raises:
        pytest.UsageError: If incompatible options are used
    """

def works_with_current_xdist():
    """
    Check compatibility with pytest-xdist version.
    
    Returns:
        bool or None: True if compatible, False if incompatible, None if not installed
    """

def is_master(config):
    """
    Determine if running on master node in distributed tests.
    
    Args:
        config: pytest configuration object
        
    Returns:
        bool: True if master node, False if worker
    """

def show_rerun(terminalreporter, lines):
    """
    Display rerun test information.
    
    Args:
        terminalreporter: pytest terminal reporter
        lines: list to append rerun information to
    """

Distributed Testing Support

Classes and hooks for pytest-xdist integration and crash recovery.

class XDistHooks:
    """Hooks for pytest-xdist integration."""
    
    def pytest_configure_node(self, node):
        """
        Configure xdist worker nodes.
        
        Args:
            node: xdist worker node
        """
    
    def pytest_handlecrashitem(self, crashitem, report, sched):
        """
        Handle crashed tests in distributed execution.
        
        Args:
            crashitem: crashed test item identifier
            report: crash report
            sched: xdist scheduler
        """

Status Database Classes

Database classes for tracking test failures and reruns across distributed test execution.

class StatusDB:
    """Base in-memory database for tracking test reruns and failures."""
    
    def add_test_failure(self, crashitem):
        """
        Record a test failure.
        
        Args:
            crashitem: test item identifier
        """
    
    def get_test_failures(self, crashitem):
        """
        Get failure count for a test.
        
        Args:
            crashitem: test item identifier
            
        Returns:
            int: Number of failures recorded
        """
    
    def set_test_reruns(self, crashitem, reruns):
        """
        Set rerun count for a test.
        
        Args:
            crashitem: test item identifier
            reruns: number of reruns allowed
        """
    
    def get_test_reruns(self, crashitem):
        """
        Get rerun count for a test.
        
        Args:
            crashitem: test item identifier
            
        Returns:
            int: Number of reruns allowed
        """

class ServerStatusDB(StatusDB):
    """Server-side socket database for distributed testing."""
    
    @property
    def sock_port(self):
        """
        Get the socket port for client connections.
        
        Returns:
            int: Port number
        """
    
    def run_server(self):
        """Run the database server to handle client connections."""

class ClientStatusDB(StatusDB):
    """Client-side socket database for distributed testing."""
    
    def __init__(self, sock_port):
        """
        Initialize client database.
        
        Args:
            sock_port: server socket port to connect to
        """

Marker Reference

@pytest.mark.flaky

Mark individual tests for automatic reruns on failure.

@pytest.mark.flaky(
    reruns=1,                    # int: Number of reruns (default: 1)
    reruns_delay=0,              # float: Delay between reruns in seconds
    condition=True,              # str|bool: Condition for when reruns should happen
    only_rerun=None,             # str|list[str]: Regex patterns for errors to rerun
    rerun_except=None            # str|list[str]: Regex patterns for errors NOT to rerun
)
def test_function():
    """Test that may be flaky and need reruns."""

Configuration

Command Line Options

  • --reruns N: Re-run failed tests N times (default: 0)
  • --reruns-delay SECONDS: Add delay between reruns (default: 0)
  • --only-rerun REGEX: Only rerun errors matching regex (repeatable)
  • --rerun-except REGEX: Skip reruns for errors matching regex (repeatable)
  • --fail-on-flaky: Fail with exit code 7 if flaky test passes on rerun

Configuration File Options

In pytest.ini or pyproject.toml:

[tool.pytest.ini_options]
reruns = "3"
reruns_delay = "1"

Priority Order

  1. @pytest.mark.flaky marker (highest priority)
  2. Command-line options (--reruns, --reruns-delay)
  3. Configuration file options (lowest priority)

Error Handling

Compatibility Restrictions

  • Not compatible with --pdb flag
  • Not compatible with --looponfail flag (pytest-xdist)
  • Not compatible with flaky plugin (choose one or the other)

Exception Filtering

Control which exceptions trigger reruns:

# Only rerun on specific exceptions
@pytest.mark.flaky(only_rerun=["AssertionError", "ConnectionError"])
def test_network_operation():
    pass

# Skip reruns for specific exceptions
@pytest.mark.flaky(rerun_except="KeyboardInterrupt")
def test_user_interaction():
    pass

Conditional Reruns

Use conditions to control when reruns happen:

import sys

# Only rerun on Windows
@pytest.mark.flaky(reruns=3, condition="sys.platform.startswith('win32')")
def test_windows_specific():
    pass

# Only rerun in CI environment
@pytest.mark.flaky(reruns=2, condition="'CI' in os.environ")
def test_ci_sensitive():
    pass

Advanced Usage

Crash Recovery

With pytest-xdist installed, the plugin can recover from hard crashes:

# Enable crash recovery with parallel execution
pytest -n auto --reruns 3

Accessing Execution Count

In custom pytest hooks, access the execution count:

@pytest.hookimpl(tryfirst=True)
def pytest_runtest_makereport(item, call):
    if hasattr(item, 'execution_count'):
        print(f"Test {item.nodeid} executed {item.execution_count} times")

Test Item Attributes

The plugin adds these attributes to test items:

  • execution_count: Number of times the test has been executed
  • _test_failed_statuses: Dict tracking failure status for each test phase
  • _terminal_errors: Dict tracking terminal errors that prevent reruns

Report Attributes

The plugin adds these attributes to test reports:

  • rerun: Rerun number for this report (0 for first run)
  • outcome: May be "rerun" for intermediate rerun reports

Types

# Test item attributes added by plugin
class TestItem:
    execution_count: int
    _test_failed_statuses: dict[str, bool]
    _terminal_errors: dict[str, bool]

# Report attributes added by plugin  
class TestReport:
    rerun: int  # 0 for first run, 1+ for reruns
    outcome: str  # may be "rerun" for intermediate reports

# Configuration constants
HAS_PYTEST_HANDLECRASHITEM: bool  # True if xdist crash handling available
RERUNS_DESC: str  # Description for --reruns option
RERUNS_DELAY_DESC: str  # Description for --reruns-delay option
suspended_finalizers: dict  # Global storage for suspended test finalizers
Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/pytest-rerunfailures@16.0.x
Publish Source
CLI
Badge
tessl/pypi-pytest-rerunfailures badge