Property-Based Testing Skill

Test in terms of properties (invariants and contracts) rather than just hand-picked examples. Let the computer generate randomized inputs to find edge cases you'd never think of.

Common Property Patterns

Step-by-Step Workflow

Step 1: Analyze for Properties

Given a function/module, identify at least 2-3 properties:

• What must always be true about the output?
• What state is conserved (length, count, sum)?
• Are there inverse operations you can pair?
• Is there an alternative implementation to compare against?
• What edge cases exist? Empty, null, negative, zero, max int, duplicates

If finding properties is hard, the function may have too many responsibilities or poorly defined contracts — consider refactoring or making side effects explicit before proceeding.

Step 2: Select the Framework

Select the standard PBT framework for the project's language (Hypothesis for Python, fast-check for TypeScript, proptest for Rust, rapid for Go, RapidCheck for C++). If the framework isn't installed, suggest adding it.

Step 3: Design Generators (Strategies)

Define strategies that cover the full valid input space — not just "nice" inputs:

• Start with primitives: integers, strings, booleans, floats
• Compose into structs: lists of, tuples of, records/dicts
• Constrain: min/max, non-empty, filtered
• Include edge values: empty string, 0, NaN, null, max int

Key generator APIs by framework: st.integers() (Hypothesis), fc.integer() (fast-check), any::<i32>() (proptest), rapid.Int() (rapid), rc::gen::arbitrary<int>() (RapidCheck).

Step 4: Write Property Tests

Write one test per property. Each test must:

1. Generate inputs via strategies
2. Act — run the function under test
3. Assert — check the invariant

Example — testing a JSON round-trip in Python with Hypothesis:

from hypothesis import given, strategies as st
import json

@given(st.dictionaries(
    keys=st.text(min_size=1),
    values=st.one_of(st.text(), st.integers(), st.floats(allow_nan=False))))
def test_json_roundtrip(data):
    """Round-trip property: serializing then deserializing yields the original."""
    serialized = json.dumps(data)
    deserialized = json.loads(serialized)
    assert deserialized == data

Step 5: Run and Analyze Failures

When a property test fails, the framework provides the minimal failing input (shrunk):

• Read the shrunk case carefully — it pinpoints the precise conditions
• The bug may not be in the function you were testing (e.g., a property test for stock levels found a bug in in_stock())
• If the property itself is wrong, refine it and re-run

Step 6: Extract Regression Unit Tests

Always extract the failing case into a concrete, deterministic unit test:

def test_json_roundtrip_regression():
    """Regression test from property-based test failure."""
    data = {"": 0}  # Minimal failing case: empty string key
    serialized = json.dumps(data)
    deserialized = json.loads(serialized)
    assert deserialized == data

This locks in the fix and ensures it can't regress when randomized inputs change.

Common Mistakes