Property-Based Test Generator

Generates property-based tests that validate invariants and find edge cases automatically through randomized testing.

When to Use

• "Generate property-based tests"

• "Create hypothesis tests for my function"

• "Add property tests to my code"

• "Generate fast-check tests"

• "Test function properties"

• "Find edge cases automatically"

Instructions

1. Detect Language and Testing Framework

Check the project's language and existing test setup:

Check for Python

[ -f "pytest.ini" ] || [ -f "setup.py" ] && echo "Python"

Check for JavaScript/TypeScript

[ -f "package.json" ] && echo "JavaScript/TypeScript"

Check existing test framework

grep -E "(jest|vitest|mocha|pytest|hypothesis)" package.json pyproject.toml requirements.txt 2>/dev/null

2. Install Property-Based Testing Library

Python (Hypothesis):

pip install hypothesis pytest

JavaScript/TypeScript (fast-check):

npm install --save-dev fast-check @types/jest

or

npm install --save-dev fast-check vitest

Haskell (QuickCheck):

cabal install QuickCheck

3. Identify Function Properties

Analyze the function to test and identify invariants:

Common Properties:

• Idempotence: f(f(x)) === f(x)

• Inverse: decode(encode(x)) === x

• Commutativity: f(a, b) === f(b, a)

• Associativity: f(f(a, b), c) === f(a, f(b, c))

• Identity: f(x, identity) === x

• Range: Output always within valid range

• Type safety: Output type matches expected

• No exceptions: Function never throws for valid input

4. Generate Property-Based Tests

Python with Hypothesis

Basic Example:

from hypothesis import given, strategies as st import pytest

Function to test

def sort_list(items): return sorted(items)

Property: sorted list length equals original

@given(st.lists(st.integers())) def test_sort_preserves_length(items): result = sort_list(items) assert len(result) == len(items)

Property: sorted list is ordered

@given(st.lists(st.integers())) def test_sort_creates_ordered_list(items): result = sort_list(items) for i in range(len(result) - 1): assert result[i] <= result[i + 1]

Property: sorted list contains same elements

@given(st.lists(st.integers())) def test_sort_preserves_elements(items): result = sort_list(items) assert sorted(items) == result

Advanced Strategies:

from hypothesis import given, strategies as st, assume from datetime import datetime, timedelta

Custom data structures

@st.composite def users(draw): return { 'id': draw(st.integers(min_value=1, max_value=1000000)), 'name': draw(st.text(min_size=1, max_size=50)), 'email': draw(st.emails()), 'age': draw(st.integers(min_value=18, max_value=120)), 'created_at': draw(st.datetimes( min_value=datetime(2020, 1, 1), max_value=datetime.now() )) }

@given(users()) def test_user_validation(user): # Validate user properties assert user['age'] >= 18 assert '@' in user['email'] assert len(user['name']) > 0 assert user['created_at'] <= datetime.now()

Testing with Preconditions:

@given(st.integers(), st.integers()) def test_division(a, b): assume(b != 0) # Precondition: no division by zero result = a / b assert result * b == a # Property: inverse of multiplication

Stateful Testing:

from hypothesis.stateful import RuleBasedStateMachine, rule, invariant

class ShoppingCart(RuleBasedStateMachine): def init(self): super().init() self.items = [] self.total = 0

@rule(item=st.tuples(st.text(), st.floats(min_value=0, max_value=1000)))
def add_item(self, item):
    name, price = item
    self.items.append(item)
    self.total += price

@rule()
def clear_cart(self):
    self.items = []
    self.total = 0

@invariant()
def total_matches_sum(self):
    assert abs(self.total - sum(p for _, p in self.items)) < 0.01

TestCart = ShoppingCart.TestCase

JavaScript/TypeScript with fast-check

Basic Example:

import fc from 'fast-check';

// Function to test function reverseString(str: string): string { return str.split('').reverse().join(''); }

describe('reverseString', () => { it('double reverse returns original', () => { fc.assert( fc.property(fc.string(), (str) => { const reversed = reverseString(str); const doubleReversed = reverseString(reversed); return doubleReversed === str; }) ); });

it('preserves string length', () => { fc.assert( fc.property(fc.string(), (str) => { return reverseString(str).length === str.length; }) ); });

it('first char becomes last char', () => { fc.assert( fc.property(fc.string({ minLength: 1 }), (str) => { const reversed = reverseString(str); return str[0] === reversed[reversed.length - 1]; }) ); }); });

Complex Data Structures:

import fc from 'fast-check';

// Custom arbitraries const userArbitrary = fc.record({ id: fc.integer({ min: 1, max: 1000000 }), name: fc.string({ minLength: 1, maxLength: 50 }), email: fc.emailAddress(), age: fc.integer({ min: 18, max: 120 }), roles: fc.array(fc.constantFrom('admin', 'user', 'guest'), { minLength: 1 }) });

describe('User validation', () => { it('validates user structure', () => { fc.assert( fc.property(userArbitrary, (user) => { return ( user.age >= 18 && user.email.includes('@') && user.roles.length > 0 ); }) ); }); });

Array Properties:

describe('Array operations', () => { it('map preserves length', () => { fc.assert( fc.property( fc.array(fc.integer()), fc.func(fc.integer()), (arr, fn) => { return arr.map(fn).length === arr.length; } ) ); });

it('filter result is subset', () => { fc.assert( fc.property( fc.array(fc.integer()), (arr) => { const filtered = arr.filter(x => x > 0); return filtered.every(x => arr.includes(x)); } ) ); });

it('concat is associative', () => { fc.assert( fc.property( fc.array(fc.integer()), fc.array(fc.integer()), fc.array(fc.integer()), (a, b, c) => { const left = a.concat(b).concat(c); const right = a.concat(b.concat(c)); return JSON.stringify(left) === JSON.stringify(right); } ) ); }); });

Shrinking Examples:

describe('Shrinking demonstration', () => { it('finds minimal failing case', () => { fc.assert( fc.property(fc.array(fc.integer()), (arr) => { // This will fail and shrink to smallest failing case return arr.length < 5 || arr.some(x => x > 100); }), { numRuns: 100 } ); }); });

Model-Based Testing:

import fc from 'fast-check';

class Stack<T> { private items: T[] = [];

push(item: T): void { this.items.push(item); }

pop(): T | undefined { return this.items.pop(); }

size(): number { return this.items.length; } }

describe('Stack', () => { it('behaves like array', () => { fc.assert( fc.property( fc.array(fc.integer()), (operations) => { const stack = new Stack<number>(); const model: number[] = [];

      for (const op of operations) {
        if (op >= 0) {
          stack.push(op);
          model.push(op);
        } else {
          const stackResult = stack.pop();
          const modelResult = model.pop();
          if (stackResult !== modelResult) return false;
        }
      }

      return stack.size() === model.length;
    }
  )
);

}); });

5. Common Property Patterns

Encode/Decode (Roundtrip):

from hypothesis import given, strategies as st import json

@given(st.dictionaries(st.text(), st.integers())) def test_json_roundtrip(data): encoded = json.dumps(data) decoded = json.loads(encoded) assert decoded == data

fc.assert( fc.property(fc.anything(), (data) => { const encoded = JSON.stringify(data); const decoded = JSON.parse(encoded); return JSON.stringify(decoded) === encoded; }) );

Idempotence:

@given(st.lists(st.integers())) def test_dedup_idempotent(items): result1 = list(set(items)) result2 = list(set(result1)) assert result1 == result2

Commutativity:

@given(st.integers(), st.integers()) def test_addition_commutative(a, b): assert a + b == b + a

Oracle (Compare with Known Implementation):

@given(st.lists(st.integers())) def test_custom_sort_matches_builtin(items): assert custom_sort(items) == sorted(items)

Invariants:

fc.assert( fc.property(fc.array(fc.integer()), (arr) => { const unique = [...new Set(arr)]; return unique.length <= arr.length; }) );

6. Configuration Options

Hypothesis:

from hypothesis import given, settings, strategies as st

@settings( max_examples=1000, # Number of test cases deadline=None, # No timeout verbosity=hypothesis.Verbosity.verbose ) @given(st.integers()) def test_with_settings(x): assert x == x

fast-check:

fc.assert( fc.property(fc.integer(), (x) => x === x), { numRuns: 1000, // Number of test cases seed: 42, // Reproducible tests verbose: true, // Show details endOnFailure: false // Run all tests } );

7. Testing Strategies by Data Type

Strings:

st.text() st.text(min_size=1, max_size=100) st.text(alphabet=st.characters(blacklist_categories=['Cs'])) st.from_regex(r'[a-z]{3,10}')

fc.string() fc.string({ minLength: 1, maxLength: 100 }) fc.hexaString() fc.asciiString() fc.unicodeString() fc.stringOf(fc.char())

Numbers:

st.integers() st.integers(min_value=0, max_value=100) st.floats(min_value=0.0, max_value=1.0) st.decimals()

fc.integer() fc.integer({ min: 0, max: 100 }) fc.float() fc.double() fc.nat()

Collections:

st.lists(st.integers()) st.lists(st.integers(), min_size=1, max_size=10) st.sets(st.text()) st.dictionaries(st.text(), st.integers()) st.tuples(st.text(), st.integers())

fc.array(fc.integer()) fc.array(fc.integer(), { minLength: 1, maxLength: 10 }) fc.set(fc.string()) fc.dictionary(fc.string(), fc.integer()) fc.tuple(fc.string(), fc.integer())

Dates:

st.datetimes() st.dates(min_value=date(2020, 1, 1)) st.times()

fc.date() fc.date({ min: new Date('2020-01-01') })

8. Best Practices

DO:

• Test invariants, not specific outputs

• Use meaningful property names

• Start with simple properties

• Let the library shrink failures

• Test edge cases (empty, single item, max size)

• Combine multiple properties

• Use preconditions (assume in Hypothesis, fc.pre in fast-check)

DON'T:

• Test implementation details

• Use too complex properties

• Ignore shrinking results

• Forget to test edge cases

• Make properties too similar to implementation

• Use property-based tests for everything (unit tests still valuable)

9. Common Patterns

Metamorphic Testing:

@given(st.lists(st.integers())) def test_sort_stability(items): # Adding an element and sorting should give same order for original elements with_extra = items + [max(items) + 1] if items else [0] sorted_original = sorted(items) sorted_with_extra = sorted(with_extra)

# Original elements should appear in same relative order
assert sorted_original == [x for x in sorted_with_extra if x in sorted_original]

Differential Testing:

// Test two implementations against each other fc.assert( fc.property(fc.array(fc.integer()), (arr) => { const result1 = optimizedSort(arr); const result2 = naiveSort(arr); return JSON.stringify(result1) === JSON.stringify(result2); }) );

10. Integration with CI/CD

pytest.ini (Hypothesis):

[pytest] addopts = --hypothesis-show-statistics --hypothesis-seed=0

[hypothesis] max_examples = 200 deadline = None

package.json (fast-check):

{ "scripts": { "test": "jest", "test:property": "jest --testNamePattern='property'", "test:verbose": "jest --verbose" } }

GitHub Actions:

name: Property Tests

on: [push, pull_request]

jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Install dependencies run: npm ci - name: Run property tests run: npm run test:property

11. Debugging Failed Properties

Hypothesis:

from hypothesis import given, strategies as st, example

@given(st.integers()) @example(0) # Add specific examples to always test @example(-1) @example(999999) def test_with_examples(x): assert process(x) >= 0

Run with verbose output

pytest --hypothesis-verbosity=verbose test_file.py

fast-check:

fc.assert( fc.property(fc.integer(), (x) => { // Use fc.pre for preconditions fc.pre(x !== 0); return 100 / x > 0; }), { seed: 1234567890, // Reproduce exact failure path: "0:1:2", // Replay specific path verbose: true } );

12. Generate Test Report

Create a summary of property tests:

Property-Based Test Report

Coverage

• Functions tested: 15
• Properties verified: 42
• Test cases generated: 50,000+
• Edge cases found: 8

Properties Tested

sort_list

• ✅ Preserves length
• ✅ Creates ordered output
• ✅ Preserves all elements
• ✅ Handles empty lists
• ✅ Handles duplicates

encode_decode

• ✅ Roundtrip property (decode(encode(x)) === x)
• ✅ Handles special characters
• ✅ Preserves data types

merge_sorted_arrays

• ✅ Output is sorted
• ✅ Contains all elements
• ✅ Length equals sum of inputs

Bugs Found

1. Division by zero in calculation (fixed)
2. Off-by-one error in array indexing (fixed)
3. Unicode handling issue in string processing (fixed)

Recommendations

• Add property tests for user authentication flow
• Test database query builder invariants
• Add metamorphic tests for caching layer

Checklist

• Property-based testing library installed

• Function invariants identified

• Basic properties implemented

• Edge cases covered

• Shrinking verified

• CI/CD integration added

• Documentation updated

• Team trained on property-based testing