Property Testing

> You are a Senior Test Architect -- you distrust example-based tests that only prove the author's assumptions, and you design properties that let the machine find the bugs you didn't imagine.

property-testing

Property-based testing (PBT) generates random inputs, checks that properties hold for all of them, and shrinks failures to minimal counterexamples. It finds edge cases that hand-written examples miss.

What You Get

• Property definitions for the target code (roundtrip, invariant, oracle, metamorphic)
• Custom generators tailored to the domain, not just random noise
• Deterministic CI configuration (fixed seeds, bounded examples)
• Shrunk counterexamples with reproduction commands

When PBT Beats Example Tests

| Use PBT | Stick with examples | |---------|---------------------| | Parsers, serializers (roundtrip property) | Simple CRUD operations | | Sorting, filtering (invariant properties) | UI rendering | | State machines, protocols (model-based) | Integration tests with external services | | Numeric algorithms (oracle: compare to reference) | Code where the property restates the implementation | | Any function with a large input space | Glue code with minimal logic |

The litmus test: Can you state what should be true for ALL inputs without restating the implementation? If yes, use PBT. If the property is just assert f(x) == implementation(x), you are testing nothing.

Workflow

1. Identify properties

Before writing any generators, identify what properties the code should satisfy. See properties.md for the full taxonomy. Common starting points:

• Roundtrip: decode(encode(x)) == x
• Invariant: len(sort(xs)) == len(xs)
• Idempotent: f(f(x)) == f(x)
• Commutative: f(a, b) == f(b, a) (when applicable)

2. Build generators

Use the language's built-in generators first. Only build custom generators when the input domain has constraints (e.g., valid email addresses, balanced trees). See generators.md.

3. Write the property test

# Python (Hypothesis)
from hypothesis import given, settings
import hypothesis.strategies as st

@given(st.lists(st.integers()))
@settings(max_examples=200, derandomize=True)
def test_sort_preserves_length(xs):
    assert len(sorted(xs)) == len(xs)

// Rust (proptest)
proptest! {
    #[test]
    fn sort_preserves_length(xs: Vec) {
        let sorted = sort(&xs);
        prop_assert_eq!(sorted.len(), xs.len());
    }
}

# Elixir (StreamData)
property "sort preserves length" do
  check all xs ` env var |
| Elixir | StreamData | `{:stream_data, "~> 1.0", only: :test}` | `--seed ` in mix test |
| TypeScript | fast-check | `npm i -D fast-check` | `fc.assert(prop, { seed })` |
| Go | gopter | `go get github.com/leanovate/gopter` | `gopter.DefaultGenParameters().Seed(n)` |
| Haskell | QuickCheck | `cabal install QuickCheck` | `quickCheckWith stdArgs{replay=Just(seed,0)}` |

## Relationship to TDD

PBT fits into the TDD workflow as a validation step after the incremental loop:

Phase 1: Planning Phase 2: Tracer Bullet (one example test) Phase 3: Incremental Loop (example tests, RED-GREEN) Phase 3.5: Property Testing 30s) | Reduce max_examples, use @settings(deadline=None) only locally | | Flaky CI from non-deterministic seeds | Always set derandomize=True or equivalent in CI | | Only testing pure functions | Use model-based testing for stateful systems |

Reading guide

| Topic | File | |-------|------| | Property taxonomy with examples | properties.md | | Custom generator patterns | generators.md | | Shrinking and debugging failures | shrinking.md |

See also

• tdd -- red-green-refactor workflow; PBT extends Phase 3
• tdd (mutation-testing sub-file) -- complementary validation (tests the tests, not the input space)

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: DROOdotFOO
• Source: DROOdotFOO/agent-skills
• License: MIT
• Homepage: https://droo.foo/

Install and usage instructions live in the source repository linked above.