Skip to content

ADR-0070: Property-Based KG Testing

  • Status: Accepted
  • Date: 2026-05-02
  • Supersedes: --
  • Superseded by: --
  • Related: ADR-0017 (ORM surface), ADR-0056 (validation modes)

Context

Trails ships @competency_question for example-based KG testing and GraphAssertions for fluent assertions. Both rely on hand-crafted instances. Edge cases in schema enforcement, SHACL constraints, and type coercion are easy to miss when every test value is chosen by the author.

Property-based testing (PBT) generates random inputs conforming to a specification and checks that invariants hold across many examples. No existing KG framework offers PBT strategies derived from node-type schemas.

Decision

Add trails.testing_properties -- a standalone module that generates Hypothesis strategies from @node_type field schemas and optional SHACL constraints (min_value, max_value, pattern, min_length, max_length, one_of).

Core surface:

  • kg_strategy(NodeTypeCls) -- returns a SearchStrategy producing random valid instances.
  • @kg_property(node_type=..., min_examples=50) -- decorator that spins up an in-memory context, generates + persists instances, and runs the wrapped assertion function.
  • kg_invariant(name, check_fn) -- register a named invariant discoverable via trails test --property.
  • field_strategy(python_type) + register_field_strategy(type, st) -- extensible type-to-strategy mapper.

CLI: trails test --property discovers and runs all @kg_property tests and registered invariants.

Hypothesis is an optional dependency; the import is guarded.

Rationale

  • Schema-aware generation catches constraint violations, coercion bugs, and boundary conditions that hand-picked examples miss.
  • SHACL constraint awareness means the generator respects min_value/max_value/pattern/one_of -- producing inputs that are valid by construction, or (via separate strategies) intentionally invalid for negative testing.
  • Separate module (testing_properties.py) avoids merge conflicts with the existing testing.py.

Consequences

  • Hypothesis becomes an optional test dependency (not required at runtime).
  • Projects using @node_type can add property tests with one decorator.
  • Future: negative-strategy generation (intentionally violating constraints) for robustness testing.