Stream: ideas

Topic: Property based testing


view this post on Zulip Matthieu Pizenberg (Aug 25 2026 at 15:01):

Does Roc already have a property-based testing story? Or a plan for it?

view this post on Zulip Anton (Aug 25 2026 at 15:26):

I did a quick AI search over our zulip history:

Standing plan (most concrete — #beginners "property based testing", 2024-08-28):
PBT should be builtin to the compiler + expect, not a bolt-on library. Plan discussed:
- Start minimal: basic generators, with an auto-derived Arbitrary implementation the same way Roc auto-derives Encoders/Decoders (Richard Feldman, Brendan Hansknecht).
- Design it so it can later plug into a real fuzz engine for deeper testing — open question was bundling a fuzz engine (Go-style) vs. making it easy to plug in an external one.
- Luke Boswell suggested filing a GitHub issue to preserve the intent for future contributors.
- No PBT support exists yet — today you can only fake it with a hardcoded seed + manual randomness.

#compiler-development "Property testing and fuzzing" (2024-01-29, revisited 2025-08):
- Brendan wrote a design doc surveying property testing/fuzzing options for Roc, and built a prototype roc-fuzz using libFuzzer with an Arbitrary-style generator.
- Richard preferred Roc eventually build its own coverage instrumentation rather than depend on LLVM's, and floated fuzzing effects by auto-generating fake host I/O.
- 2025 follow-up: a contributor (Johan Lövgren) said this thread inspired a master's thesis on fuzzing + PBT, sharing lessons about generators baking in false assumptions.

view this post on Zulip Luke Boswell (Aug 25 2026 at 20:24):

@Matthieu Pizenberg see https://github.com/lukewilliamboswell/roc-fuzz

I've been using it on everything and I am loving it :rock_on:

view this post on Zulip Matthieu Pizenberg (Aug 25 2026 at 20:43):

@Luke Boswell is it a fuzzer library or a property-based library?

view this post on Zulip Luke Boswell (Aug 25 2026 at 22:22):

So I structure my apps with most of the pure logic in a module. Then I write fuzz targets which sit next to the app root main.roc, so e.g. fuzz_property_x.roc which uses the roc-fuzz platform and my pure logic.

roc-fuzz is a platform

view this post on Zulip Luke Boswell (Aug 25 2026 at 22:26):

It uses libFuzzer for coverage guided mutations to explore deep into your app logic, but you write the properties around the invariants you want to enforce ... so I think the answer is kind of both at the same time?

Im not enough of an expert on the differences between the two categories...

view this post on Zulip Luke Boswell (Aug 25 2026 at 22:28):

We have an Arbitrary type so you can construct valid types from random inputs and assert properties on those.


Last updated: Sep 03 2026 at 15:16 UTC