Stream: compiler development

Topic: test_syntax fuzzing issue


view this post on Zulip Anthony Bullard (Jan 01 2025 at 03:56):

I am _almost done_ (I think :pray:) with the parens and commas change. Able to parse my AOC projects before any PNC changes - migrate successfully - parse and run again - and then format again in a stable fashion!

One issue I have is I'm trying to make sure there isn't any small problems I'm causing, so I'm trying to be like Luke and run the hell out of the fuzzer. The issue is I'm now only getting crashes on "not yet implemented" and other panics. I've implemented zero of these in my changes so I'm wondering what I'm doing wrong.

view this post on Zulip Anthony Bullard (Jan 01 2025 at 04:05):

In the meantime I'm running fuzzer with the module target and it seems to be churning for _way_ longer

view this post on Zulip Richard Feldman (Jan 01 2025 at 04:19):

wow, shoutout to @Joshua Warner for making the fuzzer so useful! :heart_eyes:

view this post on Zulip Anthony Bullard (Jan 01 2025 at 04:20):

Yes the fuzzer is glorious if a bit mysterious. I'm sure I'll get around to asking Joshua about all it's dark magic

view this post on Zulip Luke Boswell (Jan 01 2025 at 04:31):

Fantastic to hear :smiley:

view this post on Zulip Brendan Hansknecht (Jan 01 2025 at 06:29):

The issue is I'm now only getting crashes on "not yet implemented" and other panics. I've implemented zero of these in my changes so I'm wondering what I'm doing wrong.

Yeah, this is the pain of fuzzing something that isn't ready. Maybe there is a way to filter out this class for crashes, but generally speaking you have to remove all forms of panics before fuzzing becomes useful.

view this post on Zulip Joshua Warner (Jan 01 2025 at 12:23):

@Anthony Bullard There's a "not yet implemented" crash I fixed recently - by chance are you behind main?

view this post on Zulip Joshua Warner (Jan 01 2025 at 12:25):

If you want to zip/tar and upload test_syntax/fuzz/artifacts/fuzz_expr, I can take a look at those failures and hopefully fix them fairly shortly

view this post on Zulip Joshua Warner (Jan 01 2025 at 12:26):

Here's the rough command I've been using to quickly triage large numbers of fuzzing artifacts:

cargo build --bin minimize; for x in /Users/joshw/Downloads/artifacts\ 12/fuzz_expr/crash*  ; if /Users/joshw/src/github.com/roc-lang/roc/target/debug/minimize expr $x; break; end; end

(that's fish shell syntax, and will obviously need some adjustment if you want to run it)


Last updated: Jul 06 2025 at 12:14 UTC