Aurélien Geron (Aug 14 2026 at 00:47): Aurélien Geron (Aug 14 2026 at 00:47):A few Exercism exercises run a bit too slowly for the test runner, making it time out. To fix this, I'd like to change the test runner to use roc test --opt=speed instead of roc test --opt=dev.
I gave it a try, and indeed --opt=speed does speed up many of the slowest exercises. For example, alphametics now runs in 1.3s instead of 11.7s..
However, some exercises are much slower now, because compilation takes a very long time. For example, pov now runs in 86s instead of 0.1s.
Moreover, 3 exercises crash with --opt=speed while they work fine with --opt=dev. If you want to try this out, the exercises are sgf-parsing, sieve, and sum-of-multiples.
Luke Boswell (Aug 14 2026 at 01:07):Once all the bugs are ironed out either of these options should build and run fast
Luke Boswell (Aug 14 2026 at 01:09):If you are able to isolate fresh bugs and report them that is the easiest way to get there. I've been trying to report them as I can but it's just a bit of whack a mole until we burn them all down I guess.
Luke Boswell (Aug 14 2026 at 01:09):Fuzzing doesn't really help deep in the compiler pipelines, so it seems building real things is the most reliable way of finding these issues.
Aurélien Geron (Aug 14 2026 at 01:10):Yes, I'll try to isolate the bugs and report them
Aurélien Geron (Aug 14 2026 at 01:10):Luke Boswell said:
Fuzzing doesn't really help deep in the compiler pipelines, so it seems building real things is the most reliable way of finding these issues.
That's actually half of my motivation for working on many exercism exercises :grinning_face_with_smiling_eyes:
Luke Boswell (Aug 14 2026 at 01:11):One thing I think is helpful is having like heap of nightlies in a directory and running the examples in a matrix and measuring times etc so see where the regressions may have started
Luke Boswell (Aug 14 2026 at 01:11):The release fast nightlies generally roc check or build in milliseconds so issues stand out pretty quickly
Luke Boswell (Aug 14 2026 at 01:18):Another thing I've been doing is building a custom benchmark platform for my test harnesses in pure roc packages ... then I can add all kind of instrumentation around allocations and timings etc -- drive that with python from specs in jsonl etc and it's a really comprehensive way to find and isolate bugs at any layer.
Luke Boswell (Aug 14 2026 at 01:19):With an Agent I can say something really vague like "why are the allocations growing exponentially here, that seems bad" and a few moments later I've got a root cause and fix
Luke Boswell (Aug 14 2026 at 01:25):Or more relevant I guess in this scenario I can use the allocations in a golden snapshot and if that changes between nightlies when I upgrade I can look into it
Anton (Aug 14 2026 at 14:23):Luke Boswell said:
Or more relevant I guess in this scenario I can use the allocations in a golden snapshot and if that changes between nightlies when I upgrade I can look into it
Yeah I've been thinking about something like that too. We can probably add a fast CI workflow that tests a couple of Roc files with good coverage of the compiler like all_syntax_test.roc
Last updated: Sep 03 2026 at 15:16 UTC