Stream: ideas

Topic: roc test exit code

Anton (Feb 02 2024 at 13:07):

roc test prints "No expectations were found." and returns with exit code 2 if the provided Roc file did not contain any tests. This is pretty annoying in CI contexts where you want to test all expects in all Roc files. It's non-trivial to avoid false positives when checking if a file contains a top level expect with grep for example.

So I propose we change the returned exit code to 0 but still print "No expectations were found.".

Richard Feldman (Feb 02 2024 at 14:22):

so the motivation for this was wanting to prevent the scenario where you think all your tests are passing, but it's actually because you're running roc test in the wrong directory or something

Richard Feldman (Feb 02 2024 at 14:23):

regarding testing all expect s in all .roc files, I'd like to have roc test run all top level expects not just for the given .roc file, but also all the ones it imports (recursively)

Richard Feldman (Feb 02 2024 at 14:24):

so that way you should be able to run roc test and have it run all the tests in your project

Richard Feldman (Feb 02 2024 at 14:24):

would that address this use case?

Anton (Feb 02 2024 at 14:30):

That would work, but not for any roc files in e.g. an examples folder.

Brendan Hansknecht (Feb 02 2024 at 15:40):

Also, you can match on CI for 0 or 2. Just like with roc check. 2 is our warning exit code

Brendan Hansknecht (Feb 02 2024 at 15:40):

But yeah, less convenient but for a solid purpose

Brendan Hansknecht (Feb 02 2024 at 15:41):

Oh, also, roc test is already recursive over imports

Richard Feldman (Feb 02 2024 at 16:38):

I like the idea of trying that out!

Anton (Feb 02 2024 at 17:44):

Sorry, trying what out exactly?

Richard Feldman (Feb 02 2024 at 18:01):

checking for exit code 2 in CI

Anton (Feb 02 2024 at 18:08):

:+1: I'm setting it up like that in basic-cli

Richard Feldman (Feb 02 2024 at 18:20):

nice, sounds good!

Last updated: Jun 16 2026 at 16:19 UTC