Stream: ideas

Topic: Call only one test

Oskar Hahn (Dec 13 2023 at 12:50):

For yesterdays AoC I wrote around 20 tests. I had a lot of bugs. To debug them, I had to comment out all but one test.

Is there a way to only run one test?

Richard Feldman (Dec 13 2023 at 13:06):

not currently, but there absolutely should be! We haven't discussed a design for that, but one idea is to let roc test accept a line number, e.g. roc test foo.roc:123

Oskar Hahn (Dec 13 2023 at 13:23):

One disadvantage is, that the line number is changing a lot. If you run your test and change your code, you can not just rerun it but find and type the new line number.

Would it be possible to give a test an optional name?

expect "test myFunc"
  myFunc 7 == 14

Richard Feldman (Dec 13 2023 at 14:07):

I want to avoid that - one of the major design goals behind expect is to reduce the amount of friction and ceremony behind writing tests to as close to zero as possible, so I'd rather find a design that doesn't require changing how tests are written :big_smile:

Richard Feldman (Dec 13 2023 at 14:07):

in elm-test we have an only modifier that you can add to a test, and then only the tests with that modifier set get run

Richard Feldman (Dec 13 2023 at 14:08):

and then if they all pass it says something like "test run incomplete - all tests passed but N were skipped due to only being used" and returns a nonzero exit code so it'll fail CI if you accidentally check in an only

Richard Feldman (Dec 13 2023 at 14:09):

so we could do something similar for expect

John Murray (Dec 13 2023 at 16:48):

I think having filename/line number filtering would be dope!

expect.only or expect.skip would probably cover anything else specific

Ryan Bates (Dec 13 2023 at 20:21):

Line number would be awesome. If you use VS Code it's pretty easy to add a task which runs the test at the current line. For example:

{
  "label": "roc test current line",
  "type": "shell",
  "command": "roc test ${file}:${lineNumber}",
  "runOptions": {
    "reevaluateOnRerun": false
  }
}

You can bind it to a key command for some quick testing.

Pearce Keesling (Dec 13 2023 at 21:03):

John Murray said:

I think having filename/line number filtering would be dope!

expect.only or expect.skip would probably cover anything else specific

expect.fails is also useful so I can check in a failing test and then know when it stops failing

Notification Bot (Dec 13 2023 at 21:05):

This topic was moved here from #beginners > Call only one test by Richard Feldman.

Eli Dowling (Dec 13 2023 at 22:39):

I would personally much prefer to have the ability to name tests. I don't think it adds friction, especially if it's optional. Infact if I had a project I would enforce all tests to have a name. I think a test name provides important context and allows seeing what's being tested and what should be added in a quick glance.

If you have a really large project with a huge test suite or if some tests are integration style tests that take a lot of time to run because they talk to the outside world and do IO. Being able to have test groupings is super nice so you can exclude slow tests or tests not relevant to your change while you iterate.
Eg, Talking to your real database, starting your lsp server and sending it real commands etc

A simple way is to just have '--include #mytag' which requires that that string exists in the test name, and it's mirror '--exclude #mytag'
Eg: "#integration Renaming in AWS s3 works"

Lastly, I think one of my favourite workflows for programming is the ocaml expect test experience. Just running 'dune test --watch' gives continuous test feedback everytime you save. This would be pretty annoying in a larger project without good filtering because it would take too long and ruin the experience.

Richard Feldman (Dec 13 2023 at 22:44):

Eli Dowling said:

I think a test name provides important context and allows seeing what's being tested and what should be added in a quick glance.

that's already possible! You can add a ## comment above any expect and it'll get printed out if the test fails so you can see that added context

Richard Feldman (Dec 13 2023 at 22:46):

what I don't want to do is to create a situation where it does more than provide context - because then there are some features you can only use if you write those, at which point it's de facto mandatory if you want to use all the normal testing features

Richard Feldman (Dec 13 2023 at 23:12):

Eli Dowling said:

one of my favourite workflows for programming is the ocaml expect test experience. Just running 'dune test --watch' gives continuous test feedback everytime you save. This would be pretty annoying in a larger project without good filtering because it would take too long and ruin the experience.

we should totally have roc test --watch - that would be sweet! :smiley:

Probably check and build too, maybe even dev and watch :thinking:

Richard Feldman (Dec 13 2023 at 23:15):

Eli Dowling said:

A simple way is to just have '--include #mytag' which requires that that string exists in the test name, and it's mirror '--exclude #mytag'
Eg: "#integration Renaming in AWS s3 works"

I think something like this is plausible, although I'd like to think about tradeoffs around having it search the source code of the expect, the doc comment (if there is one), or perhaps both!

Luke Boswell (Dec 13 2023 at 23:50):

Richard Feldman said:

we should totally have roc test --watch - that would be sweet! :smiley:

I've added Issue #6278 to track this.

Agus Zubiaga (Dec 14 2023 at 14:41):

Richard Feldman said:

Probably check and build too, maybe even dev and watch :thinking:

I'd love have roc dev --watch for server platforms!

Agus Zubiaga (Dec 14 2023 at 14:51):

@Richard Feldman Are there any plans to have something like describe or to create blocks? I feel like in other languages, I almost always use only with a set of tests (for a function maybe).

Another benefit of grouping is that often you have helpers that are only ever used in tests, so it might be nice to nest those. With module params, it could also be beneficial to have inline imports for a group of tests where effects are all mocked the same way. Or just to have names in exposing that you wouldn't like to pollute the rest of your code with.

Richard Feldman (Dec 14 2023 at 15:18):

my thinking for that would be to just put groups of tests in their own modules, and then run roc test on that module

Richard Feldman (Dec 14 2023 at 15:18):

since it doesn't require new language features, I'd like to try that and see how it goes!

Last updated: Jun 16 2026 at 16:19 UTC