Stream: ideas

Topic: testing

Richard Feldman (Dec 07 2021 at 19:08):

https://mobile.twitter.com/antfu7/status/1468233216939245579 but at the function level :smiley:

Just like how Vite works in the browser, Vitest also knows the graph of your modules, which makes it able to do smart detection and only rerun the related tests. Feels almost like HMR but for tests :heart_eyes: https://twitter.com/antfu7/status/1468233216939245579/photo/1

- Anthony Fu (@antfu7)

Matthias Devlamynck (Dec 07 2021 at 19:16):

It would also be nice to show what code the editor considers to be related to the test and what test is related to some piece of code.

jan kili (Aug 27 2022 at 23:37):

I was just about to ask everyone what testing libraries we'd like to see a Roc equivalent of, and here (above in this topic) is one answer! Vitest! I'm envisioning a third-party testing library/framework, and I think it could do that level of "hot function reloading"... but maybe that should be built into the Roc CLI! I don't know.

At a high level, any other favorite frameworks/libraries that any of you would like to see for Roc?

Robot Framework? Jest? Testify? Cucumber?

jan kili (Aug 27 2022 at 23:40):

Alternatively, if you don't want an existing testing experience copy/pasted to Ro, what kind of testing principles/help/automation would you like to see in the Roc ecosystem?

jan kili (Aug 27 2022 at 23:41):

I really enjoyed watching a talk on youtube recently on how Test-Driven Development has lost its way (and acquired a bad reputation as a result), and I agree - I'd like to see smooth+scalable TDD actually be as easy as it promises.

jan kili (Aug 28 2022 at 07:43):

Time to learn the (new?) expect keyword :smile:

jan kili (Aug 28 2022 at 07:56):

It looks like the current plan for first-party testing support is to encourage both app & platform authors to colocate function tests in the top level of the same files as the functions themselves?

Richard Feldman (Aug 28 2022 at 14:01):

exactly!

jan kili (Aug 28 2022 at 14:05):

Sweet. Clear and simple!

Ayaz Hafiz (Sep 07 2022 at 19:07):

Has there already been discussion of naming expect tests? For example, I would like to have something like

expect "div by zero is an error"
    Num.divChecked 5 0 == Err DivByZero

If not, what do folks think of this? I am strongly in favor for the usual reasons of discoverability and test filtering. I can't see any immediate downside, but maybe I'm missing something

Qqwy / Marten (Sep 07 2022 at 19:16):

I wonder how far we can push expect on its own before we need some higher-level wrappers to allow for things like setup + cleanup around tests

Qqwy / Marten (Sep 07 2022 at 19:16):

Or testing many values at once by having a dict of cases to try. (There was a discussion about this in Zulip a few days back)

Qqwy / Marten (Sep 07 2022 at 19:16):

Or of course property-based testing

Brian Hicks (Sep 07 2022 at 19:31):

I was wondering about property-based testing too. It feels like it's possible to do a really good job with that here.

Richard Feldman (Sep 07 2022 at 20:25):

so, quick note: you can add a comment over it and the comment will appear in the output if the test fails

Richard Feldman (Sep 07 2022 at 20:25):

e.g.

# div by zero is an error
expect
    Num.divChecked 5 0 == Err DivByZero

Ayaz Hafiz (Sep 07 2022 at 20:41):

if we can name tests in their preceding comments, I think that's great too. The key for me is just having a way to name them, and being able to provide tooling around that (e.g. filtering)

jan kili (Sep 07 2022 at 20:41):

Nice - I was about to say "that sounds a lot like a code comment", but I imagine there are broader uses, too

Richard Feldman (Sep 07 2022 at 21:19):

a thing Ruby's rspec lets you do is to run tests by filename and line number, which I think would be nice

Brendan Hansknecht (Sep 07 2022 at 22:02):

Why would you want to run a test based on line number?

Brian Hicks (Sep 07 2022 at 22:20):

it's pretty handy to be able to focus only on a single test, or to retry with only the failing tests. By far the biggest win in rspec (for me at least) is being able to run only a single describe block at once. I don't think there's an analogous thing with expect.

Richard Feldman (Sep 07 2022 at 22:27):

the nice thing about running a test based on filename + line number is that it uniquely identifies the test, whereas sometimes test descriptions can be repeated across disparate tests in different files, leading to more tests being run than what you want

Brendan Hansknecht (Sep 07 2022 at 23:12):

I guess I am used to file/package/module/namespace name + test name or something of that nature

Richard Feldman (Sep 07 2022 at 23:25):

I want to preserve the invariant that tests don't have to have names, because it opens up design space for interesting alternative ways to specify tests (e.g. via tables)

Richard Feldman (Sep 07 2022 at 23:26):

also because I want to make writing new tests as low-friction as possible!

Brendan Hansknecht (Sep 07 2022 at 23:30):

I don't see tables as a contradiction to having names

Brendan Hansknecht (Sep 07 2022 at 23:30):

Though I guess it is some for of friction

Qqwy / Marten (Sep 08 2022 at 09:06):

Richard Feldman said:

so, quick note: you can add a comment over it and the comment will appear in the output if the test fails

:thinking: Do we want to overload normal comments for this? It feels a bit dirty (as in: abuse of notation).
Maybe it's nicer if they are required to be 'documentation' comments (that starts with a double ##)?

Richard Feldman (Sep 08 2022 at 13:14):

my thinking there is that doc comments should be for actual documentation, which means they should be written with a certain level of care - e.g. a doc comment like ## works with an empty list looks wrong to me, because that's not at the level I expect for documentation! Whereas a comment like # works with an empty list both looks fine, and also is the style I would want to encourage for tests

Richard Feldman (Sep 11 2022 at 14:57):

I just thought of a counterargument: what if I want to use a comment to explain the reasoning behind the way the expect is used the way it is, but I don't want that explanation to appear in the failure message?

e.g.

# we're using `== Ok {}` here so that if it's an `Err` instead,
# we'll see the contents of the `Err` in the failure message
expect taskResult == Ok {}

this seems like a good argument for using ## comments for failure messages and # comments for notes like this

Richard Feldman (Sep 11 2022 at 15:00):

anyone have thoughts on this?

Brian Carroll (Sep 11 2022 at 16:01):

I have a thought on it but I'm not sure if it's just purely pedantic or if it matters!
This is not really a comment at all! A comment has no semantic meaning, but in this case it does. So really it's a special string syntax that's only used with the expect keyword.
This kind of jars with me. It feels like it's an extra thing squeezed in to the language as a special case rather than fitting in nicely. Like "this language can have arbitrary rules in some places".

Qqwy / Marten (Sep 12 2022 at 18:31):

Very much agreed! You are able to explain it much better than I was in my previous message :happy:

J.Teeuwissen (Sep 12 2022 at 18:39):

Optionally it could be a separate package/tool like doctest for Haskell. This way the language stays "clean" but those who want tests in their comments can do so (in a way the package sees fit).

Last updated: Jun 16 2026 at 16:19 UTC