Stream: ideas

Topic: Roc cli workflow

Luke Boswell (Dec 07 2023 at 22:43):

It has been discussed a few times previously, I've skimmed back through zulip and it's not easy to make a summary as there are various conflicting ideas, even between tutorial and roc --help.

So here is my idea.

Developing an app using roc main.roc workflow:

• Runs using dev backends instead of llvm
• Ignores both errors and warnings
• Prints dbg values
• Checks inline expect assertions

Running an app with roc run main.roc workflow:

• Runs using llvm with some optimisation (--optimize is reserved for full opt level)
• Blocks and doesn't run with any errors, however will continue with warnings
• Ignores dbg and expect

Optional -- what if we removed roc dev in favour of just roc as it is somewhat redundant and only adds to confusion?

edit moved roc up to development workflow. Propose removing roc dev instead

Brendan Hansknecht (Dec 07 2023 at 22:47):

roc main.roc is roc dev main.roc

Brendan Hansknecht (Dec 07 2023 at 22:47):

personally I perfer that.

Brendan Hansknecht (Dec 07 2023 at 22:47):

The shortest command should be the equivalent of dev imo

Luke Boswell (Dec 07 2023 at 22:47):

Oh nice, I didn't know that

Luke Boswell (Dec 07 2023 at 22:48):

I'll edit my proposal above because I like that more... I hope this wont confuse anything. Just moving roc up to dev workflow. How would people feel about removing roc dev then instead?

timotree (Dec 07 2023 at 22:50):

What is the use case for the latter configuration with "some optimizations"? I only see the need for roc main.roc and roc --optimize main.roc. When developing I would always want a fast dev loop, I would want release optimizations otherwise. If some optimizations are needed to make the app usable (like with a game, for example), then I would want those enabled while developing as well.

Brendan Hansknecht (Dec 07 2023 at 22:51):

This is my thought.

I think we should remove both dev and run.
Instead:

• roc main.roc should do what roc dev main.roc currently does (long term runs with dev backends)
• roc main.roc --optimize should do what roc run --optimize main.roc currently does
• If you need optimized with dbg statements, that should be --optimize with an extra flag.

Also, we should add a general flag to pick llvm vs dev backend. Even if it is almost never used it will be useful for debugging.

Brendan Hansknecht (Dec 07 2023 at 22:55):

as a note, roc build should also keep around dbg statements without --optimize that is planned, just not implemtented yet.

Luke Boswell (Dec 07 2023 at 22:57):

Proposal 2 based on Brendan's idea.

Developing workflow roc main.roc:

• Runs using dev backends (llvm as a fallback until they are ready)
• Ignores both errors and warnings
• Prints dbg values
• Checks inline expect assertions

Running workflow roc main.roc --optimize:

• Runs using llvm full optimisation level
• Blocks and doesn't run with any errors, however will continue with warnings
• Ignores dbg and expect

Add a flag --backend llvm or --backend dev to assist with debugging, so you can use choose which backend to run.

Q: dev backend doesn't have any optimisation levels right?
Q: Can we also accept --optimise for those non-american english speakers?

Brian Carroll (Dec 07 2023 at 23:02):

Luke Boswell said:

Q: dev backend doesn't have any optimisation levels right?

That's right. It's designed to compile as quickly as possible. Optimisations take time, so we don't do any, and there's nothing to configure.

Richard Feldman (Dec 07 2023 at 23:14):

so here is an important consideration: if someone wants to use !#/usr/bin/env roc to mark a .roc file as executable, how should that work?

Richard Feldman (Dec 07 2023 at 23:15):

the use case there would be: someone has roc installed and I want to give them a script to run

Richard Feldman (Dec 07 2023 at 23:16):

so the end user isn't developing (e.g. they might not want expect to run) but they still care about (build time + execution time) because it's not a compiled binary

Richard Feldman (Dec 07 2023 at 23:20):

like for example should we have a roc script command?

Richard Feldman (Dec 07 2023 at 23:25):

previously I thought just plain roc would be best for that, since that's how it works in scripting languages (e.g. you just run python foo.py, not python script foo.py)

Richard Feldman (Dec 07 2023 at 23:25):

but having roc dev for development seems to be confusing in many cases

Richard Feldman (Dec 07 2023 at 23:27):

one idea to consider: what if roc with no subcommand was for scripts, but it looked for the #! at the start of the file (or maybe that the executable flag is set?), and if it's missing, it says "hey you might want one of these subcommands instead..."

Brendan Hansknecht (Dec 07 2023 at 23:50):

Thinking about how this works in python for example, all debug prints have to be manually removed before distributing

Brendan Hansknecht (Dec 07 2023 at 23:51):

So that is kinda a valid option, but yeah, a bit weird

Anton (Dec 08 2023 at 10:48):

Q: Can we also accept --optimise for those non-american english speakers?

Sounds good!

LoipesMas (Dec 08 2023 at 12:33):

I would find it confusing (or at least surprising), if roc main.roc would ignore errors. And even more so if roc main.roc --optimize didn't

Richard Feldman (Dec 08 2023 at 12:37):

errors as in failed expects?

LoipesMas (Dec 08 2023 at 12:45):

Build errors. If roc main.roc worked like roc run main.roc, it would panic at runtime, instead of failing at compile time

Brendan Hansknecht (Dec 08 2023 at 15:59):

Idk, I guess it depends on context. I like roc working as a scripting language.

Luke Boswell (Dec 08 2023 at 19:11):

It seems to me the best tradeoff here is to keep roc dev for developing, and roc for using programs.

Luke Boswell (Dec 08 2023 at 19:14):

I think the only reason it is confusing right now is because we have roc run, and the difference between them is the actual problem. If we define it more clearly, I dont think anyone will have any issues. Particularly if the default roc is strict with errors and slower compile time than using roc dev

Brendan Hansknecht (Dec 08 2023 at 19:17):

I think dbg if left in the main app should just run by default.

Brendan Hansknecht (Dec 08 2023 at 19:17):

And just use roc

Brendan Hansknecht (Dec 08 2023 at 19:18):

Just like and extra print in python or similar

Brendan Hansknecht (Dec 08 2023 at 19:18):

I think this is also important cause we have to consider build

Brendan Hansknecht (Dec 08 2023 at 19:18):

Would it be required to do build-dev to get dbgs and expects

Brendan Hansknecht (Dec 08 2023 at 19:19):

Also, a script probably wants to run all the expects anyway

Brendan Hansknecht (Dec 08 2023 at 19:19):

Just to catch errors and what not

Luke Boswell (Dec 08 2023 at 19:20):

And a script probably wants to compile fast, runtime perf is probably not that important

Luke Boswell (Dec 08 2023 at 19:26):

The main issue I see with not having dev is the default behaviour for when there are errors. I think it is helpful to be able to run an invalid program when developing.

Also the defaults for that workflow re optimisation and assertions expect are helpful to have separate from the non-dev workflow.

Brendan Hansknecht (Dec 08 2023 at 19:29):

I think any roc run related commands probably can just run with errors. We have build and check if you don't want to run with errors

Brendan Hansknecht (Dec 08 2023 at 19:30):

And we could add a flag to avoid that. Or if we want, reverse that and add a flag to enable it

Luke Boswell (Dec 08 2023 at 19:39):

Brendan Hansknecht said:

I think any roc run related commands probably can just run with errors. We have build and check if you don't want to run with errors

This seems like the root of our discussion.

I have just assumed that we always want valid programs, but when I think about it more I dont think that is always true.

Brendan Hansknecht (Dec 08 2023 at 19:50):

The big question is, what is the consequence of an invalid program and when do you want to ensure your program is valid

The consequence long term is that your program maybe crashes (today you might also crash or hang the compiler). It should only crash if the invalid path is hit during execution.

Brendan Hansknecht (Dec 08 2023 at 19:51):

As for when do you want to ensure a program is valid.

I think the roc dev workflow and languages like python show that while people do enjoy types, for development, running with minor errors is quite useful

Brendan Hansknecht (Dec 08 2023 at 19:52):

For released apps, I would hope they are either compiled roc build will always fail on error, or at least you/your ci runs roc check before release. Which again will ensure that the app is valid before being released.

LoipesMas (Dec 08 2023 at 20:40):

Personally, I want to only run valid programs. Maybe ignoring errors in unused functions would be ok, but I can also just comment them out. But perhaps I am in the minority here

Luke Boswell (Dec 08 2023 at 20:49):

@LoipesMas how would you feel about roc build main.roc && ./main?

I guess if you have the source, then you are also able to build/check the program is valid.

Elias Mulhall (Dec 08 2023 at 20:58):

Or even roc check main.roc && roc main.roc

Luke Boswell (Dec 08 2023 at 21:41):

Updated with added context from discussion above

Proposal 3

Run roc (modified)

• Run using dev backends by default (fallback to llvm without optimisations)
• Prioritise ease of use and productivity, panic/crash at run time
• Prints Errors and Warnings
• Prints dbg values
• Proceed to run an invalid program
• Check inline expect assertions and panic/crash if any fail
• Supports scripting use-case with !#/usr/bin/env roc
• Note roc run command removed as redundant

Build roc build (modified)

• Build an executable using llvm with full optimisations by default
• Prioritise safety and performance, panic/crash at compile time
• Print any Warnings
• Reports any Errors and will only build a valid program
• Ignores dbg and strips from binary
• Ignores expect and strips from binary

Check roc check (no change)

• Typecheck and report any Errors and Warnings

Test roc test (no change)

• Run all top-level expect tests

Format roc format (no change)
REPL roc repl (no change)
Version roc version (no change)
Docs roc docs (no change)
Glue roc glue (no change)

Flags

• --dev (removed)
• --optimize (removed)
• --backend <llvm/dev> (new) compile binary using llvm or dev backend instead of default
• --opt-none (new) compile without optimisations, llvm only
• --opt-size (no change) smaller binary, llvm only
• --profiling (no change) keep debug info in binary, llvm only
• --emit-llvm-ir (no change) emit llvm IR .ll file, llvm only
• --bundle (no change) archive a package
• --max-threads (no change) limit compiler threads
• --lib (no change) emit C library, dont link with platform
• --no-link (no change) emit an Object, dont link with platform
• --target (no change) compiler for an achitecture
• --time (no change) display time information
• --linker (no change) surgical or legace linker
• --prebuilt-platform (no change) don't rebuild platform (remove in future, platform responsibility)
• --check (no change) typecheck and report any errors
• --stdin (no change) read input from stdio
• --stdout (no change) output formatted file to stdio
• --wasm-stack-size-kb (no change) set WASM stack size
• --output (no change) specify output path

LoipesMas (Dec 08 2023 at 22:05):

Luke Boswell said:

LoipesMas how would you feel about roc build main.roc && ./main?

I guess if you have the source, then you are also able to build/check the program is valid.

A flag would be nicer (for example --strict), but build/check && run will do. I'll just make an alias and forget about it

Brendan Hansknecht (Dec 08 2023 at 22:10):

To clarify part of my opinion, I would be totally fine if roc did not run by default with errors. I would just put running with errors behind a flag in that case. This flag would be separate from anything dbg or expect related

Brendan Hansknecht (Dec 08 2023 at 22:16):

In the proposal, why is --optimize removed?

I don't think our users should need to think about whether or not they are using llvm or something else by default. So I would leave --optimize in, remove --opt-none, but still add --backend <llvm/dev> to enable us compiled engineers to switch for debugging purposes.

Luke Boswell (Dec 08 2023 at 22:49):

I switched it around based on prioritising perf in the build workflow. So the idea was that by default llvm built with full optimisation on, and then you opt out.

Luke Boswell (Dec 08 2023 at 23:10):

I also tossed up renaming --optimize to --opt-fast, maybe we add that back. It makes sense to be able to use when doing roc --opt-fast --backend llvm main.roc

Brian Carroll (Dec 08 2023 at 23:13):

I don't like --opt-fast because I don't know whether the fast refers to compile time or run time

Brian Carroll (Dec 08 2023 at 23:21):

I'm also not keen on "backend" because that is an implementation detail of the compiler.

Brendan Hansknecht (Dec 08 2023 at 23:23):

Maybe backend should only be exposed in debug builds as a ROC_BACKEND flag?

Brendan Hansknecht (Dec 08 2023 at 23:24):

keep it out of the cli

Declan Joseph Maguire (Dec 09 2023 at 03:08):

Brian Carroll said:

I don't like --opt-fast because I don't know whether the fast refers to compile time or run time

Maybe --opt-speed instead? I don't think I'd get confused in that case like I might with --opt-fast.

Brian Carroll (Dec 09 2023 at 10:06):

Brendan Hansknecht said:

Maybe backend should only be exposed in debug builds as a ROC_BACKEND flag?

That sounds better. But it's adding more logic to maintain and might make it more likely we'll test situations that don't match what app devs will see.
What's the drawback of compiler devs just using the same flags as app devs? Always worked ok for me, but maybe it's different for stuff you work on.

Brendan Hansknecht (Dec 09 2023 at 17:29):

I mostly think it would be useful to quickly tell if a bug is from the dev backend itself or something higher up the stack. But yeah, may not be needed.

Brendan Hansknecht (Dec 09 2023 at 17:30):

I say lets just assume we won't expose that for now and if in debugging someone need the environment varible, they can add it.

Eli Dowling (Dec 13 2023 at 23:05):

I feel like this is a lot of these options are complexity that is meaningless to most people most of the time and having a few sensible defaults would be lovely.

Eg: When I give a built version of my app to someone I'd like to just run roc build --release and it just does all the usual release things, no debug stuff, llvm backend, optimise.
When I test my app I'd want it to be the same so roc --release runs with the same flags

This has an additional benefit: If you add a feature later that is opt in, that you believe should always go into release applications you just update what the --release flag does. Not everyone is going to read the release notes for new versions and without this they just miss out on these features.

Luke Boswell (Dec 13 2023 at 23:42):

Thank you for those who have commented and provided feedback. I have updated my proposal for further consideration.

Proposal 4

Run roc (modified from current implementation)

• Prioritise ease of use and productivity, panic/crash at run time
• Run using dev backends by default (fallback to llvm without optimisations)
• Prints Errors and Warnings
• Prints dbg values
• Proceed to run an invalid program
• Check inline expect assertions and panic/crash if any fail
• Supports scripting use-case with !#/usr/bin/env roc
• Note roc run and roc dev commands removed as redundant

Build roc build (modified from current implementation)

• Prioritise safety and performance, panic/crash at compile time
• Build an executable using llvm with full optimisations by default
• Print any Warnings
• Reports any Errors and will only build a valid program
• Ignores dbg and strips from binary
• Ignores expect and strips from binary

Flags

• --backend <llvm/dev> (new flag) compile binary using llvm or dev backend instead of default
• --opt-none (new flag) compile without optimisations, llvm only
• --opt-size (no change) smaller binary, llvm only
• --opt-speed (renamed from --optimize) compile with optimisations, llvm only

Some comments from discussion above

I don't think our users should need to think about whether or not they are using llvm or something else by default.

Agree. The approach I proposing is to distinguish between two defaults by a guiding principal; either "Ease of use and Productive" or "Safe and Performant".

Most users should then only have choose between one of these based on their desired outcome.

Ease of use and Productive is chosen for the most common use case roc and to simplify the user experience. This supports scripts that do not require strict defaults. This supports development workflows where the program may not yet be completed, and as a result there is no need for a sub-command for e.g. roc dev vs roc run.

Maybe --opt-speed instead? I don't think I'd get confused in that case like I might with --opt-fast.

I like that this works well with opt-none and opt-size.

these options are complexity that is meaningless to most people most of the time and having a few sensible defaults would be lovely.

Agree, good defaults is how I propose to address this. The other flags are there to support specific needs such as debugging and won't be needed most of the time. I've included them here to ensure we are still supporting the full range of user needs.

But it's adding more logic to maintain and might make it more likely we'll test situations that don't match what app devs will see. What's the drawback of compiler devs just using the same flags as app devs?

I didn't quite follow this discussion. I'm not sure if the preferred option was to include or not to include a --backend flag. I think it is helpful to include it for release builds too, so I have left it in my updated proposal.

Brendan Hansknecht (Dec 14 2023 at 00:11):

All sounds good except to me if we just remove --backend. Backend will automatically be selected based on the --opt-* param.

If we need backend selection for compiler debuging purposes, we will add an environment variable for it. No need to expose it on CLI.

Brendan Hansknecht (Dec 14 2023 at 00:12):

Oh, one other thought, since it is only valid to pick one of --opt-size, --opt-none, and --opt-speed, maybe we should merge them to --opt <none/size/speed>

Isaac Van Doren (Dec 14 2023 at 14:11):

I pretty much always want errors to be reported at compile time. I like that the current behavior of the ‘roc’ command will either report errors or run the program if it is okay. That way I don’t have to switch between using roc check and roc. So I would miss losing that ability if the new roc command reports errors at runtime.

Brendan Hansknecht (Dec 14 2023 at 15:31):

Just to clarify one thing, it will always report the error in both cases. It would just execute anyway if the errors are small enough that we can make an executable.

For example, if you make a when .. is with a missing branch. it will report a warning, It would still run. The program may even complete successful, but either way, you can deal with the missing branch later.

Isaac Van Doren (Dec 14 2023 at 15:36):

Oh okay cool

Luke Boswell (Jan 11 2024 at 02:49):

I've ben thinking about this proposal, could we consider this accepted? can I create an issue to track this discussion and these changes? I am happy to update proposal and continue discussion if there are other things we want to consider

Richard Feldman (Jan 11 2024 at 02:52):

I'd like to introduce --watch (I have that almost working on a branch) and see how that affects things

Richard Feldman (Jan 11 2024 at 02:52):

because I suspect roc dev might want to automatically watch things, and I think that will affect what behavior makes sense for the other commands

Richard Feldman (Jan 11 2024 at 02:52):

e.g. if people normally just leave roc dev running, does that mean roc (no subcommand) should actually prioritize development?

Luke Boswell (Jan 11 2024 at 03:10):

That aligns nicely with the above proposal;
roc development ease of use and productivity (including --watch)
roc build produce safe and performant code

Luke Boswell (Apr 11 2024 at 01:46):

@Richard Feldman I've been thinking about this proposal, just wondering if it is worth documenting in an Issue. Are we happy with this approach?

Richard Feldman (Apr 11 2024 at 01:48):

sure! :+1:

Luke Boswell (Apr 11 2024 at 02:21):

Added #6637

Richard Feldman (Jul 12 2024 at 16:08):

I've been thinking about this a bunch; I have some more design thoughts, and some unanswered design questions :sweat_smile:

Richard Feldman (Jul 12 2024 at 16:12):

Always keep dbg

I think there are always going to be situations where you want dbg, so none of the CLI commands should result in dbgs getting dropped.

For example, at Vendr we always did roc build because we were compiling .roc files to be called from an existing Node codebase. So if roc build doesn't keep dbgs, then we just would not have been able to use dbg at all.

At my current job, I sometimes run (our Rust project) with --optimize when I'm working on something where the program that will grind to a halt if I run it without optimizations, and I won't even be able to work on the thing I'm trying to do. Obviously in those situations, even though I'm not building for a release, I still want dbg access - because it's a "debug build," just one with optimizations.

Unanswered design question: how can we help you identify when you left stray dbgs in your code base, so that you don't end up accidentally shipping them to production when you didn't want to?

Richard Feldman (Jul 12 2024 at 16:17):

expects are ignored iff optimizations are enabled

Basically, decouple expect from subcommands.

• If you run with optimizations enabled, then expects are optimized out.
• If you don't enable optimizations, then expect will work as intended.

It's an essential part of expect's design that they get optimized out, so you don't have to weigh any production tradeoffs of sprinkling them around the code base. You can always rest assured that they will be completely optimized away. As long as that's true, then yeah, let's run them.

Unanswered design question: Given the reality that sometimes you need to run with --optimize when doing local development, should there be a way to opt into "optimize, but keep expects around because this isn't a production build" or something like that? Keep in mind the tradeoff that if this exists, it's possible that adding back in the expects makes the local builds too slow for the use case, even with optimizations enabled, at which point performance of expects becomes a thing you have to think about more.

Richard Feldman (Jul 12 2024 at 16:34):

Minimal number of subcommands

We've had a problem with people not being sure which subcommands to use. I propose that this is the minimal set of subcommands we could support:

• roc check checks the project for errors and reports them, but doesn't build a binary
• roc build builds a binary but doesn't run it
• roc builds the binary and runs it

That's it. Concretely, this means no more roc dev or roc run subcommands.

Richard Feldman (Jul 12 2024 at 16:34):

roc foo.roc

This builds the foo.roc application into a binary in a tempdir (or in memory on Linux) and runs it. It would use the dev backend if available, but would not bother generating debuginfo (assuming we do that in the future) because you're obviously not attaching a debugger to a file that only exists for the duration of the run. This is also the desired build configuration when being run via #!/bin/env roc - that is, dev backend and no debuginfo generated.

In the future, I would like this to always run even if there are errors - like roc run does today. However, I think today it should work like roc dev and only run if there are no errors.

Years ago, before we had roc dev, there was a common complaint that the desired workflow was "I really only want to run if there are no errors." At first we recommended roc check && roc run but that felt clunky so we introduced roc dev.

One thing that's changed since then is that now we have a language server. In a lot of cases, when you go to run from the command line, you already know there are no errors because your editor told you about them and you already fixed them before you even thought about running.

One thing that has not changed is the other complaint: "run even though there are errors" sometimes works, but often crashes the compiler - without having had a chance to print out problems like type mismatches first. So you don't even have a way to tell that the reason for the crash was the "run even though there are errors" failing; the compiler crashed and you don't know why.

This is why I think we should have the behavior work differently in the short term vs the long term. Until we get the "run even though there are errors" design working, I think we shouldn't even offer it as a subcommand or a feature flag. For now it should probably be an environment variable like ROC_EXPERIMENTAL_DEFER_ERRORS=1 where the name conveys that this is something that's currently unreliable.

Once it becomes reliable, I think we can just always enable it. Between editor integrations and having it actually work reliably, I don't think there should be any remaining reasons not to have it on all the time - and that design certainly results in the simplest CLI experience.

Richard Feldman (Jul 12 2024 at 16:34):

roc check foo.roc

Works the same as today.

Richard Feldman (Jul 12 2024 at 16:35):

roc build foo.roc

Works the same as today.

Richard Feldman (Jul 12 2024 at 16:37):

No --watch flag needed

I don't think we actually want a --watch flag. Rather, if the platform specifies that it supports "watch mode" then it's just automatically activated.

For example, webservers in interpreted languages don't have a "mode" where changes to source code files aren't immediately reflected in the running program. Why would you even want that? Of course if I make changes to my files when my program is running, I'd like the program to update in realtime if that's possible - otherwise I would have stopped the program before making the changes!

Richard Feldman (Jul 12 2024 at 16:38):

anyway, so all of that is my current thinking

Richard Feldman (Jul 12 2024 at 16:38):

we've talked about having a "release" concept as distinct from "optimize" - which seems worth at least exploring in the context of the two unanswered design questions above

Richard Feldman (Jul 12 2024 at 16:41):

but overall, things I like about this general design is that:

• it minimizes the number of subcommands to learn
• running Roc programs works more similarly to how it does in other languages (e.g. if you're used to python3 webserver.py, then roc webserver.roc will probably do about what you'd expect, including fast startup time and changes to files being reflected in the running program automatically without having to rebuild or pass a flag)
• there are fewer edge cases to learn/trip over (e.g. expect is coupled to optimizations, not to subcommands, dbg is always enabled)

Richard Feldman (Jul 12 2024 at 16:41):

any thoughts on any of that welcome!

Anton (Jul 12 2024 at 17:07):

if the platform specifies that it supports "watch mode" then it's just automatically activated.

Inexperienced users may unknowingly run a production webserver with roc file.roc instead of running the binary produced by roc build. They could edit some roc files, introduce a type mismatch and bring the server down without knowing it, because it is in watch mode.

Richard Feldman (Jul 12 2024 at 17:51):

hm, I suppose - but I don't really hear about this being a problem in practice with interpreted languages :thinking:

Sam Mohr (Jul 12 2024 at 19:27):

I think this is somewhere we can take inspiration from the node ecosystem a la NODE_ENV. A --deploy-env=dev,prod flag could accompany roc build and roc that determines if dbgs and expects are included. We would default to prod when doing roc build, and dev for roc.

Sam Mohr (Jul 12 2024 at 19:28):

This is orthogonal to the last proposal I saw for the compilation target, which last I saw was --backend=dev,llvm,wasm.

Sam Mohr (Jul 12 2024 at 19:29):

Since these behaviors (whether to keep debug tools and how much to optimize) are orthogonal to each other, they need to be governed by two different things IMO

Jasper Woudenberg (Jul 12 2024 at 19:59):

Would it make sense to combine rock check and roc test as well? I group them together in my head, as "give me all the automated feedback", and often write ./test.sh scripts for projects (in other compiled languages) that run one after the other. Combining the check and test output in an intuitive way might be tricky though. Curious what folks think.

Brendan Hansknecht (Jul 12 2024 at 22:24):

Personally, I think roc check shouldn't exist. It should be part of all other commands. The fact that roc check can generate errors that roc build misses and instead crashes for is a bug. Currently, I see roc check as a workaround to get better error messages quickly.

Sam Mohr (Jul 12 2024 at 23:16):

Though it would be nice to not need Roc check, there are two things we really need it for at the moment:

• Helix truncates error messages that are too long to fit in the top of the window, so I can't see the useful content of the error 2/3rds of the time, meaning I need to run roc check | less to see everything.
• In a CI environment, you may just want to type check to avoid testing or building

Sam Mohr (Jul 12 2024 at 23:18):

Yeah, it would be nice to figure out some way to make it so that error messages are better shown in Helix. Maybe I mess with that at some point.

Luke Boswell (Jul 12 2024 at 23:18):

Could that be a flag on roc test --no-build?

Sam Mohr (Jul 12 2024 at 23:19):

We're now changing the meaning of test from "run unit/integration tests to see if the program works correctly" to "check if the program is correct"

Sam Mohr (Jul 12 2024 at 23:19):

And no build is misleading, since it really means don't run unit tests AFAICT

Luke Boswell (Jul 12 2024 at 23:23):

roc build --check-only

Sam Mohr (Jul 12 2024 at 23:26):

Yeah, maybe, but that just shows that we want to preserve roc check but we'd rather have fewer subcommands with more options available.

Sam Mohr (Jul 12 2024 at 23:26):

I appreciate the brainstorming

Sam Mohr (Jul 12 2024 at 23:27):

But I think roc check has long-term value, at least for CI/CD. @Brendan Hansknecht you're the original proponent in this thread of removing the command. Do you think the two reasons above I gave are not worth having a check command?

Brendan Hansknecht (Jul 12 2024 at 23:30):

If roc test catches everything that roc check generates, not really sure why it is needed.

Brendan Hansknecht (Jul 12 2024 at 23:30):

Though some languages have a check like feature, I feel like have never used one in practice

Brendan Hansknecht (Jul 12 2024 at 23:30):

Just build, run, or test

Brendan Hansknecht (Jul 12 2024 at 23:31):

Build and test both report everything that check would report

Sam Mohr (Jul 12 2024 at 23:31):

I use cargo check when developing if the error is complex and my LSP diagnostics are confusing, so it may be easier to read elsewhere.

Sam Mohr (Jul 12 2024 at 23:33):

I think if we discount the Helix issue I have, which @Kiryl Dziamura sent me a link for a PR that could mitigate the issue, then I think the CI/CD use-case is the only reason I'd want to still have it.

Sam Mohr (Jul 12 2024 at 23:34):

What if someone has a lot of tests? Even though running unit tests is good practice in validating software quality, do we want to force them to always run their tests?

Kiryl Dziamura (Jul 12 2024 at 23:38):

CI/CD is a valid case I think. Checking without producing binaries can potentially save both wallclock and billed CPU time

Brendan Hansknecht (Jul 12 2024 at 23:41):

I guess in my idea world, test in ci would be equivalent to a blocking check followed by test.

Brendan Hansknecht (Jul 12 2024 at 23:41):

So if check has a error, it would fail fast.

Brendan Hansknecht (Jul 12 2024 at 23:41):

Same with build in ci

Brendan Hansknecht (Jul 12 2024 at 23:41):

No need for a separate command

Sam Mohr (Jul 12 2024 at 23:44):

I agree that it should fail fast (roc test shouldn't run tests if there are errors), but they may want to avoid running tests if there are no errors. What if users are paying for CPU time?

Sam Mohr (Jul 12 2024 at 23:44):

However, I do really like simple tools, and we've already done the "add it when people complain" strategy

Brendan Hansknecht (Jul 12 2024 at 23:45):

Anyway, no harm in keeping it, but I really hate that it gives better errors than build and test

Sam Mohr (Jul 12 2024 at 23:45):

So I'd personally be okay with removing the roc check command for now until people yell and scream

Sam Mohr (Jul 12 2024 at 23:45):

Or maybe slightly before that point

Notification Bot (Jul 12 2024 at 23:51):

A message was moved here from #ideas > Roc debugging GUI by Brendan Hansknecht.

Richard Feldman (Jul 12 2024 at 23:59):

Brendan Hansknecht said:

Anyway, no harm in keeping it, but I really hate that it gives better errors than build and test

for context, the reason for this is that each stage in the compiler pipeline runs, gathers errors for later, and then moves on to the next stage. The final stage is reporting, which prints out all the accumulated errors from the previous stages, plus the total number of errors and the total build time.

we have some bugs in the building step that crash the compiler. That crash prevents the compiler from reaching the reporting stage, which means no errors get printed at all, including errors from any of the previous stages get printed.

Richard Feldman (Jul 13 2024 at 00:03):

we could print them earlier in the process, since once we finish type checking, there are no more errors to report

Richard Feldman (Jul 13 2024 at 00:04):

if we wanted to get fancy, we could print everything but the in ___ ms part to stdout without the newline, and then do that last part at the very end of the build

Richard Feldman (Jul 13 2024 at 00:05):

I guess really that would mean moving reporting to before (really in parallel with, if we like) monomorphization

Brendan Hansknecht (Jul 13 2024 at 00:07):

I had forgotten that context. Makes sense.

Matthieu Pizenberg (Jul 13 2024 at 09:13):

Not having to build or test is really convenient when coding on a lightweight machine. Especially in a language like Roc where passing typechecking usually means the program is correct and I can continue working on it without the computer being slow or drained of its battery.

Anton (Jul 13 2024 at 09:16):

but I don't really hear about this being a problem in practice with interpreted languages

People may feel like it's a silly mistake so they may not bring it up often :p

Agus Zubiaga (Jul 14 2024 at 11:25):

Brendan Hansknecht said:

I guess in my idea world, test in ci would be equivalent to a blocking check followed by test.

That seems like a good default, but ideally we should also allow running tests with errors in case you’re in the middle of a refactor and want to see if you got some part right.

David Edwards (Oct 21 2024 at 20:57):

:wave: Hello, I am looking at picking this up and @Luke Boswell suggested that it might be good to coordinate a bit here. It's quite a lengthy thread that I've had a relatively quick read through but _roughly_, I think we need to:

1. take @Luke Boswell 's latest proposal which is what is currently on the Github issue and update it with the post 11th July feedback, specifically incorporating in @Richard Feldman 's feedback from July 12th and the subsequent discussion. Essentially, summarise that, pull out any open questions and get it clarified on the Github issue. Part of that is laying out what is a "now" thing and what is a later thing, both it terms of agreed features and questions (do we have to answer them now or can we later etc).
2. Work out some sensible chunks on introducing the changes (ideally so that we can ship bits at a time)
3. Actually, err, do the work :laughing:

If that sounds good I can start having a look over the next few days :thumbs_up:

Richard Feldman (Oct 21 2024 at 21:03):

that sounds good to me!

Luke Boswell (Oct 21 2024 at 21:03):

That sounds great.

Sorry I meant to look at this, but have been spending my time polishing the roc-ray platform :sweat_smile:

I'd add to the above, if we do anything it should probably be on a branch based off the rebuild host PR. Hopefully that will land in main soon, but it has a bunch of things that will make life easier. For one the build pipeline is much easier to follow.

Richard Feldman (Oct 21 2024 at 21:15):

let's start with the issue though, to make sure we're all on the same page with where we want to end up!

David Edwards (Oct 21 2024 at 21:19):

:+1:

Sam Mohr (Oct 21 2024 at 21:52):

Thanks for looking into this David, it'll be great to polish up the first point of interaction people have with running the compiler!

David Edwards (Oct 27 2024 at 20:19):

Just to let people know, I had intended to look at this over the last week but was a bit under the weather so didn't have any time :sick: Hopefully this week though!

Luke Boswell (Oct 27 2024 at 20:20):

No worries, thanks for the update. Hope you're feeling better soon :smile:

David Edwards (Nov 07 2024 at 17:40):

Okay, I _finally_ found some time to read through this thread properly. I am basically going to repeat a bunch of what has been said above but hopefully it helps make it clearer in my head :confounded: So what we are aiming for:

• roc foo.roc builds a binary into /tmp (or TMPDIR I guess?) and then runs it.
  This will run only if there are no errors. This can be controlled through an env variable, called ROC_EXPERIMENTAL_DEFER_ERRORS (do we have other examples of using env variables like that?) This also works with "scripts" starting with the #!/bin/env roc shebang.

• roc build foo.roc builds into the current directory as it does today.

For all the above we have the following configurables:

• Optimizations are available through --opt <none/size/speed> Based on this, inline expects are kept or removed. With none they are kept, other cases removed. For roc the default is none, is that okay for roc build?
• dbgs are kept or be removed by setting a flag --dbgs <keep/rm>. For roc the default is keepfor build it is rm? A potential reason to keep dbgs in build is that if you are doing interop then you always need to build but might still need the dbgs. Do we actually want it to be an option to remove dbgs with roc`?
• compiler backend is selected by the opt flag? (I think I could do with a pointer towards what available options for the backend are and what the differences are!)

I have assumed here that for all the configurable options we want to be able to control them with flags but have different defaults on the commands (for example in the Node.js platform example it is useful to keep dbgs in build etc)

Richard Feldman (Nov 07 2024 at 19:29):

great, thanks for posting this! :smiley:

David Edwards said:

roc foo.roc builds a binary into /tmp (or TMPDIR I guess?) and then runs it.

yeah, we already have this ExecutableFile logic for that - on Linux we don't even need to make a tempdir because you can build the binary in-memory and run it, but on the other targets we put it in a tempdir.

Richard Feldman (Nov 07 2024 at 19:31):

David Edwards said:

This will run only if there are no errors. This can be controlled through an env variable, called ROC_EXPERIMENTAL_DEFER_ERRORS (do we have other examples of using env variables like that?) This also works with "scripts" starting with the #!/bin/env roc shebang.

an existing example would be the ROC_DEV_WRITE_OBJ env var.

Note that no special action needs to be taken for #!/bin/env roc to work; as long as roc accepts a foo.roc file as an argument and does what's described here, that will Just Work automatically!

Richard Feldman (Nov 07 2024 at 19:32):

David Edwards said:

Optimizations are available through --opt <none/size/speed> Based on this, inline expects are kept or removed. With none they are kept, other cases removed. For roc the default is none, is that okay for roc build?

yep! :thumbs_up:

Richard Feldman (Nov 07 2024 at 19:34):

David Edwards said:

dbgs are kept or be removed by setting a flag --dbgs <keep/rm>. For roc the default is keepfor build it is rm? A potential reason to keep dbgs in build is that if you are doing interop then you always need to build but might still need the dbgs. Do we actually want it to be an option to remove dbgs with roc`?

I think for now let's just always keep them. We've been talking about a future concept of "release" (maybe a subcommand, maybe a flag, hasn't really been discussed in depth) which might have something to say about dbg, but for now I think we should just always keep them

Richard Feldman (Nov 07 2024 at 19:36):

David Edwards said:

• compiler backend is selected by the opt flag? (I think I could do with a pointer towards what available options for the backend are and what the differences are!)

that's correct - basically the way this works is:

• if optimization is either size or speed, then we use the llvm backend
• if optimization is none, then we use a "dev backend" - which are target-specific. we have a wasm32 dev backend, an x86-64 dev backend, and an arm64 dev backend.

Richard Feldman (Nov 07 2024 at 19:38):

also just to note that we also want:

• roc check - works the same way as today
• roc format - works the same way as today
• roc test - works the same way as today, although the --opt flag should apply to it

Luke Boswell (Nov 07 2024 at 20:00):

Richard Feldman said:

David Edwards said:

• compiler backend is selected by the opt flag? (I think I could do with a pointer towards what available options for the backend are and what the differences are!)

that's correct - basically the way this works is:

• if optimization is either size or speed, then we use the llvm backend
• if optimization is none, then we use a "dev backend" - which are target-specific. we have a wasm32 dev backend, an x86-64 dev backend, and an arm64 dev backend.

I think Linux is the only one that currently uses the dev backend by default. The others you need to pass --dev to use them currently. This is because they're missing things. I'm not sure how far from using them by default for everything we are.

Jasper Woudenberg (Nov 07 2024 at 20:29):

Oh interesting! When I've been running roc using a shebang or using roc run on linux, it consistantly generates a binary next to the .roc file I'm running. Say I'm running main.roc, then I get a main binary. I've been thinking it's not so nice if a script generates files in its directory every time you run it and it was on my list to take a look at generating it in a tempdir instead. But if I understand correctly that binary shouldn't really be there in the first place :thinking:.

Richard Feldman (Nov 07 2024 at 20:34):

Luke Boswell said:

Richard Feldman said:

David Edwards said:

• compiler backend is selected by the opt flag? (I think I could do with a pointer towards what available options for the backend are and what the differences are!)

that's correct - basically the way this works is:

• if optimization is either size or speed, then we use the llvm backend
• if optimization is none, then we use a "dev backend" - which are target-specific. we have a wasm32 dev backend, an x86-64 dev backend, and an arm64 dev backend.

I think Linux is the only one that currently uses the dev backend by default. The others you need to pass --dev to use them currently. This is because they're missing things. I'm not sure how far from using them by default for everything we are.

oops, yeah that's correct - I guess I was describing how we'd like things to be rather than how they are :sweat_smile:

Richard Feldman (Nov 07 2024 at 20:35):

I think macOS uses the dev backend in the repl but not in builds maybe? I don't think there are any blockers to using it in builds

Luke Boswell (Nov 07 2024 at 20:47):

Yeah it works for macos and everything I think, just if you happen to be using something in the app or platform that isn't implemented it breaks. I can fiddle with it later to see how big the gap is.

Anton (Nov 08 2024 at 10:55):

I think Linux is the only one that currently uses the dev backend by default.

I'm not completely sure but I don't think it does

Luke Boswell (Nov 08 2024 at 10:59):

You can test it, is there any difference adding --dev compared to normal? I forgot to test this today.

Anton (Nov 08 2024 at 11:05):

Yeah, --dev does not work:

**❯ ./target/release/roc crates/cli/tests/benchmarks/nQueens.roc --dev
🔨 Rebuilding platform...
Error:

    Function, roc__mainForHost_0_caller, was not defined by the app.

Potential causes:

    - because the platform was built with a non-compatible version of roc compared to the one you are running.

        solutions:
            + Downgrade your roc version to the one that was used to build the platform.
            + Or ask the platform author to release a new version of the platform using a current roc release.

    - This can also occur due to a bug in the compiler. In that case, file an issue here: https://github.com/roc-lang/roc/issues/new/choose
**

Last updated: Jun 16 2026 at 16:19 UTC