Stream: API design

Topic: Stdin.line

Richard Feldman (Apr 21 2024 at 13:38):

so awhile back we changed Stdin.line to produce [Input Str, End]

Richard Feldman (Apr 21 2024 at 13:39):

this makes the function a lot less helpful for learning, because this doesn't work:

main =
    Stdout.line! "Type something and press Enter."
    input = Stdin.line!
    Stdout.line! "You entered: $(input)"

Richard Feldman (Apr 21 2024 at 13:39):

I wonder if we should revisit the original problem and reonsider the API

Richard Feldman (Apr 21 2024 at 13:40):

specifically, I'm wondering if we should solve it with two functions

Richard Feldman (Apr 21 2024 at 13:40):

Stdin.line goes back to producing a Str and then we have a different Stdin function for reading until EOT

Richard Feldman (Apr 21 2024 at 13:40):

thoughts?

Richard Feldman (Apr 21 2024 at 14:12):

as a concrete design idea:

## Reads bytes from stdin until a '\n'
## byte is encountered, then parses
## all the bytes before it as utf8
Stdin.line : Task Str *

## Reads bytes from stdin until an EOT
## byte is encountered, then parses
## all the bytes before it as utf8
Stdin.eot : Task Str *

Isaac Van Doren (Apr 21 2024 at 14:37):

That sounds great!

Brendan Hansknecht (Apr 21 2024 at 14:54):

I don't think that works. You often have to read in small chunks (like lines), but you also need to know when the stdin stream ends

Brendan Hansknecht (Apr 21 2024 at 14:54):

I think you fundamentally just want the end of the stream to be seen as exceptional and ignore in the normal case

Richard Feldman (Apr 21 2024 at 14:57):

ok so then we could rename the current function to like Stdin.lineOrEot or something?

Richard Feldman (Apr 21 2024 at 14:57):

and then reintroduce the previous Stdin.line design

Brendan Hansknecht (Apr 21 2024 at 14:57):

So I would propose:

Stdin.line Str [StreamEnd]

So you can do either;

# dont care
In = Stdin.line!
Stdout.line! "You entered: $(in)"

# do care
InRes = Stdin.line |> Task.result!
when inRes is
    ...

Brendan Hansknecht (Apr 21 2024 at 14:57):

Probably with a better error message, but that general idea

Richard Feldman (Apr 21 2024 at 14:58):

to me, the main motivation of this change is to have Task Str * as a beginner learning tool

Richard Feldman (Apr 21 2024 at 14:58):

I don't think we want to have to teach error handling that early

Richard Feldman (Apr 21 2024 at 14:58):

or Result

Brendan Hansknecht (Apr 21 2024 at 15:00):

I strongly think Stdin.line should do the right thing and have an error type. (Or just return an union directly)

We should be explicit if we make a version that crashes or just keeps returning empty input. That version should have the longer name
Stdin.lineUnchecked: ... or etc

Brendan Hansknecht (Apr 21 2024 at 15:01):

Also, you can hide the error handling by making the platform handle it by default

Brendan Hansknecht (Apr 21 2024 at 15:01):

Allow any error to be returned to the platform

Brendan Hansknecht (Apr 21 2024 at 15:01):

It can print the error (will be a low quality message) and crash

Brendan Hansknecht (Apr 21 2024 at 15:02):

I think that would be a better way to delay introducing error handling

Brendan Hansknecht (Apr 21 2024 at 15:03):

Stdout.line also should really return an error. Currently I think every call to stdout in basic CLI has 2 different ways to panic in rust. That is bad ux. Random crashes that the roc developer can't control.

Brendan Hansknecht (Apr 21 2024 at 15:04):

We don't have some form of try catch for platform panics. So any errors not given to roc are either fatal or silent without any way to change that from within roc.

Richard Feldman (Apr 21 2024 at 15:08):

:thinking: why would that be true of Stdin.line but not Stdout.line?

Richard Feldman (Apr 21 2024 at 15:08):

(which can also fail)

Brendan Hansknecht (Apr 21 2024 at 15:09):

It's true for both

Brendan Hansknecht (Apr 21 2024 at 15:09):

I think both should return errors to roc

Brendan Hansknecht (Apr 21 2024 at 15:09):

! lets you easily ignore them for the most part

Brendan Hansknecht (Apr 21 2024 at 15:09):

So the main code will still read just like a imperative program

Brendan Hansknecht (Apr 21 2024 at 15:09):

Just will accumulate the errors.

Richard Feldman (Apr 21 2024 at 15:10):

ok so what does hello world in Roc look like in that world?

Brendan Hansknecht (Apr 21 2024 at 15:10):

main =
    Stdout.line! "Type something and press Enter."
    input = Stdin.line!
    Stdout.line! "You entered: $(input)"

Richard Feldman (Apr 21 2024 at 15:11):

that's a type mismatch because the errors aren't handled

Brendan Hansknecht (Apr 21 2024 at 15:11):

Nope...one sec

Brendan Hansknecht (Apr 21 2024 at 15:14):

mainForHost : Task {} [] as Fx
mainForHost =
    res = main |> Task.result!
    when res is
        Ok {} -> Task.ok {}
        Err e ->
            # This could call an unsafe stderr and exit, or just use crash, or etc
            crash "Program exited with error: $(Inspect.toStr e)"

Brendan Hansknecht (Apr 21 2024 at 15:15):

So main can return any sort of error to basic cli and it will print out an crash.

Brendan Hansknecht (Apr 21 2024 at 15:16):

Of course, main can also choose to send no errors. In that case, it can handle there errors in a nicer way

Brendan Hansknecht (Apr 21 2024 at 15:16):

Now everything in main has the choice to use ! and never think about error handling if wanted

Brendan Hansknecht (Apr 21 2024 at 15:19):

Then we can have:

Stdout.line: Str -> Task {} [StdoutClosed]
Stderr.line: Str -> Task {} [StderrClosed]
Stdin.line: Task Str [StdinClosed, StdinEndOfStream]

And the end user can just write your original program with !.
Instead of maybe getting some random rust panic, They would get a roc crash (we can pick the exact behaviour) with the message: Program exited with error: StdinEndOfStream.

Richard Feldman (Apr 21 2024 at 15:19):

ahh right, I remember this design!

Richard Feldman (Apr 21 2024 at 15:19):

we talked about it once, but we didn't have Inspect back then

Richard Feldman (Apr 21 2024 at 15:19):

so now it seems more viable :+1:

Richard Feldman (Apr 21 2024 at 15:22):

and then if you want to guarantee you've handled all the errors in your application, all you have to do is to add this type annotation to your main:

main : Task {} []

Richard Feldman (Apr 21 2024 at 15:23):

and then you'll get a type mismatch if any are unhandled

Richard Feldman (Apr 21 2024 at 15:24):

but if you leave off the annotation (e.g. in tutorials) or annotate it as main : Task {} _ then it will work

Richard Feldman (Apr 21 2024 at 15:24):

hm, but in that design how do you specify exit code? :thinking:

Brendan Hansknecht (Apr 21 2024 at 15:24):

personally, I would pull the I32 out of the api and require calling Program.exit exitCode or something during the error handling.

Brendan Hansknecht (Apr 21 2024 at 15:24):

If you use the default handling, it could just exit with an exit code of 1

Brendan Hansknecht (Apr 21 2024 at 15:26):

That said, totally could still leave the exit code in if wanted. I just removed it as personal preference.

Brendan Hansknecht (Apr 21 2024 at 15:26):

mainForHost : Task I32 [] as Fx
mainForHost =
    res = main |> Task.result!
    when res is
        Ok exitCode -> Task.ok exitCode
        Err e ->
            Stderr.lineUnchecked! "Program exited with error: $(Inspect.toStr e)"
            Task.ok 1

Richard Feldman (Apr 21 2024 at 15:33):

actually something we could do is have the platform accept this:

Task {} [Exit I32]*

Brendan Hansknecht (Apr 21 2024 at 15:34):

That is much nicer

Richard Feldman (Apr 21 2024 at 15:34):

so if you return Task.err (Exit 1) then it exits with that code (just like today) but all other errors get handled in the default way

Richard Feldman (Apr 21 2024 at 15:34):

this design also makes basic-cli nicer for scripting in general!

Richard Feldman (Apr 21 2024 at 15:35):

means by default you get the "bail out on error" behavior, but you can easily opt into any amount of granular error handling and guarantees you like

Richard Feldman (Apr 21 2024 at 15:40):

I like not having exit as a Task because it affects control flow in a way that's otherwise reserved for crash

Richard Feldman (Apr 21 2024 at 15:41):

I can't name a concrete downside of that, but it feels like an invariant that shouldn't be broken lightly

Richard Feldman (Apr 21 2024 at 15:42):

so I like leaving it out of the API if we can get away with it :big_smile:

Richard Feldman (Apr 21 2024 at 15:45):

ok I'm game to try this design out!

Agus Zubiaga (Apr 21 2024 at 15:57):

This sounds perfect for basic-cli!

Brendan Hansknecht (Apr 21 2024 at 16:31):

Let's do it

Luke Boswell (Apr 21 2024 at 18:41):

I should have some time today to work in this change for basic-cli and to update the examples.

Luke Boswell (Apr 21 2024 at 18:59):

Ah, inconvenient place for this module imports bug to appear... I might have to hack something to get it to work.

── MODULE NOT IMPORTED in examples/../platform/main.roc ────────────────────────

The `Stderr` module is not imported:

33│                  Stderr.line! "Program exited with error: $(Inspect.toStr e)"
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Did you mean to import it?

────────────────────────────────────────────────────────────────────────────────

1 error and 2 warnings found in 23 ms

Brendan Hansknecht (Apr 21 2024 at 19:11):

Maybe can manually wrap/call the effect?

Will have to be different from the regular Stderr.line anyway since you want to avoid errors.
And the new Stderr.line will have an error case [StderrClosed]

Luke Boswell (Apr 21 2024 at 20:41):

Yeah, been playing with that... this bug is a bit nebulous and is proving hard to work around. I'm leaning towards just leaving the "print error" part out for a few days and we can add it back when Agus' PR lands which resolves this specific issue I think.

Richard Feldman (Apr 21 2024 at 20:45):

yeah can just use crash for now

Luke Boswell (Apr 21 2024 at 20:46):

The other wrinkle, is that to use ! we need to have Task imported in the app.

But if it's just main = Stdout.line! "Hello, World!" we get **UNUSED IMPORT**: Nothing from **Task** is used in this module which is a different incantation of the same bug.

Luke Boswell (Apr 21 2024 at 20:47):

Well, at least that this only a Warning so it still runs

Luke Boswell (Apr 21 2024 at 20:48):

But other than a stray Warning, it's working ok.

I'll update the examples.

We can clean it up later to have the nicer Inspect printing.

Richard Feldman (Apr 21 2024 at 20:48):

oh that's funny - it's actually not a bug

Luke Boswell (Apr 21 2024 at 20:49):

Well, I guess we need builtin Task before we can resolve that

Richard Feldman (Apr 21 2024 at 20:49):

in the sense that if the only uses of ! are trailing (which desugar to nothing) then after desugaring there really is no use of Task in the module

Richard Feldman (Apr 21 2024 at 20:49):

yeah task as builtin takes care of that nicely

Luke Boswell (Apr 21 2024 at 20:50):

For the short term (like next 24hours) experience, users may get an unused warning in the specific case where they dont use anything from Task in thier app.

Richard Feldman (Apr 21 2024 at 20:51):

yeah which seems fine

Luke Boswell (Apr 21 2024 at 21:40):

https://github.com/roc-lang/basic-cli/pull/184

Luke Boswell (Apr 21 2024 at 21:41):

^^ PR for the basic-cli changes... will require tomorrow's nightly to pass CI.

Luke Boswell (Apr 21 2024 at 21:41):

Looks like there may be another issue too, so I'll plug away at it until I'm reasonably confident it's just waiting on nightly for Anton

Luke Boswell (Apr 21 2024 at 21:41):

I'll also have a crack at our https://github.com/roc-lang/examples

drew (Apr 22 2024 at 00:36):

Wait, so is ! already released?

Richard Feldman (Apr 22 2024 at 00:38):

yeah, you can use it right now!

Richard Feldman (Apr 22 2024 at 00:38):

the benefits of all-nightly releases :grinning_face_with_smiling_eyes:

Richard Feldman (Apr 22 2024 at 00:38):

no need to wait for a "stable" release if we don't have them :laughing:

Luke Boswell (Apr 22 2024 at 00:39):

drew said:

Wait, so is ! already released?

I updated the roc-wasm4 examples... it looks awesome with ! suffix :hearts: https://github.com/lukewilliamboswell/roc-wasm4/blob/main/examples/rocci-bird.roc#L36

Luke Boswell (Apr 22 2024 at 00:41):

The other platform that will be nice boost with the ! syntax is roc-ray for drawing graphics etc. Looking forward to upgrading to zig-12 with that too.

Luke Boswell (Apr 22 2024 at 00:42):

And I should probably make an actual release for macos/linux so people can use from just a URL. I was thinking it might be possible to do that using GH Actions somehow... I've never looked at that before, but it's just running zig build on the different arch's and then packaging up the .a files with the roc platform files.

drew (Apr 22 2024 at 00:47):

also fwiw strong agree that Stdin.line shouldn’t return a tagged type. the benefits don’t outweigh the costs imo. i was very confused when i randomly saw this API two days ago.

Luke Boswell (Apr 22 2024 at 00:48):

Yeah, I just changed it in
https://github.com/roc-lang/basic-cli/pull/184

to Stdin.line : Task Str [End]

Richard Feldman (Apr 22 2024 at 00:53):

instead of just End, we should probably make a type alias for StdinErr : [IoErr, Oom, End] (we can add more later)

drew (Apr 22 2024 at 00:53):

how are task errors handled?

Richard Feldman (Apr 22 2024 at 00:53):

and have it be Task Str [StdinErr StdinErr]

drew (Apr 22 2024 at 00:53):

can you simply ignore them?

Richard Feldman (Apr 22 2024 at 00:53):

essentially yeah

drew (Apr 22 2024 at 00:54):

but can opt in to handling, i suppose, and then it is exhastive

Richard Feldman (Apr 22 2024 at 00:54):

right, give or take the OS introducing new types of errors that Roc doesn't know about yet :laughing:

drew (Apr 22 2024 at 00:55):

nice

drew (Apr 22 2024 at 00:55):

seems like a good compromise

Luke Boswell (Apr 22 2024 at 01:15):

Made an issue https://github.com/roc-lang/basic-cli/issues/185 to track this.

Richard Feldman (Apr 22 2024 at 03:31):

speaking of updates, while working on the tutorial I noticed something awkward: if Stderr.line can error, then gracefully handling errors (e.g. by printing a message to stderr and then doing an Exit) becomes pretty cumbersome because you have to do like Stderr.line "Error writing to stdout" |> Task.onErr! \_ -> Task.ok {}

Richard Feldman (Apr 22 2024 at 03:31):

what if instead we made it be Exit I32 Str?

Richard Feldman (Apr 22 2024 at 03:32):

and then the platform prints that Str to stderr, while automatically disregarding any errors ("if stderr is broken, you just don't get any output, sorry")

Richard Feldman (Apr 22 2024 at 03:33):

that's also nice in that it makes the errors even more concise to handle, because you can just say:

Exit 1 "Error writing to stdout"

Richard Feldman (Apr 22 2024 at 03:33):

instead of needing two lines, one of which is always a Stderr.line!

Richard Feldman (Apr 22 2024 at 03:34):

also makes it harder to make the mistake of using stdout (by force of habit) instead of stderr for error reporting :big_smile:

Isaac Van Doren (Apr 22 2024 at 03:53):

That sounds nice!

Isaac Van Doren (Apr 22 2024 at 03:54):

I wonder if it would be confusing that if you do Exit 0 "success!" the message would print to stderr. The message could print to stdout if the exit code is 0, but that might encourage people to use the tag instead of just using Stdout.line.

Richard Feldman (Apr 22 2024 at 04:00):

yeah that's probably fine

Richard Feldman (Apr 22 2024 at 04:00):

doing stdout for 0 makes sense to me :thumbs_up:

Richard Feldman (Apr 22 2024 at 04:07):

I guess a counterargument would be that it's a weird edge case you could trip over, and maybe you'd want to use Exit 0 "Warning: ..." if you were translating an error into a warning and didn't want to change the exit code

Brendan Hansknecht (Apr 22 2024 at 04:11):

I think it should always print to stderr with Exit

Brendan Hansknecht (Apr 22 2024 at 04:12):

stderr tends to also be used for status updates and terminal outputs for pipelined applications

Luke Boswell (Apr 22 2024 at 04:12):

I'm not really following, Stderr.line cant throw an error right now. Are we wanting to update the rust impl to return an error somehow?

Brendan Hansknecht (Apr 22 2024 at 04:14):

yeah. eprintln! just crashes on errror

Luke Boswell (Apr 22 2024 at 04:14):

I guess we could handle this

Panics if writing to io::stderr fails.

Writing to non-blocking stderr can cause an error, which will lead this macro to panic.

Brendan Hansknecht (Apr 22 2024 at 04:14):

let stderr = io::stderr();
let mut handle = stderr.lock();

let result = handle.write_all(b"hello world");

Richard Feldman (Apr 22 2024 at 04:15):

yeah exactly :100:

Richard Feldman (Apr 22 2024 at 04:15):

in general, writing to a file descriptor can fail in various ways

Richard Feldman (Apr 22 2024 at 04:15):

and writing to stdout and to stderr is just writing data to a file descriptor

Richard Feldman (Apr 22 2024 at 04:16):

not all of those possible failure modes apply to stdout and stderr, but plenty of them do!

Richard Feldman (Apr 22 2024 at 04:16):

so in this new design, I think we should have Stdin.line, Stdout.line, and Stderr.line all have errors

Brendan Hansknecht (Apr 22 2024 at 04:18):

Also, lets please make sure to either make them wrapping or descriptive of the filed descriptor they come from. End alone is quite unclear. Stderr End is much much more useful.

Richard Feldman (Apr 22 2024 at 04:18):

I'd like to try out the convention of naming those errors Err and just always fully qualifying them:

Stdout.line : Str -> Task {} [StdoutErr Stdout.Err]

Stderr.line : Str -> Task {} [StderrErr Stderr.Err]

Stdin.line : Task Str [StdinErr Stdin.Err]

Richard Feldman (Apr 22 2024 at 04:18):

yeah I think End would be part of the Stdin.Err tag union in this design

Brendan Hansknecht (Apr 22 2024 at 04:23):

Sounds great!

Brendan Hansknecht (Apr 22 2024 at 04:24):

Though I guess it only works if everything in Stdin has the exact same Err type.

Richard Feldman (Apr 22 2024 at 04:28):

yeah I can't think of any cases where that wouldn't be true! :big_smile:

Brendan Hansknecht (Apr 22 2024 at 04:32):

I thought file would be a case, but from discussion elsewhere, I guess everything is just io error there

Luke Boswell (Apr 22 2024 at 04:46):

Do we have any ideas around the specific errors here? Rust just has a big std::io::ErrorKind with lots of variants

Luke Boswell (Apr 22 2024 at 04:47):

So the updates I had been making were trending towards using that everywhere... but in the above API it looks like we want to tailor the tags for the specific function, and then maybe have an Other to catch the rest

Richard Feldman (Apr 22 2024 at 04:47):

yeah I think for now we can just put in a couple of them and add more later

Richard Feldman (Apr 22 2024 at 04:48):

the main thing is just switching over to the new "shape" if that makes sense

Luke Boswell (Apr 22 2024 at 04:48):

Also, I would like to use an <u8> for the error tag between rust and roc instead of generating glue for these if that's ok

Luke Boswell (Apr 22 2024 at 04:49):

It's just much simpler. Regenerating glue is very handraulic at the moment because it doesn't generate valid Rust and I have to manually fix things

Luke Boswell (Apr 22 2024 at 04:51):

So stderrLine : Str -> Effect (Result {} U8) instead of stderrLine : Str -> Effect (Result {} InternalStderrError)

Richard Feldman (Apr 22 2024 at 04:52):

as in just on the platform <-> host side?

Richard Feldman (Apr 22 2024 at 04:52):

not visible to application authors

Luke Boswell (Apr 22 2024 at 04:53):

correct

Richard Feldman (Apr 22 2024 at 04:53):

sure, seems reasonable! :thumbs_up:

Luke Boswell (Apr 22 2024 at 04:58):

Actually I'm going to use a RocStr, so I can return a description of the error from Rust

Last updated: Jul 06 2025 at 12:14 UTC