Stream: ideas

Topic: chaining syntax

Richard Feldman (Jan 08 2024 at 19:14):

here is a proposal for a very significant syntax change

Richard Feldman (Jan 08 2024 at 19:14):

any feedback welcome!

Tim (Jan 08 2024 at 20:11):

{} <- any feedback welcome |> Task.await

Tim (Jan 08 2024 at 20:12):

the new syntax makes total sense

Tim (Jan 08 2024 at 20:21):

could there be a way to show explicitly that ? is Result.try in the example function?

instead of just with Result.try, something like using ? for Result.try?

Tim (Jan 08 2024 at 20:22):

some way to make the ? syntax more guessable like you described for !

Isaac Van Doren (Jan 08 2024 at 20:25):

This is great! Very in favor.

The biggest downside I see is that it might make some type errors more confusing because of the more complicated desugaring.

Richard Feldman (Jan 08 2024 at 20:34):

Tim said:

could there be a way to show explicitly that ? is Result.try in the example function?

instead of just with Result.try, something like using ? for Result.try?

which example are you referring to?

Richard Feldman (Jan 08 2024 at 20:34):

Isaac Van Doren said:

The biggest downside I see is that it might make some type errors more confusing because of the more complicated desugaring.

I have some ideas about that, but actually I suspect it will let us improve error message friendliness :big_smile:

Tim (Jan 08 2024 at 20:42):

chomp4digits = \bytes -> with Result.try
    (digit1, rest1) = chompDigit? bytes

Johan Lövgren (Jan 08 2024 at 20:55):

I do like how more concise the code becomes. But I do find that I have to spend more mental energy to desugar the code, as you say.

Johan Lövgren (Jan 08 2024 at 20:57):

With some indentation I find the "?" to be a nice balance between verbosity and readability:

storeEmail = \path ->
    with Task.await
        url = File.readUtf8? path
        user = Http.get? url Json.codec
        dest = "$(user.name).txt"
        File.writeUtf8? dest user.email
        Stdout.line? "Wrote email to $(dest)"

It has a bit more of a "block" feel, which I feel like backpassing also provided. Makes it easier to spot from afar as well, that this is a special section of code.

Johan Lövgren (Jan 08 2024 at 20:58):

Similar to do-blocks in Haskell, or computation expressions in F#.

Johan Lövgren (Jan 08 2024 at 21:02):

You note that backpassing can make the use of Random.andThen nicer, and that we would lose that. But should not ? and with be able to work for Random.andThen as well?

Brendan Hansknecht (Jan 08 2024 at 21:05):

I like the proposal overall, but my biggest concern is that you sometimes want to handle errors where they happen. This is especially true of production grade code (which is mostly non-existent in roc currently).
This proposal promotes treating Tasks hitting errors as the exceptional case.

In current roc, I can write:

# Imagine other tasks with `!` before and after this code block.

bytes <- File.readBytes file |> Task.attempt

config =
    bytes
    |> Result.try Decode.decode
    |> Result.withDefault myDefaultConfig

Note the use of Task.attempt instead of Task.await with backpassing.
This is critical any time you want to handle an error locally as if it where a result.

In the new proposal, you would have to do something like:

# Imagine other tasks with `!` before and after this code block.

bytes = File.readBytes file
    |> Task.onErr! \_ ->
        Encode.encode myDefaultConfig
        |> Task.ok

config =
    bytes
    |> Decode.decode
    |> Result.withDefault myDefaultConfig

Note: Please try not to focus too heavily on the specific example. In general larger and more production ready code bases will hit things similar to this.

Eli Dowling (Jan 08 2024 at 21:32):

I have two thoughts:

1. Have you considered putting the symbol before the function?
   eg:

storeEmail = \path ->
    url = !File.readUtf8 path
    user = !Http.get url Json.codec
    dest = "$(user.name).txt"
    !File.writeUtf8 dest user.email
    !Stdout.line "Wrote email to $(dest)"

  # Create the build directory
    if !File.exists "build" then
        !Dir.deleteAll "build"
    else
        !Task.ok {}

I prefer this for a number of reasons.
I think it's easier to see where IO is happening, I find the ! gets a little lost when it's after the function, but it being at the start of the line makes it stand out

It's more like the await people are used to.

It makes it less like special builtin syntax and more standard, if you could make custom infix operators, and you had some operator precedence rules, you could make this in roc code :(!)= Task.await.

Some downsides:
The flow of thinking, "Id like to read some input" then tying "input=Stdout.read" then thinking I need it now, and just adding a "!" is nicer than having to go back to the start of the assignment, but I think it's a pretty minor thing, especially to people coming from async land.

2. Often I'd like to use tasks and results together, I understand hardcoding Task.await to ! allows that, but could there maybe be a more general solution eg: with Result.try as ?, Task.await as !, Option.try as ?* or some such.

Or if you combine my two proposals, it might be simpler to just allow this:(?)=Result.trybut I do like the explicitness of the "with" syntax

Brendan Hansknecht (Jan 08 2024 at 21:38):

Putting ! before can be confusing if anything that returns a Bool. These two feel very similar. One would be an awaited task for a Bool, the other would be negating a Bool returned by a function:

isDir = !Path.isDir path
isNotDir = !(Path.isDir path)

Andy Hamon (Jan 08 2024 at 21:39):

I'm still very new, so take my opinions with a grain of salt.

My immediate reaction is "great, more special symbols to learn" - symbols with special/nonstandard meanings always make learning a new language more daunting (to me at least) since they are harder to google.

If you are going to go so far as to add syntatic sugar for |> Task.await, have you considered something more discoverable, like a builtin keyword (perhaps with different associativity). I'm thinking like the await keyword in many languages. Even if its not a true character-count savings, symbols that need the shift key to type are, to me, much more annoying to type by a factor of 2 or 3.

I'm also curious about what this means for use cases like Record Builder - I've been studying this with the intention to apply similar techniques for an HTTP router with a nice builder API. Not sure if that is still possible with ? but I think it might not be.

Chris (Jan 08 2024 at 21:50):

I also don't know if i like putting ! after function name. What do you think of something like this?

storeEmail = \path ->
    url await File.readUtf8 path
    user await Http.get url Json.codec
    dest = "$(user.name).txt"
    await File.writeUtf8 dest user.email
    Stdout.line "Wrote email to $(dest)"

(Disclaimer: i have not applied this yet to all examples from the proposal, i.e. ifs, pipes, etc.)

Edit: After some giving it some thought, this is a bad idea... This would require too many rules to follow

Eli Dowling (Jan 08 2024 at 21:53):

Brendan Hansknecht said:

Putting ! before can be confusing if anything that returns a Bool. These two feel very similar. One would be an awaited task for a Bool, the other would be negating a Bool returned by a function:

isDir = !Path.isDir path
isNotDir = !(Path.isDir path)

That's a good point, You would need to pick a different symbol for sure. but there are lots to choose from ?*, $*, $, !!, all not bad options

Eli Dowling (Jan 08 2024 at 21:56):

ziutech said:

I also don't know if i like putting ! after function name. What do you think of something like this?

I definitely think removing the = from assignment is a wrong move, that makes code much harder to visually parse

Richard Feldman (Jan 08 2024 at 22:14):

Andy Hamon said:

I'm still very new, so take my opinions with a grain of salt.

My immediate reaction is "great, more special symbols to learn"

this would replace backpassing syntax - the <- symbol - so it's not strictly adding new symbols :big_smile:

Hannes Nevalainen (Jan 09 2024 at 01:37):

I like this! Im one of the ones that find backpassing mind bending and still being here :sweat_smile:
On a first glance at this proposal it seems like it would be easier to take a mental shortcut with this syntax while learning so I don’t need to fully understand how it works to use it :)

Agus Zubiaga (Jan 09 2024 at 02:19):

@Brendan Hansknecht I think in this world, instead of Task.attempt we would have something like:

asResult : Task ok err -> Task (Result ok err) []

so you can do:

config =
    file
    |> File.readBytes
    |> Task.asResult!
    |> Result.try Decode.decode
    |> Result.withDefault myDefaultConfig

There's probably a better name than asResult, but you get the idea.

Brendan Hansknecht (Jan 09 2024 at 02:27):

:laughing: |> Task.handleErrorNow!

Richard Feldman (Jan 09 2024 at 02:31):

in the proposal, I think this should be equivalent:

config =
    file
    |> File.readBytes!
    |> Decode.decode
    |> Result.withDefault myDefaultConfig

Richard Feldman (Jan 09 2024 at 02:32):

oh I guess the difference is you don't want to shortcut future tasks in the other one

Agus Zubiaga (Jan 09 2024 at 02:34):

I think that'd fail the entire task config is part of. If I understood correctly, Brendan wants to default to myDefaultConfig if File.readBytes fails

Richard Feldman (Jan 09 2024 at 02:35):

yeah makes sense!

Brendan Hansknecht (Jan 09 2024 at 02:37):

Agus Zubiaga said:

Brendan Hansknecht I think in this world, instead of Task.attempt we would have something like:

asResult : Task ok err -> Task (Result ok err) []

I'm not sure how I didn't realize that you could make that helper.

Brendan Hansknecht (Jan 09 2024 at 02:37):

Definitely cleans up the case of wanting to handle errors locally

Richard Feldman (Jan 09 2024 at 02:39):

Brendan Hansknecht said:

Agus Zubiaga said:

Brendan Hansknecht I think in this world, instead of Task.attempt we would have something like:

asResult : Task ok err -> Task (Result ok err) []

I'm not sure how I didn't realize that you could make that helper.

because it's a brand new idea! New patterns emerging already! :smiley:

Agus Zubiaga (Jan 09 2024 at 02:40):

This is another way to do it, but asResult is probably nicer in general:

config =
    file
    |> File.readBytes
    |> Task.await \bytes -> Task.fromResult (Decode.decode bytes)
    |> Task.onErr! \_ -> Task.ok myDefaultConfig

Brendan Hansknecht (Jan 09 2024 at 02:50):

Yeah, the code with asResult is much clearer to me.

It also means that at any time you can get local errors if you want:

res = File.readBytes path |> Task.asResult!

config =
    when res is
        Ok bytes ->
            Task.fromResult (Decode.decode bytes)
        Err _ ->
            # save a new config file and returns the default config
            generateNewConfigFile!

Not saying this is better, just noting you can do it and have more complex responses. Like you could have a branch for each different error type and it could be quite complex.

Anton (Jan 09 2024 at 10:20):

I was going over the build.sh port with the new syntax again and one important feature is still missing; you want to have a line number for where your error occurred. It could be "fixed" by using the future roc-script platform, which would crash on error and so should be able to provide the roc line number in the stacktrace. But, error line numbers seems like something you want to have in every platform.

Anton (Jan 09 2024 at 10:44):

With a magic wand, I'd love to have configurable options for line number, backtrace, and a "full recording". A "full recording" would contain all state you need to load up a time traveling debugger. The corresponding file could be capped in size to a chosen limit.

Anton (Jan 09 2024 at 10:44):

For a whole range of applications this easier debugging will be worth the perf cost.

Anton (Jan 09 2024 at 10:48):

Being able to trigger a full recording with an env var like RUST_BACKTRACE would be useful as well.

Richard Feldman (Jan 09 2024 at 11:34):

I think that's worth discussing, although probably in a different thread? It's not like we have line numbers today, after all :big_smile:

Fabian Schmalzried (Jan 09 2024 at 12:05):

I have to say, during AoC I did start to like doing elem <- List.map elems, so this would be missed by me. That said, I still think this proposal is great, and I don't think it will be worth it to keep backpassing once this is implemented.

Fabian Schmalzried (Jan 09 2024 at 12:07):

I was thinking about the with keyword a bit. In python for example with is something completely different. It's not clear that ? and with are connected at all. I think it's easy to learn, but maybe the connection can be more clear.
As alternative maybe it makes sense to give this concept an actual name and have a keyword that reflects that? How would you call this, when talking about it. Is it chaining? Then maybe chain as a keyword?

Sky Rose (Jan 09 2024 at 13:46):

I really like this!

I think the ! would be better at the beginning, so the task can stay as a whole expression, that the ! is applied to.

• This would become possible, where in the current proposal there's no way to write it:

task = File.readUtf8 path
bytes = !task

• It makes the mapErr example much more obvious. The ! applies to the whole task instead of to the last function that makes it:

!File.writeBytes path userData
|> Task.mapErr SaveUserDataErr

• For beginners, an important concept to teach is that Task is a noun that you have to pass to the runtime/platform to run. Putting the ! between the function and its args makes it feel more like File.readUtf8! is its own verb that does something on its own. Having a task expression that you then execute with ! is a step closer to understanding that the runtime does the execution.
• It's easier to spot at the start of the line than the middle.

Looking like boolean not seems like a manageable problem. Maybe it's fine since it shouldn't be ambiguous if you have a bool or a task. Maybe the ! could go at the end of the task expression instead of the beginning? One of them could change to a different symbol?

Anton (Jan 09 2024 at 13:50):

We could go with the not keyword instead of ! for bools, just like python.

Sky Rose (Jan 09 2024 at 13:54):

I wonder if it'd be possible to do this proposal with just one syntax/concept, instead of separate ! and with...? concepts.
One idea I haven't thought through all the way:

bytes = ! File.readUtf8 path
result = Result.try! parse(bytes)

So the wrapping function is always at the start of the line, and Task.await is the default if you skip specifying the function.

• If you're doing a ton of the same function (like in the chomp4digits example), it's a little more verbose. But it's less verbose if you're only calling it once, because you can skip the with line.
• It also allows mixing multiple different types of wrapping functions, like backpassing allows, instead of committing to a single function for the whole with block.
• I think it's also a little easier to desugar in my head than with, cuz it's a step closer to backpassing, instead of having to look at the top of the block to see what function to wrap with. (edited to add)

And a little bonus, this idea suggests having a space between the ! and the task expression, which could end up disambiguating between boolean not and task, and make it clearer that the ! applies to the whole task expression, not just the first term in it.

Sky Rose (Jan 09 2024 at 14:02):

Being able to use ! in arbitrary expressions is powerful, but could cause problems. I think starting with only allowing it at the top level as an assignment is a reasonable restriction, and could be loosened later. It makes it much clearer exactly which side effects are happening in what order. The if !File.exists path example was very compelling, but I think the increase in complexity isn't worth it.
If arbitrary expressions are alllowed, I think it should specifically be banned in guards. There's just such a tripping hazard if you could end up with an effect happening on a branch that you didn't go down. As precedent, Elixir bans effects in guards, and it's a good safety measure (though it's frequently annoying there because Elixir isn't clear about where effects happen, unlike Roc).

Agus Zubiaga (Jan 09 2024 at 14:07):

It's unclear to me whether in the proposal! can go after any identifier or if it has to be a function with arguments applied after. If it was any identifier, the following should work:

task = File.readUtf8 path
bytes = task!

I know those probably need be to handled differently in the compiler, but I assume that'd work because it seems like it would happen all the time. For example:

cwd = Os.getCwd!

Sky Rose (Jan 09 2024 at 14:09):

Last thought:
I'd be curious to see a more detailed comparison of how this proposal compares to Haskell's do notation. Obviously Roc isn't gonna use monads, but thinking about this idea as similar to monads might lead to new ways to simplify how ? and ! work. For example, can you only use ? on something similar to bind (aka andThen), or would it work on any callback like backpassing could?

Sky Rose (Jan 09 2024 at 14:11):

Fabian Schmalzried said:

during AoC I did start to like doing elem <- List.map elems, so this would be missed

with List.map
elem = elems?

:smiling_devil:

Agus Zubiaga (Jan 09 2024 at 14:13):

Sky Rose said:

Fabian Schmalzried said:

during AoC I did start to like doing elem <- List.map elems, so this would be missed

with List.map
elem = elems?

:smiling_devil:

with List.map elems? * 2

:smiling_devil: :smiling_devil:

Agus Zubiaga (Jan 09 2024 at 14:14):

(assuming !/? can go after any ident)

Richard Feldman (Jan 09 2024 at 14:43):

haha yeah it should be able to go after any identifier, not just function calls. :check:

Norbert Hajagos (Jan 09 2024 at 15:29):

I use backpassinging for nested List.walks. Please help me! :sweat_smile: .
To add something usefull:
I feel the use of backpassing in situations like the above is solely to avoid over-indentation. Seeing 3 indentation of nested "loops" is painful. But it should be so!
I feel like I am using backpassing because of the sunk-cost fallacy, since getting used to them was not easy. I would be more than fine with seeing it removed.

Agus Zubiaga (Jan 09 2024 at 15:41):

Yeah, I have tried using backpassing with List.walk, but nowadays I prefer not to for the same reason I wouldn't like to do this in an imperative language:

for (let i = 0; i < a.length; ++i) {
sum += a[i];
}

Indentation is helpful here

Brendan Hansknecht (Jan 09 2024 at 15:47):

I think it is less helpful in a pure functional language where things don't mutate.

Anton (Jan 09 2024 at 15:47):

I also just wrote this code without backpassing:

    dirListNoFilesT =
        Task.map readFirstArgT \examplesDir ->
            examplesDirPath = Path.fromStr examplesDir
            Task.map (Dir.list examplesDirPath) \dirList ->
                List.keepIf dirList \fileOrDir ->
                    !(Str.contains (Path.display fileOrDir) ".")

    dbg dirListNoFilesT

It does not look good but it required an order of magnitude less cognitive effort to write.

Brendan Hansknecht (Jan 09 2024 at 15:48):

Idk, I'm so used to back passing that It is basically a variable assignment, super easy to use wherever i want to.

Brendan Hansknecht (Jan 09 2024 at 15:49):

It is honestly one of my favorite features

Brendan Hansknecht (Jan 09 2024 at 15:49):

I would definitely prefer the two together (at a minimum waiting before removing backpassing to see how much It is still used)

Anton (Jan 09 2024 at 15:51):

I definitely see the benefits of backpassing, but it was really tripping me up in this case because I had regular assignments in between (=) and I also thought I needed to use await at first.

Agus Zubiaga (Jan 09 2024 at 15:54):

Brendan Hansknecht said:

I would definitely prefer the two together (at a minimum waiting before removing backpassing to see how much It is still used)

What would you use backpassing for that you couldn't do with with? :smile:

Brendan Hansknecht (Jan 09 2024 at 15:55):

with is only useful if you use the same function everywhere. That is a huge restriction

Brendan Hansknecht (Jan 09 2024 at 15:56):

Also, it is really verbose for anything used just a few times

Agus Zubiaga (Jan 09 2024 at 16:03):

Brendan Hansknecht said:

with is only useful if you use the same function everywhere. That is a huge restriction

It's a restriction, but in practice I think we use the same function in most cases. And in the cases we don't, I'd argue the API could be improved so that's the case. At least that was my experience with roc-pg's query builder API when I saw this proposal.

Agus Zubiaga (Jan 09 2024 at 16:04):

I ended up with a much more flexible API than the one I currently have

Brendan Hansknecht (Jan 09 2024 at 16:11):

Anton said:

I definitely see the benefits of backpassing, but it was really tripping me up in this case because I had regular assignments in between (=) and I also thought I needed to use await at first.

For your example, I would probably write:

dirListNoFiles =
    examplesDir <- readFirstArg |> Task.map
    examplesDirPath = Path.fromStr examplesDir
    dirList <- Dir.list examplesDirPath |> Task.map

    List.keepIf dirList \fileOrDir ->
         !(Str.contains (Path.display fileOrDir) ".")

dbg dirListNoFiles

Anton (Jan 09 2024 at 16:14):

Uhu, yeah, I could definitely make the conversion after I wrote it in the desugared style.

Brendan Hansknecht (Jan 09 2024 at 16:16):

With with

dirListNoFiles =
    # That is way too much tabbing for 1 or 2 uses of a function.
    # This will be problematic if I need another with block below or didn't want to hoist all lines into the with
    dirList = with Task.map
        examplesDir = readFirstArg!
        examplesDirPath = Path.fromStr examplesDir
        Dir.list examplesDirPath!

    List.keepIf dirList \fileOrDir ->
         !(Str.contains (Path.display fileOrDir) ".")

dbg dirListNoFiles

Brendan Hansknecht (Jan 09 2024 at 16:17):

Anton said:

Uhu, yeah, I could definitely make the conversion after I wrote it in the desugared style.

I guess I am too used to backpassing. I don't desugar it at this point.

Agus Zubiaga (Jan 09 2024 at 16:17):

The proposed format style is to have with in the same line as =, so you wouldn't have an extra indent

Anton (Jan 09 2024 at 16:18):

I guess I am too used to backpassing. I don't desugar it at this point.

Yeah, I think it can definitely be learned, I have not written much Task-heavy Roc yet.

Agus Zubiaga (Jan 09 2024 at 16:20):

Wouldn't this work just fine?

dirListNoFiles =
    examplesDir = readFirstArg!
    examplesDirPath = Path.fromStr examplesDir
    dirList = Dir.list! examplesDirPath

    List.keepIf dirList \fileOrDir ->
         !(Str.contains (Path.display fileOrDir) ".")

dbg dirListNoFiles

Brendan Hansknecht (Jan 09 2024 at 16:21):

Not quite

Brendan Hansknecht (Jan 09 2024 at 16:22):

dirListNoFiles =
    examplesDir = readFirstArg!
    examplesDirPath = Path.fromStr examplesDir
    dirList = Dir.list! examplesDirPath

    Task.ok
        List.keepIf dirList \fileOrDir ->
            !(Str.contains (Path.display fileOrDir) ".")

dbg dirListNoFiles

Brendan Hansknecht (Jan 09 2024 at 16:23):

At least if you want to keep the original semantics

Brendan Hansknecht (Jan 09 2024 at 16:23):

That said, I wanted to keep the original Task.map cause that was the code given.

Agus Zubiaga (Jan 09 2024 at 16:23):

Yeah, I’m confused about the original semantics. Wouldn’t that pass a task to dbg?

Brendan Hansknecht (Jan 09 2024 at 16:24):

yep

Brendan Hansknecht (Jan 09 2024 at 16:25):

That's probably why it ended with a T in the name for Task

Brendan Hansknecht (Jan 09 2024 at 16:25):

Anyway, that is besides the point. We could imagine that as any other function to force the need for with

Brendan Hansknecht (Jan 09 2024 at 16:28):

So is the current proposed syntax for a one off as:

config = with Result.try Decode.decode! bytes

# or with new line:
config = with Result.try
    Decode.decode! bytes

If so, doesn't the closure from with Result.try need to escape the block it is in....that feels off.

Brendan Hansknecht (Jan 09 2024 at 16:30):

Like how do we know if the Result.try was supposed to wrap the rest of building config (thus the code above has a bug due to not having a finishing statement). Or if Result.try was supposed to have a closure that consumes the rest of the function as a whole.

Brendan Hansknecht (Jan 09 2024 at 16:32):

I guess maybe it needs to be written as this to effect the rest of the fuction?

with Result.try config = Decode.decode! bytes

# or with new line:
with Result.try
    config = Decode.decode! bytes

But then the config variable is useless cause it won't escape the with block...so that also can't be correct.

Brendan Hansknecht (Jan 09 2024 at 16:34):

Am I missing something with the syntax? None of these feel correct. I think the syntax may make it impossible to intermix two compatible functions.

Agus Zubiaga (Jan 09 2024 at 16:38):

Why would you use Result.try to begin with if you just have one Result?

Kevin Gillette (Jan 09 2024 at 16:39):

(deleted)

Brendan Hansknecht (Jan 09 2024 at 16:40):

Imagine this uses Result.try but the rest of the function uses Result.map just trying to convey the general concept of switching between two functions that can be made compatible.

Sky Rose (Jan 09 2024 at 17:02):

It got kind of buried, so I'll plug again my previous idea for

b = Result.try! a
c = Result.map! b
f c

to allow mixing multiple functions (and solving some of the with readability problems)

Brendan Hansknecht (Jan 09 2024 at 17:05):

So here, that would be:

config = Result.try! Decode.decode bytes

! being setup in a way to not need parens surrounding the rest of the line, I assume.

Richard Feldman (Jan 09 2024 at 17:08):

so then if you had Task.await imported unqualified as just await, the build.sh port from the doc would look like this?

## Read a file, find/replace inside it, and write it back.
replaceInFile = \path, find, replace ->
    content = await! File.readUtf8 path
    File.writeUtf8 path
        (content |> Str.replaceEach find replace)

main =
    # Handle all errors by crashing
    Task.onErr run \err ->
        crash "Error: $(Inspect.toStr err)"

run =
    # Check jq version
    await! Cmd.exec "jq --version"

    # Create the build directory
    await! if await! File.exists "build" then
        Dir.deleteAll "build"
    else
        Task.ok {}

    await! Dir.create "build"

    # Copy public/ to build/
    await! Dir.copyAll "public" "build"

    # Download the latest examples
    await! Cmd.exec "curl -fL -o examples-main.zip https://…"
    await! Cmd.exec "unzip -o examples-main.zip"

    await! Dir.copyAll "examples-main/examples" "content/examples"

    # Replace links in content/examples/index.md
    await! replaceInFile "content/examples/index.md"
        "](/"
        "](/examples/"

    # Clean up examples artifacts
    await! Dir.deleteAll "examples-main"
    await! File.delete "examples-main.zip"

    # Download design assets
    ...

Richard Feldman (Jan 09 2024 at 17:10):

as opposed to:

## Read a file, find/replace inside it, and write it back.
replaceInFile = \path, find, replace ->
    content = File.readUtf8! path
    File.writeUtf8 path
        (content |> Str.replaceEach find replace)

main =
    # Handle all errors by crashing
    Task.onErr run \err ->
        crash "Error: $(Inspect.toStr err)"

run =
    # Check jq version
    Cmd.exec! "jq --version"

    # Create the build directory
    if File.exists! "build" then
        Dir.deleteAll! "build"
    else
        Task.ok! {}

    Dir.create! "build"

    # Copy public/ to build/
    Dir.copyAll! "public" "build"

    # Download the latest examples
    Cmd.exec! "curl -fL -o examples-main.zip https://…"
    Cmd.exec! "unzip -o examples-main.zip"

    Dir.copyAll! "examples-main/examples" "content/examples"

    # Replace links in content/examples/index.md
    replaceInFile! "content/examples/index.md"
        "](/"
        "](/examples/"

    # Clean up examples artifacts
    Dir.deleteAll! "examples-main"
    File.delete! "examples-main.zip"

    # Download design assets
    ...

Anton (Jan 09 2024 at 17:17):

Yeah, the second one looks a lot better

Agus Zubiaga (Jan 09 2024 at 17:21):

The second one also probably looks better when pipes are involved. I would give an example but I’m on mobile rn :smile:

Sky Rose (Jan 09 2024 at 17:50):

For Task.await specifically, it could keep the special case where a plain ! defaults to Task.await!

Sky Rose (Jan 09 2024 at 17:52):

    ...
    ! Dir.create "build"
    ! Dir.copyAll "public" "build"
    ! Cmd.exec "curl -fL -o examples-main.zip https://…"
    ! Cmd.exec "unzip -o examples-main.zip"
    ...

Richard Feldman (Jan 09 2024 at 17:54):

I think it's predictable that if we have a prefix ! which means anything other than Bool.not, it will be confusing to everyone who's used a mainstream programming language :sweat_smile:

Richard Feldman (Jan 09 2024 at 17:54):

unless you count Bash :sweat_smile:

Eli Dowling (Jan 09 2024 at 23:03):

maybe use an operator that is generally not used elsewhere? <! or !> or ~ I quite like the idea of tilda it's not used elsewhere and has a nice flowy feel

Brendan Hansknecht (Jan 09 2024 at 23:06):

I think figuring out a nice prefix operator would be really nice for new developers learning roc.

Then you never have to explain:

Dir.create "build"
    |> Task.onErr! WrapWithErr

The marker will always go in the same location no matter what.

Agus Zubiaga (Jan 09 2024 at 23:23):

I guess you could put it at the end of the expression:

Dir.create "build"!
Dir.create "build"
    |> Task.onErr WrapWithErr!
Dir.current!
    |> Dir.copyAll "build"!

Brendan Hansknecht (Jan 09 2024 at 23:50):

I think that would still be confusing the new users

Brendan Hansknecht (Jan 09 2024 at 23:51):

Specifically why no ! after the Dir.create on the second line.

Brendan Hansknecht (Jan 09 2024 at 23:51):

If you put It before It is consistent whether or not you pipe through a modifier function

Eli Dowling (Jan 09 2024 at 23:52):

One random thought on the "with" syntax, I think prefixing the operator with a name when using a non-standard mapping, like first= try~ List.first [1,2] might be a good way to avoid confusion and lots of weird operators. A possible option is to make the "with" syntax load all of a module's contents so they are available using this prefix syntax. eg:

#backpassing
addAtIndex =\list,str->
    {before:idxS,after:numS}<-str|>Str.splitFirst ","|>Result.try
    idx<-Str.toNat idxS |>Result.try
    num<-Str.toU8 numS |>Result.try
    listNum<-list|>List.get idx |>Result.map
    listNum+num

# with syntax
addAtIndex =\list,str-> with Result
    {before:idxS,after:numS}= try~ str|>Str.splitFirst ","
    idx= try~ Str.toNat idxS
    num= try~ Str.toU8 numS
    listNum= map~ list|>List.get idx
    listNum+num

dbg addAtIndex [10,30]"1,3"
##prints 33

Agus Zubiaga (Jan 09 2024 at 23:52):

Because the Task is not ready to run yet :smile:

Agus Zubiaga (Jan 09 2024 at 23:53):

I know beginners might not have a good intuition on how Task works, but I don't think this is worse than the status quo

Brendan Hansknecht (Jan 09 2024 at 23:55):

That's fair. I would rather just improve on the status quo by just hiding that.

Agus Zubiaga (Jan 09 2024 at 23:56):

I get prefixing is maybe a little more intuitive, but I think it's bad for the ergonomics of piping. I think it's really nice to able to do this:

Dir.current!
    |> Dir.copyAll "build"!

Agus Zubiaga (Jan 10 2024 at 00:00):

Brendan Hansknecht said:

That's fair. I would rather just improve on the status quo by just hiding that.

Can you really hide it, though? I feel like by the time they're making Task transformations with functions such as mapErr, they already have to understand that Dir.create doesn't actually perform the operation.

Brendan Hansknecht (Jan 10 2024 at 00:01):

Trying to think if we can get both without a way more complex logic here.

Like can we make this work (with whatever character we land on in the end):

! Dir.current
    |> Task.onErr (\_ -> Task.ok "/tmp")
    |> ! Dir.copyAll "build"

It would be more complex, but I don't see why It couldn't work.

Brendan Hansknecht (Jan 10 2024 at 00:03):

Can you really hide it, though?

This is why I am still in favor of #ideas > chaining but with Task.attempt and ? for error returns

Brendan Hansknecht (Jan 10 2024 at 00:04):

I think working directly on the Effect type, giving the user a Result and then having them opt into propagation with ? is a lot clearer and more similar to standard async and await in other languages.

Brendan Hansknecht (Jan 10 2024 at 00:05):

You would never have a reason to modify the Effect before running it, you would always do it after

Brendan Hansknecht (Jan 10 2024 at 00:05):

This would become something like:

Dir.current!
    |> Result.withDefault "tmp"
    |> Dir.copyAll!? "build"

Brendan Hansknecht (Jan 10 2024 at 00:08):

You still can do the error mapping then propagation case as well:

Dir.current!
    |> Result.mapErr? WrappingTag
    |> Dir.copyAll!? "build"

Brendan Hansknecht (Jan 10 2024 at 00:13):

This is all very close to symmetrical with what someone would see in Rust for example:

let dir = dir_current().await.context("extra error wrapping info")?;
dir_copy_all(dir, "build").await?;

Brendan Hansknecht (Jan 10 2024 at 00:16):

In rust the function would be async and return a result of some sort. In roc, it would be a normal function that returns an Effect (Result ...) of some sort

Brendan Hansknecht (Jan 10 2024 at 00:17):

Of course, Effect (Result k v) can still be aliased as Task k v, but we fundamentally would be chaining with Effect.after by default (which is equivalent to Task.attempt) for something accumulating a result.

Richard Feldman (Jan 10 2024 at 00:48):

:thinking: what would the type of Stdout.line be in that world? Would it use Result or would you have to |> Effect.map Ok to get it to coexist with those?

Richard Feldman (Jan 10 2024 at 00:50):

I guess if you have Task ok err : Effect (Result ok err) then you can just make everything be Task for compatibility

Brendan Hansknecht (Jan 10 2024 at 00:50):

It has no err case, so it would just be a Str -> Effect {}

Brendan Hansknecht (Jan 10 2024 at 00:50):

You would use it as:

Stdout.line! "some str"

You would never need to use ? with it

Richard Feldman (Jan 10 2024 at 00:52):

oh I missed the !? earlier

Richard Feldman (Jan 10 2024 at 00:52):

I'm not a fan of how !? makes all the statements look sort of confused and exasperated to me :sweat_smile:

Richard Feldman (Jan 10 2024 at 00:53):

I'm curious to explore the idea of everything returning Result though - maybe we could slightly change how the "statement !" works such that it would accept either Effect {} or else something like Effect [Ok {}, Err *] which is essentially equivalent to Effect {} in terms of being "safe to discard"

Richard Feldman (Jan 10 2024 at 00:54):

so like maybe the rule could be that if you write Stdout.line! "blah" without assigning it to anything, instead of checking to make sure it's Effect {}, we instead check to make sure it's an Effect whose type parameter "contains no information" - which would also be true of Effect [Ok {}, Err *]

Richard Feldman (Jan 10 2024 at 00:54):

(aka Task {} *)

Richard Feldman (Jan 10 2024 at 00:55):

so another way of saying this is that I think we could explore the idea of having all the existing types in basic-cli be the same, except that Task is a type alias for Effect (Result ok err)

Richard Feldman (Jan 10 2024 at 00:55):

and then if chaining is done on Effect, then this would work:

Dir.current!
    |> Result.withDefault "tmp"
    |> Dir.copyAll!? "build"

Richard Feldman (Jan 10 2024 at 00:57):

(btw I've been using Haskell syntax highlighting on examples with ! because it highlights ! and ? instead of showing them as errors, or Ruby if I need # comments)

Brendan Hansknecht (Jan 10 2024 at 00:59):

Also, totally agree that !? is not the prettiest. I just don't have a better idea at the moment without like an await keyword, but wrapping keywords don't feel great either. So it just represents my current best idea.

Richard Feldman (Jan 10 2024 at 01:00):

one idea could be to define ? to be for Effect (Result ok err) rather than Result ok err

Richard Feldman (Jan 10 2024 at 01:00):

so then it could be

Dir.current!
    |> Result.withDefault "tmp"
    |> Dir.copyAll? "build"

Richard Feldman (Jan 10 2024 at 01:01):

! meaning "await this" and ? meaning "await this and then unwrap the Result by short-circuiting on Err"

Brendan Hansknecht (Jan 10 2024 at 01:03):

Oh yeah. I like it

Brendan Hansknecht (Jan 10 2024 at 01:05):

Though I assume in totally task free code It may be nice to use both ! and ? directly on results. (If with can be used for both of those, it would be amazing)

Richard Feldman (Jan 10 2024 at 01:05):

:thinking: I think at that point people could get the behavior of the original ! proposal by just using ? everywhere they would have used !

Brendan Hansknecht (Jan 10 2024 at 01:05):

Yep :+1:

Richard Feldman (Jan 10 2024 at 01:07):

so I guess another way to reformulate the idea is to say Task is still opaque, but ! is for "await" and "? is for "attempt"

Richard Feldman (Jan 10 2024 at 01:08):

(or vice versa with which operator does which thing)

Richard Feldman (Jan 10 2024 at 01:08):

but I think the application code ends up looking the same either way

Richard Feldman (Jan 10 2024 at 01:11):

because basically if there's a concise way to do both, then people will do whichever they prefer on a case-by-case basis either way

Agus Zubiaga (Jan 10 2024 at 01:12):

Seems like ? should be the one that returns the Result because it’s like asking “how did it go?”

Brendan Hansknecht (Jan 10 2024 at 01:14):

Oh boy....that makes total sense but will confuse anyone from languages where ? basically means either propagate the error case to the caller or It means unwrap and crash on the error case.

Brendan Hansknecht (Jan 10 2024 at 01:14):

So ? leads to values without errors in other languages.

Agus Zubiaga (Jan 10 2024 at 01:14):

Any other than Rust?

Brendan Hansknecht (Jan 10 2024 at 01:20):

Zig

Brendan Hansknecht (Jan 10 2024 at 01:20):

I think there is at least one more....

Agus Zubiaga (Jan 10 2024 at 01:21):

Hm, that’s tough :sweat_smile:

Brendan Hansknecht (Jan 10 2024 at 01:21):

Haha.... Yeah, maybe it is les common than i think

Brendan Hansknecht (Jan 10 2024 at 01:22):

I guess on most languages ? Is still just ternary

Agus Zubiaga (Jan 10 2024 at 01:22):

This is pretty cool, but I’m sad I won’t be able to use ? to chain other types and we are back at nesting lambdas there

Brendan Hansknecht (Jan 10 2024 at 01:25):

So in this proposal, both ! and ? have direct definitions on Task. Theoretically they could substitute other defintions on Result or etc other types.

Richard Feldman (Jan 10 2024 at 01:26):

another thought:

Dir.current!
    |> Result.withDefault "tmp"
    |> Dir.copyAll? "build"

...could be rewritten in the original proposal as:

Dir.current
    |> Task.withDefault! "tmp"
    |> Dir.copyAll! "build"

(assuming there's a Task.withDefault : Task ok err, err -> Task ok *)

Richard Feldman (Jan 10 2024 at 01:28):

I'm pretty surprised how close the semantics are in practice here

Agus Zubiaga (Jan 10 2024 at 01:29):

Yeah, but what if you want to case on the err?

Richard Feldman (Jan 10 2024 at 01:29):

|> Task.attempt! right?

Richard Feldman (Jan 10 2024 at 01:30):

like trying to rewrite whatever you want to be expressed from one style to the other seems to be achievable in the same verbosity level, which surprises me

Agus Zubiaga (Jan 10 2024 at 01:30):

That wouldn’t work as it’s currently defined, but we can make a Task ok err -> Task (Result ok err) [] helper as I mentioned yesterday

Brendan Hansknecht (Jan 10 2024 at 02:00):

If we had both ! and ? how often if ever would we still need modify a task before using ! or ??

How often would we see something like:

Dir.current
    |> Task.withDefault! "/tmp"

Brendan Hansknecht (Jan 10 2024 at 02:01):

Trying to see if those two operators would be enough to avoid the delayed application ! which I think will confuse beginners.

Kevin Gillette (Jan 10 2024 at 16:08):

Tim said:

could there be a way to show explicitly that ? is Result.try in the example function?

instead of just with Result.try, something like using ? for Result.try?

Or maybe:

with ? as Result.try

! and ? would be the only symbols accepted before as, and would have defaults as originally proposed

Sky Rose (Jan 10 2024 at 17:44):

That solves knowing which function is being used in the with, but I think it'd still be better to specify it line by line. To compare to a pipeline, you don't say at the beginning of a pipeline "I'm gonna do a bunch of List.maps." Instead, you write on each line "I'm gonna use a |> List.filter, then |> List.first, then |> Result.mapErr". So I don't think we need a with block at all.

There's a bunch of different functions floating around this discussion, but they're all callbacks. With backpassing, there didn't need to be multiple different types of backpassing for different callbacks. I think it's possible for us to end up with a single syntax for callbacks here that's both generic for all callbacks and ergonomic for the common cases.

Brendan Hansknecht (Jan 10 2024 at 17:52):

Technically speaking, we could just modify backpassing if that is the goal:

current backpassing is val <- task |> await

we could simply do backpasssing as val = task |> await! with val = as optional

Brendan Hansknecht (Jan 10 2024 at 17:54):

That would be a much simpler change that gets rid of the most complained about part of backpassing <- while still having full power of backpassing and explicit names.

Brendan Hansknecht (Jan 10 2024 at 17:57):

Of course that also requires some extra magic if we want it to work in more complex cases like the file example:

This is not just normal backpassing with a new syntax, it is a lot more complex.

if File.exists "build" |> await! then
        Dir.deleteAll "build" |> await!
    else
        Task.ok  {} |> await!

Brendan Hansknecht (Jan 10 2024 at 17:59):

Also, it would probably be too flexible, and lead to some really confusing code. Backpassing with List.map is weird. This new syntax with List.map is totally unnatural.

(x, y) = points |> List.map!
x + y

Brendan Hansknecht (Jan 10 2024 at 18:01):

but I think it'd still be better to specify it line by line

I think it is important to realize that in almost all functions, just 1 of these will be used. It will be the andThen equivalent function for the specific return type. So this isn't a value that is expected to change line by line. It is expected to almost always be the same for entire functions and to fill a very common pattern.

Brendan Hansknecht (Jan 10 2024 at 22:34):

Brendan Hansknecht said:

If we had both ! and ? how often if ever would we still need modify a task before using ! or ??

How often would we see something like:

Dir.current
    |> Task.withDefault! "/tmp"

After thinking about this more, I think we should just stick with the original defined mappings in original doc. ! is Task.await. ? is either not used or maps to Result.try.

I think that no matter what, there will be this type of delayed mapping. It is a required feature that we need to teach cause roc builds up tasks and then requests the platform execute it. Roc never runs a task itself and will need more complex control at certain points like with concurrent tasks.

I think helpers can get any of the features I am interested in (like early error handling). It also has a clear accumulation story with tasks/results. Of course we need to add a number of helpers to task and teach those.

I guess it leaves in some unnecessary cases where we wrap something in a task/result even though it can't error, but that isn't really a big deal at all.

I think the only really case that can't be recovered with a helper would be Task.map. That said, it just means your final return needs to have a Task.ok call which is no big deal.

Richard Feldman (Jan 10 2024 at 22:37):

cool, that reasoning all makes sense to me! :+1:

Agus Zubiaga (Jan 11 2024 at 03:07):

Would ? still use with or always mean Result.try?

Agus Zubiaga (Jan 11 2024 at 03:08):

I vote to keep with, but I’m biased :upside_down:

Brendan Hansknecht (Jan 11 2024 at 03:16):

I think we should probably just use ! with with. And if that is the case, not add ?. But idk. Maybe it is useful for both to have a default and be reassignable in context

Brendan Hansknecht (Jan 11 2024 at 04:52):

I just feel if you can reassign both ! and ? at the same time It may lead to less clear code due to more context to follow.

Also, no major gain in using both. They really do the same thing just on different types.

Brian Carroll (Jan 11 2024 at 05:50):

Yeah I'd prefer to get rid of the special case for Task. There's no need for it. We can just always require an explicit with Task.await. A little more verbose but worth it because it's more explicit, and reduces the number of things to learn/remember.

Brian Carroll (Jan 11 2024 at 05:50):

But I think the doc suggested that we start with that version anyway.

Sven van Caem (Jan 11 2024 at 17:52):

I wonder if we could keep ! to exclusively mean Task.await, (just like arithmetic operators are Num-specific syntax sugar), and then bring back the ability to have lambda bodies be on the same line as their definition, e.g:

value =
    x |> f |> Result.try \y ->
    y |> g |> Result.try \z ->
    z + 1

So you'd keep the common case of Task.await maximally concise, while still allowing a slightly more awkward but also easier to figure out version of backpassing for when you want to avoid too much indentation.

IIRC, the point of flipping the lambda was to make backpassing resemble normal assignment to try to make using Tasks look more like imperative code, so if we don't need it for that purpose anymore, maybe this way of writing lambdas might be worth reconsidering again?

Brendan Hansknecht (Jan 11 2024 at 18:11):

I don't think it is fair to call Task a more common case than Result. In fact, long term, i would expect Result to be more common.

Brendan Hansknecht (Jan 11 2024 at 18:12):

Result will be used in libraries and all non-effectful code in roc. Task is just the top level effectful layer

Richard Feldman (Jan 11 2024 at 18:31):

Brendan Hansknecht said:

I don't think it is fair to call Task a more common case than Result. In fact, long term, i would expect Result to be more common.

personally I anticipate that Task.await will be used more than every other use case put together, and I don't think it will be close

Richard Feldman (Jan 11 2024 at 18:31):

that's the way it has turned out in Haskell and I can't think of a reason Roc would be different :big_smile:

Richard Feldman (Jan 11 2024 at 18:32):

although it's true Result is used in more places, I expect in practice that Result.try will be used a lot less than Task.await

Richard Feldman (Jan 11 2024 at 18:33):

because main usually needs a Task, which means every new Task needs to be combined somehow with the previous one, and await is by far the most common way to do that

Richard Feldman (Jan 11 2024 at 18:33):

whereas it's not the case that every program builds up to a giant Result :big_smile:

Richard Feldman (Jan 11 2024 at 18:34):

there are plenty of ways to go from Result ok err to either ok or err, whereas pretty much the only way to go from Task ok err to either ok or err is something like await (and await is the most common of the different ways of doing that)

Brendan Hansknecht (Jan 11 2024 at 18:34):

Hmm... But won't essnetially every library call build up to result?

Richard Feldman (Jan 11 2024 at 18:37):

depends on the library, but let's take something like a parser

Richard Feldman (Jan 11 2024 at 18:38):

it's really common to run a parser and then pattern match on Ok parsed -> ... and Err parseErr -> ... and then both of those branches evaluate to something that isn't a Result

Richard Feldman (Jan 11 2024 at 18:38):

so even though the parser ultimately returns a Result, Result.try is never called

Richard Feldman (Jan 11 2024 at 18:39):

whereas if you're similarly handling errors in a Task, you still end up with both the success and failure branches evaluating to a Task

Richard Feldman (Jan 11 2024 at 18:39):

which sometimes involves an await call, but even when not, it's more likely that one of those branches will go on to be awaited

Richard Feldman (Jan 11 2024 at 18:40):

whereas in the Parse case, the Result is gone, so there's no more opportunity for it to be chained

Richard Feldman (Jan 11 2024 at 18:41):

but at the end of the day, I'm mainly basing this prediction on how things have gone in practice in Haskell :big_smile:

Richard Feldman (Jan 11 2024 at 18:41):

for example, this article notes that:

Since do notation is used almost everywhere IO takes place, newcomers quickly believe that the do notation is necessary for doing IO

Richard Feldman (Jan 11 2024 at 18:42):

like in practice, using do notation (Haskell's equivalent of backpassing or !) with I/O is way more common than using it with anything else

Richard Feldman (Jan 11 2024 at 18:43):

second place would actually be parsers if I had to guess, not Result (which Haskell calls Either)

Brendan Hansknecht (Jan 11 2024 at 18:44):

Got it

Brendan Hansknecht (Jan 11 2024 at 18:47):

Then I change my statement to:

I really dislike how this reads and would gladly pick backpassing or flexible ! over it.

value =
    x |> f |> Result.try \y ->
    y |> g |> Result.try \z ->
    z + 1

Kiryl Dziamura (Jan 12 2024 at 01:09):

I don't like what I‘m about to propose, but just a thing to consider: the only new operator ? can rely on an ability (let’s say Try). For the Result, it would be the Result.try function, for the Task, it’s Task.await.
So the operator behavior relies on the type ability and not on the special with syntax that applies to the whole function.
The downside is obvious implicitness and the possibility to write weird stuff like Dir.copyAll?? "build". And, of course, potential ambiguity.
The paradigm here is “whatever it is - try to do something with an underlying value”. But shortcuts starting with the word “whatever” are a terrible idea :grinning:
Just brainstorming

Kiryl Dziamura (Jan 12 2024 at 01:19):

I also like to think about “?” vs “!” from an emotional perspective. Like, how would I feel the language? Doubtful and cautious or confident and straightforward?
Because this punctuation will be everywhere.

Brendan Hansknecht (Jan 12 2024 at 01:39):

I'm pretty sure that in current roc, that ability is not possible. It is a higher order ability.

Brendan Hansknecht (Jan 12 2024 at 01:39):

Would require special language integration or expansion of abilities to support higher order functions.

Brendan Hansknecht (Jan 12 2024 at 01:42):

Something like this where Wrapper would also need to be a type variable and not a concrete type.

try: Wrapper a err, (a -> Wrapper b err) -> Wrapper b err

Brendan Hansknecht (Jan 12 2024 at 01:46):

Hmm...it may also not be generic enough. Cause if you want to use ! with a PRNG that has no failure case it would be:

next: Generator a, (a -> Generator b) -> Generator b

Not sure I quite have the right type, but it has no failure type variable and I believe the generator encodes what type to produce from the rng...

Kiryl Dziamura (Jan 12 2024 at 03:03):

Feels like the same problem would be for PRNG and with. Then ? makes sense only for Result because of the early return ability.

And then ! can be used to unwrap (or pull?) anything based on the type. Like, Task.await or PRNG.next. But it would require a mechanism to define the default “puller”.

What I don't like about with is that it’s effectively an operator overload. I would prefer no overload at all rather than that. I anticipate situations where you spend energy thinking about what is the best behavior for the ! in a context and then have to refactor it anyway.

with can be a concept similar to let/where in haskell btw

Brendan Hansknecht (Jan 12 2024 at 03:05):

Yeah, with is a block scope operator overload for a function that takes a specific type and a closure.

Brendan Hansknecht (Jan 12 2024 at 03:06):

I'm not sure what you mena by require a mechanism to define a default puller?

Kiryl Dziamura (Jan 12 2024 at 03:23):

It was a continuation of the type-based operator behavior idea. If it’s used with Task - it calls await, for a generator it calls next etc. Yes, it's still implicit, too specific, and not possible with the current state of the world. Just thinking aloud.

Agus Zubiaga (Jan 12 2024 at 12:24):

Kiryl Dziamura said:

What I don't like about with is that it’s effectively an operator overload.

That’s true, but for me, the main downside of operator overloading is that it’s unclear what the operator will do in a given context.
Using something like type classes for this seems to make the problem only worse, where the function that will be run is most commonly specified in a whole different module or even package.
In the case of with, the farthest it could be is at the top of the top-level definition you’re looking at.

Richard Feldman (Jan 12 2024 at 12:30):

true, although to be fair it's usually specifying a function (e.g. Task.await) that lives in another module anyway :big_smile:

Agus Zubiaga (Jan 12 2024 at 12:34):

I should’ve said “which function will run”. I obviously don’t have a problem with calling functions in general :laughing:

Agus Zubiaga (Jan 12 2024 at 12:36):

I can easily look at the docs for Task.await by just hovering. While finding this specific implementation of the ability will probably require looking at the source.

Agus Zubiaga (Jan 12 2024 at 12:38):

Unless abilities required using only exposed functions, so they can be surfaced in the docs :thinking:

Agus Zubiaga (Jan 12 2024 at 12:41):

Anyway, I’m not completely against using type classes for this. I was cool with it in the earlier discussions. I just don’t see how they make it less of an operator overloading problem.

Kevin Gillette (Jan 12 2024 at 14:25):

Thought: "chaining" isn't the term I'd reach for to describe this (I'd probably think that it refers to |>). Perhaps "shorthand await" if we only have a fixed ! ? Maybe "managed dispatch" or something if we adopt the original proposal?

Kevin Gillette (Jan 12 2024 at 14:33):

Other note: another way to describe the desugaring is via a composing lambda:

f! a

# becomes
(\x -> f x |> Task.await) a

It's syntactically heavier, but has the benefit of keeping the full transform in its original position (though that might not hold for more complex examples), rather that a simpler desugaring that rearranges more of the code. This kind of in-place transform may click better for some readers if offered as a secondary explanation, although, at least for me, chaining syntax as described in the proposal doc clicked pretty well already.

Brendan Hansknecht (Jan 12 2024 at 15:42):

I think the biggest advantage of locking to just Task.await for ! and Result.try for ? would be ease of teaching. If they are locked to specific functions, you probably don't have to teach the desugaring at all.

I think it would be easy to explain them away to beginners as how you run a task and how you propagate the error from a result. Probably with more precise wording, but none the less easier to teach.

I think many people who use roc consistently and are not beginners will want the full power of with. Or if with isn't in the language, they would want backpassing or similar.

Richard Feldman (Jan 14 2024 at 22:01):

here's a simple idea for unifying them:

• there's only one operator, namely !
• it desugars to something different depending on the type it's used on
• it does this by looking up the module where the opaque type is originally defined. If that module exposes something named andThen, that's what will be used as the chaining function.

Richard Feldman (Jan 14 2024 at 22:01):

so essentially, "! is sugar for andThen"

Richard Feldman (Jan 14 2024 at 22:02):

it wouldn't work for Result, because Result is a tag union and not an opaque type, so "the module where it's defined" isn't a thing.

Personally I don't think that's a problem...neither Elm nor OCaml have syntax sugar for chaining Results and it's fine.

Fabian Schmalzried (Jan 14 2024 at 22:03):

Would be possible to add andThen to Result to make it work.

Richard Feldman (Jan 14 2024 at 22:04):

no, because Result is just a type alias for [Ok ok, Err err]

Richard Feldman (Jan 14 2024 at 22:04):

the idea only works on opaque types, and if Result were opaque, you couldn't pattern match on it

Brendan Hansknecht (Jan 14 2024 at 22:10):

I'm not really a fan of that, but I get it. I just feel that after task, result is the most likely us case.

After that, probably custom command types like would be used in roc-pg. That said, there is no guarentee those types have to be opaque.

It also sounds kinda annoying in general to need to pick between pattern matching and this feature.

Brendan Hansknecht (Jan 14 2024 at 22:11):

Probably fine, bit feels like an unnecessary restriction

Brendan Hansknecht (Jan 14 2024 at 22:12):

It also will be harder to follow than using the with syntax. The with syntax will clearly show what function us used. This will hide that in the type definition (at the same time as long as it isn't abused it would hopefully be clear).

Brendan Hansknecht (Jan 14 2024 at 22:13):

Given adding this feature is removing backpassing, I am definitely in favor of keeping it flexible

Isaac Van Doren (Jan 14 2024 at 22:18):

I would definitely miss not being able to use it with results

Richard Feldman (Jan 14 2024 at 22:19):

interesting! Do you use Result.try with backpassing currently?

Isaac Van Doren (Jan 14 2024 at 22:33):

Actually I guess I don't use it that frequently. But it is very cool. So "definitely miss" is probably an exaggeration :sweat_smile:

Brendan Hansknecht (Jan 14 2024 at 22:42):

I have use the equivalent feature a lot in rust ?

Brendan Hansknecht (Jan 14 2024 at 22:42):

I don't use It as often in roc cause for most things I do in roc they are quick scripts where I just crash on error.

Brendan Hansknecht (Jan 14 2024 at 22:42):

I would expect to use It more in the future as I eventual do library work or larger apps.

Richard Feldman (Jan 14 2024 at 22:57):

I also use it a lot in rust, but almost always with I/O

Richard Feldman (Jan 14 2024 at 22:58):

in other words, my use of it in Rust is more analogous to Task.await than Result.try

Brendan Hansknecht (Jan 14 2024 at 23:02):

I think my use has been much more split. Plenty of non-task functions

Richard Feldman (Jan 14 2024 at 23:40):

here's a variation on the type-directed idea, which could work with Result

Luke Boswell (Jan 15 2024 at 00:42):

I like this variation.

Brendan Hansknecht (Jan 15 2024 at 00:51):

Yeah, I think that is reasonable overall. Worth testing at least. A user can always wrap result if they want to test the alternative format with Result.try

Brendan Hansknecht (Jan 15 2024 at 00:52):

My one concern is:

Only builtin abilities can use compound type variables. If other abilities try to use them, the result is a compiler error which links to an explanation of the various considerations leading to that design.

Not having higher kinda abilities is already a hot topic. Do you think this will really annoy a set of users? Essentially kinda having the feature but not giving it to the users.

Brendan Hansknecht (Jan 15 2024 at 00:54):

It's kinda proof the feature is needed in some cases.

Brendan Hansknecht (Jan 15 2024 at 00:55):

Doesn't bother me personally at all, but I know it is a common topic

Richard Feldman (Jan 15 2024 at 01:12):

it's possible, but I figure we can follow the usual process - talk about motivating use cases, tradeoffs, etc.

Isaac Van Doren (Jan 15 2024 at 01:27):

I like this variation also. Nice that it uses a single operator and is very concise

Kiryl Dziamura (Jan 15 2024 at 02:41):

Ha! That's what I meant under type/ability-based behavior, but articulated in adequate language! :grinning_face_with_smiling_eyes:

Speaking of chaining of Results in Rust. There is at least one place where your code is ?-driven: wasm that communicates with Web API in browsers (via wasm-bindgen). Tons of things can throw or not be implemented in a browser, so andThen and ? become your best friends. But yeah, it's IO in terms of roc, so probably not relevant.

Agus Zubiaga (Jan 15 2024 at 03:12):

The main downside I see is that beginners will have to learn abilities to fully understand how simple Roc programs work. However, I like how this is just one feature that once learned can be applied usefully in a lot of cases.

Agus Zubiaga (Jan 15 2024 at 03:17):

FWIW I showed the examples using ! vs backpassing to a friend who has no FP experience, and he told me he found the former much less intimidating

Agus Zubiaga (Jan 15 2024 at 03:18):

which is I guess not surprising but I wanted to mention it :)

Richard Feldman (Jan 15 2024 at 03:18):

yeah I did the same thing too before proposing it :big_smile:

Richard Feldman (Jan 15 2024 at 03:20):

Agus Zubiaga said:

The main downside I see is that beginners will have to learn abilities to fully understand how simple Roc programs work.

I was thinking of teaching ! in terms of Task first, and then introducing abilities with Set, and then from there working up to the non-Task uses of !

Agus Zubiaga (Jan 15 2024 at 03:22):

In addition to AndThen and Map2, could we also get Wrap for wrapping a value in the type? That way we can simplify record builders even further

Agus Zubiaga (Jan 15 2024 at 03:25):

Also for the elseless-if, as you mentioned

Brendan Hansknecht (Jan 15 2024 at 03:25):

Would all 3 of those be overloaded to the same symbol? Would that be confusing?

Agus Zubiaga (Jan 15 2024 at 03:27):

Hm, not all things with map2 naturally have andThen

Agus Zubiaga (Jan 15 2024 at 03:29):

For example, roc-pg Selection has map2 because you can merge them, but they can’t have andThen because that’d require somehow running roc code in the middle of a query to determine the next selections

Agus Zubiaga (Jan 15 2024 at 03:34):

(That can happen at the command level, but that’s a different concept)

Richard Feldman (Jan 15 2024 at 03:37):

Agus Zubiaga said:

In addition to AndThen and Map2, could we also get Wrap for wrapping a value in the type? That way we can simplify record builders even further

I was thinking about this, but I'd like to try it without first and see how it goes

Agus Zubiaga (Jan 15 2024 at 03:38):

Ok cool, it’s a minor improvement anyway

Richard Feldman (Jan 15 2024 at 03:38):

when using record builder with tasks, I think I'd prefer an explicit Task.ok { ... } over having it be inferred, so you can see it's returning a task

Richard Feldman (Jan 15 2024 at 03:39):

because in every other case we do it that way :big_smile:

Agus Zubiaga (Jan 15 2024 at 03:40):

I see, that’s a good point

Richard Feldman (Jan 15 2024 at 03:41):

also I realized Wrap wouldn't be needed for "if without else, because as long as that's not the very end of the expression (e.g. it's in the middle of defs), then there's more chaining happening in there anyway, so the whole intermediate conditional can be expressed in terms of andThen

Agus Zubiaga (Jan 15 2024 at 03:43):

Ohh, that’s right!

Agus Zubiaga (Jan 15 2024 at 03:44):

and I guess we wouldn’t allow the last thing to be an “if without else” anyway

Agus Zubiaga (Jan 15 2024 at 03:45):

That’d be weird in an expression oriented language

Johan Lövgren (Jan 15 2024 at 08:12):

I also like this variation. Makes me curious to see how the record builder syntax will look.

Kevin Gillette (Jan 15 2024 at 16:33):

Richard Feldman said:

here's a variation on the type-directed idea, which could work with Result

Aside: I'm confused by the [Nullable a, NotNull] example. If something is Nullable but has a value, that suggests that it's not null, whereas if something is NotNull, presumably it should have a value, but it can't!

Brendan Hansknecht (Jan 15 2024 at 16:34):

Probably should be : Nullable a : [ NotNull a, Null ]

Richard Feldman (Jan 15 2024 at 17:06):

oops, fixed!

Sky Rose (Jan 15 2024 at 19:03):

What happens if you use ! on the last "statement" of a function, instead of leaving it off? In the old proposal there was an implied Task.ok {} that would get added for the last callback. With the ability, how does it know how to handle that case?

Also, does the new proposal keep the feature you liked from earlier where you can omit the assignment iff the function returns a Task {} _?

Richard Feldman (Jan 15 2024 at 20:28):

Sky Rose said:

does the new proposal keep the feature you liked from earlier where you can omit the assignment iff the function returns a Task {} _?

yeah, that should still Just Work

Sky Rose said:

What happens if you use ! on the last "statement" of a function, instead of leaving it off? In the old proposal there was an implied Task.ok {} that would get added for the last callback. With the ability, how does it know how to handle that case?

good question! If we wanted to do that, we'd have to add an extra Wrap ability, so I'd like to try it without and see if it's worth it

Richard Feldman (Jan 15 2024 at 20:28):

honestly I'd separately been second-guessing that part of the original proposal, because it feels like it could get confusing as soon as you started modifying tasks with |> Task.mapErr! and using them concurrently etc.

Richard Feldman (Jan 15 2024 at 20:29):

like maybe it would become considered bad style even if it were allowed

Johan Lövgren (Jan 15 2024 at 20:44):

Would it be worthwhile to consider adding wrap to the andThen and map2 abilities? Like return for monads and pure for applicatives

Brian Carroll (Jan 15 2024 at 20:45):

My opinion is that there should be nothing implicit and no Wrap ability. The last expression in your function... must have the type that your function signature says it returns! (In fact this is so incredibly obvious I can't believe I'm saying it! :laughing: ) There simply is no problem to solve here.

Brian Carroll (Jan 15 2024 at 20:47):

If your function returns a Task then you need to return the value in a Task. So you might need to do Task.ok to do that. That is fine. There is no problem to solve.

Richard Feldman (Jan 15 2024 at 20:48):

yeah that's kind of how I'm feeling about it

Brian Carroll (Jan 15 2024 at 20:54):

It would be different if we were allowing higher-kinded types in user space, like Haskell does. I think.

Rene Mailaender (Jan 16 2024 at 21:19):

Very interesting discussion. At first I was very pro new syntax, but then again something felt a little off to me.

Actually, I don't think it's the back passing syntax what gives beginners a hard time. I think it is the concept of back passing. The fact that x looks like a assignment when it's a parameter instead. And that the function body follows in the next line.

x <- File.readBytes path |> Task.await
# body of the function

It feels like a decoupling of a function call (and its parameter) and its body.

Because of that, to my own surprise, I actually prefer the old syntax for one simple reason. It makes it very clear and obvious that it's not a simple assignment, but something different. And I think that is good.

I mean correct me if I'm wrong. Even with the new syntax, at it's core it's still a kind of back passing, right? My concern is, that the ! syntax looks and feels to similar to a kind of assertion in other languages. Especially when it's compared with a = which usually suggests an assignment.

I would argue, that this makes it even harder for beginners to fully understand the implications of back passing. Even worse, it possibly would make it quite easy to fall into the trap of thinking understanding it, even one is not.

Restricting it to only work on the andThen ability seems to be a good idea, but I think people who still want to use the back passing on something different, actually can.

x = File.readBytes path |> Task.attempt! Task.ok

when x is
    Ok _ -> Task.ok {}
    Err _ -> Task.err {}

I think this would work, right? But it makes it way less obvious, that x actually is a parameter of a function call, in my opinion.

Brendan Hansknecht (Jan 16 2024 at 21:43):

all your points are correct around the underlying implementation and the potential ways to use it.

Isaac Van Doren (Jan 17 2024 at 00:43):

What if the rest of the proposal was the same but instead of using = it still used <- ? That would make it more clear that uses of ! aren’t assignments

Agus Zubiaga (Jan 17 2024 at 00:56):

Hm, but then you wouldn’t be able to use it inline such as in if conditions or nested inside expressions

Brendan Hansknecht (Jan 17 2024 at 01:00):

I mean we could just allow leaving off {} <- technically.

Brendan Hansknecht (Jan 17 2024 at 01:01):

After working on Rocci bird I have two thoughts:

1. This is very clearly not an assignment in terms of perf and code gen.
2. The current systax can be really heavy handed at times and ! would be more readable.

Richard Feldman (Jan 17 2024 at 01:31):

Brendan Hansknecht said:

This is very clearly not an assignment in terms of perf and code gen.

true, but I think the relevant part here is scanning for ! suffixes vs <-s outside record builders

Brendan Hansknecht (Jan 17 2024 at 01:32):

:+1:

Richard Feldman (Jan 17 2024 at 01:35):

Rene Mailaender said:

Because of that, to my own surprise, I actually prefer the old syntax for one simple reason. It makes it very clear and obvious that it's not a simple assignment, but something different. And I think that is good.

there's certainly some value to that, but of course the most obvious is not to have any syntax sugar at all and write out normal lambdas :big_smile:

Richard Feldman (Jan 17 2024 at 01:36):

so I appreciate the point that backpassing looks more like lambdas, but the thing is, we've tried that experiment for a couple of years and the experimental results are that a lot of beginners struggle with them

Richard Feldman (Jan 17 2024 at 01:37):

so maybe it turns out that beginners struggle even more with ! syntax, but I think there's enough justification here that it might be better that it seems worth trying out

Richard Feldman (Jan 17 2024 at 01:38):

it's definitely encouraging that showing people with no functional programming background samples of both syntaxes, they seem to find the ! syntax easier to follow

Richard Feldman (Jan 17 2024 at 01:38):

I assume they aren't intuitively guessing that it desugars to lambdas, but I don't know if that's actually essential to beginner understanding

Richard Feldman (Jan 17 2024 at 01:39):

to me, the essential part is that beginners understand how to put basic Roc programs together that work

Richard Feldman (Jan 17 2024 at 01:40):

there are several levels of understanding around how tasks work, one of which is that it's lambdas all the way down, probably ending with the in-memory representation the host sees, but I don't think it's necessarily a problem if beginners don't grasp that right away—provided they're making good progress toward the more important beginner milestone of being able to build Roc programs on a basic level

Brendan Hansknecht (Jan 17 2024 at 01:44):

Yeah, I definitely think that ! enables beginners to mustly ignore it until it becomes more relevant.

Brendan Hansknecht (Jan 17 2024 at 01:44):

seeing <- people will immediately question it

Rene Mailaender (Jan 18 2024 at 21:22):

I see your points and I'm aware of back passing being part of the language for some years now and that beginners, me included, struggle with it. But I realise I might have come across, as if I was making an argument for keeping the old syntax in general. That wasn't my intention. Only by direct comparison of both syntaxes (old and new), I would slightly prefer the old one. And only because it let's you pause and question what is actually happening.

That being said, I think it's a good idea to try something new. :)

Richard Feldman said:

I assume they aren't intuitively guessing that it desugars to lambdas, but I don't know if that's actually essential to beginner understanding

to me, the essential part is that beginners understand how to put basic Roc programs together that work

I agree on that. But I think it's also important, a beginner has a clear understanding, they might not understand back passing yet. In my experience it leads to way worse decisions when some thinks them understand something, when they are actually not.

Richard Feldman said:

Rene Mailaender said:

Because of that, to my own surprise, I actually prefer the old syntax for one simple reason. It makes it very clear and obvious that it's not a simple assignment, but something different. And I think that is good.

there's certainly some value to that, but of course the most obvious is not to have any syntax sugar at all and write out normal lambdas :big_smile:

haha, true.
though, I think it actually provides some real value, so getting it right is something worth "fighting" for. :D

For me the value lies in the reduction of indention, for the price of being "trapped" in the last lambda. So it's a nice trade off.

In that regard, I had the idea to instead of back passing the lambda, just use a regular lambda, bad mark it as "unindented" or "taking over". This could be marked by "blocking" a regular lambda arrow with e.g. a |. So a -> would become a ->|

So a chain with regular lambdas:

doSmth = \path ->
    File.readBytes path |> Task.await \bytes ->
        Stdout.line „some logging“ |> Task.await \{} ->
            Task.ok bytes

would become this:

doSmth = \path ->
    File.readBytes path |> Task.await \bytes ->|
    Stdout.line „some logging“ |> Task.await  \{} ->|
    Task.ok bytes

That way the familiar lambda "direction" stays the same, but with the benefit of no indention.

And to build a records would look like this now:

await getApples \apple ->|
await getBanana \banana ->|
Task.ok {apple, banana}

Brendan Hansknecht (Jan 18 2024 at 21:28):

I think the problem of syntaxes that just don't indent lambdas is that they put very important information somewhere that you don't know to look. It is all the way at the end of the line.

Brendan Hansknecht (Jan 18 2024 at 21:32):

While this is technically the case with regular lambdas, with regular lambdas:

1. We keep them simple. If they get large, we extract them into other functions. This doesn't fit well with the long nested chaining of tasks.
2. Relatedly, variables often have obvious/trivial names because they are sort lived.
3. The indentations lets us know that we need to look at the previous line for names
4. You can always format a normal lambda differently and it will look fine.

For 4 specifically think of this:

Task.await
    (File.readBytes path)
    \bytes ->|

# Does that continue here or tabbed in, just kinda strange as a syntax with multiple line inputs.

Fabian Schmalzried (Jan 18 2024 at 22:20):

Rene Mailaender said:

I agree on that. But I think it's also important, a beginner has a clear understanding, they might not understand back passing yet. In my experience it leads to way worse decisions when some thinks them understand something, when they are actually not.

In this specific case, I would disagree about it beeing harmful. I might even teach it "wrong" to beginners that are not used to functional:
"!, let's you execute a task and gives you the result, similar to await in other languages. Later you will learn, that it can do more."

Last updated: Jun 16 2026 at 16:19 UTC