Richard Feldman (Sep 06 2024 at 21:13): Richard Feldman (Sep 06 2024 at 21:13):here's a version of the proposal that I'm feeling much better about than the previous 2 drafts! (See #ideas > Purity Inference proposal v2 for the previous discussion.)
It ended up with .roc files today not changing much except for the type signatures, but a bunch of things get nicer:
https://docs.google.com/document/d/1ZVD3h5jLpQNFSDXTg2RkzPhNXz5EErUXBBjN8TuyiqQ/edit?usp=sharing
any thoughts welcome!
Sam Mohr (Sep 06 2024 at 21:52):"We discussed how adding a space resolved this issue:" there's no space after the ? here
Luke Boswell (Sep 06 2024 at 22:31):Could you please add a link to the previous version(s) in the doc?
Richard Feldman (Sep 06 2024 at 22:42):fixed and fixed!
Sam Mohr (Sep 07 2024 at 00:29):Another (obligatory?) thought train:
=> in place of -> for function definitions. It's also good for when branches, but in general it tells us what's effectful without any more code written, as in same # of chars, no type signatures needed. I think this was in v2 but I just missed it?) Sam Mohr (Sep 07 2024 at 00:29):=> in when branches seems much less important to me than => as the lambda operator. when branches are not very long, and the ! is already there. And now I need to replace my -> with => when I change what I write... Not a strong opinion, but I lean a bit against as long as the lambda operator is now => when the function is effectful. Richard Feldman (Sep 07 2024 at 00:30):yeah I don't think it's necessary in the when branches either haha
Richard Feldman (Sep 07 2024 at 00:30):I'd be fine with then in those :shrug:
Richard Feldman (Sep 07 2024 at 00:30):when color is Red then makes sense to me
Richard Feldman (Sep 07 2024 at 00:31):in terms of how it reads
Richard Feldman (Sep 07 2024 at 00:31):when color is
Red then ...
Yes, then is now common for control flow syntax. Less terse, but good
Sam Mohr (Sep 07 2024 at 00:33):?> operator is really nice! I think it doesn't blow our "operator soup" budget, and it extends the Result-centric operator pool, meaning we aren't giving tools that will get exploited. It seems like having it would interfere with standard ? usage. In a normal pipeline:data =
Http.get! "https://api.com/"
|> Json.decode?
|> Result.mapErr \err -> DecodeError err
This is invalid, because the decoding error is already returned after the 3rd line, so we don't make it to the Result.mapErr. I think it'd be better to do this:
data =
Http.get! "https://api.com/"
|> Json.decode
?> \err -> DecodeError err
otherwise, we need to make the first example work by changing how ? works semantically
Sam Mohr (Sep 07 2024 at 00:35):It'd go from "return an error early here" to maybe "return an error at the end of this pipeline, unless there are two ?'s, in which case just don't do that?"
Sam Mohr (Sep 07 2024 at 00:45):! work in two different ways, a.k.a. sometimes like !?. Though it removes our occasional interrobang problem, it means that we have a "polymorphism" that we don't really have anywhere else. The cited example of polymorphism is with tags, but I don't think that's the same thing. When I can write Result.mapErr loadResult LoadErr and LoadErr, the type of LoadErr is always [LoadErr [SpecificErr]]. However, polymorphic ! can no longer be used to return Results that are evaluated later, unless you wrap the effectful code in a closure:File.tryRead : Result (Result Str [FailedToRead]) [OtherErrs]
readAttempt =
File.tryRead! "file.txt"
I either need File.tryRead to return a nested Result, or wrap readAttempt in a closure.
Sam Mohr (Sep 07 2024 at 00:48):This is a pretty niche situation, but I'd say that File.tryRead!? is also niche, especially if the new ?> operator does the early returning for us. I'd rather have more numerous, explicit operators over variably-behaving operators.
Anton (Sep 07 2024 at 12:28):Hah, I love that there is a wikipedia article on interrobang :p
Sky Rose (Sep 07 2024 at 13:49):+1 to Sam's point 4. Making ! short circuit Results is saying that we expect nearly every effect to be short circuited. This is true in scripts, but: One reason Roc's type system is so good is it guides you into correctly handling all errors. By making short circuiting the only thing that's easy to do, this proposal now tries to make it so you _don't_ do anything interesting with your errors.
I think this proposal does work really well for the goal of making scripts easy and natural to write, though. I like the ?> operator. It really smoothly fits with ? and |>. It feels natural and not arbitrary.
Sky Rose (Sep 07 2024 at 13:49):What is when! ? It shows up in the proposal but I didn't see it explained.
Richard Feldman (Sep 07 2024 at 14:37):oops, that was leftover from an earlier revision - removed!
Richard Feldman (Sep 07 2024 at 14:45):maybe it's just me, but personally I find !? so ridiculous-looking that I can't even imagine wanting to get up on a stage and show code on a slide that includes it. Or write a blog post about it. I dislike it so much that I think it would make me feel ashamed to try to promote the language.
Richard Feldman (Sep 07 2024 at 14:46):I do get that it's a straightforward design in terms of composition, but there's a line where code that is logically reasonable looks too ridiculous to accept as a normal thing in the language, and for me !? is so far over that line I can't even see the line anymore
Richard Feldman (Sep 07 2024 at 14:49):the criticism of ! always short-circuiting is totally reasonable (e.g. it's polymorphic in an unprecedented way, and it assumes that propagating errors is the best default when maybe it's not) but I think if we want to discuss alternatives, !? is not an alternative we should spend time discussing because at this point I just can't imagine a world where that ends up being the final design :sweat_smile:
Richard Feldman (Sep 07 2024 at 14:52):Sky Rose said:
Making
!short circuit Results is saying that we expect nearly every effect to be short circuited. This is true in scripts, but: One reason Roc's type system is so good is it guides you into correctly handling all errors. By making short circuiting the only thing that's easy to do, this proposal now tries to make it so you _don't_ do anything interesting with your errors.
an important note here is that short-circuiting doesn't cause any more or fewer errors to be handled; what it does is change what code runs (or doesn't) if there's an error
Richard Feldman (Sep 07 2024 at 14:52):if no short-circuiting happens, then you get back a Result
Richard Feldman (Sep 07 2024 at 14:52):no error has been handled! Result is explicitly saying "there is an unhandled error here which must be handled before you can access the non-error value"
Richard Feldman (Sep 07 2024 at 14:54):so to me, short-circuiting is about whether we think deferred or immediate error handling is more likely to be desired - e.g. handling multiple errors in one central place versus handling them right away
Richard Feldman (Sep 07 2024 at 14:56):and we don't have to speculate about this; back in the backpassing days (and before backpassing was introduced) you had to use either Task.await to defer errors from effects, or Task.onErr to handle them immediately. In practice, deferring (Task.await) was so common that I'm not even sure how much awareness there was of Task.onErr as an alternative :sweat_smile:
Richard Feldman (Sep 07 2024 at 14:56):so I agree that this design assumes that deferring the error for later will be the most commonly desired thing to do, but that's because we already know in practice that this is the most commonly desired thing to do, and not just in scripts :big_smile:
Richard Feldman (Sep 07 2024 at 14:58):as an aside, the way I ended up at this version of the proposal was that I kept being bothered by how the previous version seemed to result in code that looked less nice than status-quo code
Richard Feldman (Sep 07 2024 at 15:00):like today ! is used all over the place to do deferred error handling, because that's what makes the most sense in the code, and it's also used when there is no error to defer at all (e.g. a Task which never fails) and it's been a total non-issue to have ! used even when there is no possibility of failure, yet it's also been nice to have those marked as effectful (e.g. the Rocci Bird example from the doc).
Richard Feldman (Sep 07 2024 at 15:02):that's where the idea of having ! Just Work (and not short-circuit) with effectful functions that return a non-Result came from: it's unusual, for sure, but the point is that it preserves the nice-looking code we have today with Task while removing the need for Task.attempt, Task.result, Task.fromResult, etc.
Richard Feldman (Sep 07 2024 at 15:03):the other thing that having ! work two different ways reminds me of is type variables in open tag unions
Richard Feldman (Sep 07 2024 at 15:05):depending on when you got into the language, you may or may not remember the common and very long discussions we used to have about "open tag unions" because we used to be syntactically explicit about polymorphism in tag unions - e.g. when you put Foo into the repl, it would print [Foo]* - and although this was more explicit than today where it just prints [Foo] (it's still polymorphic, we just don't show it visibly anymore because it turns out to be not only unnecessary but actively confusing/unhelpful) it's just resulted in an overall better experience for everyone
Richard Feldman (Sep 07 2024 at 15:05):(as evidenced by the fact that we no longer have recurring long discussions about what's going on with those things!)
Richard Feldman (Sep 07 2024 at 15:06):this reminds me of that because although I get that ! working differently for Result vs not is unprecedented, explaining it as "hey it works like ? if you give it a Result and just never short-circuits if you don't give it a Result" seems like a simple explanation in the same way that "tags can be used where a function is expected and it Just Works" is also a simple explanation
Richard Feldman (Sep 07 2024 at 15:09):Sam Mohr said:
polymorphic
!can no longer be used to returnResults that are evaluated later, unless you wrap the effectful code in a closure:File.tryRead : Result (Result Str [FailedToRead]) [OtherErrs] readAttempt = File.tryRead! "file.txt"
this is true, but having when conditions not short-circuit the entire when is explicitly designed to address this - so that you don't need a tryRead and can just do this:
when File.read! "file.txt" is
this is definitely not convenient in the following very specific scenario:
ResultResult Richard Feldman (Sep 07 2024 at 15:13):in that scenario there are a few ways you can get the Result:
result =
when File.read! "file.txt" is
Ok val -> Ok val
Err val -> Err val
result =
answer = File.read! "file.txt"
answer
result = (\{} => File.read! "file.txt") {}
none of these are what I'd consider beautiful or elegant solutions, but my hypothesis is that wanting a Result but not wanting to handle it immediately comes up almost never in practice
Richard Feldman (Sep 07 2024 at 15:14):the reason I wanted to change the when behavior is again based on past experience: the pattern I've seen used overwhelmingly often is the one that's on the homepage example:
when Richard Feldman (Sep 07 2024 at 15:18):because this is the way it's basically always done, the complaints I've heard over the years have mostly been that trying to follow this pattern has been unergonomic, e.g.
<- and then later in the line |> so the arrows going in both directions on the same line)Task to handle the errors, but not being sure when/how to convert it into a Result in conjunction with the chaining that was happening elsewhere. (I like when runEffects! is because there isn't a special function you have to call, you just make the call and pattern match on the result immediately.) Richard Feldman (Sep 07 2024 at 15:32):I just added to the end of the doc the current homepage example before/after this proposal - I think it illustrates this pretty nicely!
Anton (Sep 07 2024 at 15:35):I just added to the end of the doc the current homepage example before/after this proposal
The arrows there still need to be updated to =>
Richard Feldman (Sep 07 2024 at 15:35):thanks, fixed! :thumbs_up:
Richard Feldman (Sep 07 2024 at 15:37):today's main:
main =
Path.fromStr "url.txt"
|> storeEmail
|> Task.onErr handleErr
without the pipeline:
main =
Task.onErr (storeEmail (Path.fromStr "url.txt")) handleErr
proposed:
main = \arg =>
when storeEmail! (Path.fromStr arg) is
Ok dest -> Stdout.line! "Wrote email to $(dest)"
Err err -> handleErr! err
I think for a beginner, the when version is easier to understand than either the pipeline or non-pipeline versions
Richard Feldman (Sep 07 2024 at 15:40):like "I run storeEmail! and then depending on whether it returns Ok or Err I do one thing or another"
Jasper Woudenberg (Sep 07 2024 at 15:41):Would the "when-expression-doesnt-short-circuit" rule be confusing when calling an effectful function that returns tags in a result?
To clarify, as I understand it the following wouldn't compile:
readStoplightById : U64 -> Result [Red, Orange, Green] Str
main = \_ ->
when readStopLightById! 12 is
Red -> Stdout.line! "Hit the brakes!"
Orange -> Stdout.line "Hit the accelerator!"
Green -> Stdout.line "Good to go!"
that's correct, it would have to be Ok Red -> etc.
Jasper Woudenberg (Sep 07 2024 at 15:43):How would you write that code snippet above correctly if you don't want to handle errors?
Richard Feldman (Sep 07 2024 at 15:43):that's a reasonable concern, but the rule has to be one or the other (either it short-circuits the surrounding when or it doesn't) and it seems like the risk is there no matter which behavior we choose
Richard Feldman (Sep 07 2024 at 15:43):as in, either way someone might expect it to do the other thing :big_smile:
Richard Feldman (Sep 07 2024 at 15:44):Jasper Woudenberg said:
How would you write that code snippet above correctly if you don't want to handle errors?
as in, short-circuit the error? This would do that:
readStoplightById : U64 -> Result [Red, Orange, Green] Str
main = \_ =>
color = readStopLightById! 12
when color is
Red -> Stdout.line! "Hit the brakes!"
Orange -> Stdout.line! "Hit the accelerator!"
Green -> Stdout.line! "Good to go!"
this would cause main to return Result though, which may or may not be desirable haha
Jasper Woudenberg (Sep 07 2024 at 15:51):Fair enough, and the solution looks nice!
I'm curious if folks will come up with that solution, it requires an active awareness that the behavior of ! is different in between when .. is then outside of it, otherwise you won't know it helps to lift the expression out of the when. It's not something folks might have active awareness for if we don't emphasize it in documentation because we're banking on the code intuitively doing the right thing.
But it's maybe something we have to try out to know whether it's going to trip people up.
Richard Feldman (Sep 07 2024 at 15:52):yeah agreed
Richard Feldman (Sep 07 2024 at 15:52):also a thing I think is important is that, although you might not necessarily realize right away how to do it, I think it's clear in both cases what the code is doing to a beginner
Richard Feldman (Sep 07 2024 at 15:54):ideally it would be both clear what it's doing and easy to figure out that that's what you should do, but given how much time has now gone into trying to find a design like that, I'm increasingly coming to think this is one of these "you want N things to be true about the design, but none of the possible designs we can find have more than N-1 of them at the same time"
Jasper Woudenberg (Sep 07 2024 at 15:54):Yeah, I agree readability is great
Notification Bot (Sep 07 2024 at 17:15):2 messages were moved from this topic to #ideas > opting out of short-circuiting by Richard Feldman.
Kevin Gillette (Sep 07 2024 at 19:04):This unlocks the ability in the future for platforms to specify effectful operations which don't go through the state machine.
How would such code be tested? Would tests just simulate the effect?
Richard Feldman (Sep 07 2024 at 19:11):yeah all the application code would look identical either way, including tests
Richard Feldman (Sep 07 2024 at 19:12):it would just be a behind-the-scenes platform implementation detail with performance tradeoffs
Sky Rose (Sep 07 2024 at 21:22):Richard Feldman said:
so I agree that this design assumes that deferring the error for later will be the most commonly desired thing to do, but that's because we already know in practice that this is the most commonly desired thing to do, and not just in scripts :big_smile:
If pushing people towards collecting errors and handling them all at once is intentional, then that addresses my main concern. :thumbs_up:
The example in the proposal with storeEmail! does this centralized handling by wrapping the effectful code and the handleErr in separate functions so that when storeEmail! path is fits on one line. Is it possible to do that handling within the same function, which would be shorter and more readable in some situations? Would this work?
main : Str => U8
main = \path =>
result =
str = File.readUtf8! path ?> FileReadError
num = Str.fromU32? str ?> ParseError
File.writeUtf8! path (Str.toU32 num) ?> FileWriteError
when result is
Ok _ -> 0
Err (FileReadError _) -> 1
Err (ParseError _) -> 2
Err (FileWriteError _) -> 3
that should work, yeah!
Richard Feldman (Sep 07 2024 at 22:09):Sam Mohr said:
data = Http.get! "https://api.com/" |> Json.decode ?> \err -> DecodeError err
I hadn't thought about using this operator in pipelines - what if we made it be |? instead of ?> so it looks nicer in pipelines?
data =
Http.get! "https://api.com/"
|> Json.decode
|? DecodeError
data =
Http.get! "https://api.com/" |> Json.decode |? DecodeError
data =
Http.get! "https://api.com/" |? HttpGetFailed
I think the ?> looks nicer in the pipeline.
Sam Mohr (Sep 07 2024 at 22:52):Yes, the |? visually looks better to me because of the left side alignment, but the > is the most important character IMO because it implies the piping direction
Sam Mohr (Sep 08 2024 at 00:41):Just a sidenote, but ignoring the tedium of how much work purity inference will be to implement, I think it'll actually make things conceptually simpler in the compiler as well as in the Roc dev's mind! Tasks require a lot of thunk bundling, whereas here we just call functions, and color values/functions as effectful.
Kevin Gillette (Sep 09 2024 at 17:04):I like the idea of then instead of ->/=> for when cases. The syntactical inconsistency between if and when always felt like an esoteric or arbitrary choice. Generally, the "why not both" approach to choosing between keywords and symbols in control flow never felt stylistically great to me.
Kevin Gillette (Sep 09 2024 at 17:08):Also, given that ! would need to be used in all effectful calls (thus is pretty good effect signaling), even if the formatter were to do a good job translating between => and -> in when cases, it still feels like unnecessary syntactic coupling; anyone who is writing code without use of an autoformatter would waste time making sure the ! and => have proper "verb agreement".
I agree that => makes sense for function syntax. I disfavor the same for when because I feel that then is a much better choice.
Richard Feldman (Sep 09 2024 at 17:28):I kinda lean that direction too, and I do think that the status quo of -> feels weird in a world where there's a distinction between -> and => in functions
Richard Feldman (Sep 09 2024 at 17:28):how do others feel about when ... is ... then?
Sam Mohr (Sep 09 2024 at 17:33):I think my problem with when-then is actually a formatting problem. The main benefit I get from -> over then is that -> is visually sparse, so it's easy to skip over. Less so with a keyword:
rgb = when color is
Red -> (255, 0, 0)
Green -> (0, 255, 0)
Blue -> (0, 0, 255)
# versus
rgb = when color is
Red then (255, 0, 0)
Green then (0, 255, 0)
Blue then (0, 0, 255)
But I think if we did the golang struct formatting approach, then its more clear:
rgb = when color is
Red then (255, 0, 0)
Green then (0, 255, 0)
Blue then (0, 0, 255)
But even without the formatting change, I think it's not as visually distinct between the pattern and the expression in the then over the -> case, but then is more consistent, so it's probably the right move...
Sam Mohr (Sep 09 2024 at 17:35):And part of my worry comes from us using the Elm formatter instead of a Roc one, so I think having the then in a keyword color fixes my problem too.
Alex Nuttall (Sep 09 2024 at 18:19):I think it would be a shame to lose the table-like quality to the when block. A non-word symbol is much better at representing a column boundary than a word
Brendan Hansknecht (Sep 09 2024 at 19:15):I find it much harder to read with then. I think if ... then works cause it is used mostly standalone and has the precursor if.
Brendan Hansknecht (Sep 09 2024 at 19:16): someComplexExpression a b c d then does not work
Brendan Hansknecht (Sep 09 2024 at 19:16):Or at least not nearly as well for visual parsing
Brendan Hansknecht (Sep 09 2024 at 19:16):tabbing would still be enough info, but it is much harder to parse quickly
Sam Mohr (Sep 09 2024 at 20:06):I agree with Brendan. I think I'm leaning towards then for consistency with if, not because I think it's more readable. I think -> is easier to parse at a glance. There are plenty of languages that do -> or => for matching, so it seems like we have enough precedent there.
Musab Nazir (Sep 09 2024 at 20:07):^ my thoughts as well. Status quo seems cleaner to read and matches other languages
Luke Boswell (Sep 09 2024 at 20:32):I like when-then even if it's a little quirky. Aligning the branches using tabs is a neat idea too.
Isaac Van Doren (Sep 09 2024 at 21:47):I prefer the current syntax for when
Isaac Van Doren (Sep 09 2024 at 22:58):I see how then is more consistent with the if syntax, but it looks really chunky to me. There’s also enough use of the arrow in other languages for similar uses that I think it switching to then might make Roc seem more foreign
Richard Feldman (Sep 10 2024 at 02:09):this feels like another "not a blocker, default to status quo for now with the option to revisit later" situation :big_smile:
Richard Feldman (Sep 10 2024 at 02:09):always happy to trim the scope of the initial implementation!
Agus Zubiaga (Sep 10 2024 at 23:48):(deleted)
Sam Mohr (Sep 17 2024 at 05:13):I know that we're not 100% sure on syntax and all, but the underlying mechanism seems to be decided already. I have a pretty good idea of what implementing this would look like (though unverified), so if anyone wants to take a stab at initial work for effectful function typing and type inference, feel free to reach out and I can write something up for you.
Isaac Van Doren (Sep 17 2024 at 19:51):Has there been any thought about if/how this proposal will impact effectful and simulation testing once it gets implemented?
Richard Feldman (Sep 17 2024 at 20:02):yeah the main thing is that it gets nicer
Richard Feldman (Sep 17 2024 at 20:02):in that expect can Just Work for either
Richard Feldman (Sep 17 2024 at 20:03):instead of a separate keyword for "the Task version of expect"
Richard Feldman (Sep 17 2024 at 20:04):but module params mean you still have to actually get the effectful functions from somewhere, and if you're in a test and aren't getting a "real" effect from the platform, the only option available is to provide simulated ones
Isaac Van Doren (Sep 17 2024 at 22:07):Nice!
Isaac Van Doren (Sep 17 2024 at 22:08):Are there any plans to support something like mocking?
Richard Feldman (Sep 17 2024 at 22:12):kinda - this design means you basically don't need a separate concept of mocking
Richard Feldman (Sep 17 2024 at 22:12):I should really make an example of this haha
Agus Zubiaga (Sep 17 2024 at 22:23):Sam Mohr said:
I know that we're not 100% sure on syntax and all, but the underlying mechanism seems to be decided already. I have a pretty good idea of what implementing this would look like (though unverified), so if anyone wants to take a stab at initial work for effectful function typing and type inference, feel free to reach out and I can write something up for you.
I hadn't said it before because there isn't a final design yet, but I have been working on the core of this proposal. We're discussing a lot of different changes, and things like try can be separate work streams, but I'd like to work on the implementation of effectful functions (syntax, typechecking, and compiling to Task).
Ayaz Hafiz (Sep 17 2024 at 22:41):i would suggest writing out in detail what compiling to Task actually looks like before trying to implement it. It is not trivial especially if you want to preserve optionality for polymorphic effects in the future. Happy to get on a call to provide insights if necessary but some of the papers i mentioned previously can also be helpful.
Agus Zubiaga (Sep 17 2024 at 22:50):That'd be great, thanks! I'm starting with syntax and type checking for now which are more straightfoward, but I'll probably take you up on that when it's time to compile :smile:
Sam Mohr (Sep 17 2024 at 22:53):So unless I'm missing something, since a Task represents a thunk that returns a Result ok err, but most of these functions return any old value, I don't think that we should be targeting a Task for the compilation unit of effectful functions. It seems that we could simply maintain a flag on (recursive) functions and closures that tracks if they are effectful, and then "color" functions during type unification.
We'd just need to tell LLVM which functions are pure and which aren't, I think right now we tell them they're all pure except for Tasks?
Sam Mohr (Sep 17 2024 at 22:54):Actually, I expect it's when we do ForeignCalls, so the above change should just fall into place
Sam Mohr (Sep 17 2024 at 22:54):Anyway, if we target Task, which is inherently lazy, we now have to eagerly run every Task right after wrapping it, which seems like an unnecessary "indirection"
Sam Mohr (Sep 17 2024 at 22:55):That's why I expect that supporting Task as a component of effectful functions would be a difficult marriage, since they should work differently
Agus Zubiaga (Sep 17 2024 at 23:02):Yeah, I didn't mean we will desugar into Task before type checking. The type system would be fully aware of the purity of a function, but when we compile they'll work just like Task. At least that's what it was proposed.
Sam Mohr (Sep 17 2024 at 23:03):I agree that desugaring to Task would not be the right behavior. But it seems that even acting as Task "under the hood" is unnecessary. Tasks are lazy, the new design is eager, right?
Brendan Hansknecht (Sep 17 2024 at 23:03):Are there any concerns with how this merges into effect interpreters? With effect interpreter, we will fundamentally be building a tag based state machine for all of the tasks. So any effectful function would compile to that somehow.
Agus Zubiaga (Sep 17 2024 at 23:03):It still needs to be compiled to the effect state machine so that the platform can control their execution
Brendan Hansknecht (Sep 17 2024 at 23:05):Agus Zubiaga said:
It still needs to be compiled to the effect state machine so that the platform can control their execution
Just a note, today this really isn't the case. Today, the platform launches the first task, but roc then runs everything and has all control. It does blocking synchronous calls to tasks like Stdout.line
Agus Zubiaga (Sep 17 2024 at 23:05):Yeah, I know. I'm just saying that it's an orthogonal change.
Sam Mohr (Sep 17 2024 at 23:05):We'd need to compile everything to continuations, right?
Agus Zubiaga (Sep 17 2024 at 23:05):yeah, that doesn't change
Agus Zubiaga (Sep 17 2024 at 23:07):From the point of view of the platform, purity inference doesn't change how effects work. We can make the effect interpreters change separately.
Agus Zubiaga (Sep 17 2024 at 23:07):There might be small differences in how hosted modules look
Sam Mohr (Sep 17 2024 at 23:08):Yeah, those will probably change aesthetically but basically work the same
Sam Mohr (Sep 17 2024 at 23:10):If you have a good idea of how things will run, then I'm happy to defer to you, the person that has been working on this. I just think that Task as it works now may not be the right way to go.
Sam Mohr (Sep 17 2024 at 23:10):Though I can see a way that Task could function as a continuation on its own
Sam Mohr (Sep 17 2024 at 23:11):Either way, best of luck with this! I'm sure you'll need it
Agus Zubiaga (Sep 17 2024 at 23:13):Oh, I didn't come up with this approach. This is what was proposed by @Richard Feldman in the doc . Happy to discuss alternatives, but I think we need to do something very similar to Task to preserve the characteristics we have now.
Sam Mohr (Sep 17 2024 at 23:14):Are you mainly referring to this version of the Purity Inference doc?
Sam Mohr (Sep 17 2024 at 23:14):I should re-read before I suggest alternatives
Sam Mohr (Sep 17 2024 at 23:15):If you don't remember, I can check myself
Agus Zubiaga (Sep 17 2024 at 23:15):I think that part was the same in all the versions
Sam Mohr (Sep 17 2024 at 23:16):Okay, I remembered some elision
Sam Mohr (Sep 17 2024 at 23:16):I'll go check later tonight
Agus Zubiaga (Sep 17 2024 at 23:20):I think v3 doesn't mention it because it's written as an amend of the previous proposals, but v2 says:
(Behind the scenes, "effectful functions" would compile to the same builtin Task data structure we have today, but the Task type would no longer be user-facing. Instead, it would become a compiler implementation detail.)
Sam Mohr (Sep 17 2024 at 23:22):V3 mentions:
This unlocks the ability in the future for platforms to specify effectful operations which don't go through the state machine. This is beneficial for things like getting the current time (especially if going for high-precision e.g. for fine-grained timing), which is actively harmed by having to go through the async machinery instead of happening synchronously. The overhead of doing this through an effectful function call, compared to having no choice but to wrap it in Task, is a strict perf upgrade.
Sam Mohr (Sep 17 2024 at 23:22):I don't know how that's possible if everything is a thunk/Task
Agus Zubiaga (Sep 17 2024 at 23:22):Yeah, I was going to mention that next. In the future, some effects might be sync and those would not be compiled to Task.
Sam Mohr (Sep 17 2024 at 23:23):When you say "we use Task under the hood", do you mean "we use the machinery that generates a thunk that returns a value", or something else?
Sam Mohr (Sep 17 2024 at 23:24):Because that machinery is still good, but to my knowledge it's really just FFI: https://github.com/roc-lang/roc/blob/3215a8f3d73daf881185e4f7474972f2059754b6/crates/compiler/can/src/task_module.rs#L57-L84
Sam Mohr (Sep 17 2024 at 23:24):And the "thunk" part, if mandated, is my issue
Agus Zubiaga (Sep 17 2024 at 23:24):Right, but that will change with effect interpreters
Sam Mohr (Sep 17 2024 at 23:25):Yes
Sam Mohr (Sep 17 2024 at 23:25):Are you trying to implement the effect interpreters basically in the same thrust?
Sam Mohr (Sep 17 2024 at 23:25):That would make this make sense to me
Agus Zubiaga (Sep 17 2024 at 23:26):No, I think that’s a separate change. I want to work on this incrementally.
Sam Mohr (Sep 17 2024 at 23:28):Okay, I think I'll just wait for a PR then, or maybe a branch if you ever feel like sharing something early
Sam Mohr (Sep 17 2024 at 23:29):I'm not sure how to resolve this dissonance without seeing the code...
Agus Zubiaga (Sep 17 2024 at 23:34):So Task are user level now, the proposal (as I understand it at least, @Richard Feldman can confirm) is to make them an implementation detail of effectful functions. There's a separate longstanding effort to change how Task works behind the scenes (effect interpreters), but that isn't directly affected by how the user performs effects from the surface.
Richard Feldman (Sep 17 2024 at 23:40):that's right, yeah
Sam Mohr (Sep 17 2024 at 23:42):The tricky thing with using Task is how it interacts with control flow. Right now, we need to do some real work to extract generated variables from their when or if or |> contexts. That wouldn't be necessary, I think, if they weren't thunks
Richard Feldman (Sep 17 2024 at 23:47):so in this design, I think we have a type variable behind the scenes which tracks which of these 4 function types a given function has (only 2 of the types are visible, namely pure vs effectful - but we need to track all 4 as distinct from one another behind the scenes, in order to compile the way we want):
Time.now! and just about everything in the wasm4 platform :big_smile: Task equivalent so that the function actually returns at that point, and one of the things it returns is "everything that happens next" inside a closure. (Today we call that Task.await.) Examples: Http.get!, Result.parallel!, etc.List.mapParallel and such - from a type-checking perspective, this counts as pure, but from a compilation perspective, it's exactly the same as "effectful and async" in that it compiles down to a state machine entry. (The host doesn't care about the distinction between pure and effectful, it just cares about the distinction between sync and async.) Agus Zubiaga (Sep 18 2024 at 00:21):@Sam Mohr Did you mean that until we support effect interpreters we don't actually need to compile to Task because everything is sync anyway?
Sam Mohr (Sep 18 2024 at 00:21):Yes, basically
Agus Zubiaga (Sep 18 2024 at 00:23):Ok yeah, that makes sense. Sorry I didn't follow earlier, I was stuck on the "effectful and async" way of thinking which is what most task will be.
Sam Mohr (Sep 18 2024 at 00:23):You're good
Brendan Hansknecht (Sep 18 2024 at 00:24):effectful and synchronous
I'm actually a bit surprised we plan to support this at all.
Sam Mohr (Sep 18 2024 at 00:25):I'm just making sure that we don't tunnel vision on following Task because it's contrary to my mental model of how these things work in my head. Effectful functions don't always return Result, and they can be synchronous
Brendan Hansknecht (Sep 18 2024 at 00:25):I think it just make a bigger hassle around passing extra pointers into roc and such (at least in the future effect interpreter world)
Richard Feldman (Sep 18 2024 at 00:26):Brendan Hansknecht said:
effectful and synchronous
I'm actually a bit surprised we plan to support this at all.
I think it's pretty important for some use cases
Richard Feldman (Sep 18 2024 at 00:26):like I think all the wasm4 applications would suffer worse perf if we didn't
Richard Feldman (Sep 18 2024 at 00:27):and with time specifically, it's definitely undesirable to have extra call stack/conditional machinery around getting the actual time value, makes it less precise and therefore less useful
Brendan Hansknecht (Sep 18 2024 at 00:29):True, though for most platforms that will use a system call to get the time, it shouldn't make a meaningful difference.
Sam Mohr (Sep 18 2024 at 00:30):The only reason I think we'd want to target Task/thunk behavior is if the effect interpreter behavior would be really hard to add to a synchronous version of effectful functions. I don't know how that would look
Brendan Hansknecht (Sep 18 2024 at 00:31):But yeah, I'm sure it will be measurable and matter somewhere. Though now it is a platform specfic tradeoff to figure out. For wasm4 specifically, I don't think it matters. I think the actually effect calling overhead will be much lower than many other things in wasm4.
Sam Mohr (Sep 18 2024 at 00:31):But control flow desugaring would be greatly simplified if we went for synchronous effectful functions first
Brendan Hansknecht (Sep 18 2024 at 00:32):But control flow desugaring would be greatly simplified if we went for synchronous effectful functions first
Yeah, if it is synchornous, it is just ffi. No state machine in interpreter. Though the control flow in roc isn't really harder, the state machine is executed by the platform
Richard Feldman (Sep 18 2024 at 00:34):fair point about wasm4 specifically, but really the category of platforms I'm thinking of are resource-constrained ones where there will be absolutely no benefit to the state machine, just overhead, and overhead in an already resource-constrained environment is extra undesriable
Brendan Hansknecht (Sep 18 2024 at 00:36):For sure
Agus Zubiaga (Sep 18 2024 at 00:38):Sam Mohr said:
But control flow desugaring would be greatly simplified if we went for synchronous effectful functions first
Yeah, that's a good observation. We can probably skip it for now, but we will have to do it eventually.
Sam Mohr (Sep 18 2024 at 00:40):To be clear, you're saying that we can skip simplifying the desugaring for now, but will have to do it eventually?
Agus Zubiaga (Sep 18 2024 at 00:40):It wouldn't be desugaring really, but yeah, we have to compile to continuations for async effects when we support that
Brendan Hansknecht (Sep 18 2024 at 00:42):Yeah, roughly compiling to:
Op : [
StdoutLine Str \{} -> Op,
StdinLine \Str -> Op,
Done,
]
yeah actually we can simplify the representation compared to Task, which is nice
Agus Zubiaga (Sep 18 2024 at 00:42):I was thinking of that as crucial for the implementation of the proposal, because initially I didn't know we planned to support synchronous effects at all (even though that's all we have now). However, if we are going to support that, we can ship an initial version that does FFI directly.
Richard Feldman (Sep 18 2024 at 00:43):like just compile directly to the state machine instead of a wrapper around it, since we wouldn't need to offer a user-facing API (e.g. Task.map etc.) that would care about the wrapper
Richard Feldman (Sep 18 2024 at 00:44):which is nice as an implementation simplification, although that technique for turning Task into a state machine is so wild I kinda hope another use for it comes up eventually now that I know it exists :laughing:
Agus Zubiaga (Sep 18 2024 at 00:46):I'm sure another Richard in an alternative universe is using it
Ayaz Hafiz (Sep 18 2024 at 02:13):The only way to compile algebraic effects of the form here into a state machine (which is necessary for the event loop in the platform + enforcing controlled execution of effects) is via the continuation or free monad. Task is the continuation monad, so it is natural to compile to Task directly (obviously by the point where you do this in the implementation Task would have already be compiled to an intermediate form, so really you are compiling to the continuation monad).
You also must do this if you ever want mutli-shot continuations in the host, which is a separate problem that I am not sure is really resolved in Roc (I believe the current state of the world means it is not safe to call a continuation more than once).
There is a separate question here about whether you want to compile to the continuation monad (Task) or the free monad. I would suggest the continuation monad though. It makes optimizations easier.
Brendan Hansknecht (Sep 18 2024 at 02:18):(I believe the current state of the world means it is not safe to call a continuation more than once)
The only issue is needing to increment lamba capture refcounts, right?
Ayaz Hafiz (Sep 18 2024 at 02:18):yes
Sam Mohr (Sep 18 2024 at 02:20):So it seems like we need two forms of compilation: for sync effects, we want to basically just call FFI. For async effects, we want to compile to something shaped like a Task. Is there a way to do both without needing to have two entirely different effect-handling mechanisms?
Ayaz Hafiz (Sep 18 2024 at 02:21):why do you want sync effects?
Sam Mohr (Sep 18 2024 at 02:21):For time sensitive ops, like Utc.now!
Ayaz Hafiz (Sep 18 2024 at 02:21):uhhh
Brendan Hansknecht (Sep 18 2024 at 02:21): Sam Mohr (Sep 18 2024 at 02:21): Sam Mohr (Sep 18 2024 at 02:21):lol
Ayaz Hafiz (Sep 18 2024 at 02:22):the problem is that you can't really have both and preserve invariants about a state machine though
Ayaz Hafiz (Sep 18 2024 at 02:22):i'm not sure the perf argument holds up. Is the concern about the overhead of a function call?
Sam Mohr (Sep 18 2024 at 02:23):I believe so
Brendan Hansknecht (Sep 18 2024 at 02:24):I think we could make this work, but all of the sync functions would require to be passed into roc (this is what we plan to do long term for roc_alloc and such anyway).
I think the main cost of task is the suspension of intermediate state into a closure, not the function call.
Ayaz Hafiz (Sep 18 2024 at 02:24):A function call would be required regardless unless you do inlining during linking. In which case there is no difference between sync vs async effects.
Ayaz Hafiz (Sep 18 2024 at 02:24):Got it
Sam Mohr (Sep 18 2024 at 02:25):Well, if we compile to a Task, that's now two function calls, right?
Ayaz Hafiz (Sep 18 2024 at 02:25):yes
Sam Mohr (Sep 18 2024 at 02:25):It may be minor, but it's still a cost
Ayaz Hafiz (Sep 18 2024 at 02:25):but the calls are cheap and the target predictor will do it's job
Ayaz Hafiz (Sep 18 2024 at 02:25):the suspension to closure state is a reasonable point
Sam Mohr (Sep 18 2024 at 02:25):It does seem like supporting both would be a pretty high complexity cost
Ayaz Hafiz (Sep 18 2024 at 02:25):but this seems really complicated
Ayaz Hafiz (Sep 18 2024 at 02:26):and you lose the state machine
Ayaz Hafiz (Sep 18 2024 at 02:27):If you have hard realtime requirements anything that passes over an FFI might be the wrong choice
Sam Mohr (Sep 18 2024 at 02:27):Fair
Sam Mohr (Sep 18 2024 at 02:27):In which case, @Agus Zubiaga I think you should forget what I was pushing for haha
Sam Mohr (Sep 18 2024 at 02:28):It seems like our ability to support effect interpreters would be a pretty simple push on top of Task
Brendan Hansknecht (Sep 18 2024 at 02:28):Yeah, the thought is that most effect use the state machine, but some effects could opt into being synchronous (by default all allocator related effects will be this way to allow for explicit allocators). Whenever a roc function is called, it is passed in an extra arg that is essentially a list of function pointers. That is used for anything synchoronous.
I am honestly not sure if this is the right choice. Especially given that by adding allocation to the state machine, that would also enable the platform to swap out the allocator whenever it chooses. It can use an arena allocator for just a subset of allocation by setting a config on the interpreter.
Ayaz Hafiz (Sep 18 2024 at 02:30):you're also giving up things like testability, you can no longer simulate e.g. time changes with Utc.time in tests unless you support compiling conditionally to a state machine in tests vs. direct in prod
Brendan Hansknecht (Sep 18 2024 at 02:30):oh yeah, that is huge. Hadn't thought about that one.
Agus Zubiaga (Sep 18 2024 at 02:30):I think the idea is that those would be mocked via module params
Sam Mohr (Sep 18 2024 at 02:31):Yes
Ayaz Hafiz (Sep 18 2024 at 02:32):but if you mock via a module param you have to provide a constant function right? vs. if I write a handler for the state machine in a test, I can say the first Utc.time is one date and the next Utc.time is 1 day later, for example
Sam Mohr (Sep 18 2024 at 02:33):Unit tests that are testing effects would be simulated and provided via a module params interface. Integration tests would use the full platform anyway
Ayaz Hafiz (Sep 18 2024 at 02:34):So if you want to test effects with state you have to use the whole platform? you cannot write a stateful fake?
Sam Mohr (Sep 18 2024 at 02:35):Not without Stored, no
Sam Mohr (Sep 18 2024 at 02:35):But we should consider bringing that back for effectful testing, though it has the potential to be used for evil
Ayaz Hafiz (Sep 18 2024 at 02:35):i think it should be strongly considered
Ayaz Hafiz (Sep 18 2024 at 02:36):IMO the real power of effects is it basically gives you dependency injection for free
Ayaz Hafiz (Sep 18 2024 at 02:36):you can write unit tests that simulate a filesystem or an external service without having to inject that FS/service/etc into the runtime implementation of the function
Sam Mohr (Sep 18 2024 at 02:37):I think it's that plus module params that really provide DI
Ayaz Hafiz (Sep 18 2024 at 02:37):that way you can provide fakes in your test that simulate real business logic, without having to write mocks
Ayaz Hafiz (Sep 18 2024 at 02:38):that's true, but module params are a bit different because they cannot preserve state between calls
Sam Mohr (Sep 18 2024 at 02:38):I'm just saying they're the interface for the "injection" part that makes it ergonomic
Sam Mohr (Sep 18 2024 at 02:39):But yes, they're basically syntax sugar for extra func args
Brendan Hansknecht (Sep 18 2024 at 02:41):Yeah, full simulation by just implementing a state machine would be huge.
Brendan Hansknecht (Sep 18 2024 at 02:42):It is definitely a strong reason to consider not allowing synchronous effects (that or having a special way to inject them that allows for state)
Richard Feldman (Sep 18 2024 at 02:56):my current thinking regarding state in tests is to support something like this:
expect \{ get!, set! } ->
and those are effectful functions you can call in the test to set and get whatever state you want for simulation purposes
Richard Feldman (Sep 18 2024 at 02:57):so much simpler than Stored and only available in tests
Sam Mohr (Sep 18 2024 at 02:57):Only available in tests is great
Agus Zubiaga (Sep 18 2024 at 02:57):That would only work for one type, right?
Richard Feldman (Sep 18 2024 at 02:58):sure but you can make that one type be a record
Sam Mohr (Sep 18 2024 at 02:59):Another place I want simpler lenses...
Richard Feldman (Sep 18 2024 at 03:00):the exact record passed in would be { get! : {} => a, set! : a => {} }
Brendan Hansknecht (Sep 18 2024 at 03:00):so module params that pass in lambdas that use get and set to enable state?
Richard Feldman (Sep 18 2024 at 03:00):right
Richard Feldman (Sep 18 2024 at 03:00):so they can simulate whatever they like
Daniel Schierbeck (Oct 11 2024 at 09:12):Just an idea from an old Ruby afficionado: in Ruby it's pretty common to use or and and for control flow, e.g.
authenticate! or raise HttpError.new(401)
authorize! :can_read, @article or raise HttpError.new(403)
(note how ! is used for "dangerous" or effectful methods by convention)
The difference between e.g. || and or is just the precedence, but in practice it's a very nice and concise way to control flow without nested if statements.
I thought of that when I read this proposal and saw the ?> operator, which gave me a bit of a Haskell feel :smile:
Would it make sense to use or instead of ?>?
File.writeUtf8! path or WriteFailed
File.readUtf8! path or \err ->
when err is
FileNotFound then IncorrectPathError
...
@Daniel Schierbeck my original proposal in #ideas>`try-else` error context adding syntax was to use the else keyword here, which is symmetric with if-else
Sam Mohr (Oct 11 2024 at 09:43):The benefit of ? is that it doesn't overlap with control flow, so you don't have to use it in conjunction with try for it to make sense
Sam Mohr (Oct 11 2024 at 09:43):But or might conflict with boolean statements, though probably not a big deal
Sam Mohr (Oct 11 2024 at 09:44):Though we're now leaning into ? being an error-related character, since ?? now means Result.withDefault
Daniel Schierbeck (Oct 11 2024 at 09:55):I may just be generally averse to fancy operators, I went back to an old Haskell project after ~10 years and couldn't make heads or tails of it :laughing:
Daniel Schierbeck (Oct 11 2024 at 09:58):I've also seen orElse used, but I _think_ that may have been in VB or something. I guess withDefault and mapErr are very similar, with the difference really just being whether a function or a value is passed on the RHS?
I must admit that ?? did not at all made me think of withDefault. It might be because I haven't really written any Rust.
Daniel Schierbeck (Oct 11 2024 at 09:59):foo or \x -> y
foo orElse 42
?
Nathan Kramer (Oct 11 2024 at 10:06):I think ?? is quite a good choice for Result.withDefault, since it's the same as the null coaslecing operator in JS
Daniel Schierbeck (Oct 11 2024 at 10:11):Not writing enough JS, either, it seems...
Nathan Kramer (Oct 11 2024 at 10:12):Oh, that's not a bad thing :laughing:
Carson (Oct 14 2024 at 17:40):Richard Feldman said:
maybe it's just me, but personally I find
!?so ridiculous-looking that I can't even imagine wanting to get up on a stage and show code on a slide that includes it. Or write a blog post about it. I dislike it so much that I think it would make me feel ashamed to try to promote the language.
This may be a cursed suggestion, but if the problem is only with the aesthetics of !?, you could steal an idea from the uiua programming language. Uiua is in the APL tradition of PLs, so it uses a lot of unicode symbols. Instead of using a special keyboard layout, the preferred method of input is to type out the english name of the operator and the formatter will convert the english name to the corresponding unicode symbol.
So in Roc, if the user types out !? to represent both an effectful and fallible call, the formatter could convert them into a single symbol, e.g. ‽. That way there's no hidden control flow decided by a remote definition, you would be in control of the aesthetics of the resulting code, and everything else about the proposal could stay the same.
Richard Feldman (Oct 14 2024 at 18:05):I don't think we should use non-ASCII symbols in Roc's syntax :big_smile:
drew (Oct 16 2024 at 13:48):I recognize that this might be an annoying comment, but after using gleam and seeing how complicated these proposals make the language (based on the complexity of choices and no clear winner given the tradeoffs), have we stepped back and compared to back passing again?
drew (Oct 16 2024 at 13:49):I _really_ like use in gleam. It is amazingly flexible and general while being very clear syntactically.
Anton (Oct 16 2024 at 14:22):Does use work exactly the same as backpassing in roc?
Brendan Hansknecht (Oct 16 2024 at 14:25):Yeah, use is back passing
pub fn with_use() {
use username <- result.try(get_username())
use password <- result.try(get_password())
use greeting <- result.map(log_in(username, password))
greeting <> ", " <> username
}
clearly "hard to learn" is the primary downside to backpassing, but honestly the combination of these new proposals feels equally hard to learn, less general, and seems to introduce a large amount of syntactic noise to the language. at least with backpassing, once you learned it, it was consistent and the only solution to all of this class of problems.
Brendan Hansknecht (Oct 16 2024 at 14:29):and seeing how complicated these proposals make the language (based on the complexity of choices and no clear winner given the tradeoffs)
I think most of this proposal won't be visible to an application developer. Besides a tiny bit of syntax, I think that we could 100% swap over today and most people would barely notice.
Most of the debating on these proposals has been about minor syntax compared to the goal of the proposals as a whole.
drew (Oct 16 2024 at 14:30):i don't think two new, special case operators, that constantly interact in your code feels like a "tiny bit of syntax"
Brendan Hansknecht (Oct 16 2024 at 14:30):! and ??
drew (Oct 16 2024 at 14:30):as someone who pops in and out, things are staring to feel a bit operator-soup-y, which as i understand it was one of the things roc wanted to avoid from, say, haskell
drew (Oct 16 2024 at 14:31):Brendan Hansknecht said:
!and??
yes. like -- is there a space? are they keywords or functions? where do i use them?
Richard Feldman (Oct 16 2024 at 14:31):the current plan (which is most of the way implemented btw, so we'll be able to try it out soon!) is to not have !, ?, or <-
drew (Oct 16 2024 at 14:31):it feels very ad hoc
drew (Oct 16 2024 at 14:31):ah, i see
Brendan Hansknecht (Oct 16 2024 at 14:31):! and ? are tangential to this proposal. They could be removed while keeping the rest of the proposal
Brendan Hansknecht (Oct 16 2024 at 14:31):They also are already in roc today
Richard Feldman (Oct 16 2024 at 14:32):the current plan is to have a return keyword that works the way it works in imperative language, and a try keyword which is sugar for "give me a Result, unwrap it if it's Ok, and if it's Err then return that Err"
Brendan Hansknecht (Oct 16 2024 at 14:32):But I totally see the argument for removing them and going back to backpassing
drew (Oct 16 2024 at 14:32):Richard Feldman said:
the current plan is to have a
returnkeyword that works the way it works in imperative language, and atrykeyword which is sugar for "give me aResult, unwrap it if it'sOk, and if it'sErrthenreturnthatErr"
so tasks remain as is today? how do we deal with IO?
Brendan Hansknecht (Oct 16 2024 at 14:33):We still have !, right? It is just part of a function name?
drew (Oct 16 2024 at 14:34):yeah, so for me my comments still hold, we've just replaced some symbols with new keywords. i think stepping back i still like where things were with backpassing better than now, but please take this with a grain of salt as i am very far from the day to day of the language.
Richard Feldman (Oct 16 2024 at 14:35):here are the main tradeoffs as I see them:
Task is not technically a new primitive bc it's an ordinary opaque type, no return or try)Task wrapper is no longer required to do effects, but can still be opted into if desired), monadic APIs are potentially less ergonomic Richard Feldman (Oct 16 2024 at 14:36):I can understand why it might seem like the backpassing world is easier to learn, but I have extremely high confidence that the purity inference world will be easier for beginners to learn, and I don't think it's remotely close :big_smile:
Richard Feldman (Oct 16 2024 at 14:37):it's always possible I will turn out to be wrong about that, but I would be absolutely stunned if it turned out that after we see how things go in the purity inference world, the consensus is that the backpassing world was easier for beginners to learn
drew (Oct 16 2024 at 14:37):got it. for me, the code snippets are starting to feel like rust, which worries me
Richard Feldman (Oct 16 2024 at 14:37):to me the main selling points of the backpassing world are smaller number of language primitives (which I absolutely do value!) and having sugar for monadic APIs
drew (Oct 16 2024 at 14:38):yep i hear you.
Richard Feldman (Oct 16 2024 at 14:39):we're getting close to a point where we can look at actual before/after code bases with e.g. basic-cli and can see more directly how it compares to today!
Dan G Knutson (Oct 16 2024 at 14:45):As someone else who's popped in and out, I think part of the issue is not knowing what's past roc, current roc, abandoned future roc, and planned future roc. I don't mind !? but I think I'd probably like return better. It sounds like the current plan is less operator soup and more procedural keywords with sugar? Like try, return, and I guess soon for. I'm curious about the effects not showing up in the return type as Task thing, that seems out of place.
Richard Feldman (Oct 16 2024 at 14:54):effects show up in the arrow rather than the return type itself
Richard Feldman (Oct 16 2024 at 14:55):e.g. Str -> Str is a pure function, and Str => Str is an effectful one (today, the latter would be Str -> Task Str *)
Richard Feldman (Oct 16 2024 at 14:56):I appreciate that it's not clear what's current vs past vs present vs future, but the main reason Roc doesn't have a version number yet is wanting to communicate really clearly that things are in flux! :big_smile:
Dan G Knutson (Oct 16 2024 at 14:58):I'm looking forward to the talk about it!
Richard Feldman (Oct 16 2024 at 14:58):thanks - happening tonight! :smiley:
https://hfpug.org/event/richard-feldman-the_functional_purity_inference_plan/
drew (Oct 16 2024 at 15:32):ooo
drew (Oct 16 2024 at 15:32):awesome
drew (Oct 16 2024 at 16:21):will this talk be recorded on youtube? very cool
Richard Feldman (Oct 16 2024 at 16:26):I believe so!
Ray Myers (Oct 19 2024 at 22:20):@drew here's the link
Just saw the talk, which was my first exposure to this proposal - I LOVE it!
Ray Myers (Oct 19 2024 at 22:27):Lean 4's do notation was one of the most convenient ways I'd seen of getting regular loops and re-assignment into a purely-functional context, but I think you've probably outdone them! Still - might be a source of ideas for hurdles that come up if there are any remaining.
Luke Boswell (Oct 21 2024 at 04:20):Just messing with an example for roc-ray using the new purity inference stuff... and came across something that I thought I might raise. Not sure if I am doing this right? it looked a little strange to me.
# definition
drawWorld : Model -> ({} => {})
# then at the call-site
# RENDER WORLD
RocRay.drawMode2D! model.cameraID (drawWorld model)
# definition of drawRectangle
drawRectangle! : { rect : Rectangle, color : Color } => {}
$ roc check examples/2d_camera.roc
0 errors and 0 warnings found in 30 ms
In current roc land... this DrawMode2D takes a Task to action. The idea here was that the platform could automatically include both the Begin and End call preventing users from making that mistake.
I think I'm going to change the API here anyway to require people to be more explicit and require an RocRay.endMode2D!, and have a runtime error instead.
But I thought this might be interesting from a purity-inference perspective.
Brendan Hansknecht (Oct 21 2024 at 05:32):Those types all look reasonable to me. Take a model, don't run any affects, then give me a task to run effects.
Brendan Hansknecht (Oct 21 2024 at 05:33):That said, I would probably just change it to: Model => {} and pass the Model into RocRay.drawMode2D! as well
Brendan Hansknecht (Oct 21 2024 at 05:33):I think that is more natural
Brendan Hansknecht (Oct 21 2024 at 05:33):# definition
drawWorld : Model => {}
# then at the call-site
# RENDER WORLD
RocRay.drawMode2D! model.cameraID model drawWorld
but either is fine
drew (Oct 22 2024 at 02:02):Ray Myers said:
drew here's the link
Just saw the talk, which was my first exposure to this proposal - I LOVE it!
watching this now
drew (Oct 22 2024 at 02:03):is var effectively ST from haskell? can i use var without my function using =>?
drew (Oct 22 2024 at 02:04):oh ha, question was just asked. it is pure!
Brendan Hansknecht (Oct 22 2024 at 03:01):can i use
varwithout my function using=>?
you can use var without =>
drew (Oct 22 2024 at 14:59):just finished the talk, very enjoyable. do you see a world in which backpassing is in the language for advanced cases (monadic APIs)
Richard Feldman (Oct 22 2024 at 15:07):it's conceivable but unlikely I think
Richard Feldman (Oct 22 2024 at 15:09):e.g. Elm doesn't have it and it's been totally fine, and the main difference between Elm and Roc is that in Roc there's way more chaining of I/O specifically
drew (Oct 22 2024 at 15:39):got it. thanks.
drew (Oct 22 2024 at 15:39):i do find your argument that backpassing works in gleam because you don’t need it for IO compelling
drew (Oct 22 2024 at 15:40):which would be the case in roc post the changes described
Brendan Hansknecht (Oct 23 2024 at 01:08):I don't have min repro yet, but I think I found a bug with purity inference and _ = ...
This destructure assignment doesn't introduce any new variables:
21│ _ = W4.saveToDisk! [Num.addWrap saves 1]
^
If you don't need to use the value on the right-hand-side of this
assignment, consider removing the assignment. Since Roc is purely
functional, assignments that don't introduce variables cannot affect a
program's behavior!
If I remove the _ =, I get:
The result of this call to W4.saveToDisk! is ignored:
21│ W4.saveToDisk! [Num.addWrap saves 1]
^^^^^^^^^^^^^^
Standalone statements are required to produce an empty record, but the
type of this one is:
Result {} [SaveFailed]
If you still want to ignore it, assign it to _, like this:
_ = File.delete! "data.json"
I just wanted to say that I'm loving how pervasive the ! in the name thing is, and having there it shows up in places it didn't before and makes it so much easier to see what is happening.
Also, I think the call to not use => also in the function implementation was a good call. It took me a few minutes at most to adjust to the annotation being => but I like how the muscle memory isn't interrupted when writing a lambda.
Brendan Hansknecht (Oct 23 2024 at 02:21):Also, my issue can be reproduced by pulling roc-wasm4 on the purity-inference branch and then running roc check examples/basic.roc
Sam Mohr (Oct 23 2024 at 02:33):Luke Boswell said:
Also, I think the call to not use
=>also in the function implementation was a good call. It took me a few minutes at most to adjust to the annotation being=>but I like how the muscle memory isn't interrupted when writing a lambda.
I was gonna leave a comment on the vid to say that it was a typo or something because I thought we needed it to indicate whether a function sans signature was effectful, but the exclamation does the job!
Agus Zubiaga (Oct 23 2024 at 02:34):Good catch @Brendan Hansknecht! I’ll look into that tomorrow
Agus Zubiaga (Oct 23 2024 at 02:38):That and warnings for missing ! in arguments are the only things missing afaik
Brendan Hansknecht (Oct 23 2024 at 02:49):Also, I would like to report that with purity inference and direct io calls, I was able to get rocci-bird to compile down to 44kb (40kb if I also run wasm-opt on it). The old version was 64kb (60kb with wasm-opt).
Also the removal of the nested lambas from task ended up fixing some bugs that I had to explicit workarounds for in the past.
Isaac Van Doren (Oct 23 2024 at 02:58):grug warn closures like salt, type systems and generics: small amount go long way, but easy spoil things too much use give heart attack
Perhaps grug was right all along :laughing:
Richard Feldman (Oct 23 2024 at 03:04):I had the author of that (the grug post) on Software Unscripted a year or so ago, really friendly guy!
Richard Feldman (Oct 23 2024 at 03:04):he also made htmx
Isaac Van Doren (Oct 23 2024 at 03:09):Yeah he’s great! I haven’t seriously used HTMX, but his writing about Hypermedia, REST, and his book Hypermedia systems has completely changed the way I see web dev.
Brendan Hansknecht (Oct 23 2024 at 03:57):To be fair, the real issue is that we don't compile chains of tasks into an async state machine. That would also fix the issue here. So the main issue was naive closures that keep moving state around
Brendan Hansknecht (Oct 23 2024 at 15:38):Two other comments from updating roc wasm4.
1) Simply calling tasks from withing branches without wrapping is awesome (specifically look at flapSoundTask.
before:
{ yVel, nextAnim, flapSoundTask } =
if !prev.lastFlap && flap && flapAllowed prev.frameCount prev.rocciFlapAnim then
anim = prev.rocciFlapAnim
{
yVel: jumpSpeed,
nextAnim: { anim & index: 0, state: RunOnce },
flapSoundTask: W4.tone flapTone,
}
else
{
yVel: prev.player.yVel + gravity,
nextAnim: updateAnimation prev.frameCount prev.rocciFlapAnim,
flapSoundTask: Task.ok {},
}
flapSoundTask!
after
{ yVel, nextAnim } =
if !prev.lastFlap && flap && flapAllowed prev.frameCount prev.rocciFlapAnim then
anim = prev.rocciFlapAnim
W4.tone! flapTone
{
yVel: jumpSpeed,
nextAnim: { anim & index: 0, state: RunOnce },
}
else
{
yVel: prev.player.yVel + gravity,
nextAnim: updateAnimation prev.frameCount prev.rocciFlapAnim,
}
2) This does get a bit weird if you don't really have an else, but I'm sure we'll change this at some point:
if gainPoint > 0 then
W4.tone! pointTone
else
{}
Yeah, I now allow standalone statements as long as they return {} and are effectful.
Agus Zubiaga (Oct 23 2024 at 16:04):Changing how if works felt out of scope, but I agree it’d be nice to support else-less if as long as it returns {} too
Agus Zubiaga (Oct 23 2024 at 16:04):We can do that separately
Agus Zubiaga (Oct 24 2024 at 03:55):Brendan Hansknecht said:
I don't have min repro yet, but I think I found a bug with purity inference and
_ = ...This destructure assignment doesn't introduce any new variables: 21│ _ = W4.saveToDisk! [Num.addWrap saves 1] ^ If you don't need to use the value on the right-hand-side of this assignment, consider removing the assignment. Since Roc is purely functional, assignments that don't introduce variables cannot affect a program's behavior!If I remove the
_ =, I get:The result of this call to W4.saveToDisk! is ignored: 21│ W4.saveToDisk! [Num.addWrap saves 1] ^^^^^^^^^^^^^^ Standalone statements are required to produce an empty record, but the type of this one is: Result {} [SaveFailed] If you still want to ignore it, assign it to _, like this: _ = File.delete! "data.json"
This is fixed now! I moved the warning from can to the type checker and it will now only complain if the RHS is pure. So this is fine: https://github.com/roc-lang/roc/blob/3bd90a51b6fde5e24af70ec1b95233730fe0a544/crates/cli/tests/effectful/ignore_result.roc#L7
Agus Zubiaga (Oct 24 2024 at 03:57):I also added lots of error messages and warnings today for missing ! and common mistakes such as forgetting to call an effectful function
Agus Zubiaga (Oct 24 2024 at 03:58):Got to 0 known bugs!
Agus Zubiaga (Oct 24 2024 at 04:00):Luke did find an issue with ? while porting basic-cli. It no longer desugars properly because I now keep statements around until the type checker for better errors.
Agus Zubiaga (Oct 24 2024 at 04:00):I could fix that but it would be kind of pointless because Result.try wouldn’t work with effectful functions anyway
Agus Zubiaga (Oct 24 2024 at 04:01):We need the return based try keyword for purity inference
Sam Mohr (Oct 24 2024 at 04:01):Working on that this weekend!
Agus Zubiaga (Oct 24 2024 at 04:02):Amazing!
Agus Zubiaga (Oct 24 2024 at 04:02):Since purity inference is backwards compatible with Task, I’m going to make the PR ready for review tomorrow
Sam Mohr (Oct 24 2024 at 04:02):Let me make a thread after rehearsal ends on finishing off return. Return is working in the PR, but the refcounting is wrong
Agus Zubiaga (Oct 24 2024 at 04:02):We should be able to merge it separately and we can update the platforms when we have try
Sam Mohr (Oct 24 2024 at 04:03):I agree
Luke Boswell (Oct 24 2024 at 04:06):It's a baby step... but we've got one working example in basic-cli in https://github.com/roc-lang/basic-cli/pull/257
$ roc examples/time.roc
🔨 Rebuilding platform...
Completed in 1504447000ns
Why is the time in nanoseconds?
Luke Boswell (Oct 24 2024 at 05:25):Lol, idk. I guess that's just how we wrote that example.
Brendan Hansknecht (Oct 24 2024 at 05:43):Oh, that is the print out from the time example....I thought that was roc printing how long it took to compile the program
Oskar Hahn (Oct 24 2024 at 06:18):Agus Zubiaga said:
Yeah, I now allow standalone statements as long as they return
{}and are effectful.
This comment makes me think, if dbg should be renamed to dbg!. I would say, that dbg is effectful. With purity inference, dbg could be handled as any other effectful function, with the only exception, that every platform has to implement it.
Brendan Hansknecht (Oct 24 2024 at 06:21):I don't think so. You don't want a debug to poison a stack of otherwise pure functions...but I'm not fully sold either way.
Oskar Hahn (Oct 24 2024 at 06:23):Brendan Hansknecht said:
I don't think so. You don't want a debug to poison a stack of otherwise pure functions...but I'm not fully sold either way.
That is true. I did not think about the change in the function signature.
Eli Dowling (Oct 31 2024 at 12:53):I just watched Richard talk on this feature and I must say I'm extremely impressed!
I wasn't around for the discussion about it, but I could be more happy with how it turned out.
I think for me the most significant win(of many) is the signals it sends to folks outside the language.
To me, this sweeping change makes it abundantly clear that the roc community intends to make a pragmatic language that is great for real people to solve real problems in. That adherence to a particular style is far less important than making coding easy and clear.
I'm also super keen on the framework this lays for future expansion into more algebraic effects territory. I actually don't think that is something that needs doing, but the fact that this approach makes improvements in that area mostly seamless feels like the right move for giving roc space to grow in the future should the need arise.
Awesome work everyone :tada:
Jared Cone (Nov 08 2024 at 05:48):Would var work with destructuring?
var ctx_ = ctx
{ ctx_, blah1 } = doSomething
{ ctx_, blah2 } = doSomethingElse
I suppose doSomething wouldn't be able to return a record like that, but maybe a tuple?
jan kili (Nov 08 2024 at 06:02):Is record an invalid return type? Or cannot a record key end with underscore? Presumably there's a syntax to rename like { ctx: ctx_, ...
Jared Cone (Nov 08 2024 at 19:00):Will you be able to specify a param as var? Something like:
doSomething = \var ctx_ ->
instead of
doSomething = \ctx ->
var ctx_ = ctx
I think the plan is to support it in all identifier patterns
Last updated: Jun 16 2026 at 16:19 UTC