Jasper Woudenberg (Aug 28 2024 at 18:44): Jasper Woudenberg (Aug 28 2024 at 18:44):I like the idea overall. Not needing to convert between Result and Task is a pretty nice bonus!
I wonder if it would make more sense to annotate effects using the arrow instead of a bang in the result type, given it's the function itself that's effectful.
So instead of this: {} -> Result! U64 Str
We might have this: {} !> Result U64 Str
(I don't know if !> is the best operator here or something else would be better, you get the idea)
If an additional rule is that in a type signature with multiple !> operators all those operators have the same implicit effect parameter, then there's no need for a separate => operator, so that might reduce the syntax footprint a bit.
Plus, it'd avoid needing to explain why this isn't a valid type signature:
effectfullNumber: U64!
@Jasper Woudenberg I think -!> implies that something is happening during the creation of a return value, but I'm generally on the same page.
Richard Feldman (Aug 28 2024 at 18:49):yeah I agree that it would be nicer if the arrow itself described the effectfulness (or not), although I'm personally not a fan of !>
Richard Feldman (Aug 28 2024 at 18:49):I'm also not generally a fan of infix operators that take more than 2 characters
Richard Feldman (Aug 28 2024 at 18:50):the three that seem reasonable to me are -> and => and ~>
Richard Feldman (Aug 28 2024 at 18:51):another problem is that there has to be syntax for "named effectfulness variable" (e.g. !fx in the doc) - even if it comes up super rarely in practice, syntax for it has to exist for type inference to work
Richard Feldman (Aug 28 2024 at 18:51):(otherwise, for example, you could put valid code into the repl which happens to have those relationships and we just couldn't print an inferred type for it)
Richard Feldman (Aug 28 2024 at 18:51):and I'm not sure how to sneak a variable into the arrow haha
Richard Feldman (Aug 28 2024 at 18:52):that said, I also separately like the idea of effectfullNumber: U64! being something you can't even write syntactically
Richard Feldman (Aug 28 2024 at 18:53):Jasper Woudenberg said:
If an additional rule is that in a type signature with multiple
!>operators all those operators have the same implicit effect parameter, then there's no need for a separate=>operator, so that might reduce the syntax footprint a bit.
in that world, we could replace ! with => and then have:
-> means pure function=> means "potentially effectful function, although it could be pure if it's higher-order" Agus Zubiaga (Aug 28 2024 at 18:56):Why is => needed? If you have a higher-order function like:
List.map : List a, (a -> b) -> List b
The only way for List.map to return a b is by calling the passed function, so we can infer that it's equally effectful as the passed function.
Richard Feldman (Aug 28 2024 at 18:56):there's a section in the doc on that
Agus Zubiaga (Aug 28 2024 at 18:57):Oh, I didn't see that in the latest version
Sam Mohr (Aug 28 2024 at 18:57):I agree, even though the doc covers it, I don't think it's that big of a deal that an effectful (a -> b) can be used, it will just propagate up an async section.
Sam Mohr (Aug 28 2024 at 18:57):But the cost of this is now people have to write functions knowing that they're handling async-able functions
Agus Zubiaga (Aug 28 2024 at 18:58):I'm worried it will create an unnecessary split in the ecosystem, too
Agus Zubiaga (Aug 28 2024 at 18:59):maybe not so much a split, but it'd be frustrating if you were using a higher-order function from a library and you couldn't use it with effects because the author forgot to use =>
Sam Mohr (Aug 28 2024 at 18:59):So I'd say unless it's really a big issue that we just function color things all the way down, the cost is what Agus points out, which is that the default way to use functions is now overly constrictive for potentially async usage, and the special way is better but requires an understanding of async coloring
Richard Feldman (Aug 28 2024 at 19:00):here's the section I had in an earlier draft, regarding that idea:
The proposal here is to make the current
List.maptype signature Just Work (that is, be effect-polymorphic) by adopting the following rule:If a higher-order function type annotation contains exactly one function type, then it is effect-polymorphic.
This rule preserves being able to tell whether a function is pure by looking only at its type:
* if there's a!after its->then the function is definitely effectful
* if there's a!type variable after the->or if the function type has exactly one other function (and that function is an argument, not in the return type) then it's effect-polymorphic and whether or not it performs effects will depend on what type you pass it at the call site.
* otherwise, it's a pure functionImportantly, this rule is about function types that appear visibly in the type annotation. Function types that are behind a type alias or opaque type don't apply to this rule.
This rule lets common higher-order functions have types that look the same as today, while being more useful than today because they're effect-polymorphic.
Richard Feldman (Aug 28 2024 at 19:01):so the rule in the doc for => definitely feels simpler to explain: "their effectfulnesses are equal"
Sam Mohr (Aug 28 2024 at 19:02):At that point, would it maybe be better to have -> now act as => does, and -> act as a "pure-only" version?
Sam Mohr (Aug 28 2024 at 19:02):If possible, the default should be the best option, so the default function signature should be more general if possible
Richard Feldman (Aug 28 2024 at 19:03):yeah so one potential variation of this "higher-order functions are special" design is to say that "-> always means pure" and "=> sometimes means effectful but not always" (instead of the other way around)
Sam Mohr (Aug 28 2024 at 19:04):Yeah, if that's possible to implement in a way that doesn't require much thought from the user and that could be the default function type we teach in the tutorial, then I'm in
Richard Feldman (Aug 28 2024 at 19:04):so there's a broad question of which of these three designs is better:
->, =>, and ! and they all mean different things Sam Mohr (Aug 28 2024 at 19:04):Does your design handle multiple different colorings? What about:
doubleMap : a, (a => b), (b => c) => c
I think you'd need the explicit type variable in that case
Richard Feldman (Aug 28 2024 at 19:05):depends on what you want it to do
Sam Mohr (Aug 28 2024 at 19:05):Yeah, yeah
Richard Feldman (Aug 28 2024 at 19:06):you wouldn't need the explicit type variable if both functions are being called
Sam Mohr (Aug 28 2024 at 19:06):I'm trying to come from a perspective of "How do we make things Just Work:tm:"
Richard Feldman (Aug 28 2024 at 19:06):because they all have the same effectfulness
Agus Zubiaga (Aug 28 2024 at 19:06):but what if you pass an effectful one and a non-effectful one?
Agus Zubiaga (Aug 28 2024 at 19:07):This puts the pressure in the higher-order function author, but they might not know how it's used
Richard Feldman (Aug 28 2024 at 19:08):Agus Zubiaga said:
but what if you pass an effectful one and a non-effectful one?
still the same
Richard Feldman (Aug 28 2024 at 19:08):non-effectful unifies to effectful
Agus Zubiaga (Aug 28 2024 at 19:08):ah, nice
Richard Feldman (Aug 28 2024 at 19:08):I think if you try writing it out with type variables it'll be come apparent :big_smile:
Richard Feldman (Aug 28 2024 at 19:08):like if both are being called, and they have different variable names, how would you put both variable names in the overall function's return type at the same type?
Richard Feldman (Aug 28 2024 at 19:08):(you wouldn't, you'd use the same variable name for all of them)
Richard Feldman (Aug 28 2024 at 19:11):to go back to Jasper's original point about trying to remove ! in favor of changing the arrow, here is one option that takes the same semantics as the current proposal but with different syntax:
List.first : List elem -> Result elem [ListWasEmpty]
List.map : List a, (a ~> b) ~> List b
File.readUtf8 : Str => Result Str [FileReadErr File.ReadErr]
I don't love how ~> looks though haha
Agus Zubiaga (Aug 28 2024 at 19:13):So -> means pure, ~> means equally efffectful, and => means effectful?
Sam Mohr (Aug 28 2024 at 19:13):My worry with -> vs => is that a newcomer won't really get that there's much of a difference. I feel like ~> or something like it implies something special happening, which I'd say effects are
Sam Mohr (Aug 28 2024 at 19:14):I guess I like three arrow operators over two arrows and a bang
Agus Zubiaga (Aug 28 2024 at 19:14):I feel like that's too many symbols for functions :smiley:
Richard Feldman (Aug 28 2024 at 19:15):yeah, although I don't love the "higher-order functions work differently" exception, I do like the idea of having -> and => be the only two options more than I like the idea of having 3 different arrows
Agus Zubiaga (Aug 28 2024 at 19:18):Why do you think it's important for a higher-order function to specify that it can work with effectful functions? It doesn't have to do anything different from the surface
Richard Feldman (Aug 28 2024 at 19:19):if we go with => meaning "potentially effectful function" and -> meaning "pure function" , a neat thing that happens is that in TypeScript it's always =>, so for people coming to Roc from TypeScript, a learning guide can say "in Roc you can annotate the function just like in TypeScript, using =>, which means it might do effects...but now you can additionally annotate it using -> which guarantees it's pure"
Sam Mohr (Aug 28 2024 at 19:19):I'm gonna re-skim the doc in case I'm missing something, but it feels like Task and ! might be simpler to understand over all this function coloring stuff.
Sam Mohr (Aug 28 2024 at 19:20):That's a little inflammatory, actually. Take the above comment with a grain of salt.
Richard Feldman (Aug 28 2024 at 19:20):Agus Zubiaga said:
Why do you think it's important for a higher-order function to specify that it can work with effectful functions? It doesn't have to do anything different from the surface
say I'm inside an effectful function and I'm trying to track down where a piece of outside state is changing (e.g. a database or file or something)
A thing I really value is being able to go through my calls one at a time, just glance at the type of the function being called, and say "that one is pure, so there is zero chance whatsoever that it's responsible for this effect I'm trying to track down, and I can move on to the next call without even glancing at that function's implementation"
Richard Feldman (Aug 28 2024 at 19:21):for that to work, it has to be possible in all cases to look at a function's type signature, and only its type signature, and answer the question "is this function definitely pure or potentially effectful, meaning I need to investigate further?"
Agus Zubiaga (Aug 28 2024 at 19:21):but it's not the higher-order function that isn't pure, it's the effectful function you passed to it, right?
Sam Mohr (Aug 28 2024 at 19:22):I'm not sure how much Roc code will have a type signature written down because we provide such good type inference. If you're saying you'll use your editor to do this, can't we just give a code action to tell you where the effects are happening?
Richard Feldman (Aug 28 2024 at 19:22):Agus Zubiaga said:
but it's not the higher-order function that isn't pure, it's the effectful function you passed to it, right?
it's both
Richard Feldman (Aug 28 2024 at 19:23):like if I'm in the code, and I see a code to List.map foo bar
Richard Feldman (Aug 28 2024 at 19:23):I may have gotten the effectful function bar passed in from elsewhere
Agus Zubiaga (Aug 28 2024 at 19:23):I know what you mean, but wouldn't it already be obvious if in the same line you're passing an effectful function?
Richard Feldman (Aug 28 2024 at 19:23):but passing around a function doesn't do any effects until some function actually cals it
Agus Zubiaga (Aug 28 2024 at 19:23):I see
Agus Zubiaga (Aug 28 2024 at 19:23):but if you're looking at the type of List.map, why wouldn't you look at the type of bar?
Agus Zubiaga (Aug 28 2024 at 19:24):oh, I guess your argument is that you'd stop at List.map because you'd see that it doesn't have a ! or => in its annotation
Richard Feldman (Aug 28 2024 at 19:25):well, that would be the most convenient - but in the design where higher-order functions are treated differently, if I'm aware of that rule I can know to look at the type of bar before deciding whether or not to move on
Agus Zubiaga (Aug 28 2024 at 19:26):Ok, I see your argument now
Agus Zubiaga (Aug 28 2024 at 19:30):I just think that in practice all higher-order functions will use => and then you'd have to check the passed functions anyway
Richard Feldman (Aug 28 2024 at 19:30):totally
Richard Feldman (Aug 28 2024 at 19:30):I think in general, a function being effect-polymorphic means you have to check it
Richard Feldman (Aug 28 2024 at 19:30):and also in general, almost all higher-order functions will end up being effect-polymorphic
Richard Feldman (Aug 28 2024 at 19:30):the only counterexample I could see to that is stuff like Parser
Agus Zubiaga (Aug 28 2024 at 19:31):right, but then there isn't much benefit to having a separate syntax, is there?
Richard Feldman (Aug 28 2024 at 19:31):because to be polymorphic, the effect type variable would have to appear in the type as a type parameter, and I don't think Parser a fx wouldn't be worth it just to be able to run effects in the middle of your parser (which...who actually wants to do that anyway?!)
Sam Mohr (Aug 28 2024 at 19:32):Which means to get your benefit of needing less time to find an issue, we'd want to teach people to avoid notating functions as => where they want to avoid allowing effects.
Sam Mohr (Aug 28 2024 at 19:32):That seems like something you may want in app code, but libraries should be discouraged from doing.
Richard Feldman (Aug 28 2024 at 19:32):Agus Zubiaga said:
right, but then there isn't much benefit to having a separate syntax, is there?
to me, the main benefit of having a separate syntax for "equally effectful" is not to the functions which use it, but to the functions that don't use it
Agus Zubiaga (Aug 28 2024 at 19:33):which higher-order functions would not use it?
Richard Feldman (Aug 28 2024 at 19:33):so let's say I have this type:
... -> Str
because any other function is either effectful or not, and you can tell that from ! in the annotation
Richard Feldman (Aug 28 2024 at 19:34):in the world where there's a separate "equally effectful" syntax, I'm like "that right there is a pure function, no further information needed"
Agus Zubiaga (Aug 28 2024 at 19:34):there's no !, so it's defnitely pure
Richard Feldman (Aug 28 2024 at 19:35):what if the full function type is this?
(Str -> Str) -> Str
so it's actually a higher-order function and therefore now potentially effectful if I pass it a higher-order function
Richard Feldman (Aug 28 2024 at 19:35):so ... -> Str is no longer enough information to tell for sure that the function is pure
Agus Zubiaga (Aug 28 2024 at 19:36):right, but if it's indeed effectful, the function where you call it would have to have a !
Richard Feldman (Aug 28 2024 at 19:36):right, but I now always need to read the entire function type to be able to tell that
Richard Feldman (Aug 28 2024 at 19:36):in the design where there is an "equally effectful syntax", ... -> Str is enough information. It does not matter at all what the arguments are
Sam Mohr (Aug 28 2024 at 19:36):I can see that as a benefit, yes
Richard Feldman (Aug 28 2024 at 19:36):in the design without it, I always have to look at the arguments no matter what, even if it's not higher-order
Richard Feldman (Aug 28 2024 at 19:36):because I can't know if it's higher-order without looking at the arguments
Richard Feldman (Aug 28 2024 at 19:37):so now I always need to consider all the arguments and the return type to know whether it's pure, no matter what
Richard Feldman (Aug 28 2024 at 19:37):even if I never call a higher-order function in my entire life (somehow), I still need to check all the arguments every single time
Richard Feldman (Aug 28 2024 at 19:37):because there's no other way to tell if they're higher-order
Richard Feldman (Aug 28 2024 at 19:38):that's what I mean about the main benefit of having a separate "equally effective" syntax mainly benefiting functions which don't even use that syntax
Richard Feldman (Aug 28 2024 at 19:38):the benefit to those functions is that I only have to look in one place to know that the function is pure, instead of always having to look at the args to figure out if it's higher-order or not, and therefore potentially only sometimes pure
Richard Feldman (Aug 28 2024 at 19:39):I'm not saying that's a deal-breaker btw, I just want to make sure the downside is clear
Agus Zubiaga (Aug 28 2024 at 19:44):Yeah, I can see how that helps
Agus Zubiaga (Aug 28 2024 at 19:47):I'm still not sure how much difference it would make in the common case, but it's hard to know this until you have lived with the feature
Sam Mohr (Aug 28 2024 at 19:47):Actually, question: with this design, I don't think whether a function is effectful can effect affect if it will fail early, right? It just works like a Result that might take a while to return, yes?
Sam Mohr (Aug 28 2024 at 19:48):If an effectful function won't cause early returns in a way different from Results, then I really don't think there's a need for the above ability to go find where the failure happened
Richard Feldman (Aug 28 2024 at 19:48):so what if we tried this as the design?
=> means there is definitely an effect-> with a visible function type (so, not hidden behind an alias or opaque type) is effect-polymorphic and will run an effect if and only if you pass it a function that runs effects (and otherwise it will be pure)-> in all other cases guarantees the function is pure Richard Feldman (Aug 28 2024 at 19:48):so that means -> basically means "this can definitely be called from a pure function"
Richard Feldman (Aug 28 2024 at 19:49):but if you call it from an effectful function, it might do an effect
Richard Feldman (Aug 28 2024 at 19:49):"pure functions can always call -> functions and never => functions" is true in this design
Sam Mohr (Aug 28 2024 at 19:49):Works for me!
Brendan Hansknecht (Aug 28 2024 at 19:50):Can you show the list.map type with that specification?
Notification Bot (Aug 28 2024 at 19:50):105 messages were moved here from #ideas > Purity Inference by Richard Feldman.
Sam Mohr (Aug 28 2024 at 19:50):It would be nice to have a more effect-implying symbol for function signatures than =>, but I can't think of anything.
>> could work? It's probably fine
Richard Feldman (Aug 28 2024 at 19:51):Brendan Hansknecht said:
Can you show the list.map type with that specification?
it's the same as today:
List.map : List a, (a -> b) -> List b
I think that is a bad idea
Brendan Hansknecht (Aug 28 2024 at 19:52):I think it hides effectfulness and that is just confusing
Brendan Hansknecht (Aug 28 2024 at 19:52):Cause that looks pure, but it isn't guaranteed to be.
Agus Zubiaga (Aug 28 2024 at 19:53):It's as guaranteed pure as the function you pass
Sam Mohr (Aug 28 2024 at 19:53):It does hide it, but doesn't not requiring ! hide it as well, in a way? I think this whole subject is a way to hide Tasks, basically
Brendan Hansknecht (Aug 28 2024 at 19:54):Especially given, it might be pure maybe (a -> b) is a deprecated argument and not actually called. Maybe it always returns an empty list
Brendan Hansknecht (Aug 28 2024 at 19:54):There is no guarantee here
Brendan Hansknecht (Aug 28 2024 at 19:54):So I think it really needs to be in the type especially as apis get more conplex
Brendan Hansknecht (Aug 28 2024 at 19:57):Sam Mohr said:
It does hide it, but doesn't not requiring
!hide it as well, in a way? I think this whole subject is a way to hide Tasks, basically
That is actually my biggest concern of this entire proposal. In most (all?) languages with async and await, await is required. This is for understandability. There is no reason it actually is needed in the code. Calling an async function is enough to know it must be awaited. Await could be hidden away.
I think it is probably a bad idea that roc is fully hiding away !. Roc doesn't require types. As such, I think it is rather important to have ! always be in the code for clarity.
Agus Zubiaga (Aug 28 2024 at 19:58):I thought about requiring ! in expressions still, but it'd be awkward if you had to unwrap the Result too, you'd end up with a lot of !?
Brendan Hansknecht (Aug 28 2024 at 19:59):Easily telling if code is effectful or not is important. One of the biggest benefits of a pure language like roc. Really important to be able to see in the middle of a code function which class might be introducing impurity. In a long function seeing a File.read! is a reminder that that specific call might return varying results. Removing ! loses that and greatly hurts debugability
Richard Feldman (Aug 28 2024 at 19:59):Brendan Hansknecht said:
Sam Mohr said:
It does hide it, but doesn't not requiring
!hide it as well, in a way? I think this whole subject is a way to hide Tasks, basicallyThat is actually my biggest concern of this entire proposal. In most (all?) languages with async and await, await is required. This is for understandability. There is no reason it actually is needed in the code. Calling an async function is enough to know it must be awaited. Await could be hidden away.
Go is a notable exception; all I/O in Go is async behind the scenes and there's no await keyword (or async keyword for that matter) and I've generally seen that described as a selling point of Go
Brendan Hansknecht (Aug 28 2024 at 20:00):That is not a correct view of go
Brendan Hansknecht (Aug 28 2024 at 20:00):And it is the reversw probalm
Brendan Hansknecht (Aug 28 2024 at 20:00):Async to sync is always safe. Sync to async is not
Richard Feldman (Aug 28 2024 at 20:00):Brendan Hansknecht said:
Easily telling if code is effectful or not is important. One of the biggest benefits of a pure language like roc. Really important to be able to see in the middle of a code function which class might be introducing impurity. In a long function seeing a
File.read!is a reminder that that specific call might return varying results. Removing!loses that and greatly hurts debugability
a design we could consider is requiring ! at the end of the name if a function wants to do effects
Brendan Hansknecht (Aug 28 2024 at 20:00):Cause you can always make an async call and then block
Brendan Hansknecht (Aug 28 2024 at 20:00):That creates a sync api
Richard Feldman (Aug 28 2024 at 20:01):I gotta go for a bit, but will check back in later!
Sam Mohr (Aug 28 2024 at 20:01):I think this proposal gives us two things:
! in its signatureTask and normal values too.@Brendan Hansknecht I think you don't like the first point and its implications, which I'm also moving toward. But the second point isn't really possible in Roc, and would be nice to support as a way to make more code more useful.
Brendan Hansknecht (Aug 28 2024 at 20:01):Go everything is sync with explicit threading. The thread just may block on an async call (which makes it async)
Sam Mohr (Aug 28 2024 at 20:02):I think something along the lines of List.map : List a, (a -> b ! e) -> List b would be nice
Brendan Hansknecht (Aug 28 2024 at 20:02):Yeah, I would be fine with 2. I would just require using ! with maybe effectful functions.
Sam Mohr (Aug 28 2024 at 20:04):So at the very least, I presume you'd be okay with changing function signatures from Stdout.line: Str -> Task Str * to Stdout.line: Str -> Result! Str *?
Sam Mohr (Aug 28 2024 at 20:04):Part of 1) above is hiding the concept of Task, which is separate from hiding effects
Jasper Woudenberg (Aug 28 2024 at 20:05):Richard Feldman [said]
a design we could consider is requiring
!at the end of the name if a function wants to do effects
I kind of like this, but I wonder if it goes against the goal of not bothering newcomers with the difference between effecfull/pure functions, because now they'll have to know when to put a bang in their function names and when not.
drew (Aug 28 2024 at 20:33):prior art in unison here https://www.unison-lang.org/docs/language-reference/abilities-and-ability-handlers/#abilities-in-function-types. they put effects “on the arrow” and a standard arrow is polymorphic on abilities. meaning higher order functions don’t need an ability and non-ability version (sorry for bringing up unison so often, not an expert but they seem to have solved some similar problems)
Jasper Woudenberg (Aug 28 2024 at 20:36):Jasper Woudenberg said:
Richard Feldman [said]
a design we could consider is requiring
!at the end of the name if a function wants to do effectsI kind of like this, but I wonder if it goes against the goal of not bothering newcomers with the difference between effecfull/pure functions, because now they'll have to know when to put a bang in their function names and when not.
Come to think of it, maybe it's a benefit, the learning curve passing through the difference between pure/effectull functions before types.
drew (Aug 28 2024 at 20:41):drew said:
prior art in unison here https://www.unison-lang.org/docs/language-reference/abilities-and-ability-handlers/#abilities-in-function-types. they put effects “on the arrow” and a standard arrow is polymorphic on abilities. meaning higher order functions don’t need an ability and non-ability version (sorry for bringing up unison so often, not an expert but they seem to have solved some similar problems)
Here's the map signature, FYI https://share.unison-lang.org/@unison/base/code/releases/3.17.0/latest/terms/@ko93h54ensirkthhekqb7898lk1j1klhv5mplfpcui1bh03dcrg9i2tp3ibfk3r1qp36uhie79el2fbc15rpm6bkgl7l1tfahst2m6g. the {e} doesn't need to be explicitly written
drew (Aug 28 2024 at 20:48):example usage https://gist.github.com/drewolson/14c36dca67c4b59b05d1099f6df45bf5
Agus Zubiaga (Aug 28 2024 at 20:48):Is the {e} required for all higher-order functions?
Agus Zubiaga (Aug 28 2024 at 20:49):Or passed functions, I guess
Agus Zubiaga (Aug 28 2024 at 20:52):That might be a good compromise. All passed functions need either the "equally effectful" syntax or the effectful polymorphism variable
Agus Zubiaga (Aug 28 2024 at 20:53):so you can't forget to do it and you can always pass an effectful function to a higher-order one
drew (Aug 28 2024 at 20:56):a -> b is syntactic sugar for a ->{e} b
drew (Aug 28 2024 at 20:57):yes, equally effectful is right
Sam Mohr (Aug 28 2024 at 20:57):We could allow that, but then Rich's point of "I want to be able to read the signature and know what's happening"
drew (Aug 28 2024 at 20:57):yeah, true, it definitely doesn't help there at all
Sam Mohr (Aug 28 2024 at 20:57):I'd rather some concise but explicit way to do it
Sam Mohr (Aug 28 2024 at 20:59):I think at least I prefer a ->{e} b over a -> b ! e because it has that "during" implication to it
Richard Feldman (Aug 28 2024 at 21:00):drew said:
prior art in unison here https://www.unison-lang.org/docs/language-reference/abilities-and-ability-handlers/#abilities-in-function-types. they put effects “on the arrow” and a standard arrow is polymorphic on abilities. meaning higher order functions don’t need an ability and non-ability version (sorry for bringing up unison so often, not an expert but they seem to have solved some similar problems)
unless I'm missing something, it sounds like Unison has this design when it comes to -> (although they don't have => because they enumerate which effects a function does, so a generic "this does effects" wouldn't make sense for them):
Richard Feldman said:
so what if we tried this as the design?
=>means there is definitely an effect->with a visible function type (so, not hidden behind an alias or opaque type) is effect-polymorphic and will run an effect if and only if you pass it a function that runs effects (and otherwise it will be pure)->in all other cases guarantees the function is pure
Agus Zubiaga (Aug 28 2024 at 21:01):looks like it
Richard Feldman (Aug 28 2024 at 21:02):I guess an alternative way to say it is:
=> means there is definitely an effect-> means "equally effective"-> in the function type, the function must be pure Richard Feldman (Aug 28 2024 at 21:02):when I say it that way I like it better
Richard Feldman (Aug 28 2024 at 21:02):feels a lot simpler to look at a function and tell if it's pure
Richard Feldman (Aug 28 2024 at 21:05):"is there one ->? if so, it's definitely pure"
Agus Zubiaga (Aug 28 2024 at 21:05):It would make a lot of sense if the syntax for the effectful type var was a b> c, but I guess that looks weird :upside_down:
Agus Zubiaga (Aug 28 2024 at 21:07):(where b is the "effectfulness")
Sam Mohr (Aug 28 2024 at 21:07):So now this seems like a good solution for making the default function signature -> effect agnostic while still allowing explicit annotation via =>. Is there a way to make Brendan happy w.r.t. obvious await via !?
Richard Feldman (Aug 28 2024 at 21:18):to explore the "-> and => only, no ! in types" design further - in that world, what would the syntax be for when there's a type variable involved?
Richard Feldman (Aug 28 2024 at 21:18):-a-> :stuck_out_tongue:
Sam Mohr (Aug 28 2024 at 21:18):Like actually
Richard Feldman (Aug 28 2024 at 21:19):yeah I meant that as a joke but now that I see it I actually kinda dig it :big_smile:
Richard Feldman (Aug 28 2024 at 21:19):it's pretty self-explanatory haha
Sam Mohr (Aug 28 2024 at 21:19):If you're not a fan of >2 chars long operators, then I see why it's off the table
Richard Feldman (Aug 28 2024 at 21:19):in general I don't like them, but this feels different
Agus Zubiaga (Aug 28 2024 at 21:19):we can call it the brochette operator
Richard Feldman (Aug 28 2024 at 21:19):like splitting up a 2-char operator
Sam Mohr (Aug 28 2024 at 21:20):Something you were getting at with the "I want to look at the ... -> Str part of the function is the localization of useful info
Sam Mohr (Aug 28 2024 at 21:20):I think that -a-> localizes the effect info to where it's happening
Sam Mohr (Aug 28 2024 at 21:21):I've already been pushing this idea, so I probably sound like a broken record, but having stuff by the arrow Just Makes Sense to me
Sam Mohr (Aug 28 2024 at 21:22):I think our options are:
a -e-> ba -> b ! ea ->{e} ba -> b {e} Agus Zubiaga (Aug 28 2024 at 21:24):my preference: 1 > 3 > 2 > 4
Sam Mohr (Aug 28 2024 at 21:24):Literally same
Sam Mohr (Aug 28 2024 at 21:24):I think the ! makes sense if we keep the ! suffix for awaiting effectful code, but is vestigially the only part of Roc with ! if we get rid of the !
Sam Mohr (Aug 28 2024 at 21:27):The terse options (a.k.a. 1 and 3) seem less nice if we have long effectful type variables: -effect-> is still legible, but not as nice as -e-> IMO.
Sam Mohr (Aug 28 2024 at 21:27):But I still think it's obvious what it means: stuff that happens "during" execution
Richard Feldman (Aug 28 2024 at 21:27):so putting together the most recent discussed design ideas:
List.first : List elem -> Result elem [ListWasEmpty]
List.map : List a, (a -> b) -> List b
# alternatively, if you want to write out the variables:
List.map : List a, (a -fx-> b) -fx-> List b
File.readUtf8 : Str => Result Str [FileReadErr File.ReadErr]
Looks GREAT to me! :heart_eyes:
Agus Zubiaga (Aug 28 2024 at 21:31):Yeah, I really like this!
Richard Feldman (Aug 28 2024 at 21:31):notable features of this design:
! in types, which means no need to explain why foo : Str! is invalid; you can't even attempt to write that-> functions and can never call => functions-> means 'equal effectfulness', which implies that if there's only one ->, the function is pure" Sam Mohr (Aug 28 2024 at 21:32):I guess the thing that I'm not sure that you, Rich, are convinced of, is whether -> inferred to mean -fx-> is good with you. And if you're tentatively agreed pending code examples, what condition would cause us to change the -> behavior?
Richard Feldman (Aug 28 2024 at 21:33):I'm actually good with it now
Richard Feldman (Aug 28 2024 at 21:33):the new wording of the rule (which is technically slightly different semantically than the old one, but not in a way that I think is likely to matter ~ever) seems much better to me
Richard Feldman (Aug 28 2024 at 21:33):there's less thinking involved because you can just scan the type to count the number of ->s
Sam Mohr (Aug 28 2024 at 21:34):Great!
Agus Zubiaga (Aug 28 2024 at 21:34):I don't know if you saw but a compomise I proposed is to always require -fx-> for passed functions
Richard Feldman (Aug 28 2024 at 21:34):put another way, it doesn't have ... -> Str but it does have ... -> ... -> Str meaning it's effect-polymorphic
Richard Feldman (Aug 28 2024 at 21:34):Agus Zubiaga said:
I don't know if you saw but a compomise I proposed is to always require
-fx->for passed functions
I'd rather not haha
Richard Feldman (Aug 28 2024 at 21:34):I think higher-order functions are so common, it's worth not having to put extra variables in them
Richard Feldman (Aug 28 2024 at 21:34):especially because they look complicated enough just from being higher-order
Richard Feldman (Aug 28 2024 at 21:34):without having to make them even longer
Agus Zubiaga (Aug 28 2024 at 21:35):yeah, makes sense
Agus Zubiaga (Aug 28 2024 at 21:35):that's what I prefer too
Richard Feldman (Aug 28 2024 at 21:36):Richard Feldman said:
put another way, it doesn't have
... -> Strbut it does have... -> ... -> Strmeaning it's effect-polymorphic
maybe another way to say what I like about this rule is that it feels like "-> means equality of effects, and then there's a natural special-case that falls out of that where functions are pure" is nicer than "-> normally means pure EXCEPT! there's a gotcha to be aware of where sometimes it doesn't, which we introduced for convenience"
Richard Feldman (Aug 28 2024 at 21:36):even if in practice they have almost exactly the same semantics :big_smile:
Sam Mohr (Aug 28 2024 at 21:51):Wait a second: maybe I'm crazy, but how to we handle non-function Tasks??
Sam Mohr (Aug 28 2024 at 21:52):If my code is
main : Result {} *
main =
Stdout.line "Hello, world!"
Where do I put my effect annotation?
Sam Mohr (Aug 28 2024 at 21:53):Crazy idea, but between this issue and the inability to specialize non-function values causing issues in our platform linking, maybe we just ban Tasks that aren't returned by functions?
Sam Mohr (Aug 28 2024 at 21:54):This might need a new topic...
Agus Zubiaga (Aug 28 2024 at 21:56):That's not possible, you have to make it a thunk
Agus Zubiaga (Aug 28 2024 at 21:57):I mean the proposal specifies only functions can be effectful
Sam Mohr (Aug 28 2024 at 21:57):Well currently the actual type of main in basic-cli is Task I32 _, not {} -> Task I32 {}
Agus Zubiaga (Aug 28 2024 at 21:58):yeah, it'd have to become a function
Sam Mohr (Aug 28 2024 at 21:58):Okay, so yeah, we're banning bare Tasks, good to be on the same page
Richard Feldman (Aug 28 2024 at 21:58):the doc has a section titled "Thunks" that talks about this a bit
Sam Mohr (Aug 28 2024 at 21:58):Okay, reading
Brendan Hansknecht (Aug 28 2024 at 23:05):One piece I don't get about the current proposal to have -> ambiguous over effectfulness.
If I see:
Str -> Str couldn't it do an effect? Like it could use the first Str to call file.ReadUtf8?
Brendan Hansknecht (Aug 28 2024 at 23:09):Feels like I would never know if a function is pure
Brendan Hansknecht (Aug 28 2024 at 23:09):Also, I feel like sometime you don't want to allow any sort of effects in your higher order functions (though maybe that is a wrong opinion...not sure)
Richard Feldman (Aug 28 2024 at 23:20):Brendan Hansknecht said:
One piece I don't get about the current proposal to have
->ambiguous over effectfulness.If I see:
Str -> Strcouldn't it do an effect? Like it could use the first Str to call file.ReadUtf8?
the most recent idea is to have all ->s in a type signature share an effect variable
Richard Feldman (Aug 28 2024 at 23:21):so if there's only one -> in the type then it's definitely pure
Sam Mohr (Aug 28 2024 at 23:21):Brendan Hansknecht said:
One piece I don't get about the current proposal to have
->ambiguous over effectfulness.If I see:
Str -> Strcouldn't it do an effect? Like it could use the first Str to call file.ReadUtf8?
From Rich's comment, "-> means equality of effects".
Meaning if I have a function List.map : List a, (a -> b) -> List b, then we inherit all effectfulness from our inputted callbacks. If the a -> b mapper callback is effectful, it "pollutes" all -> arrows and now the parent function is effectful. If there's only one -> like in Str -> Str, then it can't be "polluted" by any effectful callbacks
Brendan Hansknecht (Aug 28 2024 at 23:21):I don't think that follows from your first statement, So Str -> Str would be Str -fx> Str and fx could be effectful.
Brendan Hansknecht (Aug 28 2024 at 23:21):I think this is definitely an inconsistency
Richard Feldman (Aug 28 2024 at 23:22):to demonstrate why it's not, how would you implement an effectful version of that function?
Richard Feldman (Aug 28 2024 at 23:22):like that's the type, what's the body?
Sam Mohr (Aug 28 2024 at 23:23):I think you have two sources for effectfulness: the body of the function, and callbacks provided to the function. If the body is effectful, we infer the whole function as =>. If the callbacks are potentially effectful, we get -fx->.
In Str -> Str, if the body is effectful, we'd infer Str => Str, if it isn't, we infer Str -> Str.
Richard Feldman (Aug 28 2024 at 23:23):(fx is * in Str -> Str)
Sam Mohr (Aug 28 2024 at 23:23):And if you annotate Str => Str as Str -> Str, you'd get a type error
Richard Feldman (Aug 28 2024 at 23:24):yeah so if you have foo : Str -> Str I claim it's impossible to write an implementation of foo that does effects :big_smile:
Richard Feldman (Aug 28 2024 at 23:25):(because if you call an effect, that function's effect variable would have to become bound, and therefore its inferred type would have to be Str => Str, and Str -> Str would be too general)
Sam Mohr (Aug 28 2024 at 23:25):The only way to do it would be to write a stdlib function with effects lol
Brendan Hansknecht (Aug 28 2024 at 23:25):I don't think I follow:
x : Str -fx> Str
x = \str ->
file.readUtf8 str
fx is parametric and roc would infer that fx must be effectful due to the call to file.readUtf8
Richard Feldman (Aug 28 2024 at 23:25):Str -fx-> Str is like saying Str -*-> Str
Richard Feldman (Aug 28 2024 at 23:25):that type is incorrectly claiming that the function is more flexible than its implementation is
Brendan Hansknecht (Aug 28 2024 at 23:26):This still doesn't sit right
x : Str -fx> Str
x = \str ->
str
If that is the case, this is also too general
Brendan Hansknecht (Aug 28 2024 at 23:26):Cause it is forcing fx to be no effects
Richard Feldman (Aug 28 2024 at 23:26):that's the point though!
Brendan Hansknecht (Aug 28 2024 at 23:26):Which also is more general than being unsure about effects or not
Richard Feldman (Aug 28 2024 at 23:27):so if I have a function that does effects, I can call both pure functions and effectful functions from it
Brendan Hansknecht (Aug 28 2024 at 23:27):Like both specifically effectful and specifically pure are not as general as maybe effectful
Richard Feldman (Aug 28 2024 at 23:27):the way that works (using type unification) is that pure functions have an unbound effect variable, and effectful ones have it bound (to {} or something, doesn't really matter what as long as it's concrete)
Richard Feldman (Aug 28 2024 at 23:28):so actually a pure function is as general as "maybe effectful" because otherwise you couldn't call a pure function from an effectful function
Richard Feldman (Aug 28 2024 at 23:28):which of course wouldn't be desirable :sweat_smile:
Sam Mohr (Aug 28 2024 at 23:28):Let's say I have this function:
doubleMap : a, (a -fx-> b), (b -fx-> c) -fx-> c
doubleMap = \a, mapToB, mapToC ->
a
|> mapToB
|> mapToC
If mapToB is not effectful but mapToC is, then effects are being done, so the three fx vars unify to "effectful"
Brendan Hansknecht (Aug 28 2024 at 23:29):Ok, I guess I get it. Only works cause pure implicitly can convert to effectful
drew (Aug 28 2024 at 23:46):i still don’t understand why => is required in this proposal, but honestly i’m just a fly on the wall here
Richard Feldman (Aug 28 2024 at 23:47):the most recent idea has => meaning what ! did in the doc
Richard Feldman (Aug 28 2024 at 23:48):so in this design, => means "definitely effectful"
Richard Feldman (Aug 28 2024 at 23:48):I'll make a v2 of the proposal at some point based on the discussions here
Richard Feldman (Aug 28 2024 at 23:57):two slight tweaks I just realized are necessary
Richard Feldman (Aug 28 2024 at 23:58):one, similarly to how we elide unbound type variables in open tag unions, we need to do the same for unbound effect variables in type aliases and opaque types, because otherwise you couldn't do this:
Foo := (Str -> Str)
it would have to be
Foo a := (Str -a-> Str)
which would be really annoying
Richard Feldman (Aug 28 2024 at 23:59):second, and related, we should say that -> is sugar for -fx-> across an entire function annotation, not across an entire type annotation
Richard Feldman (Aug 28 2024 at 23:59):otherwise this wouldn't work either:
Parser a := [
Succeed a,
Foo (Parser a -> Parser a),
Bar (Parser a -> Parser a),
]
it would have to be:
Parser a fx := [
Succeed a,
Foo (Parser a -fx> Parser a),
Bar (Parser a -fx> Parser a),
]
which again would be undesirable :big_smile:
Richard Feldman (Aug 29 2024 at 00:02):actually that second one might not be necessary, if we wanted to allow hiding the effect variable in general - even if it's bound :thinking:
Richard Feldman (Aug 29 2024 at 00:02):I don't think we have a precedent for that, but a variable is a variable...maybe it's fine
Brendan Hansknecht (Aug 29 2024 at 00:05):I think this is my exact concern. This really is losing the ability in roc to tell what is pure or not
Richard Feldman (Aug 29 2024 at 00:07):how so? In the tweak I suggested, these are 100% definitely both pure:
Parser a := [
Succeed a,
Foo (Parser a -> Parser a),
Bar (Parser a -> Parser a),
]
because the -> sharing variables only applies within a function definition, not the surrounding tag union
Brendan Hansknecht (Aug 29 2024 at 00:08):Foo := (Str -> Str)
... I thought it was pure and safe to use here without magical changes, but actually it ran an effect and changed a bunch of my code to effectful
Brendan Hansknecht (Aug 29 2024 at 00:09):Becuase I used foo which I thought could only wrap a pure function, it lead to weird bugs due to expecting purity that wasn't there.
Richard Feldman (Aug 29 2024 at 00:10):oh, definitely if you don't put a type variable in Foo you wouldn't be able to do that
Richard Feldman (Aug 29 2024 at 00:10):(I should have mentioned that explicitly earlier!)
Sam Mohr (Aug 29 2024 at 00:11):List.fooStrings : List Str, Foo -> List Str
If Foo contains effectful code, then this is now effectful even with only one visible ->. That is a issue, yeah
Richard Feldman (Aug 29 2024 at 00:11):like Foo fx := (Str -fx-> Str) would let you put an effectful function in there
Richard Feldman (Aug 29 2024 at 00:11):but Foo := (Str -> Str) would only accept pure functions
Brendan Hansknecht (Aug 29 2024 at 00:13):Richard Feldman said:
it would have to be
Foo a := (Str -a-> Str)which would be really annoying
I guess I misunderstood :point_up:. I thought you were saying that you wanted this to work implicitly causing adding the type variable for effectfulness would be annoying.
Richard Feldman (Aug 29 2024 at 00:14):oh no, sorry haha
Richard Feldman (Aug 29 2024 at 00:14):I just meant that you should be able to put pure functions in type aliases and opaque types (like today) without adding a type parameter to them
Richard Feldman (Aug 29 2024 at 00:14):(and if you want it to be polymorphic, also like today, you do have to add a type parameter)
Sam Mohr (Aug 29 2024 at 00:15):Even if we force type aliases/opaque types to annotate their effect-polymorphism, it's still
List.fooStrings : List Str, Foo fx -> List Str
which isn't visibly effect-polymorphic unless the arrow is changed to -fx->
Sam Mohr (Aug 29 2024 at 00:16):We could do some analysis during type checking to say that if you reference a type that is effect-polymorphic, then you need to explicitly annotate your return arrow as effect-polymorphic?
Sam Mohr (Aug 29 2024 at 00:17):This seems better than being allowed to write code that "looks pure" (meaning has one arrow), or not doing said analysis and then needing to know things about all used aliases
Brendan Hansknecht (Aug 29 2024 at 00:17):I found a simple example that gets really weird:
memoize: Dict a b, (a -> b) -> b
Suddenly memoize doesn't work as expected
Brendan Hansknecht (Aug 29 2024 at 00:20):Cause a -> b is not a pure function
Brendan Hansknecht (Aug 29 2024 at 00:21):So memoization is not actually safe, but the writer assumed only pure functions could be passed in and that is really what the api seems to suggest
Richard Feldman (Aug 29 2024 at 00:27):hm, can we find another example? I don't think memoize with that type can do what it says it would :sweat_smile:
Richard Feldman (Aug 29 2024 at 00:28):I guess it could be this:
memoize : Dict a b, (a -> b) -> (Dict a b, b)
Sorry:
memoize: Dict a b, a, (a -> b) -> (Dict a b, b)
yeah something like that
Richard Feldman (Aug 29 2024 at 00:29):right, so yeah you'd want to be able to say "this function has to be pure"
Agus Zubiaga (Aug 29 2024 at 00:29):That’s also unsafe because you could pass different functions in later calls
Richard Feldman (Aug 29 2024 at 00:29):so this would work I think:
memoize : Dict a b, a, (a -x-> b) -y-> (Dict a b, b)
(or this, but I'm trying to avoid using * since we might remove it)
memoize : Dict a b, a, (a -*-> b) -*-> (Dict a b, b)
I thought about that but would that compile?
Brendan Hansknecht (Aug 29 2024 at 00:30):Well that is the issue, I think the problem is that you have to specify that. You could also just use -*>
Agus Zubiaga (Aug 29 2024 at 00:30):I think you couldn’t call the function inside
Agus Zubiaga (Aug 29 2024 at 00:30):Because the type says it might return an effect, but you cannot return that effect from the higher order function
Richard Feldman (Aug 29 2024 at 00:31):hm, I don't follow
Agus Zubiaga (Aug 29 2024 at 00:31):I don’t follow myself actually :sweat_smile:
Brendan Hansknecht (Aug 29 2024 at 00:31):I think -*> works, but I think the issue is that a user has to specify it.
It means that the users has to definsively program. Any function they take in as an input by default might be impure
Brendan Hansknecht (Aug 29 2024 at 00:32):I have to always assume impurity in any code I look at online with higher order functions. This is bad
Agus Zubiaga (Aug 29 2024 at 00:33):if we had a separate syntax it would still have to be chosen defensively
Richard Feldman (Aug 29 2024 at 00:35):well this is the entire goal though, right?
Richard Feldman (Aug 29 2024 at 00:35):like we can't possibly have it both ways
Richard Feldman (Aug 29 2024 at 00:36):either by default higher-order functions are effect-polymorphic or they aren't
Brendan Hansknecht (Aug 29 2024 at 00:36):I think the original => was a lot better
Brendan Hansknecht (Aug 29 2024 at 00:36):It at least was opt in
Agus Zubiaga (Aug 29 2024 at 00:36):but people would still use it for every higher order function
Brendan Hansknecht (Aug 29 2024 at 00:36):I don't think so
Agus Zubiaga (Aug 29 2024 at 00:36):so they would have to go outside the norm to enforce pureness
Brendan Hansknecht (Aug 29 2024 at 00:37):I think most people will use the default of -> and never think about it.
Richard Feldman (Aug 29 2024 at 00:37):I don't think that would last
Richard Feldman (Aug 29 2024 at 00:37):if your package offers higher-order functions that aren't effect-polymorphic (like the builtins would be), people would complain and ask you to change it
Brendan Hansknecht (Aug 29 2024 at 00:38):This hypothesis stuns me
Sam Mohr (Aug 29 2024 at 00:38):So we need at least an opt-out
Brendan Hansknecht (Aug 29 2024 at 00:38):Imagine a world where we don't have this change
Brendan Hansknecht (Aug 29 2024 at 00:38):Most higher order functions would not use tasks, would they?
Brendan Hansknecht (Aug 29 2024 at 00:39):And if my function doesn't return a task, I can't call an inter function that returns a task
Richard Feldman (Aug 29 2024 at 00:39):probably not, but I think the main reason for that is that people don't want to duplicate all their higher-order functions and name them ___Task :big_smile:
Brendan Hansknecht (Aug 29 2024 at 00:39):The only minor exception to that is where a user has control of the state
Brendan Hansknecht (Aug 29 2024 at 00:40):I think most higher order functions never have a need to be impure
Agus Zubiaga (Aug 29 2024 at 00:40):Also, in the Task world you can still use walk-like functions to compose them
Richard Feldman (Aug 29 2024 at 00:40):well that's still true
Richard Feldman (Aug 29 2024 at 00:40):just do it with thunks instead of tasks and then run all the thunks
Agus Zubiaga (Aug 29 2024 at 00:40):Only by passing effectul functions, right?
Richard Feldman (Aug 29 2024 at 00:40):you can always build up as many thunks as you want using pure functions
Richard Feldman (Aug 29 2024 at 00:40):doesn't matter if the thunks are effectful or not
Richard Feldman (Aug 29 2024 at 00:41):defining a function is pure
Richard Feldman (Aug 29 2024 at 00:41):only running it can possibly do effects
Agus Zubiaga (Aug 29 2024 at 00:41):Yeah, but you cannot compose them with a function that disallows effects
Richard Feldman (Aug 29 2024 at 00:41):like for example, today I can do List.map strings File.readUtf8 and I get back a List (Task Str _)
Richard Feldman (Aug 29 2024 at 00:41):and I can then walk that list to run all the tasks
Richard Feldman (Aug 29 2024 at 00:42):similarly, in the effect world I can do List.map strings \str -> \{} -> File.readUtf8 str
Agus Zubiaga (Aug 29 2024 at 00:42):You cannot run from inside List.walk though
Agus Zubiaga (Aug 29 2024 at 00:42):Unless it allows effectful functions
Richard Feldman (Aug 29 2024 at 00:42):I mean even if that wasn't allowed in this world, you can still recurse
Richard Feldman (Aug 29 2024 at 00:42):and pull them out of the list one at a time if you have to
Agus Zubiaga (Aug 29 2024 at 00:43):Like with a recursive function?
Agus Zubiaga (Aug 29 2024 at 00:43):That works for list but wouldn’t work with opaque types from a package
Richard Feldman (Aug 29 2024 at 00:44):ok but if that exposes a pure walk function I can use that to obtain a List and recurse etc.
Richard Feldman (Aug 29 2024 at 00:45):and even if List didn't support that, you could build your own cons list out of tag unions
Richard Feldman (Aug 29 2024 at 00:45):there is no way to prevent people from using pure functions to assemble sequences of effectful thunks and then run them, I promise :big_smile:
Richard Feldman (Aug 29 2024 at 00:45):it's just a matter of how convenient that is
Agus Zubiaga (Aug 29 2024 at 00:45):Haha yeah, I know
Brendan Hansknecht (Aug 29 2024 at 00:45):With this prososal, why are we keeping roc pure at all?
It doesn't make sense to me. Roc is a functional language. In roc, higher order functions are super common and all over the place. If every higher order function is ambiguous to purity by default, what are the benefits of having a pure language at at all? Almost all functions would be maybe impure.
Agus Zubiaga (Aug 29 2024 at 00:45):It’s just mean it’s more ergonomic in the Task world
Richard Feldman (Aug 29 2024 at 00:46):Brendan Hansknecht said:
With this prososal, why are we keeping roc pure at all?
It doesn't make sense to me. Roc is a functional language. In roc, higher order functions are super common and all over the place. If every higher order function is ambiguous to purity by default, what are the benefits of having a pure language at at all? Almost all functions would be maybe impure.
oh I don't think so - I expect the percentage of pure functions in a Roc program to be unchanged by this
Brendan Hansknecht (Aug 29 2024 at 00:47):Please explain, I definitely do not understand.
Brendan Hansknecht (Aug 29 2024 at 00:47):To me this definitely feels like so much convenience and implicitness such that the langauge may as well be impure.
Brendan Hansknecht (Aug 29 2024 at 00:48):Like I will have to be just as defensive as a programmer using an impure language cause I use higher order functions all over the place and any of them could be impure now.
Richard Feldman (Aug 29 2024 at 00:48):oh not at all
Richard Feldman (Aug 29 2024 at 00:48):like ok so let's say I write a function with this type:
foo : Str -> Str
foo = \str ->
in an imperative language, the body of this function could be doing effects all over the place
Richard Feldman (Aug 29 2024 at 00:49):there's no version of this proposal where that's true
Richard Feldman (Aug 29 2024 at 00:49):100% for sure, calling foo will perform zero effects
Sam Mohr (Aug 29 2024 at 00:49):Brendan, you are probably already thinking this, but I think the goal here is to think like a startup: let's try something and be idealistic in our design, and if it works out, cool, if not, we move on. We have to be a little "unreasonable" to get outside of our comfort zone, but that doesn't mean we'll stay in crazy land if we can't make it work, all said and done.
Richard Feldman (Aug 29 2024 at 00:49):and if foo tries to do any effects, either by calling a higher-order function or not, it will result in a compile error
Brendan Hansknecht (Aug 29 2024 at 00:50):Yep. But in practice, I think many of my functions are higher order.
Richard Feldman (Aug 29 2024 at 00:50):ok so let's pick one of those higher-order functions!
Richard Feldman (Aug 29 2024 at 00:50):let's say I am authoring this higher-order function:
Richard Feldman (Aug 29 2024 at 00:51):bar : (Str -> Str) -> Str
bar = ...
100% for sure, the implementation of bar does not call any effectful functions
Richard Feldman (Aug 29 2024 at 00:51):I can tell you that just from the type
Richard Feldman (Aug 29 2024 at 00:51):otherwise it would be a compile error
Richard Feldman (Aug 29 2024 at 00:51):just like today, no different at all
Richard Feldman (Aug 29 2024 at 00:51):because if it did call any, the type would have to change =>
Richard Feldman (Aug 29 2024 at 00:51):the only difference is that now bar can be passed an effectful function, and if so, then calling bar becomes an effect
Richard Feldman (Aug 29 2024 at 00:52):but you could only possibly do that from a function that was already effectful anyway
Richard Feldman (Aug 29 2024 at 00:52):just like today
Richard Feldman (Aug 29 2024 at 00:52):like I couldn't call bar from foo unless I passed it a pure function
Richard Feldman (Aug 29 2024 at 00:52):in which case no effects would be performed
Richard Feldman (Aug 29 2024 at 00:53):to me, it is absolutely essential to preserve that property - that is the main benefit of pure functional programming to me
Richard Feldman (Aug 29 2024 at 00:53):is that I can rule things out by looking at a function's type and knowing its purity based on that
Richard Feldman (Aug 29 2024 at 00:54):and the "polymorphic effects" part of this doesn't change that meaningfully
Richard Feldman (Aug 29 2024 at 00:54):because I can already call List.map things \thing -> File.readUtf8
Richard Feldman (Aug 29 2024 at 00:55):and that's pure because all it's doing is building up a list of tasks, not actually running them
Richard Feldman (Aug 29 2024 at 00:55):I can only run them from within a Task
Richard Feldman (Aug 29 2024 at 00:55):and similarly, if I'm inside a pure function in this design, I can only build up a list of effectful thunks using List.map, I can't actually perform any effects except from within an effectful function
Richard Feldman (Aug 29 2024 at 00:56):that's why I don't think the percentage of pure code would change
Richard Feldman (Aug 29 2024 at 00:56):if I tried to introduce an effect somewhere, it would cause exactly the same cascading ripple of type changes as today
Richard Feldman (Aug 29 2024 at 00:56):except instead of all those functions having to change to Task, they'd change to =>
Richard Feldman (Aug 29 2024 at 00:56):that's the only difference
Brendan Hansknecht (Aug 29 2024 at 01:01):But if bar calls the effectful function passed into it twice (assuming it is cheap to run and not effectful), it could get two different results
Sam Mohr (Aug 29 2024 at 01:01):I'm not sure if there is no difference. If I wrote this code:
main = \{} ->
List.range { start: At 1, end: At 5 }
|> List.map \num ->
Stdout.line "now is $(Num.toStr num)"
Sleep.millis 1000
Stdout.line "done!"
Would "done!" print before or after my count? Would the count take 5 seconds to run, or 1 second followed by 5 simultaneous prints?
Brendan Hansknecht (Aug 29 2024 at 01:02):So bar needs to be written defensively assuming that the function passed in might be effectful and is only valid to call once with the same inputs
Brendan Hansknecht (Aug 29 2024 at 01:02):Like a compare function could return different results on every call
Richard Feldman (Aug 29 2024 at 01:03):hm, I see your point but I have a hard time imagining the bug report count resulting from people not even thinking about that being more than a rounding error
Richard Feldman (Aug 29 2024 at 01:03):like if I can't call effectful functions directly, why would I call the same function twice?
Richard Feldman (Aug 29 2024 at 01:03):that's just worse than calling it once and caching the answer
Brendan Hansknecht (Aug 29 2024 at 01:04):I think if I have assumed cheap higher order function, I would call it multiple times without worrying about it.
Brendan Hansknecht (Aug 29 2024 at 01:04):That doesn't feel far-fetched
Richard Feldman (Aug 29 2024 at 01:04):I mean it's definitely possible, I just don't see it happening a significant amount of the time in practice
Sam Mohr (Aug 29 2024 at 01:04):This is reminding me of Koka's caveats around resumption. You don't know if effect handlers will return zero times, once, or many times. Because effect handlers do whatever they want, you have to plan around them.
And here, so long as code is effect-polymorphic, you can't write code assuming it has consistent output, yes.
Brendan Hansknecht (Aug 29 2024 at 01:04):It might even be better in some cases due to tradeoffs between compute and memory or due to a change that might happen but isn't guaranteed
Brendan Hansknecht (Aug 29 2024 at 01:05):I'm mostly trying to point out that this opens the door to those pains
Brendan Hansknecht (Aug 29 2024 at 01:05):They may be rare, but they are definitely a new kind of pain that didn't exist before and also is not captured by the type system
Sam Mohr (Aug 29 2024 at 01:05):Andrew Kelley has some perf-based videos around how much faster computation is over memory storage/retrieval
Richard Feldman (Aug 29 2024 at 01:05):I appreciate that
Brendan Hansknecht (Aug 29 2024 at 01:06):If I was going to run an effect I might place it in a different location in my function where it is best for a potential async suspensión to happen
Richard Feldman (Aug 29 2024 at 01:07):but I think it's also fair that, to Agus's point, for this to happen kinda requires someone not to be thinking about this consideration and going through the motions without realizing the implications of effect-polymorphism
Brendan Hansknecht (Aug 29 2024 at 01:07):Otherwise, the effect may capture a metric ton of intermediate data (and that can be super expensive as seen by problems with roc-wasm4)
Richard Feldman (Aug 29 2024 at 01:07):and even if we make it opt-in, I think it will be so common for higher-order functions to do this that the "I'm not thinking about it and just going through the motions" reflex will be to annotate it as effect-polymorphic anyway
Richard Feldman (Aug 29 2024 at 01:08):although to be fair, if the function wasn't higher-order and then became higher-order, and that required changing syntax, that might naturally give someone pause as they go to change the syntax
Richard Feldman (Aug 29 2024 at 01:08):to ponder the implications
Richard Feldman (Aug 29 2024 at 01:09):but now we're talking about an even smaller delta of outcomes as the conditions for what sequence of unusual events has to transpire for a bug to occur gets longer and longer :sweat_smile:
Richard Feldman (Aug 29 2024 at 01:10):incidentally, one of the other reasons I don't think this would make a difference in the proportion of pure functions people write is that in Clojure and OCaml you can just run side effects from any function, and it's still a very strong cultural norm to write as many pure functions as possible
Richard Feldman (Aug 29 2024 at 01:10):in both communities
Richard Feldman (Aug 29 2024 at 01:10):so we're talking about having a language where not only would that cultural norm be at least as strong as it would be there, but also changing from pure to effectful requires going around and changing all the types of all the callers (just like today)
Sam Mohr (Aug 29 2024 at 01:12):Is there some way that we could estimate how much people will want the average higher-order function to be effect-polymorphic vs pure?
It seems like if it was (hyperbolically) 99% of people, then it definitely should be the default, and it wouldn't be worth making people always opt-in when it really is the default.
But if it's 1% of people, then it should be opt-in, and then I think Brendan's issue goes away.
But we are talking from different realities here, because we seem to disagree on what that number is, and therefore what our design goals should be.
Richard Feldman (Aug 29 2024 at 01:12):and I've never heard about a problem of people in Clojure or OCaml accidentally calling things in higher-order functions multiple times and then having effects get repeated, even though that's always been possible with all higher-order functions in those languages
Richard Feldman (Aug 29 2024 at 01:13):I guess Unison people might have even more apples-to-apples data on that, since Unison literally has the higher-order function design we're currently discussing :big_smile:
Sam Mohr (Aug 29 2024 at 01:14):Good suggestion! I'll see if I can find anything.
Sam Mohr (Aug 29 2024 at 01:15):Just curious, Rich(ard) [do you have a preference?], if it really was that most people wanted to pass pure functions to libraries, and only rarely pass effectful functions, do you think you would want the -> to default to pure, and require annotation to become effect-polymorphic?
Richard Feldman (Aug 29 2024 at 01:19):it's a good question
Richard Feldman (Aug 29 2024 at 01:22):so one possibility I hadn't considered: what if we went with the design where you have to use explicit -fx-> and we just didn't make most higher-order functions effect-polymorphic
Richard Feldman (Aug 29 2024 at 01:22):like for example, give it to List.walk but not to List.map
Richard Feldman (Aug 29 2024 at 01:23):so then the List.map signature is the same as today because it works exactly the same way as today
Richard Feldman (Aug 29 2024 at 01:23):that design definitely has the simplest rule set, which I appreciate
Richard Feldman (Aug 29 2024 at 01:23):-> means it's pure=> means it's effectful-var-> means it's polymorphic Richard Feldman (Aug 29 2024 at 01:23):that's it
Sam Mohr (Aug 29 2024 at 01:24):Which is to say, make library authors have to decide when to opt-in for effect polymorphism?
Richard Feldman (Aug 29 2024 at 01:24):and if in practice people weren't going to use List.map for effects except in very rare cases anyway, are we really missing out?
Agus Zubiaga (Aug 29 2024 at 01:25):Yeah, I think this would be a good compromise as I mentioned before
Sam Mohr (Aug 29 2024 at 01:26):I agree that List.map isn't a function that people would really use to perform effects with, but it does mean that authors might get pestered to make their higher-order functions more flexible.
Agus Zubiaga (Aug 29 2024 at 01:26):I would expect a lot of packages to default to effect polymorphic higher order functions just in case
Sam Mohr (Aug 29 2024 at 01:26):But like the Unicode situation with Roc, some things really do need proper consideration, this might just be one of those
Richard Feldman (Aug 29 2024 at 01:26):Agus Zubiaga said:
I would probably still expect a lot of packages to still default to effect polymorphic higher order functions just in case
I actually predict most of them will follow the example of the stdlib, and follow whatever convention is established there
Richard Feldman (Aug 29 2024 at 01:27):e.g. if the convention is that you make walk polymorphic and that's it, I'd expect libraries to do that too
Agus Zubiaga (Aug 29 2024 at 01:27):Good point
Richard Feldman (Aug 29 2024 at 01:27):this direction is pretty interesting
Sam Mohr (Aug 29 2024 at 01:27):We should recommend making package functions effect-polymorphic unless there's a good reason not to in our tutorial
Agus Zubiaga (Aug 29 2024 at 01:27):Some functions are more likely to need effects than other
Sam Mohr (Aug 29 2024 at 01:27):Yep
Richard Feldman (Aug 29 2024 at 01:28):like I think an interesting observation is that today, there's no real demand for List.mapTask but I've wanted List.walkTask on several occasions
Richard Feldman (Aug 29 2024 at 01:28):(but that hasn't been implementable because Task wasn't a builtin)
Agus Zubiaga (Aug 29 2024 at 01:29):Most data structure packages could allow effects in their most powerful function and default to pure for the rest
timotree (Aug 29 2024 at 01:29):One example to consider is List.sortWith : List a, (a, a -> [LT, Eq, GT]) -> List a. Maybe that function wants to be always pure instead of polymorphic. It doesn't make a lot of sense to pass a non-deterministic comparison function, so you're not losing a lot by making it always pure, and one downside of having it be effect polymorphic is it means that changing the sorting algorithm is easily observable rather than just an implementation detail. (Although I guess that's true either way because ill-behaved pure comparison functions can probe the sorting algorithm too)
Richard Feldman (Aug 29 2024 at 01:30):another thing that appeals to me about this direction is that it simplifies the decision space
Richard Feldman (Aug 29 2024 at 01:30):like I don't have to think "hm, is there a way I could write this effectful thing using effect-polymorphic List.map in a cool way?"
Richard Feldman (Aug 29 2024 at 01:31):it's just like "nope that only works with pure functions, ok one less possibility to consider among N equivalent possible ways to write it"
Richard Feldman (Aug 29 2024 at 01:31):(which is the corresponding benefit to less flexibility when the flexibility is a convenience)
Sam Mohr (Aug 29 2024 at 01:33):@timotree the example Brendan gave earlier was a memoizing function. I think the same thing applies here, we have cases where we want to ensure we're not running someone else's effectful code.
Richard Feldman (Aug 29 2024 at 01:35):at the risk of opening up a can of worms (but it is very relevant to this discussion I think) I have a WIP proposal that would let you do a for loop style based on giving it a walk function
Richard Feldman (Aug 29 2024 at 01:36):and in that world, if walk is effect-polymorphic, you could use the for syntax for a lot more things without having to call walk directly
Sam Mohr (Aug 29 2024 at 01:37):I'll avoid delving into that then, but yes, that's where walk comes in handy, and map doesn't
Richard Feldman (Aug 29 2024 at 01:38):which in turn would mean even less demand for using effects in functions like List.map (which are already convenience wrappers around walk) because the main point of for is to be an ergonomic wrapper around walk :big_smile:
Richard Feldman (Aug 29 2024 at 01:39):Richard Feldman said:
so one possibility I hadn't considered: what if we went with the design where you have to use explicit
-fx->and we just didn't make most higher-order functions effect-polymorphic
[...]
Richard Feldman said:
->means it's pure=>means it's effectful-var->means it's polymorphic
@Brendan Hansknecht when you get back, I'm curious what you think of this idea
Isaac Van Doren (Aug 29 2024 at 01:39):This is definitely my favorite of the proposed directions!
Richard Feldman (Aug 29 2024 at 01:39):yeah it's the new frontrunner for me too
Sam Mohr (Aug 29 2024 at 01:41):Works for me!
drew (Aug 29 2024 at 01:55):are there many effect types in roc or only one?
Sam Mohr (Aug 29 2024 at 01:56):Just the one (for now)
drew (Aug 29 2024 at 01:57):as the person who initially offered the prior art, i will say this all does feel quite complicated :sweat_smile:
drew (Aug 29 2024 at 01:59):haskell works without effect polymorphic functions with foldM basically
Sam Mohr (Aug 29 2024 at 02:00):Unfortunately for us, this is an intrinsically complex domain, so Roc's goal to manage complexity while still making it known what's happening is just hard
drew (Aug 29 2024 at 02:00):very true
Sam Mohr (Aug 29 2024 at 02:01):FP has more ornate solutions that we are forgoing because we don't want to allow wizardry
Brendan Hansknecht (Aug 29 2024 at 02:02):Richard Feldman said:
->means it's pure=>means it's effectful-var->means it's polymorphic
I am much happier with that. I still am nervous around it only being represented in the type, but I have no quals with seeing what it feels like to only have it in the type.
Richard Feldman (Aug 29 2024 at 02:02):drew said:
haskell works without effect polymorphic functions with foldM basically
that's essentially the same idea as "make walk effect-polymorphic and all the other ones pure unless there's a specific reason not to (e.g. parallel should be effect-polymorphic too)"
drew (Aug 29 2024 at 02:03):i think it’s more the walkTask equivalent
drew (Aug 29 2024 at 02:03):haskell has a bunch of *M variants
Richard Feldman (Aug 29 2024 at 02:04):that's what I mean, yeah
drew (Aug 29 2024 at 02:04):but i guess higher kinded types means only one variant forever
Isaac Van Doren (Aug 29 2024 at 02:04):haskell works without effect polymorphic functions with foldM basically
I think this proposal could make working with effects significantly nicer than it is in Haskell
Richard Feldman (Aug 29 2024 at 02:04):foldM being equivalent to either walkTask or effect-polymorphic walk
drew (Aug 29 2024 at 02:05):yep yep
drew (Aug 29 2024 at 02:05):except foldM can’t take a pure function
drew (Aug 29 2024 at 02:05):you need Identity nonsense
drew (Aug 29 2024 at 02:08):i guess foldM is polymorphic over a higher-kinded effect, but it must exist, whereas ->{e} in unison is polymorphic over a potentially empty set of effects
drew (Aug 29 2024 at 02:10):like ->{e} could actually mean ->{e1,e2,...} which i suppose makes this latest suggestion forwards compatible with an N-effect roc world
drew (Aug 29 2024 at 02:10):speaking of which, does => become useless when the number of effects becomes greater than one?
drew (Aug 29 2024 at 02:11):is => simply syntactic sugar for -Task->
Isaac Van Doren (Aug 29 2024 at 02:11):I don't think there is any plan to add any effects other than Task
drew (Aug 29 2024 at 02:11):if that's the case, -?-> could be "effect or no effect"
drew (Aug 29 2024 at 02:12):though it is annoying to type :smile:
Ayaz Hafiz (Aug 29 2024 at 02:12):Hey all - sorry, unfortunately I don't have the bandwidth to read everything in this thread, keep up the great energy :) I just want to add one perspective to this that I've talked to Richard about before. Some of it may already be in the thread, again, sorry that I haven't read everything.
To whatever extent possible, I would really encourage avoiding any polymorphism in effects in the syntax visible to the language user. Obviously effect polymorphism is necessary, but I believe there are approaches here that can hide that entirely from the syntax/having to think about it most of the time.
More polymorphism means more things to think about. If author must now always think about whether their function is effect polymorphic or not, I wonder if that ends up putting the language in the same spot it started with - the dichotomy of task/not task is removed and so is some syntax/types, sure, but it's been traded it for this other thing.
Roc's type system can be difficult for some to get started with because it is novel/unfamiliar/complex in some ways (think of open tag unions, polymorphic unions, and optional record fields). To whatever extent possible, I think it is prudent to avoid adding another dimension unless absolutely necessary. It also opens the gate to ecosystem splits like a fully effect-polymorphic/rigidly pure ecosystem, even if the sight of that future isn't visible now.
If the goal of polymorphism is to gain insight into what functions are definitely pure, I would consider focusing on other ways to surface that specifically. For example, from my perspective the benefit of that goal is that when I am debugging function X, I know I only need to look at y_1, ... y_n function calls. If that's the case, what if effectful calls are signified at the call site instead of the declaration site? So for example, List.map ls f is List.map! ls f if f is effectful. Then there is no need for polymorphism in the declarations in the context of this use case specifically. Not saying that's definitely the way, but pushing really hard on avoiding type system complexity via polymorphism is a very valuable metric, if it can be achieved.
Richard Feldman (Aug 29 2024 at 02:14):Ayaz Hafiz said:
I just want to add one perspective to this that I've talked to Richard about before.
for context, and credit where due, it was a previous conversation with Ayaz where he advocated for reconsidering algebraic effects that led to this proposal in the first place!
drew (Aug 29 2024 at 02:19):Ayaz Hafiz said:
So for example,
List.map ls fisList.map! ls fif f is effectful.
Yeah, this honestly makes a ton of sense to me.
drew (Aug 29 2024 at 02:20):it's also pretty clear at the call site
drew (Aug 29 2024 at 02:21):i also think if there's ~no chance of having more than 1 effect, maybe *Task functions are fine? They are certainly explict.
drew (Aug 29 2024 at 02:21):anyway, curious what you all come up with!
Richard Feldman (Aug 29 2024 at 03:04):some brief thoughts on some of these points:
Ayaz Hafiz said:
To whatever extent possible, I would really encourage avoiding any polymorphism in effects in the syntax visible to the language user. Obviously effect polymorphism is necessary, but I believe there are approaches here that can hide that entirely from the syntax/having to think about it most of the time.
I think if it's only visible in the types of e.g. walk and parallel, and walk ends up being primarily used via for syntax sugar, then people will encounter the types extremely rarely. So I don't think in that design it's something people would be thinking about a lot, but I could be wrong!
Richard Feldman (Aug 29 2024 at 03:05):Ayaz Hafiz said:
More polymorphism means more things to think about. If author must now always think about whether their function is effect polymorphic or not, I wonder if that ends up putting the language in the same spot it started with - the dichotomy of task/not task is removed and so is some syntax/types, sure, but it's been traded it for this other thing.
Regarding "More polymorphism means more things to think about. If author must now always think about whether their function is effect polymorphic or not..." - I think this cuts both ways!
As noted earlier, if (for example) List.map and friends are all effect-polymorphic, then as a non-library author I'm thinking about a much larger number of ways to express the same thing...and there are a lot more library users than authors! Also, I predict authors will follow the precedent the stdlib sets (libraries being "idiomatic" is generally considered a good thing in most languages), which means I don't think they'll need to think about it all that much on a per-function basis.
I do think this is a reasonable consideration, but in general I don't think we can get away with "somebody is thinking about effect polymorphism" if it's a thing that exists at all.
Richard Feldman (Aug 29 2024 at 03:11):Ayaz Hafiz said:
Roc's type system can be difficult for some to get started with because it is novel/unfamiliar/complex in some ways (think of open tag unions, polymorphic unions, and optional record fields). To whatever extent possible, I think it is prudent to avoid adding another dimension unless absolutely necessary.
I think both tag unions and record fields are interesting related examples here, in that:
walk actually do so in practice). Richard Feldman (Aug 29 2024 at 03:14):Ayaz Hafiz said:
It also opens the gate to ecosystem splits like a fully effect-polymorphic/rigidly pure ecosystem, even if the sight of that future isn't visible now.
I think that is a totally valid concern, although I just don't think the demand for it would be high enough. The point that started us down the path that led to the current design idea was that in practice it would probably be the case that even if List.map supported being passed an effectful function, 99+% of the time people would pass a pure function to it anyway.
if that hypothesis is correct, and there isn't that much demand for effect-polymorphic higher-order functions outside of a few outliers like walk and parallel, then it stands to reason that there wouldn't be much demand for a whole alternative effect-polymorphic ecosystem either
Richard Feldman (Aug 29 2024 at 03:15):Ayaz Hafiz said:
what if effectful calls are signified at the call site instead of the declaration site? So for example,
List.map ls fisList.map! ls fif f is effectful. Then there is no need for polymorphism in the declarations in the context of this use case specifically. Not saying that's definitely the way, but pushing really hard on avoiding type system complexity via polymorphism is a very valuable metric, if it can be achieved.
this is an interesting idea, although a notable nice feature of the current design idea is that it's sort of the bare minimum
Richard Feldman (Aug 29 2024 at 03:16):like no matter how the design ends up evolving over time, the minimum starting point has to include:
every other idea builds on this foundation, and the current frontrunner design is basically just "ship this foundation"
Richard Feldman (Aug 29 2024 at 03:17):and maybe it ends up being that in practice we don't like it and want to change things, but given that we have good reason to suspect this is the nicest design overall long-term anyway (and it certainly seems like the simplest one we've discussed), I think there's a good case to be made that it's the most logical starting point
Richard Feldman (Aug 29 2024 at 03:17):and then we can see how these various hypotheses play out in practice and discuss what changes to make (if any) based on seeing what the experience of using it is actually like in practice
Brendan Hansknecht (Aug 29 2024 at 04:10):So for example,
List.map ls fisList.map! ls fif f is effectful.
This is definitely still my preference for all effectful functions. Just require the ! still like other languages would require await. Makes it visible at the callsite to the end user.
even if
List.mapsupported being passed an effectful function, 99+% of the time people would pass a pure function to it anyway.
I still think it would be reasonably common to use it for effects. Specifically for things where you deal with querying websites. Being able to run a list of queries, and get a list of results sounds like it would be wanted. Though that would probably be List.mapTry to be fair. Like a webscaper that grabs n pages. Loads there links into a new list and the loops.
Brendan Hansknecht (Aug 29 2024 at 04:11):Oh, nvm, I see what you mean. You would want parallel to run them all at the same time. Not map.... So map would be pretty rare due to the thread limitation.
Brendan Hansknecht (Aug 29 2024 at 04:11):But I still could see lists of tasks to run in general in some cases.
Brendan Hansknecht (Aug 29 2024 at 04:23):a syntax for defining effect-polymorphic function types with a named type variable
I don't think this is required.
If a function is effectful, you must use ! to call it.
If a function is maybe effectful and any of its args are effectful, you must use ! to call it.
Otherwise, everything must be pure and it is called normally.
Brendan Hansknecht (Aug 29 2024 at 04:25):This would mean that a function taking in a higher order function would not know if that function is pure or not (which I'm not the biggest fan of), but it would mean that the call site for any effectful direct function would use !. It also forces simplicity in the type system but has some things that are impossible to expresss where it would treat something as effectful even though it may not truly be effectful. But that is technically valid cause all pure functions can implicitly be converted to effectful.
Brendan Hansknecht (Aug 29 2024 at 04:29):Not saying :point_up: is the correct solution, but I think that constraint may be designing ourselves into extra complexity that isn't actually required. We can force simplicity if we accept some restrictions.
Jasper Woudenberg (Aug 29 2024 at 06:33):Another thougth with regards to removing effect-polymorphic function notation (i.e. -var-> syntax, in the latest proposal):
As I understand it, we'd only need that when a single type signature might contain multiple type-polymorphic functions, like in this example:
fn : (a -e-> b), (c -f-> d) -> { one: a -e-> b, two: c -f-> d }
I'm curious if we can come up with real-ish examples of library functions that might want to use this syntax, to get a sense of what we'd lose if the language didn't support those types of functions.
Because similar to how the the single-implicit-effect-parameter in => allows us to combine what would otherwise be an effectful and non-effectful version of List.walk into one function, I would imagine a function with multiple explicit effect-type-parameters can be replaced with a couple of different functions with that have at most one polymorphic effect parameter each. And if that avoids a very complicated type, maybe it's the better design?
Jasper Woudenberg (Aug 29 2024 at 10:30):Oh wait, the problem is you can write a function fn = \one, two -> { one, two }, the compiler might infer the complicated type, and if it's illegal we'd need to explain folks why their seemingly reasonable function is not allowed :thinking:
(just parroting what Richard already said before)
Richard Feldman (Aug 29 2024 at 10:47):yeah the variable name has to exist, at a minimum
Richard Feldman (Aug 29 2024 at 10:48):the question is whether we add optional syntax sugar in addition to it :big_smile:
Richard Feldman (Aug 29 2024 at 10:50):regarding ! in expressions, one thing we could try is making them optional in the initial release
Richard Feldman (Aug 29 2024 at 10:51):I know the whole point would be to make them mandatory, but having them be optional would let us try out both styles and see how they feel in practice
Richard Feldman (Aug 29 2024 at 10:52):like for example we can see how often !? comes up and how it feels, and we can also separately try out implementations with no ! in the function body and see how that feels too, and compare them
drew (Aug 29 2024 at 13:12):i'm assuming you all have already read this, but the frank paper seems like good prior art as well (https://arxiv.org/abs/1611.09259)
drew (Aug 29 2024 at 13:13):most notably, from the introduction:
drew (Aug 29 2024 at 13:13):
- a novel approach to effect polymorphism which avoids men- tioning effect variables in source code, crucially relying on the observation that one must always instantiate the effects of an operator being applied with the ambient ability, that is, pre- cisely those algebraic effects permitted by the current typing context;
Richard Feldman (Aug 29 2024 at 13:19):I think this is what Unison's system is based on
drew (Aug 29 2024 at 13:20):yep, correct, though frank seems to make some different decisions in the paper
drew (Aug 29 2024 at 13:20):and it's boiled down
drew (Aug 29 2024 at 13:22):it seems that the concept of "suspension" is deeply built into both langauges, i'm not sure how isomorphic this is to Task
Ayaz Hafiz (Aug 29 2024 at 13:22):+1 on the frank paper, it is excellent foundational material I would encourage reading, as well as the Leijen and Xie papers on effects in Koka, and https://dl.acm.org/doi/pdf/10.1145/3485479 and https://arxiv.org/pdf/1812.11664. The last one shows a rigorous effect system in OCaml without type extensions - OCaml today also has effects without having effect polymorphism (or even the idea of effects) in the type system (besides some minor things for continuations which are irrelevant to Roc)
Richard Feldman (Aug 29 2024 at 13:34):it's interesting which things different implementations place value on
Richard Feldman (Aug 29 2024 at 13:35):to me, approximately the entire benefit is being able to tell from the type annotations which functions are guaranteed to be pure and which are not
Richard Feldman (Aug 29 2024 at 13:35):so not having effects in the type system at all strikes me as kinda "then what's the point?"
Richard Feldman (Aug 29 2024 at 13:36):but I can appreciate that in OCaml's case that would be a gigantic breaking change to the entire ecosystem, so maybe that was never a realistic option :sweat_smile:
Ayaz Hafiz (Aug 29 2024 at 13:38):i think it's worth reading about why they added it/using it in a language like OCaml, and their perspective on the benefits, because it could change the set of things that are considered for this design/validate them/etc
Brendan Hansknecht (Aug 29 2024 at 16:15):Richard Feldman said:
yeah the variable name has to exist, at a minimum
I disagree. pure functions can always be promoted to impure. So the variable is not required
Brendan Hansknecht (Aug 29 2024 at 16:16):You can force that if a higher order impure function is passed in, all functions passed in must be treated as higher order impure functions
Brendan Hansknecht (Aug 29 2024 at 16:16):If you do that, no variable is needed to be exposed to the users
Brendan Hansknecht (Aug 29 2024 at 16:16):Given this is theoretically a very rare usecase I think you can get simplicity while still having most of the functionality.
Richard Feldman (Aug 29 2024 at 16:22):(copy/pasting this from another thread)
if there's no effectfulness variable and I put this into the repl, what's the inferred type?
\fn1, fn2 ->
(fn1, fn2, (\arg -> fn1 "$(arg)!"), (\arg -> fn "$(arg)!")))
in all the current designs, the inferred type would be:
(Str -fx1-> Str),
(Str -fx2-> Str)
-> (
(Str -fx1-> Str),
(Str -fx2-> Str),
(Str -fx1-> Str),
(Str -fx2-> Str),
)
because the outer function doesn't call either of the functions it receives, it just passes them trough and creates new functions with them, none of which are effectful
Brendan Hansknecht (Aug 29 2024 at 16:23):Whatever our syntax for this is:
(Str -fx1-> Str),
(Str -fx1-> Str)
-> (
(Str -fx1-> Str),
(Str -fx1-> Str),
(Str -fx1-> Str),
(Str -fx1-> Str),
)
No need to print out the type variable cause they are all the same.
Brendan Hansknecht (Aug 29 2024 at 16:23):If either of the inputs are impure, all functions returned are treated as impure
Brendan Hansknecht (Aug 29 2024 at 16:24):Basically one impure function poisons the well.
Sam Mohr (Aug 29 2024 at 16:26):That can be solved by splitting this into two functions, which probably should be done anyway since there seem to be two unrelated operations happening, but it's not ideal that we affect "unrelated operations"
Sam Mohr (Aug 29 2024 at 16:27):But if this is what it takes for a cleaner syntax, it's a worthwhile cost to force people to split this kind of function into two
Richard Feldman (Aug 29 2024 at 16:31):Brendan Hansknecht said:
Whatever our syntax for this is:
(Str -fx1-> Str), (Str -fx1-> Str) -> ( (Str -fx1-> Str), (Str -fx1-> Str), (Str -fx1-> Str), (Str -fx1-> Str), )No need to print out the type variable cause they are all the same.
oh so you mean in the specific case where they're all fx1 we don't need to say fx1 everywhere?
Brendan Hansknecht (Aug 29 2024 at 16:36):I'm saying that we ban having multiple effectfulness variables in a single function
Brendan Hansknecht (Aug 29 2024 at 16:36):Roc will always unify to exactly one effectfulness variable for a given set of inputs and outputs to a single function.
Brendan Hansknecht (Aug 29 2024 at 16:37):Which is always possible to do cause pure functions can be implicitly converted to impure functions
Brendan Hansknecht (Aug 29 2024 at 16:40):If you write:
\fn1, fn2 ->
(fn1, fn2, (\arg -> fn1 "$(arg)!"), (\arg -> fn "$(arg)!")))
if both fn1 and fn2 are pure, all outputs are pure.
If either of fn1 or fn2 are impure, all outputs are impure.
if both fn1 and f2 are impure, all outputs are impure.
Richard Feldman (Aug 29 2024 at 16:42):Brendan Hansknecht said:
I'm saying that we ban having multiple effectfulness variables in a single function
but literally what happens if you put that expression into the repl
Richard Feldman (Aug 29 2024 at 16:44):is the idea that if you take 2 polymorphic functions and pass them into a function which then returns them, even though they never interacted with each other except for having passed through this function, they are now coupled to one another just by having been passed to the same function and then returned by it?
Brendan Hansknecht (Aug 29 2024 at 16:44):yep
Richard Feldman (Aug 29 2024 at 16:44):that definitely seems like incorrect behavior haha
Brendan Hansknecht (Aug 29 2024 at 16:46):It is a behavior to simplify the exposed surface of the type system. We have mentioned that we expect multiple effectfulness variables to be exceptionally rare. So I am suggesting we ban them to simplify the type system.
Brendan Hansknecht (Aug 29 2024 at 16:47):It is a concept users never have to learn.
Sam Mohr (Aug 29 2024 at 16:47):I think we still probably want some syntax (maybe ~>?) to denote effect polymorphic functions, but then we don't need vars, yes
Brendan Hansknecht (Aug 29 2024 at 16:47):They just learn that higher order functions can be effect polymorphic, but if any higher order function is impure all must be treated as impure.
Sam Mohr (Aug 29 2024 at 16:48):We don't need ~> if we do Ayaz's proposal
Sam Mohr (Aug 29 2024 at 16:49):This also means aliases/opaque types don't need vars! They just write Mapper := (Str ~> Str)
Richard Feldman (Aug 29 2024 at 16:50):I don't think for the initial version of this it's a reasonable idea to not have named variables at all
Richard Feldman (Aug 29 2024 at 16:50):if it turns out they're so rarely used in practice that we want to consider taking them out later because we realize it'll be harmless and simplify the learning case, that's something we can discuss
Richard Feldman (Aug 29 2024 at 16:51):but not having them at all, when in every other case type inference relies on them, feels like a "we're just gonna take a small part out of the foundation, what could go wrong?" and that feels like a proposition that's so fundamentally scary I don't think it could possibly justify the benefits for v1 of this :big_smile:
Richard Feldman (Aug 29 2024 at 16:52):like if we want to have them but then choose to intentionally not use them at all in any of the builtins, and maybe not even teach them except in an obscure corner of the langref, that also seems fine
Richard Feldman (Aug 29 2024 at 16:52):but I think not having them at all breaks assumptions about type inference that I think are extremely risky to break
Richard Feldman (Aug 29 2024 at 16:54):(e.g. the cascading effects of making that repl example earlier infer to have all the type variables equal to each other could very easily break a bunch of examples that are totally unrelated and which we would expect to work, just because it would be such an unprecedented change to how type unification of those variables works)
Sam Mohr (Aug 29 2024 at 16:56):Also, I know that you're against algebraic effects, but the current design sets us up to handle those, which is very exciting for me! Maybe someday...
Sam Mohr (Aug 29 2024 at 16:56):This keeps our options open
Richard Feldman (Aug 29 2024 at 16:59):well this is a form of algebraic effects, just without the "multiple types of effects" and without the "handlers" concepts
Sam Mohr (Aug 29 2024 at 16:59):Exactly, yes!
Sam Mohr (Aug 29 2024 at 17:00):Handlers is more what I was pointing to yes
Ayaz Hafiz (Aug 29 2024 at 17:00):to be pedantic, this is algebraic effects with both multiple effect types and handlers. the distinction is that handlers are only defined by the platform.
Richard Feldman (Aug 29 2024 at 17:00):Brendan Hansknecht said:
Whatever our syntax for this is:
(Str -fx1-> Str), (Str -fx1-> Str) -> ( (Str -fx1-> Str), (Str -fx1-> Str), (Str -fx1-> Str), (Str -fx1-> Str), )No need to print out the type variable cause they are all the same.
If either of the inputs are impure, all functions returned are treated as impure
Basically one impure function poisons the well.
so what do you think about the design we talked about at some point where the syntax for "they are all the same" is just -> on its own?
Brendan Hansknecht (Aug 29 2024 at 17:00):If that is the case, and with what I have read so far (I'm sure I'm missing pieces), I would vote for this personally as a starter setup:
->: definitely pure
=>: definitely effectful
-var>: effect polymorphic
all effectful functions require ! to be called.
Effect polymorphic functions require ! unless the type variable can be resolved to always pure.
(as in, List.walk can be called without ! if it is passed in a pure function).
The ! can be removed later if we think it is simply noise or otherwise not worth having.
I really heavily want to push for starting with !. I think that given the number of new/beginner users in roc, if we start without !, many users will never realize that it exists. They won't see the benefits of it and it will just not use it cause they don't know the tradeoff. I think we really need to get a feeling for if ! is good or bad. I personally think it is super important for readability and keeping a strong separation between effectful and pure.
Richard Feldman (Aug 29 2024 at 17:02):I think one of the downsides of requiring ! that we should discuss is that in practice, there will be a ton of !? in this design
Richard Feldman (Aug 29 2024 at 17:02):specifically, everywhere you see ! in today's Roc will become !?
Brendan Hansknecht (Aug 29 2024 at 17:03):Not quite everywhere, but yeah, most. Cause we have a lot of error wrapping that would become task! ... |> Result.mapErr? WrapperErr
Richard Feldman (Aug 29 2024 at 17:03):true
Richard Feldman (Aug 29 2024 at 17:03):also like Env.dict! would not need it because it wouldn't return Result, but that's a rare case
Sam Mohr (Aug 29 2024 at 17:03):I see mapErr more than bare !
Richard Feldman (Aug 29 2024 at 17:04):really? :face_with_raised_eyebrow:
Sam Mohr (Aug 29 2024 at 17:04):In user code, yes
Richard Feldman (Aug 29 2024 at 17:04):oh you mean in this design it would be more common to see |> Result.mapErr? than bare !
Sam Mohr (Aug 29 2024 at 17:04):Yes
Richard Feldman (Aug 29 2024 at 17:05):I thought you meant that today mapErr! is more common than bare ! :laughing:
Sam Mohr (Aug 29 2024 at 17:05):Lol, I'm lazy when typing on my phone
Richard Feldman (Aug 29 2024 at 17:05):yeah I agree, I'd expect something! ... |> Result.mapErr? would be more common than bare !
Brendan Hansknecht (Aug 29 2024 at 17:07):My thought on !?:
something! ... |> Result.mapErr?! and ? separately.! sooner. So it will lead to the starter test running faster. Sam Mohr (Aug 29 2024 at 17:07):I'm also definitely a proponent of ! being explicit if at all possible. I'm the guy at work that wraps function args in a struct so that the call site is readable without looking at the function signature.
Sam Mohr (Aug 29 2024 at 17:08):Which ! gives us
Richard Feldman (Aug 29 2024 at 17:08):I buy that, but I think the better way to run that experiment is to make ! optional (as in, just have it be accepted by the parser and then ignored) and then we can try the code both ways and see what we prefer
Richard Feldman (Aug 29 2024 at 17:08):then we can either remove it or make it mandatory based on how that goes
Sam Mohr (Aug 29 2024 at 17:09):That's a great idea!
Sam Mohr (Aug 29 2024 at 17:09):Then we can duke talk it out (I enjoy comedic hyperbolics more than the next person)
Richard Feldman (Aug 29 2024 at 17:09):well I'd say compare and contrast, but sure :laughing:
Sam Mohr (Aug 29 2024 at 17:09):And then it's super easy to do what Agus did with the new import syntax, which is parse both and format to the desired one once we change
Richard Feldman (Aug 29 2024 at 17:10):yeah exactly
Richard Feldman (Aug 29 2024 at 17:10):makes incrementally transitioning (to either end state) easier
Brendan Hansknecht (Aug 29 2024 at 17:11):(as in, just have it be accepted by the parser and then ignored)
I don't think that will do justice to the experiment. A new user will probably never use !. They won't know the tradeoffs.
Brendan Hansknecht (Aug 29 2024 at 17:12):I mean if we format to auto add !, I guess I would be fine with the parser accepting either.
Jasper Woudenberg (Aug 29 2024 at 17:12):A random thought: if we're going to make it so calling an effectful function requires adding a bang, then another way we could mark functions as effectful is by insisting impure function names end with a bang. So:
fn : {} -> Result U64 Str # pure function
fn!: {} -> Result U64 Str # effectfull function
So the ! is not special syntax, it's part of the function name itself. Ruby is a sort-of precedent here, there's a cultural norm of letting 'dangerous' functions end with a bang.
Some advantages:
foo!? becomes easier to explain. The bang is there simply because it's part of the function name, so really nothing special going on here. Brendan Hansknecht (Aug 29 2024 at 17:13):We still have maybe effectful functions though. What do you write for List.map?
Jasper Woudenberg (Aug 29 2024 at 17:13):The "rule" there would be that if you define a higher level pure function, Roc will give you an effectful version with a bang for free. So we basically pretend there's two functions, one written by the programmer and one auto-generated.
Richard Feldman (Aug 29 2024 at 17:14):I think it's very safe to assume that beginners will overwhelmingly react more positively to ? than to !? (I don't think we need to run an experiment to confidently say that!) - I think the relevant question is whether advanced users value the ! in the expressions enough to justify the downside of the increased learning curve
Richard Feldman (Aug 29 2024 at 17:15):that's why I like the idea of making ! optional - it lets advanced users try it out without impacting the beginner experience at all
Richard Feldman (Aug 29 2024 at 17:15):and then we can determine whether we (advanced users) like it enough to justify the learning curve cost etc.
Sam Mohr (Aug 29 2024 at 17:15):Which we'd learn from them reading other people's code and complaining, I'd guess
Richard Feldman (Aug 29 2024 at 17:16):yeah I mean I've done a lot of "showing syntax to people who only use mainstream languages and asking what they think of it" over the years and this is one of the few cases where I can imagine asking which one they prefer and the response being something like "...you're kidding, right?" :sweat_smile:
Richard Feldman (Aug 29 2024 at 17:16):but that doesn't mean it's the wrong design overall!
Richard Feldman (Aug 29 2024 at 17:16):just that it's super predictable which one beginners will prefer
Richard Feldman (Aug 29 2024 at 17:16):and that has tradeoffs with other pros and cons etc.
Sam Mohr (Aug 29 2024 at 17:18):Agreed. I'm saying we'd gauge the value of ! based on how much new users dislike it. I agree they're guaranteed to dislike it
Brendan Hansknecht (Aug 29 2024 at 17:18):"...you're kidding, right?"
No no no, trust me, it's better for you, by changing all code into emojis it enables all people regardless of literacy to understand
Richard Feldman (Aug 29 2024 at 17:19):someone should record a video where they try to sell people on BF with a straight face as the future of programming
Sam Mohr (Aug 29 2024 at 17:19):If you got someone like Jon Gjengset to do that, people would take it seriously
Brendan Hansknecht (Aug 29 2024 at 17:20):we (advanced users) like it enough
If we are benchmarking leaning towards advanced users, I am much more open to this. Cause I really do think this will be a care where new users (who probably don't even fully know if they want pure functions in general) would miss the benefits
Richard Feldman (Aug 29 2024 at 17:20):yeah agreed
Richard Feldman (Aug 29 2024 at 17:20):put another way - we already know the benefits of the current system; if we have the option of taking out the ! how much do we miss it?
Richard Feldman (Aug 29 2024 at 17:21):(in practice I mean)
Matthieu Pizenberg (Aug 29 2024 at 17:38):Richard Feldman said:
just that it's super predictable which one beginners will prefer
Well if you ask beginners why rust needs a "mut" whenever you mutate things, they will definitely tell you "are you kidding, just let the compiler figure out if this thing is mutated, why bother with the annotation"
Richard Feldman (Aug 29 2024 at 17:38):I don't think those are on the same level :big_smile:
Richard Feldman (Aug 29 2024 at 17:39):I wouldn't expect an "are you kidding" reaction from mut
Richard Feldman (Aug 29 2024 at 17:40):but anyway, the broader point is that the advanced/experienced user preference is the thing I think we want to experiment with in this case
Agus Zubiaga (Aug 29 2024 at 19:49):Having the formatter automatically add ! isn’t super easy, it would need to run the type checker :upside_down:
Agus Zubiaga (Aug 29 2024 at 19:56):Coming next April 1st to a store near you: The official Roc keyboard with a U+203D key
Sam Mohr (Aug 29 2024 at 19:59):Agus Zubiaga said:
Having the formatter automatically add
!isn’t super easy, it would need to run the type checker :upside_down:
Yeah, I thought of that after, let's just assume we'll reformat to remove if we remove, and give a type error in the other case.
Brendan Hansknecht (Aug 29 2024 at 19:59):Format to ‽
Brendan Hansknecht (Aug 29 2024 at 19:59):But how will I know if that is ?! or to !?
Brendan Hansknecht (Aug 29 2024 at 19:59):Haha
Agus Zubiaga (Aug 29 2024 at 20:00):That doesn’t work I think, you need to pass an argument to the effectful function inside the result :smiley:
Agus Zubiaga (Aug 29 2024 at 20:01):? ()! ?
Brendan Hansknecht (Aug 29 2024 at 20:01):Problem solved. I see exactly zero downside of interrobang
Agus Zubiaga (Aug 29 2024 at 20:01):ikr
Agus Zubiaga (Aug 29 2024 at 20:09):Jokes aside, if ! was required we wouldn’t need thunks would we?
Agus Zubiaga (Aug 29 2024 at 20:10):we would need some kind of non-arrow syntax for annotations
Sam Mohr (Aug 29 2024 at 20:16):Agus Zubiaga said:
we would need some kind of non-arrow syntax for annotations
I don't understand what you mean by this. I presume you're not referring to record builders?
Agus Zubiaga (Aug 29 2024 at 21:56):No, just in general. The reason only functions can be effectful with purity inference is that you need to be able to decide when to run them.
If we require !, I think you can refer to them without performing them, like Task.
Agus Zubiaga (Aug 29 2024 at 21:57):instead of Env.dict! {}, it could just be Env.dict!
Agus Zubiaga (Aug 29 2024 at 21:58):If you want to pass the effect without running it, you omit the !
Agus Zubiaga (Aug 29 2024 at 21:59):Not saying it’s a good idea, but maybe thunks are redundant in this world
Richard Feldman (Aug 29 2024 at 21:59):I actually like the thunks personally
Richard Feldman (Aug 29 2024 at 22:00):I think it makes it more intuitive to understand when things are vs aren't actually running yet
Richard Feldman (Aug 29 2024 at 22:00):because intuitively, when you define a function it obviously doesn't run yet
Richard Feldman (Aug 29 2024 at 22:00):and then when you call it, it intuitively does run
Agus Zubiaga (Aug 29 2024 at 22:03):Yeah, makes sense
Last updated: Jun 16 2026 at 16:19 UTC