Brendan Hansknecht (Nov 11 2024 at 19:41): Brendan Hansknecht (Nov 11 2024 at 19:41):While I am an overall fan of #ideas > static dispatch - proposal , I think the proposal needs to work with an ability like concept not remove one.
Warning: Please try to ignore syntax. I am trying to explain a concept not pick a good syntax or name for it.
Note: for this doc I will be saying self to mean the current nominal or opaque type. I always will be saying opaque type from here forward, but they could be opaque or nominal types.
Static dispatch is fundamentally a system that allows adding methods to opaque types. A method is simply a function defined in the same module as an opaque type with self being the first arg. The method can be exposed but it does not have to be (can be for local use only). The special thing about methods is that they can be called with dot notation: self.method(arg2, arg3). A method can still be called like a normal function as well: method(self, arg2, arg3).
This may seem like a small syntax change, but it actually has a very large ramification. A function is allowed to operate generically over methods.
# This only works with lists
func = \x ->
List.append x 123
# This works with any opaque type with an append method (a, Num b -> c)
func = \x ->
x.append(123)
This unlocks a new level of generic flexibilty. It enables implicit generic function dispatch based on methods that an opaque type has. This is very similar to golang interfaces where if you happen to have the right named and typed methods, you implicitly follow the interface. There is no opt-in or opt-out for golang interfaces. You simply have the interface if you have the right shape.
This is actually more flexible than golang interfaces. There is no explicit interface defined anywhere in this picture. The interface is implicit based on the use. Golang is already considered a more flexible system than what is seen in most other languages and Roc with #ideas > static dispatch - proposal is even more flexible than that. I think that we may accidentally be opting into too much flexibility with this design.
This system is exceptionally flexible for one core reason: Roc wants to make it possible for a user to never need to write a type signature.
Generally this can seen most clearly by asking: "what type would the repl spit out if I typed this expression?"
Lets look at a direct example expanded from above:
func = \x ->
x |> List.reserve 5 |> List.append 123 |> List.reverse
This is pretty easy. x is used with a bunch of list functions. It also has a element type that matches the literal 123. So the output type is: List (Num a) -> List (Num a)
Lets do the same with method call syntax:
func = \x ->
x.reserve(5).append(123).reverse()
Hmm, this one is harder. We know little about the 3 functions called here. They weren't imported from anywhere. They also aren't local functions. Let's break down the pieces:
reserve is a function a, Num b -> cappend is a function c, Num d -> ereverse is a function e -> fPutting that all together, I guess that func is a:
func : a -> f where
a implements { reserve: a, Num b -> c },
c implements { append: c, Num d -> e },
e implements { reverse: e -> f }
I'm not sure about you, but that is not how I would want this function to be typed. It is a mess and will get completely unwieldy for longer functions. This is what I mean when I say that static dispatch may accidentally be too flexible.
I think that for static dispatch to be reasonable, we need to reign in the flexibility. I think this can be done with minimal loss of functionality by two parts:
Explicit qualification is pretty simple of a concept. It is the root discussion of #ideas > static dispatch - method/function call syntax. I'm not gonna repeat everything there, but either you need to explicitly import the methods to use them unqualified or used them with module qualification.
func = \x ->
import List exposing [append, reserve, reverse]
x.reserve(5).append(123).reverse()
# Or
func = \x ->
# maybe there would be some other shorthand to remove the repeated `List:`,
# but I am ignoring that here.
x.List:reserve(5).List:append(123).List:reverse()
Just having explicit qualification is not enough by itself. It allows us to use the dot method syntax, but it completely removes the "static dispatch" part of the #ideas > static dispatch - proposal. This is a rigid system. Without abilities or interfaces or something of that nature, it is exceptionally restricted.
We could use the above system with the abilities of today, but I think we actually could make the system closer to what was proposed in the static dispatch proposal without the giant mess of types.
To do this, I think we should add a feature that replaces abilities that is essentially what interfaces are in go. For simplicity in this doc, I am just going to call these interfaces, but name and syntax totally up for change.
The core idea of an interface is that it is a group of functions that an opaque type implements. An interface like in go is automatically implemented if a type exposes all of the functions with correct name and signature as is required by the interface. There is no opt-in or opt-out.
Warning: Again, please try to ignore syntax. Just focus on how this feature functions.
Let's look at the example from above changing from strictly Lists to any generic container with the right methods:
# This is an interface definition
Container implements {
# self is a special type variable in the definition referring to the opaque type
append: self, elem -> self,
reserve: self, U64 -> self,
reverse: self -> self,
# Other methods...
}
func = \x ->
# Explicitly expose the methods otherwise requires qualification by interface name.
import Container exposing *
x.reserve(5).append(123).reverse()
The type of func is now simply a -> a where a implements Container. Any opaque type with all of those functions will just work here.
This is simple, understandable, and still flexible.
It also leads to easy composition of interfaces and a simple way to support more complex functions that don't use method dot syntax:
ReadWriter implements {} where self implements Reader & Writer
Complex implements {
decode: List U8 -> Result self error
}
I think this has a lot more promise without crazy types and too much flexibility
Important side note: I think we could use method dot syntax with current abilities. It would be very similar to interfaces, but with more restrictions. I think the flexibility of interfaces is an nice middle ground between the fully flexible static dispatch and the more restrictive abilities. I think it fits in nicer with the rest of the static dispatch proposal.
I think that static dispatch as proposed in #ideas > static dispatch - proposal is too flexible and that is problematic. I think it will lead to overly complex and messy types in many cases. This will make code harder to reason about and bugs more common. I think that we need to meet at a middle ground. That middle ground is probably some form of interfaces that are required to use static dispatch.
I definitely think there is still room to improve this in various places, but I think it is a better base to iterate off of. I think that working around roc not requiring any type signatures at all makes parts of this proposal rougher than I would like. Hopefully others have ideas around how to make this nicer.
I honestly want to write something closer to this to make interfaces nicer to use even without a type signature:
func = \x implements Container ->
x.reserve(5).append(123).reverse()
So lots of room for improvement, including in syntax. Please try to keep this thread focus on the core concept for now. If the core idea seems wanted, we can create spinoff threads to figure out a design that really makes the syntax nice to use.
Thoughts?
Richard Feldman (Nov 11 2024 at 19:54):thanks for writing this up! I'm on mobile, but briefly - I see where you're coming from, but I have the opposite conclusion :big_smile:
Richard Feldman (Nov 11 2024 at 19:54):I'll write more when I'm back at a keyboard!
Derin Eryilmaz (Nov 11 2024 at 19:56):I do think the flexibility should be left to autocomplete, so that everything is still fully qualified unless you really don't want it to be
Derin Eryilmaz (Nov 11 2024 at 20:55):@Brendan Hansknecht I'm wondering how a decode function would look with your syntax? or any ability with the relevant type (the one that the module depends on) in the return type of the function
Brendan Hansknecht (Nov 11 2024 at 21:28):It would look essentially identical to decode today. Look at Complex above. It just is part of the interface. There are no restrictions on the interface functions.
Derin Eryilmaz (Nov 11 2024 at 21:34):so when calling it, you would do Complex.decode?
Brendan Hansknecht (Nov 11 2024 at 21:34):Yep
Richard Feldman (Nov 11 2024 at 21:58):Brendan Hansknecht said:
This is actually more flexible than golang interfaces. There is no explicit interface defined anywhere in this picture.
So I would break down the difference between Roc and Go into two parts:
A, and also by the way I'm inferring and defining an interface named A and adding it to scope for you"—but if you're going to support type inference for these types anyway, that would feel a lot clunkier than having a syntax where you can specify the inferred interface right there in the type!To demonstrate that this gap is about type inference and not static dispatch, consider this:
Brendan Hansknecht said:
func = \x -> x.reserve(5).append(123).reverse()[ ... ]
I guess thatfuncis a:func : a -> f where a implements { reserve: a, Num b -> c }, c implements { append: c, Num d -> e }, e implements { reverse: e -> f }I'm not sure about you, but that is not how I would want this function to be typed.
I completely agree, but this is nothing new - we already have many instances of "that inferred annotation is not how I would want this function to be typed, because it's too generic", and the solution is always "...and so I will add a type annotation that represents how I actually do want the function to be typed."
For example, here's the inferred type of an un-annotated function that rotates the coordinates in a 3D point:
» \point -> { point & x: point.y, y: point.z, z: point.x }
<function> : { x : a, y : a, z : a }b -> { x : a, y : a, z : a }b
I'm not sure about you, but that is not how I would want this function to be typed either! :big_smile:
But of course the reason it has that inferred type is just that I wrote something very flexible, and the type inference system correctly inferred the most generic (aka principal) type that could be inferred based on what I wrote.
Outside the repl, unless I'm writing a quick script or something, I'd probably add a type annotation saying that these fields are numbers—possibly even a more specific number type. But the only way to avoid the experience where I don't get this type inferred is to not have type inference used on it (which is to say, I annotate the type).
I think the flexibility concerns expressed here about methods are just as valid for records. For example:
I think that for records to be reasonable, we need to reign in the flexibility. I think this can be done with minimal loss of functionality by two parts:
1. Requiring explicit qualification of which (nominal) struct type is being accessed wherever field access syntax is used.
2. Having groups of fields ("interfaces") that are explicitly predefined and required to be used.
Records, like methods, are essentially saying "I expect the following group of [fields/methods] to both exist and to have these types." There's no need for a type to formally opt into either, and there's also no need to name the group ahead of time; you can do it on the fly.
This is consistent with mainstream languages, which also don't have different annotation requirements for field access and method access. Either type annotations are required for both (e.g. Go, which requires specifying struct and/or interface types), or for neither (e.g. Python). Going back to this example:
func = \x ->
x.reserve(5).append(123).reverse()
This a function declaration anyone can write in Python, JavaScript, Ruby, and any of their statically-typed equivalents; the only difference is that if you want type-checking, their type-checkers might require more annotations to get any actual benefit from the type-checker. (TypeScript's infers the type "any" for x, for example.) If it were instead record fields...
func = \x ->
x.foo.bar.baz
...the same thing would be true: it would look and work the same way in Python, JS, Ruby, etc.—but the statically-typed equivalents of those languages might require a type annotation to specify where the fields are coming from. (TypeScript infers the type "any" for x in this example too.)
I don't see a reason why those tradeoffs should be different for Roc; it seems to me that either both record fields and methods should require type annotations, or else neither should!
Let's look at the longer example:
Brendan Hansknecht said:
# This is an interface definition Container implements { # self is a special type variable in the definition referring to the opaque type append: self, elem -> self, reserve: self, U64 -> self, reverse: self -> self, # Other methods... } func = \x -> # Explicitly expose the methods otherwise requires qualification by interface name. import Container exposing * x.reserve(5).append(123).reverse()The type of
funcis now simplya -> a where a implements Container. Any opaque type with all of those functions will just work here.This is simple, understandable, and still flexible.
We can compare this to what's proposed (using the syntax proposed in a different thread, which I think is an improvement over the syntax used in the proposal doc). I also took out append because supporting the additional elem type variable in there (which would amount to "parameterized abilities") would actually be an entire proposal unto itself, and it's not relevant for this topic.
# This is a type alias
Container self : where self.{
# self is just a type variable name I chose to make the diff smaller
reserve: self, U64 -> self,
reverse: self -> self,
# Other methods...
}
# Explicitly annotate the function
func : x -> x where x.Container
func = \x ->
x.reserve(5).append(123).reverse()
The diff here is:
Container implements { compared to Container self : where self.{import Container exposing * compared to func : x -> x where x.ContainerThe diff between the composition example is even smaller:
It also leads to easy composition of interfaces and a simple way to support more complex functions that don't use method dot syntax:
ReadWriter implements {} where self implements Reader & Writer
Again using the proposed syntax from the other thread, it would be:
ReadWriter self : where self.Reader, self.Writer
So in my mind, a simpler (but essentially equivalent) version of this proposal would be:
"Functions that use methods can have very flexible inferred types, so we should require type annotations for them."
As far as I can tell, this would have the same semantic consequence as the proposal in this thread, but it wouldn't require adding new syntax or language concepts. It could just be a compiler warning, and in fact the compiler could print out the inferred type as a starting point so you could just go through and make parts of the type annotation more specific as desired.
But, as exemplified by the record example earlier, I don't think this concern is unique to methods, and I don't see why methods should be treated differently. It would feel weird to me to say "in general, we can infer types for you, including when it comes to methods, but instead of it just being a best practice to annotate your top-level types, in this one specific subset of functions we're going to give you a warning so that CI fails if you don't annotate them. But all the other functions in the language don't require annotations."
Personally, I think it's valuable to allow functions like this to be written as-is in quick scripts:
func = \x ->
x.reserve(5).append(123).reverse()
I don't think requiring the author of a quick script to annotate this (or to use syntax that's equivalent to an annotation) makes the script author's life better, and anyone who isn't writing a quick script can (and likely should) choose to write an explicit type annotation anyway.
As a bonus, having type inference fully supported in the feature means that editors can at least infer the types for you and deterministically generate a valid annotation for you to tweak, which would save time compared to having to figure out what type information you need to specify completely by hand.
Anyway, sorry for the wall of text, but those are my thoughts! :big_smile:
Brendan Hansknecht (Nov 11 2024 at 22:21):I think comparing to records really drops the amount of complexity and verbosity that methods generate. Records are fundamentally simple in Comparison to the amount of noise and constraints in functions. Also, functions are chained in ways to lead to much bigger chains of type variable compared to a record.
I think that is a large part of her season they are bad to try and mold into the same thing
Richard Feldman (Nov 11 2024 at 22:22):Brendan Hansknecht said:
I think comparing to records really drops the amount of complexity and verbosity that methods generate.
but I think the more important point is that in practice, people don't tend to be confronted with those types either way :big_smile:
Richard Feldman (Nov 11 2024 at 22:22):like I don't tend to see types like { x : a, y : a, z : a }b -> { x : a, y : a, z : a }b in practice, even though that's what is inferred in the repl example
Brendan Hansknecht (Nov 11 2024 at 22:23):I think this would make it pretty common.
Richard Feldman (Nov 11 2024 at 22:23):hm, why would it be different? :thinking:
Brendan Hansknecht (Nov 11 2024 at 22:23):If you use a method in a top level function, you will hit it
Brendan Hansknecht (Nov 11 2024 at 22:23):Cause you will want to annotate the function
Richard Feldman (Nov 11 2024 at 22:23):but if the function is annotated, you won't see something that generic
Richard Feldman (Nov 11 2024 at 22:24):oh, well if the annotation disagrees with the implementation, we can give a more specific message - like "foo is not exposed from that module" or "the foo function exposed by that module has a different type than how you're using it here, namely: ___"
Brendan Hansknecht (Nov 11 2024 at 22:24):I think it would make it much more common to see adhoc groupings of methods that lead to complex type signature compared to abilities where it is always forced to be aggregated into a predefined groupings
Richard Feldman (Nov 11 2024 at 22:25):hm, so is the concern mostly about error message quality?
Richard Feldman (Nov 11 2024 at 22:25):or a different scenario
Brendan Hansknecht (Nov 11 2024 at 22:25):Error message, complexity of written type signatures, accidentally being overly generic in ways that leads to bugs when incorrect types are used that only look to follow an API but don't actually in practice.
Brendan Hansknecht (Nov 11 2024 at 22:26):The kinds of bugs you get from python duck typing, but a tiny bit more managed
Brendan Hansknecht (Nov 11 2024 at 22:26):Cause static dispatch is a form of duck typing
Richard Feldman (Nov 11 2024 at 22:26):well complexity of written type signatures is definitely not affected...if you do choose to annotate, the delta between the proposed syntax in this thread and the proposed syntax in the other thread is +0/-0 lines and just slightly different syntax for how the declaration starts
Richard Feldman (Nov 11 2024 at 22:27):and accidentally being overly generic can only happen if you aren't annotating
Brendan Hansknecht (Nov 11 2024 at 22:27):Requiring it to be predefined reduces the complexity compared to each function potentially having its own adhoc set of methods
Richard Feldman (Nov 11 2024 at 22:27):if you do choose to annotate, the doc is also identical to the proposal in this thread
Brendan Hansknecht (Nov 11 2024 at 22:27):And many beginners (and probably intermediate folks) will just annotate with a slightly cleaned up version of whatever the repl spits out
Brendan Hansknecht (Nov 11 2024 at 22:28):So it will lead to more mess
Richard Feldman (Nov 11 2024 at 22:28):oh I very strongly doubt that haha
Richard Feldman (Nov 11 2024 at 22:28):func = \x ->
x.reserve(5).append(123).reverse()
I think everyone would annotate this as List (Num a) -> List (Num a) or something more specific, like List U64 -> List U64
Richard Feldman (Nov 11 2024 at 22:29):I would be totally shocked if any significant number of people (beginners, intermediate, or advanced) annotated it as something more flexible than that! :big_smile:
Brendan Hansknecht (Nov 11 2024 at 22:29):Sure, but what if they want it to be generic. Or flexible (even if it isn't required which many people do)
Brendan Hansknecht (Nov 11 2024 at 22:29):Like they want it to work with any sort of list not just the standard library one.
Brendan Hansknecht (Nov 11 2024 at 22:29):People will write that code generically if the gate is open to do so
Richard Feldman (Nov 11 2024 at 22:30):well this is where I think the record example is relevant - today it's totally possible to make any annotation using records more flexible, and people don't even bother to add the * in { ... }* for arguments, which would make them accept records that are a superset of what the function accepts
Richard Feldman (Nov 11 2024 at 22:30):I think if we look for precedent across the Roc code that people have written, we find the opposite
Richard Feldman (Nov 11 2024 at 22:30):that people naturally tend to choose annotations that are simpler but much less flexible
Brendan Hansknecht (Nov 11 2024 at 22:30):That is data, not methods. I think it will be treated different. People don't think about them the same way
Brendan Hansknecht (Nov 11 2024 at 22:31):Methods are easy to share access many structures. Exact data layout is not
Richard Feldman (Nov 11 2024 at 22:31):hm, I could see that being the case for parameterized ones, if we were to allow that
Brendan Hansknecht (Nov 11 2024 at 22:31):So record field sharing is rare, but method sharing is not
Richard Feldman (Nov 11 2024 at 22:31):e.g. Appendable elem -> Appendable elem or Container elem -> Container elem
Richard Feldman (Nov 11 2024 at 22:31):but we don't support that today with abilities, and I don't think we should support it here either
Brendan Hansknecht (Nov 11 2024 at 22:32):What type is this function?
fn = \x, e -> x.append(e)
I think method syntax requires supporting the type variable
Richard Feldman (Nov 11 2024 at 22:32):I do agree that when it comes to collections, people tend to want to be more flexible in a lot of cases
Richard Feldman (Nov 11 2024 at 22:33):x, e -> r where x.{ append : x, e -> r }
Brendan Hansknecht (Nov 11 2024 at 22:33):Yeah. So people will do that to support generic containers
Brendan Hansknecht (Nov 11 2024 at 22:33):They just won't use Container elem cause we don't allow it. They will use that raw type.
Richard Feldman (Nov 11 2024 at 22:34):I don't think that would work the way they think they would though
Brendan Hansknecht (Nov 11 2024 at 22:34):So arguably not support Container elem will just lead to more people using the raw signatures.
Richard Feldman (Nov 11 2024 at 22:34):hm
Brendan Hansknecht (Nov 11 2024 at 22:35):where list.{append: ..., get: ..., set: ..., drop: ...}
I have my generic container
Brendan Hansknecht (Nov 11 2024 at 22:36):I think the initial static dispatch proposal is really opening up a floodgate that is too flexible
Richard Feldman (Nov 11 2024 at 22:37):if that's the case, how is the proposal in this thread different? :thinking:
Richard Feldman (Nov 11 2024 at 22:37):to me the thing it would be opening up is the ability to say "this function accepts an Appendable"
Richard Feldman (Nov 11 2024 at 22:38):which is not something you can say today, at least not with the parameterized type
Richard Feldman (Nov 11 2024 at 22:39):I think that's a totally valid concern, but I am still struggling to understand how the proposal here is significantly different from "you get a warning if you use a method without adding a type annotation" :sweat_smile:
Richard Feldman (Nov 11 2024 at 22:39):like concretely, is there a scenario where this would be different from just requiring type annotations on functions that use methods?
Richard Feldman (Nov 11 2024 at 22:39):(and if so, what would be different about it?)
Brendan Hansknecht (Nov 11 2024 at 22:39):Oh, I would totally go for that as well. Methods require type signatures.
Richard Feldman (Nov 11 2024 at 22:39):ok cool!
Brendan Hansknecht (Nov 11 2024 at 22:40):This is just trying to fit into roc without requiring type signatures (and I think that makes the proposal less good)
Richard Feldman (Nov 11 2024 at 22:41):but at that point, I think what I'm not seeing is why it's helpful to people to say "the compiler will warn you if you don't annotate these" as opposed to either a rule of "warn if you omit top-level annotations in general" (which we could do, but feels like a separate proposal) or "have it continue to be a recommended convention, but don't enforce it with a compiler warning"
Brendan Hansknecht (Nov 11 2024 at 22:41):If we say that type signatures are required for methods, that mostly alleviates the problems listed here without as much complexity just to avoid requiring type signatures
Richard Feldman (Nov 11 2024 at 22:41):sure
Richard Feldman (Nov 11 2024 at 22:41):but couldn't we wait and see?
Richard Feldman (Nov 11 2024 at 22:41):like confirm whether there is actually a problem in practice before enforcing that
Richard Feldman (Nov 11 2024 at 22:42):we could always start with the recommendation, see how that goes, and then add compiler enforcement if the cultural convention isn't working out for some reason
Brendan Hansknecht (Nov 11 2024 at 22:43):I guess the root questions to ask are:
Brendan Hansknecht (Nov 11 2024 at 22:44): Brendan Hansknecht (Nov 11 2024 at 22:45):I think these questions have to be answered before we really dial in on a design.
Brendan Hansknecht (Nov 11 2024 at 22:47):Current roc is:
Brendan Hansknecht (Nov 11 2024 at 22:47):The static dispatch proposal is:
Brendan Hansknecht (Nov 11 2024 at 22:48):This thread's proposal is trying to be:
Brendan Hansknecht (Nov 11 2024 at 22:49):Also, this gives a bit more clarity, of a key distinction. I don't just want type annotations. I also want explicit interface definitions.
Richard Feldman (Nov 11 2024 at 22:54):Brendan Hansknecht said:
I guess the root questions to ask are:
- Are we ok with the equivalent to higher order abilities?
- Should abilities require explicit definitions or is implicit ok?
- should abilities require explicit opt in or be automatically applied if the correct methods exist?
totally agree! my general thoughts are:
Richard Feldman (Nov 11 2024 at 23:04):Brendan Hansknecht said:
Also, this gives a bit more clarity, of a key distinction. I don't just want type annotations. I also want explicit interface definitions.
I think it's important to note that the proposal supports your preference. You can absolutely create the equivalent of interfaces every time, and opt into type annotations everywhere!
Richard Feldman (Nov 11 2024 at 23:04):the question is whether everyone in every situation should be forced to do that
Richard Feldman (Nov 11 2024 at 23:04):which is what I'm currently not convinced of :big_smile:
Brendan Hansknecht (Nov 11 2024 at 23:07):I guess if we still have good error messages (mostly worried about noise here) and these really adhoc groupings come up rarely in practice. So most people are still creating logical groupings (or using concrete types) and using those in type signatures, then 2 and 3 both could be implicit without much drawback.
2 being implicit means that messy adhoc groupings are possible. If it is explicit, forcing naming and hopefully logical groupings would hopefully reduce interface abuse and lead to cleaner abstractions.
3 being implicit means that you might hit bugs due to assuming a type meets an interface, but being wrong. Its append method may not be the same as a containers append method.
I think 2 and 3 being explicit are fundamentally enforcing order and reducing the chance of accidental bugs. But it is possible that neither of those are issues in practice....
Richard Feldman (Nov 11 2024 at 23:07):totally agree! And I'd like to do the experiment to find out if they're issues in practice
Richard Feldman (Nov 11 2024 at 23:08):obviously if it sucks we can change it :big_smile:
Richard Feldman (Nov 11 2024 at 23:08):to elaborate on the question of "should Appendable be a thing?" (which is what I think #1 above boils down to) - I think the reason I'm not sure about whether it would be a good or bad thing is that I don't have firsthand experience using Go, and in all the languages where I do have experiences using things like that, the tradeoffs are different in pretty significant ways
Richard Feldman (Nov 11 2024 at 23:10):for example, I've used Appendable in Java, and while it can be cool to write some functions that are generalized to take an "appendable" rather than (for example) a more specific collection type, that comes at the cost of everyone who wants to benefit from that abstraction having to opt into it
Richard Feldman (Nov 11 2024 at 23:11):now in Go you don't have to opt into it, which definitely changes the tradeoffs (in a net positive direction I think), but I don't have any firsthand experience with that to know what it feels like :sweat_smile:
Richard Feldman (Nov 11 2024 at 23:12):I also have firsthand experience with Rust and Haskell, which have ad hoc polymorphism that supports orphans, so you can do something in between, where I can define Appendable outside the definition of the nominal type and then add it to the nominal type after the fact
Richard Feldman (Nov 11 2024 at 23:12):which also has a different set of tradeoffs, bc Rust's orphan rule mean that I have to enumerate alongside my Appendable definition all the nominal types I'm going to add it to
Richard Feldman (Nov 11 2024 at 23:13):and Haskell's orphan rule has its own set of problems
Richard Feldman (Nov 11 2024 at 23:13):so my gut feeling is that I'm cautiously optimistic that it's a net positive, but with an emphasis on the "cautious" feeling :laughing:
Richard Feldman (Nov 11 2024 at 23:15):like on the one hand, being able to say "this function takes an Appendable instead of a List" makes the type more informative in that it tells me something about what the function is doing (or perhaps what it is not doing)
Richard Feldman (Nov 11 2024 at 23:16):but on the other hand, there's the general question of "you spend X amount of time trying to figure out what's the narrowest interface you could put on this function instead of the collection type you have in mind; is the return on that investment for X positive, negative, or neutral at the end of the day?"
Richard Feldman (Nov 11 2024 at 23:18):and it's that last part where I'm not sure, and would be curious what it's like in Go in practice!
Richard Feldman (Nov 11 2024 at 23:25):also, I have a vague intuition that some of these would not be implementable without some other type system feature (higher-kinded types maybe, but also maybe something less powerful?)
Richard Feldman (Nov 11 2024 at 23:26):like for example, I'm quite sure that this would work (in any design): Concatenable a : a.{ concat : a, a -> a }
Richard Feldman (Nov 11 2024 at 23:26):that seems very obvious - it's the same type as add, sub, etc.
Richard Feldman (Nov 11 2024 at 23:27):but this is different:
Appendable a elem : a.{ append : a, elem -> a }
we've talked about this before, and @Ayaz Hafiz made the point that you can't satisfy this constraint with e.g. List.append : List elem, elem -> List elem - or at least, the type system should reject this, but currently doesn't because of a bug
Richard Feldman (Nov 11 2024 at 23:30):because in List.append, elem has a constraint between its first two arguments whereas append : a, elem -> a says they should be unrelated
Richard Feldman (Nov 11 2024 at 23:34):so if you used List.append for that implementation, you'd be claiming it fit a broader type than it actually does
Richard Feldman (Nov 11 2024 at 23:35):...all of which is to say, I think actually we'd need to do more than static dispatch to enable things like Container and Appendable, which can (and I think should) be a separate idea!
Brendan Hansknecht (Nov 11 2024 at 23:37):If they are a separate idea then this wouldn't work, right?
func = \someList, elem -> someList.append(elem)
I think they really have to come paired
Brendan Hansknecht (Nov 11 2024 at 23:38):List.append wouldn't work as a method cause it depends on a higher order relationship
Brendan Hansknecht (Nov 11 2024 at 23:40):That or we keep the bug in abilities today where we simply don't check the higher order relationship and only ever fill in the type with the concrete versions which enables append: a, elem -> a to just work.
Brendan Hansknecht (Nov 11 2024 at 23:44):Also, I worked in go for about 2 years professionally. Most of the time everything is concrete. When it isn't, you take the minimal interface as input and return the most concrete thing possible (might still be an interface). Was really nice to work with. I never ran into or heard on any complaints in practice with automatically being part of an interface. Most people just loved it. Occasionally you had to slim down the methods in an interface to enable accepting more types, but it generally just worked and was great. I think more people have a positive experience with the system even if they are theoretically uncomfortable with it.
Richard Feldman (Nov 11 2024 at 23:46):Brendan Hansknecht said:
List.append wouldn't work as a method cause it depends on a higher order relationship
That or we keep the bug in abilities today where we simply don't check the higher order relationship and only ever fill in the type with the concrete versions which enables
append: a, elem -> ato just work.
yeah I'm not sure what the feasible options are here; @Ayaz Hafiz would have a better intuition than I would!
Brendan Hansknecht (Nov 11 2024 at 23:48):Brendan Hansknecht said:
List.append wouldn't work as a method cause it depends on a higher order relationship
Cause func would have the type a, b -> a where a.{append: a,b -> a}
List.append: List elem, elem -> List elem does not satisfy the constraints of the append method which would expect List.append to have the type: List a, b -> List a.
Richard Feldman (Nov 11 2024 at 23:48):right
Richard Feldman (Nov 11 2024 at 23:48):yeah I agree, it seems like it wouldn't work without a type system change
Brendan Hansknecht (Nov 11 2024 at 23:56):I mean the funny part is that it probably would just work today if it is implemented similar to abilities.
Assuming appendable was an ability today, I think we:
append: a, b -> aList.append: List elem, elem -> List elema with List elem. Replace b with elem. Yep all good. Those are the same type signatures: List elem, elem -> List elemList elem, elem -> List elem. This has added in an extra constraint. If you fail the constraint it is a compilation error (or maybe a crash in the current compiler).Essentially, substitution allowed using the shared type variable elem which added in an extra constraints. It probably should have substituted while enforcing uniqueness of type variables. So a gets replaces with List elem. b tries to get replaced with elem, but that is already claimed so it instead gets elem2. The resulting type signature does not match and it always fails List elem, elem2 -> List elem
Derin Eryilmaz (Nov 12 2024 at 01:11):just to add something tiny: if you want to ensure that a type matches an ability, you should probably be able to do that from within the same file.
check : a -> {} where a.Ability
t : ThingInModule
ignore = \{} -> check t
that would make up for it being implicit. i'm wondering if we should make a shorthand to do this, or if there's a neater way to write it that i'm not thinking of.
Richard Feldman (Nov 12 2024 at 01:18):Brendan Hansknecht said:
I mean the funny part is that it probably would just work today if it is implemented similar to abilities.
Assuming appendable was an ability today,
Appendable isn't implementable today (well, it type checks but then works incorrectly due to the bug, but if we fixed the bug then it wouldn't type check anymore) - that's why Ayaz and I already had a whole conversation about this months ago :big_smile:
Richard Feldman (Nov 12 2024 at 01:22):the conclusion from that conversation was that if we wanted to support it, we'd need a limited subset of higher-kinded polymorphism - one which preserves principal decidable type inference (I did not previously know there existed a subset of HKP that preserved principal decidable type inference, but turns out it exists!)
Richard Feldman (Nov 12 2024 at 01:22):and that would be equally true of abilities and of static dispatch, and for the same reasons
Richard Feldman (Nov 12 2024 at 01:24):but basically you need to be able to write something like append : container elem, elem -> container elem
Richard Feldman (Nov 12 2024 at 01:32):which, I suppose, is worth considering as a separate design question since:
Richard Feldman (Nov 12 2024 at 01:33):all of which is to say, of the two concerns I raised about HKP in the FAQ, the type inference one wouldn't apply to this subset, and the other one is worth revisiting based on new context
Richard Feldman (Nov 12 2024 at 01:34):possibly, the conclusion is the same as before, which I think means static dispatch doesn't work, but it's also possible the conclusion is that in this form it could be fine, in which case it does certainly sound appealing to steal this UX from Go: :big_smile:
Brendan Hansknecht said:
Also, I worked in go for about 2 years professionally. Most of the time everything is concrete. When it isn't, you take the minimal interface as input and return the most concrete thing possible (might still be an interface). Was really nice to work with. I never ran into or heard on any complaints in practice with automatically being part of an interface. Most people just loved it. Occasionally you had to slim down the methods in an interface to enable accepting more types, but it generally just worked and was great. I think more people have a positive experience with the system even if they are theoretically uncomfortable with it.
Richard Feldman (Nov 12 2024 at 01:35):so I think (unless @Ayaz Hafiz says I'm missing something here) that means the static dispatch proposal is on hold until we figure out whether foo.append(bar) can actually work without the HKP subset :stuck_out_tongue:
Richard Feldman (Nov 12 2024 at 02:54):hm, even with HKP I'm not sure how this would work:
\arg -> arg.map(fn)
...could you give that function both a Result (which has .map, which takes a function, but which operates on Result, which has two type parameters) and List (which has .map, which takes a function, but which operates on a List, which has one type parameter)
Richard Feldman (Nov 12 2024 at 02:55):but that also seems like another @Ayaz Hafiz question :sweat_smile:
Ayaz Hafiz (Nov 12 2024 at 03:06):hello
Ayaz Hafiz (Nov 12 2024 at 03:06):is there a quick recap/tldr of the issue?
Richard Feldman (Nov 12 2024 at 03:11):yes!
Richard Feldman (Nov 12 2024 at 03:13):so in the static dispatch proposal, if you put this into the repl:
\x, e -> x.append(e)
...it seems like the inferred type would be:
x, e -> ret where x.{ append : x, e -> ret }
which could be read as "do static dispatch on the x type to figure out what module it's defined in, then call a append : x, e -> ret function from that module"
Richard Feldman (Nov 12 2024 at 03:15):separately, you can make type aliases like:
Eq a : where a.{ isEq : a, a -> Bool }
and then use them like:
Dict.insert : Dict k v, k, v -> Dict k v
where k.Eq, k.Hash
the question has to do with scenarios like trying to do this:
Appendable c : where c.{ append : c, elem -> c }
could that work as-is? would the alias need a second type parameter? would HKP be required to express the constraint between c and elem if you wanted List.append to satisfy that?
Richard Feldman (Nov 12 2024 at 03:17):and if so, would it even be possible to write an expression like names.append("blah") or would that also require HKP to work with type inference?
Ayaz Hafiz (Nov 12 2024 at 03:23):HKTs for appendable - yes
Ayaz Hafiz (Nov 12 2024 at 03:23):for name.append("blah"), name is generic?
Ayaz Hafiz (Nov 12 2024 at 03:23):as in name : a forall a
Richard Feldman (Nov 12 2024 at 03:25):well I just mean can it work at all :sweat_smile:
Richard Feldman (Nov 12 2024 at 03:26):for example, can this work?
appendBlah : List Str -> List Str
appendBlah = \names -> names.append("blah")
and if so, can it still work if you remove the type annotation but always pass it a List Str?
Richard Feldman (Nov 12 2024 at 03:27):and then can it still work if sometimes you pass it a List Str in some places, but in other places pass it a Set Str (pretending for the sake of argument that Set.append existed)
Ayaz Hafiz (Nov 12 2024 at 03:28):Richard Feldman said:
for example, can this work?
appendBlah : List Str -> List Str appendBlah = \names -> names.append("blah")
yes this is fine
Ayaz Hafiz (Nov 12 2024 at 03:28):Richard Feldman said:
and if so, can it still work if you remove the type annotation but always pass it a
List Str?
yes but presumably you want to be able to provide a type to appendBlah?
Richard Feldman (Nov 12 2024 at 03:28):yeah I just mean like if I'm writing a quick script and choose not to write any type annotations
Richard Feldman (Nov 12 2024 at 03:29):I assume the type inference can still work based on my having passed it a List Str and then used the return value as a List Str elsewhere
Richard Feldman (Nov 12 2024 at 03:29):(this one I assume works, I just mainly meant it as a lead-in to the next question about what if you call it with different types)
Ayaz Hafiz (Nov 12 2024 at 03:30):i would think about it this way
Ayaz Hafiz (Nov 12 2024 at 03:30):\names -> names.append("blah") is equivalent to \names, append -> append names "blah"
Ayaz Hafiz (Nov 12 2024 at 03:30):the latter has an appropriate type in the current language
Ayaz Hafiz (Nov 12 2024 at 03:31):the question becomes do you want the type of append to reflect the idea that the type of "blah" is related to the type of names
Ayaz Hafiz (Nov 12 2024 at 03:31):if you don't, no HKTs needed
Richard Feldman (Nov 12 2024 at 03:34):ok, so my confusion then is that this:
Appendable c : where c.{ append : c, elem -> c }
...looks to me like it could successfully resolve to List.append : List elem, elem -> List elem because it doesn't specify a relationship between c and elem (meaning elem is just *)
Brendan Hansknecht (Nov 12 2024 at 03:34):So just to clarify and ask this directly: Is it/should it be valid to pass a List elem, elem -> List elem to a function expecting a a, b -> a input?
Richard Feldman (Nov 12 2024 at 03:35):so my understanding with that one is that the reason you couldn't use List.append to satisfy it is that List.append requires a relationship between elem and c (due to List elem and elem sharing a type variable) and that function's type says that their relationship needs to be unconstrained
Richard Feldman (Nov 12 2024 at 03:36):which is why that version of Appendable wouldn't be implementable, and it would require higher-kinded types to become implementable
Richard Feldman (Nov 12 2024 at 03:36):but in that case, I'm confused about why \names -> names.append("blah") works
Richard Feldman (Nov 12 2024 at 03:37):because I would think the inferred type of that function (without an annotation) would be the same as the Appendable type, namely:
names -> ret
where names.{ append : names, Str -> ret }
all this stuff feels way too complex for roc, but idk
Derin Eryilmaz (Nov 12 2024 at 03:43):were there use cases that people wanted but couldn't get with the current abiltiies sytem? other than the implicit vs explicit thing
Richard Feldman (Nov 12 2024 at 03:48):this is just trying to figure out what's possible in the static dispatch proposal
Richard Feldman (Nov 12 2024 at 03:48):including whether it's actually implementable as described, or if there's a type system implication I missed (which it's sounding like I didn't, but I'm still not 100% sure :big_smile:)
Richard Feldman (Nov 12 2024 at 03:51):also, we are way in the weeds haha...this is how the sausage is made on a lot of features that end up feeling a lot simpler to use than they were to design!
Ayaz Hafiz (Nov 12 2024 at 03:57):i think the problem might be the implicit assumption in the semantics of appendable
Ayaz Hafiz (Nov 12 2024 at 03:57):Appendable c : where c.{ append : c, elem -> c }
this is fine if you don't care about reflecting a relationship between elem and c
Ayaz Hafiz (Nov 12 2024 at 03:59):\names, append -> append names "blah" has type a, (b, Str -> c) -> c which is a totally fine type where (b, Str -> c) can be solved by (List Str, Str) -> Str, so if you don't care about expressing the relationship between b, Str, and c, there's no problem there at all
Brendan Hansknecht (Nov 12 2024 at 04:01):if you don't care
the advantage of caring is just being more strict with possible implementations and fully defining relationships?
Cause technically append could also be solved with Str.append which is Str, Str -> Str and really is not the same type of function as List Str, Str -> List Str (or a theoretically Set.append: Set Str, Str -> Set Str)
Ayaz Hafiz (Nov 12 2024 at 04:01):yes
Richard Feldman (Nov 12 2024 at 04:01):ahh ok cool!
Richard Feldman (Nov 12 2024 at 04:02):ok in that case both of these make sense to me
Ayaz Hafiz (Nov 12 2024 at 04:03):it might make it easier to just think about this in terms of the current language with abilities
Ayaz Hafiz (Nov 12 2024 at 04:03):the semantics are equivalent
Ayaz Hafiz (Nov 12 2024 at 04:04):you cannot express anything in the current language you can't in this proposal and vice versa, the difference is syntax
Richard Feldman (Nov 12 2024 at 04:05):and the algorithm we use to select which function to use (given the type), but yeah, that's unrelated to the type system
Ayaz Hafiz (Nov 12 2024 at 04:05):but that's also equivalent to abilities
Richard Feldman (Nov 12 2024 at 04:09):right, I just mean that abilities have a different algorithm for how to find the implementation, that's all
Richard Feldman (Nov 12 2024 at 04:10):although I guess the difference is pretty small haha
Richard Feldman (Nov 12 2024 at 05:25):ok so I guess the summary from that discussion is:
Appendable (which is already implementable today as an ability, as long as it's like container, elem -> container and there's no type relationship between the container and the element) could be created and applied to builtins without the builtins having to opt into it, which in turn means it might see more usecontainer elem, elem -> container elem, that would require HKP, but we don't need to have that type to implement Appendable or for type inference on static dispatch to Just Work for something like names.append("blah") (with no type annotations) Ayaz Hafiz (Nov 12 2024 at 05:35):yes
Ayaz Hafiz (Nov 12 2024 at 05:35):i do agree with the sentiment that the syntax expressed here does seem too complicated, understandable if it's an edge case
Richard Feldman (Nov 12 2024 at 05:50):in the doc, the syntax was something more like:
Appendable c : where c.append(elem) -> c
the motivation for changing it to this...
Appendable c : where c.{ append : c, elem -> c }
...is that the first syntax doesn't have a way to express when you want to dispatch on something other than the first argument type, e.g. for decoding
Norbert Hajagos (Nov 12 2024 at 10:29):Thank you, this was an incredible discussion!
Sam Mohr (Nov 12 2024 at 11:39):We seem to have concerns about unwieldy inferred types being a side effect of allowing solely unqualified methods without a type name. I think remaining conscious that Roc devs almost all have an LSP at their disposal can help mitigate this issue, specifically with code actions.
From a prior example:
func = \x ->
x.reserve(5).append(123).reverse()
Without type annotations, there won't be much filtering in naming suggestions for any of these methods, so the dev is incentivized to put some minor annotation:
func : List (Num *) -> _
func = \x ->
x.reserve(5).append(123).reverse()
-- or alternatively
func : x -> _ where x.NumList
func = \x ->
x.reserve(5).append(123).reverse()
They can start with just the types of the args, and that is enough to get useful hints without needing to figure out even what they're returning, meaning they're still able to "code via exploration".
We are already planning on adding LSP code actions to add type signatures to definitions (GH issue), we can also add a "finish type annotation" action that replaces _ type holes with their inferred types. (Ideally, this is callable within the function body so the dev doesn't need to go backwards while typing)
All this to say that:
roc format annotate to help clarify things Sam Mohr (Nov 12 2024 at 11:44):If we'd rather find some way to allow the user to declare types as they go because we want to avoid them needing to write even a partial type signature, there are plenty of languages that embed type information next to their arguments, and our recent parentheses push would make it a small transition to get authors to write:
fn func(x: List (Num *)): _ ->
x.reserve(5).append(123).reverse()
-- or alternatively
fn func(x: impl x.NumList): _ ->
x.reserve(5).append(123).reverse()
Which preserves forward authoring of code.
Derin Eryilmaz (Nov 12 2024 at 13:13):Derin Eryilmaz said:
just to add something tiny: if you want to ensure that a type matches an ability, you should probably be able to do that from within the same file.
check : a -> {} where a.Ability t : ThingInModule ignore = \{} -> check tthat would make up for it being implicit. i'm wondering if we should make a shorthand to do this, or if there's a neater way to write it that i'm not thinking of.
bumping this--is this something we want? it seems useful for ensuring that your type matches a well known interface before you publish it in a library, for example.
Derin Eryilmaz (Nov 12 2024 at 13:14):which is one of the advantages of explicitly opting in to interfaces, like in rust
Richard Feldman (Nov 12 2024 at 13:46):hm, I don't understand the use case - can you give a more concrete example of when you'd want this? like not using arbitrary types like "Foo" but rather a specific motivating example that might come up in real-world code?
Derin Eryilmaz (Nov 12 2024 at 13:48):@Richard Feldman if you want to make sure your custom type is properly implementing Decode before you publish it as a library
Derin Eryilmaz (Nov 12 2024 at 13:48):or some other complex trait like that
Richard Feldman (Nov 12 2024 at 13:49):oh I think you could just write a test which annotates it as such
Richard Feldman (Nov 12 2024 at 13:50):that's already doable based on the original doc proposal, should Just Work!
Last updated: Jun 16 2026 at 16:19 UTC