Abilities · ideas · Zulip Chat Archive

here's a design doc I've been working on for awhile - love to get any feedback anyone has about it!

Matthias Beyer (Oct 27 2021 at 14:41):

Num.add : number, number -> number
    where number has Num

add: number, number -> number
  where number has Num

So no Num.add, because add would be now a function that is generic over its parameters and not be attached to a type (Num in this case)?

Richard Feldman (Oct 27 2021 at 14:46):

but sure, what you wrote is accurate to the way it would appear in the Num module!

Richard Feldman (Oct 27 2021 at 14:46):

Matthias Beyer (Oct 27 2021 at 15:28):

Overall this sounds a lot like traits in Rust or (I guess) like typeclasses in Haskell! I saw that you don't like these, still my impression is that abilities are the same thing as traits or typeclasses. I think this is a good idea, trait bounds in Rust are a super-power and thus I can only approve :smile: !

The following is syntax and ergonomics, feel free to ignore! :laughing:. So from an ergonomics standpoint I am not sure on the syntax in some cases (though I also do not have a FP background). Not sure whether we want to discuss this right away or rather first talk about other details... What feels tedious when defining a new ability is that you basically have to repeat where <variable> has <ability> for every function to define in the interface of the ability.

# instead of
Eq has { isEq, isNotEq }
isEq : val, val -> Bool where val has Eq
isNotEq : val, val -> Bool where val has Eq

# simply
Eq with T:
  isEq: T, T -> Bool
  isNotEq: T, T -> Bool

Where T is the type for which Eq is implemented. This would basically say there's an ability Eq, that has a fn isEq that gets two parameters of the type that Eq is implemented for and returns a Bool.

The other thing is when implementing a new type, the current syntax ties the implementations of an ability to the type (syntax-wise). If the implementation of an ability would be "stand-alone" syntax-wise, defining new abilities and attaching them to foreign types would be a nobrainer (syntax-wise of course).

So while I can understand that you don't want classification in the language, I still don't understand why abilities shouldn't be user-defined? These are the basis of generics, and (of course IMHO) generics are a good thing to have (* when designed properly, not like in C++ :laughter_tears: ). Generics in Rust, and of course the trait system, are the superpowers of Rust and being able to define generic functions where the parameters only have to suffice a certain, well-defined, interface and have the compiler ensure these guarantees makes writing Rust a breeze!

Lucas Rosa (Oct 27 2021 at 15:29):

Abilities can be user defined
"Although I think custom user-defined abilities are worth having in the language because they address Problem #7, I hope they are used rarely in practice."

Lucas Rosa (Oct 27 2021 at 15:34):

Richard Feldman (Oct 27 2021 at 15:53):

minor note: I think the most common definition of generics is parametric polymorphism, which Roc already has - e.g. List has a type parameter, and you can make your own parameterized types using tags and records! :smiley:

Richard Feldman (Oct 27 2021 at 15:54):

and yeah, the design is that abilities can be user-defined (there's a section called "Defining New Abilities" which talks about this) but I hope this will be rarely done in practice - like for example, this example of Rust traits has them basically being used in the classic OOP style (classifying Sheep as Animals, making default methods and then overriding them), which I think would be a counterproductive way to use this feature in Roc!

Brendan Hansknecht (Oct 27 2021 at 16:00):

The name abilities makes me really want to call them powers. Then in conversation with someone, ask them what super powers their type has.

Matthias Beyer (Oct 27 2021 at 16:01):

yes, I see how that example is OOPish. But traits are not OOPish at all, they are just interfaces that can be implemented for different types. There's no inheritance involved at all (the default implementation thing looks like it, but it is actually just a "copy this implementation for the type that implements the interface", no inheritance-style thing). Even static dispatch to trait-methods is possible, if the concrete type is known!

Richard Feldman (Oct 27 2021 at 16:02):

Brendan Hansknecht (Oct 27 2021 at 16:11):

With the type printout is any information gained by adding the where.
5 : number where number has Num
could just be something like:
5 : has Num

Richard Feldman (Oct 27 2021 at 16:13):

Lucas Rosa (Oct 27 2021 at 16:14):

Brendan Hansknecht (Oct 27 2021 at 16:18):

Also, do we have/want any way for a user to be able to tell that Num is an ability and not a type. Cause I feel like a new user will want to write

x: Num
x = 7

Brendan Hansknecht (Oct 27 2021 at 16:20):

Oh, random other question, can I write something like this:
NumType: number where number has Num
If so, maybe it would be fine to use an ability as a type and just assume that Num: number where number has Num is already defined. I am not sure if it would be too confusing, but I feel like I would probably use aliasing a lot if the syntax was long.

Joseph Anthony Zullo (Oct 27 2021 at 22:52):

(Rewriting this paragraph after thinking about things more) Whether Hash and Encode should be separate or not is tricky. What's different between these functions is that the internal representation that you want to work with changes. Let's say we are working with global tags. The compiler will probably represent these as ints in memory for efficiency. For virtually every use-case of Encode, we want to convert tags to their string representation, and then any encoding from there (json, sexp etc) will work off of that string. With hashing on the other hand, we probably want to be working directly with the representation in memory (some integer), because it's fast and doesn't traverse a string. These should be unifiable without performance degradation, but they are fundamentally doing very different things. I would opt to be less magical and more transparent, and make hashing and encoding separate since this way it's more clear what representation is being processed internally. Additionally if Hash and Encode are unified I think it's inevitable that issues will come up with implementation, but it's hard to describe simply.

For Encode and toStr: Python makes a distinction between its __str__ and __repr__ methods which I think is relevant here. It's possible that we would rather have our toStr function have built-in pretty printing and leave encoding to represent unmodified internal structure. For example, printing something like a linked list might benefit a lot from a string/prittyprint ability that converts it into an array first instead of something that unloads a bunch of conses and nils.

Zeljko Nesic (Oct 27 2021 at 23:44):

It adresses many pain-points.
First, it checks off my biggest concern interface wise of modules and the editor, but now I can see it clearly how it would work.
Second, get's rid of need for "deriving" annotation, which is ugly in every language, and tediously stupid activity!

Its important to teach it as a different language feature, which unlock another dimenstion in annotating data structures, not explaining the "hiearchy of things"

I understand that you might be afraid of categorization, but on the other hand people like to write types to label abilities. In Haskell it feels really nice that after you have added sane abilities, you can see which resourses a function uses to get the result needed.

getDatabaseThingy : database , SomeParam -> Task DatabaseErrors Thingy
    where database has PgCustomConnection
getDatabaseThingy = \ db, p ->
    runPgQuery db (mkQuery p)

PgCustomConnection has
        { dbCredentials : val -> Creds
        , runPgQuery : PgQuery -> Task DatabaseErrors val
        } where val has PgCustomConnection

FunctorAbility has
    { fmap : (a -> b) , val a -> val b
    , fpure : a -> val b -> val a
    }  where val has FunctorAbility

ItCanHazMonoid has
    { mempty : val
    , mappend : val -> val -> val
    , mconcat : List val -> val
    }  where val has ItCanHazMonoid

Looks nice, I am fan of having type definitions on one place when defining ability.

Eq :: { isEq : a , a -> Bool ,
        isNotEq : a , a -> Bool
       } where a :: Eq

getDatabaseThingy : database , SomeParam -> Task DatabaseErrors Thingy
    where database :: PgCustomConnection
getDatabaseThingy = \ db, p ->
    runPgQuery db (mkQuery p)

PgCustomConnection ::
        { dbCredentials : val -> Creds
        , runPgQuery : PgQuery -> Task DatabaseErrors val
        } where val :: PgCustomConnection

Otherwise, exciting, I strongly believe this is very imporant to have, even though it might make head-ache to implementit proper.

Martin Stewart (Oct 28 2021 at 07:05):

I think the Encode and Decode abilities are a mistake. We do this at work in F# and the problem is, since record field names and tag names are included in the generated json, renaming them or modifying the type in other ways will silently create a breaking change for any API that relies on that format. And someone new to the code will have no idea if a innocuous looking type is safe to change or if it's indirectly referenced in a type that gets serialized. In practice what I've seen happen is that people avoid refactoring types because they know they'll need to spend time checking a bunch of APIs to see that things still work afterwards.

I think a better approach is to have the json package provide a editor tool for generating encoder/decoder functions for a given type in the user's code. That way, if they later change the type, they'll get a compiler error since it no longer works with the code that was generated. Also generating explicit code means that a beginner will have an example of what writing encoders/decoders looks like, lessening the learning curve if they ever need to write one by hand.

Tim Whiting (Oct 28 2021 at 14:13):

My understanding is that Roc will enforce semantic versioning. So to cover that issue have your API models be in a separate library and anytime there is a breaking change that library will be forced to increment its semantic version number. Anyone using the API can use the old if they want, but the two are no longer backwards compatible. Designing servers that support both versions of an API is a harder issue in general, maybe Roc needs a first class concept of API design? However there are other solutions for API design like grpc protobufs or openapi3 specs which might be better suited for this since they are language agnostic. The grpc and openapi3 generators could then generate Encode and Decode abilities as well as the interface classes potentially as an IDE plugin that reads their specs.

Richard Feldman (Oct 28 2021 at 14:32):

another strategy you can use is to have a separate type alias for the serialized type which is initially a copy/paste of the original record type alias, but then if you have the serialization calls all use the copy/pasted alias, you'll get a type mismatch if you ever change the internal type and forget to change how the serialization/deserialization works

Brendan Hansknecht (Oct 28 2021 at 16:14):

How does Serde deal with versioning? It is successful with simple encode and decode, so we should probably learn from it.

Matthias Beyer (Oct 28 2021 at 18:22):

serde does not deal with versioning at all. You make your types de/serializable and have to deal with updating your API yourself.

Matthias Beyer (Oct 28 2021 at 18:28):

IMO, having a notion of decoder and encoder or de/serializable should not be provided by the language nor by the standard library, but by a library. Like we have in Rustland with serde. But that's just my 2ct.

Locria Cyber (Oct 28 2021 at 18:32):

Locria Cyber (Oct 28 2021 at 18:35):

Locria Cyber (Oct 28 2021 at 18:38):

serde's Serialize trait is complex because it needs to support all types of output, from regular grammar to non-regular ones.

Martin Stewart (Oct 28 2021 at 19:06):

@Tim Whiting @Richard Feldman If I understand your suggestions correctly, they both seem to assume that the type used by the API is self contained? My experience is that this often isn't the case. The type might reference many other types in various other modules that are used in many other places throughout the program. It would be infeasible to move these types and all their functions out to a package or duplicate all of those modules.

Maybe I'm misunderstanding something here but having Encode/Decode abilities caught me by surprise because it seems like implicit behavior and something where you need to know best practices in order to avoid accidentally breaking something in production. It feels like the opposite of what a language inspired by Elm would be about.

Tim Whiting (Oct 28 2021 at 19:27):

If you don't have self contained types for the interface between backend/frontend, how do you ever evolve your API? There are very limited things you can do to your API other than add new optional fields, or creating new services with new types for any change. Though you could argue that new services with new types are the only way not to break clients relying on and old version of the API. Self contained types / API could allow one client to use an older version, and another to use a newer API, but how do you let the server handle both versions of the API? That is my question.

Martin Stewart (Oct 28 2021 at 19:50):

In some cases we indeed only make limited changes. In other cases (when working with Elm) we do actually duplicate types in order to be able to have a v1 and v2 of an API call, though we don't duplicate everything, only the parts that were modified since we know what we just changed. But the crucial thing is, we make those changes when we intend to, not by accident due to someone deciding to rename a record field (again, when working with Elm, in F# we accidentally break stuff since it's all automatically encoded/decoded).

Brendan Hansknecht (Oct 28 2021 at 19:55):

I mean that sounds like a really simple case that tests deal with.
Also, if someone is explicitly labeling a type as encode/decode, other programmers should be able to tell that they are not allowed to mess with the definition unless they also deal with the frontend/consumers, or make changes optional.

Brendan Hansknecht (Oct 28 2021 at 19:57):

I don't think this is a problem with having a default encode/decode function. I think it is a problem with testing, code review, and code quality rules.

Brendan Hansknecht (Oct 28 2021 at 20:00):

Oh, i just realized the doc says encode by default and not an annotation similar to rust. That i think is a bad idea. I think it needs to be explicitly annotated somehow.

Nigel Thorne (Oct 29 2021 at 01:34):

Any type that has matching methods defined for it could be said to 'have that ability'.

Ability names can be used in place of types in parameters, thereby enabling method be defined that work for all future types that also conform to this set of function definitions.

It occurs to me that you could have explicit statements that a type implements an ability or a kind of duck typing at compile time that infers it.

Joseph Anthony Zullo (Oct 29 2021 at 03:39):

I mean, I think what is occurring to you correctly matches how abilities work? You declare an ability, and in it the type signature of methods, then when you declare a constructor with that ability it checks to see that the methods are implemented. This is very similar to traits / typeclasses in Rust/Haskell, and also Java interfaces.

Joseph Anthony Zullo (Oct 29 2021 at 03:41):

Lucas Rosa (Oct 29 2021 at 04:08):

Zeljko Nesic (Oct 29 2021 at 11:15):

@Lucas Rosa So you think that Functor has { fmap : (a -> b) , f a -> f b } where f has Functor is not allowed?

Richard Feldman (Oct 29 2021 at 12:11):

Richard Feldman (Oct 29 2021 at 12:12):

Folkert de Vries (Oct 29 2021 at 12:29):

specifically the f in f a has a kind that is not Type (also written *, pronounced "star"). E.g. if we take f = List then List itself is not a type, it needs to be applied to a type argument to form a type

Richard Feldman (Oct 29 2021 at 12:31):

In general (not just in Roc), it doesn't really work to explicitly apply something like abilities (or traits, or typeclasses, etc) to structural types like anonymous records.

origin : { x : I64, y : I64, z : I64 }
origin = { x: 0, y: 0, z: 0 }

Point : { x : I64, y : I64, z : I64 }

origin : Point
origin = { x: 0, y: 0, z: 0 }

this is equivalent because Point is a type alias - it's completely interchangeable with its definition

Richard Feldman (Oct 29 2021 at 12:32):

now, if I were to take this code where Point is a type alias and try to give it a new ability (or to override a default one) what I'd necessarily be saying is that I'm making that change for every { x : I64, y : I64, z : I64 } in the program!

Richard Feldman (Oct 29 2021 at 12:34):

I very strongly think that should be allowed. Besides the implementation challenges, it would also mean you could affect completely distant and unrelated types, including private ones in other modules, just by writing some code in a new module - a global monkeypatch, in effect. Also sometimes they could conflict. It would allow for unprecedented huge messes to be created. :sweat_smile:

Richard Feldman (Oct 29 2021 at 12:35):

So if that's not allowed, how could we get to a world where Point has Encode and Decode?

Richard Feldman (Oct 29 2021 at 12:36):

Richard Feldman (Oct 29 2021 at 12:38):

If we don't have Encode and Decode implemented by default for all structural types (e.g. records, tags, etc), then if you want to add Encode and Decode to a particular type, that type must be a newtype with getter and setter function(s) to expose the fields

Folkert de Vries (Oct 29 2021 at 12:40):

I vaguely remember that ML modules also kind of fix this, but I don't know any details

Richard Feldman (Oct 29 2021 at 13:30):

Richard Feldman (Oct 29 2021 at 13:39):

If I understand it right, the point is that basically if I have (for example) a User, which is a type alias for a record with a bunch of fields, and then I send that in a HTTP response using automatic serialization, and then later I change a field on that type alias, I won't get any compiler errors but it might still cause a production bug.

In contrast, if I make an explicit encoder function, and I change User, I'll get a type mismatch because the encoder function will no longer line up with the User record.

Richard Feldman (Oct 29 2021 at 13:45):

that would address the concern about Encode and Decode not needing to be explicitly labeled (and opt-in rather than opt-out), but it would still be the case that if you made a change to the User record, you wouldn't automatically get a type mismatch; it would break production code by default

Richard Feldman (Oct 29 2021 at 13:45):

notably, a difference between "write an encoder and decoder by hand to get this benefit" versus "copy/paste a type alias to get this benefit" is that the latter is much easier for beginners to learn how to do!

Richard Feldman (Oct 29 2021 at 13:56):

What if Encode and Decode are builtins but not automatically applied to structural types like records and tags, so that (unlike Eq for example) you can only get access to them via a newtype?

Then if you want to encode or decode a User record, you would first need to make a newtype like UserSerialized := User has [ Encode, Decode ] and then pass that to encoding and decoding functions instead of a raw User record. So, one extra line of boilerplate per type. But you still wouldn't need to actually write encoders or decoders by hand (if you didn't want to), because Encode and Decode are builtin abilities (like Eq) and the compiler just knows how to make default implementations for you

Richard Feldman (Oct 29 2021 at 13:56):

in other words, it would basically have the same pros/cons as #[derive(Serializable, Deserializable)] in Rust

Richard Feldman (Oct 29 2021 at 14:22):

could also combine the "copy/paste type alias" idea to do EncodedUser := { ...copypasted User goes here...} has [ Encode, Decode ]

Richard Feldman (Oct 29 2021 at 14:31):

Brendan Hansknecht (Oct 29 2021 at 15:07):

I think the explicit annotation is important for something like this. I do believe it will significantly help reduce bugs. It at least gives users a reminder that they have to pay attention when they modify a specific record.

Brendan Hansknecht (Oct 29 2021 at 15:13):

Also, your type alias comment is mildly concerning when thinking about the bigger picture. Cause I don't think people will want a function that explicitly takes a point to also accept any other type that also happens to be an x, y, and z. I would have assumed:

SomeFunc : { x : I64, y : I64, z : I64 } -> ...

Point : { x : I64, y : I64, z : I64 }
SomeFunc : Point -> ...

Richard Feldman (Oct 29 2021 at 15:32):

Richard Feldman (Oct 29 2021 at 15:34):

Point : literally tells the compiler "anywhere you see Point you can replace it with exactly what I wrote on the other side of the : and it will mean exactly the same thing"

Brendan Hansknecht (Oct 29 2021 at 16:01):

Would any of the uses change? Like would I have to change all of my functions to unwrap the point, or is just the equal sign the difference?

Brendan Hansknecht (Oct 29 2021 at 16:03):

Also, is := current Roc syntax or future functionality to be added, I don't think I have ever seen it before.

Zeljko Nesic (Oct 29 2021 at 16:21):

Lucas Rosa (Oct 29 2021 at 16:21):

Richard Feldman (Oct 29 2021 at 16:36):

Richard Feldman (Oct 29 2021 at 16:37):

(private tags currently do approximately the same thing; := would replace private tags)

Richard Feldman (Oct 29 2021 at 16:38):

Richard Feldman (Oct 29 2021 at 16:40):

out of curiosity, specifically with a Point type, what would be the scenario where something bad would happen if you used : and the types were interchangeable?

It's hard for me to imagine using that and actually regretting it in practice because something bad happened :big_smile:

Brendan Hansknecht (Oct 29 2021 at 20:18):

Point was probably not that great of an example. I think the bigger issue would be with different types that are likely to share field names, but I think i still have an idea:

So with type aliases it enables a record with extra fields being passed, correct? So like SomeFunc : { x : I64, y : I64 } -> ... would accept a { x : I64, y : I64, z : I64 } as an argument. There could definitely be some cases where using a quaternion or 3d point as a 2d point would give unwanted results.

Folkert de Vries (Oct 29 2021 at 20:20):

that's not accurate, { x : I64 } only accepts records with exactly those fields. To accept any record with the x field of the correct type, you'd need to explicitly open it op with { x : I64 }*

Brendan Hansknecht (Oct 29 2021 at 20:23):

Richard Feldman (Oct 29 2021 at 22:00):

I'm gonna focus on writing documentation when I get back from GOTO Copenhagen and Handmade Seattle in 2 weeks!

Martin Stewart (Oct 30 2021 at 16:58):

Could I ask what's the drawback with my suggestion? That is, have a code generator bundled with the json package (or bytes, xml, etc). To me this seems like all upsides with none of the downsides:

Brendan Hansknecht (Oct 30 2021 at 17:16):

What do you mean when you say a code generator, like something that looks at a roc record string and generates a roc function for encode and decode?

Brendan Hansknecht (Oct 30 2021 at 17:17):

Martin Stewart (Oct 30 2021 at 17:20):

Correct, by code generator I mean an editor plugin that generates Roc code when the user requests it. That code is then just like any user written code.

Richard Feldman (Oct 30 2021 at 22:34):

yeah, this was actually the original thing I was planning to do (pre-Abilities) - have the editor be able to generate implementations for you, which you can then tweak by hand as necessary

Richard Feldman (Oct 30 2021 at 22:36):

one of the problems with that plan was what to do about nested types - e.g. if I have a User record which has inside it an Email opaque type, how does the editor know what it should use to decode that? What if there's more than one decoder that could work?

Abilities give an answer to that: if Email has Decode, then that tells the editor exactly what implementation to use

Richard Feldman (Oct 30 2021 at 22:40):

I was going to say the drawback was the one I mentioned a moment ago (how does the editor know what implementation to use?) but then I realized abilities can solve this by specifying which encoding/decoding function to use, even if they aren't involved in the implementation of that function :big_smile:

Richard Feldman (Oct 30 2021 at 22:41):

Richard Feldman (Oct 30 2021 at 22:46):

User :
    {
        name : Str,
        email : Email,
        address : Address,
    }

Address :
    {
        street : Str,
        city : Str,
        postcode : Str,
    }

Richard Feldman (Oct 30 2021 at 22:49):

decode : input, Decoder input err -> Result User err

where input is something like Json (like in serde, it works with multiple encodings; after all, why not?)

Richard Feldman (Oct 30 2021 at 22:51):

Richard Feldman (Oct 30 2021 at 22:52):

so now let's say I add a field to User - this previously-generated decode function will now break, because it's no longer specifying that field

Richard Feldman (Oct 30 2021 at 22:53):

this was the goal! I got a compile error instead of silently breaking production. The system works! :thumbs_up:

Richard Feldman (Oct 30 2021 at 22:53):

however, now let's say I add a field to Address. This will not just break the User decoder, but also every other decoder that's using Address.

Richard Feldman (Oct 30 2021 at 22:54):

how many of those are there in my code base? If there are a lot of them, it could be painful to go through and update them all. Is that okay?

Richard Feldman (Oct 30 2021 at 22:55):

in contrast, in the "Encode and Decode are automatically implemented for all primitive types" design, adding a field to Address would not actually cause a compile error, because Address is entirely primitive types - so although it would be less painful to update all the broken decoders, it could very easily silently break production

Richard Feldman (Oct 30 2021 at 22:56):

Brendan Hansknecht (Oct 31 2021 at 23:06):

I think automatically deriving Encode and Decode is fine, but I think it needs to be explicitly annotated. I don't think User should be allowed to rely on Address encoding if Address doesn't specify it has encode.

Martin Stewart (Nov 02 2021 at 19:33):

I've written a code generation tool that for Elm that does essentially what I'm advocating here (it generates codecs for elm-serialize instead). The way it works is that it just makes a guess when it comes determining what implementation to use. In your example, if it sees a single function with the type signature Codec e Address then it will use that. It could probably also do something like find functions called toString: Address -> String and fromString : String -> Result e Address and use those to build the rest of the codec.

This might seem like a really sketchy approach to code generation, but in practice I've found it works well (we use it at work and my colleagues seem happy with it too). The reason for this is, it only needs to generate the code once, which means any mistakes can be corrected by the user while still sparing them from 99% of the work.

I think this approach could work in Roc as well. I guess there's no harm in also using abilities to figure out what functions to use instead of a heuristic but my point is that it's quite flexible.

If the user has many unique encoders/decoders for address then I think it's important that they look at all of them. If they aren't unique then the user should probably just have a single function instead. As mentioned above, the code gen should try to reuse existing implementations before it generates its own copy.

Brendan Hansknecht (Nov 02 2021 at 20:33):

I think it would definitely be worth trying @Martin Stewart's idea. I think it is much easier to go from that to automatically deriving encode than the other direction. That being said, this probably will take significantly more work to get off the ground than automatically deriving (requires well functioning editor with plugins vs some minor compiler functions).

Richard Feldman (Nov 02 2021 at 21:35):

Richard Feldman (Nov 10 2021 at 22:21):

Brendan Hansknecht (Nov 12 2021 at 08:20):

Quick question on the encode trait. What roughly would be its API? An encode function that takes an encoder and specific type that is implementing encode for. Then encoders would implement a different ability? Just trying to understand how the encode trait would target multiple formats.

Zeljko Nesic (Nov 12 2021 at 12:04):

Thingy : { name : Str, age: U32 }

it = { name = "Ye", age = 42 }

main : Str
main =
    Encode.stringEncoder it

Brendan Hansknecht (Nov 12 2021 at 17:34):

Joseph Anthony Zullo (Nov 12 2021 at 19:13):

I think there will have to be some polymorphic compiler magic. E.g. there has to be some function/preprocessor that takes in an arbitrary value and spits out an s-expression/ast, and then you can work from there. I'm not sure how Rust and Haskell handle this though.

Brendan Hansknecht (Nov 12 2021 at 19:22):

So i think the core of my question is being missed. Sure encode may be automatically derived for some struct, but a user still has to be able to write a new encoder. What if i want to encode to proto, flatbuffer, xml, string, etc. Sure the standard library or common packages may generate a few of those encoders like json, but as a user what needs to be implemented to create a new encoder that targets a new serialization format?

Abilities so far only talk about the encode part of the API, not the encoder part.

Richard Feldman (Nov 12 2021 at 19:30):

I haven't put any significant amount of thought into it, but as a baseline my thinking was to start by looking at how serde does them and go from there!

Ayaz Hafiz (Mar 06 2022 at 14:35):

jan kili (Mar 06 2022 at 19:05):

I just read the abilities proposal for the first time since really using Roc, and I'm excited for it!
I only have one request: I think has should be renamed to can.

jan kili (Mar 06 2022 at 19:15):

Bool.isEq : val, val -> Bool
    where val can Eq

Dict k v can only
    [ Eq, Hash, Sort ]

Brendan Hansknecht (Mar 06 2022 at 19:56):

I feel the issue is more with short ability names. If it instead said: where val has Equality that would look proper.

Brendan Hansknecht (Mar 06 2022 at 19:57):

Richard Feldman (Mar 06 2022 at 20:01):

Richard Feldman (Mar 06 2022 at 20:02):

Bool.isEq : val, val -> Bool
    | val can Eq

Brian Carroll (Mar 06 2022 at 20:22):

The can terminology fits nicely with the name "abilities" too.
But I'm also just reading the doc now and realising that can Num sounds weird! Naming is hard!

» 5
5 : number
    where number can Num

Derek Gustafson (Mar 06 2022 at 20:25):

jan kili (Mar 06 2022 at 20:26):

Derek Gustafson (Mar 06 2022 at 20:27):

jan kili (Mar 06 2022 at 20:29):

Derek Gustafson (Mar 06 2022 at 20:30):

I know. Do we really want to distinguish between things that can be added and things that can be subtracted?

jan kili (Mar 06 2022 at 20:37):

Looked at another way, do we really want to require custom types to implement subtraction just to use builtin addition?

Brian Carroll (Mar 06 2022 at 20:37):

Brian Carroll (Mar 06 2022 at 20:39):

Separate abilities for addition and subtraction and multiplication and log and whatever just seems like a lot of complexity. The custom type example doesn't seem like an edge case that should drive it.

Brian Carroll (Mar 06 2022 at 20:41):

I think basic numbers have to work nicely because essentially all programs will use that. Custom addition doesn't really have to be as nice. You can make a function for it and say addMyType : MyType, MyType -> MyType

Brian Carroll (Mar 06 2022 at 20:42):

Brian Carroll (Mar 06 2022 at 20:44):

The can keyword is nice in many cases but we can use something else if it doesn't work or use punctuation.

Derek Gustafson (Mar 06 2022 at 20:47):

As a mathematician, seeing addition as an ability says to me "this is a semigroup" and has me thinking I should be approaching the language that way

jan kili (Mar 06 2022 at 20:48):

Derek Gustafson (Mar 06 2022 at 20:51):

Richard Feldman (Mar 06 2022 at 20:52):

I think encode and decode should be separate because sometimes it makes sense to have the one but not the other; e.g. maybe I want to specify how to encode my Document type into a .docx file but I don't want to implement a .docx parser to turn one of those back into a Document type :sweat_smile:

Derek Gustafson (Mar 06 2022 at 20:54):

jan kili (Mar 06 2022 at 20:55):

I'm open to addition and subtraction being inextricably linked, btw :) just trying to unlearn my OOP training

Derek Gustafson (Mar 06 2022 at 20:55):

jan kili (Mar 06 2022 at 20:55):

I'm most interested in whether we should encourage developers to make custom abilities be bundled or unbundled as a best practice

Richard Feldman (Mar 06 2022 at 20:56):

Richard Feldman (Mar 06 2022 at 20:57):

I wonder if there are other examples besides Num that sound weird with can :thinking:

Derek Gustafson (Mar 06 2022 at 21:00):

Linguistically, can should be followed by a verb. Anything that is better described by a noun than a verb will probably have this issue

Richard Feldman (Mar 06 2022 at 21:00):

Richard Feldman (Mar 06 2022 at 21:01):

if only there were a word for "can do all the things a number can do" :sweat_smile:

Derek Gustafson (Mar 06 2022 at 21:02):

Yeah. Unfortunately all those terms, that I know of, use classification terminology

Richard Feldman (Mar 06 2022 at 21:04):

there's also the problem that numbers are really basic and come up a lot, so choosing a word that people don't immediately grasp is a significant downside

Folkert de Vries (Mar 06 2022 at 21:07):

Derek Gustafson (Mar 06 2022 at 21:09):

Folkert de Vries (Mar 06 2022 at 21:10):

because I actually know how to work with monads. They have better type signatures

Richard Feldman (Mar 06 2022 at 21:11):

Derek Gustafson (Mar 06 2022 at 21:13):

Richard Feldman (Mar 06 2022 at 21:26):

the nice thing about has is that it can be read as "has the ____ ability" and whatever name you chose for the ability fits naturally into that blank

jan kili (Mar 06 2022 at 21:30):

Num.add : x, x -> x
    | x can Add

Num.add : x, x -> x
    | x has Num

jan kili (Mar 06 2022 at 21:31):

(but maybe I'm misunderstanding the most-common ability syntax appearances for noobs)

Richard Feldman (Mar 06 2022 at 21:31):

Richard Feldman (Mar 06 2022 at 21:33):

» 1
1 : a | a has Num

Richard Feldman (Mar 06 2022 at 21:33):

Richard Feldman (Mar 06 2022 at 21:34):

» 1
1 : a | a has Add, Sub, Div, Mul, Pow, Rem

Richard Feldman (Mar 06 2022 at 21:34):

» 0x1
1 : a | a has Int

» 0.1
0.1 : a | a has Frac

Richard Feldman (Mar 06 2022 at 21:35):

another downside is that if they are separate abilities, it makes it more likely that people will overload + and such for DSLs

Richard Feldman (Mar 06 2022 at 21:35):

jan kili (Mar 06 2022 at 21:36):

Richard Feldman (Mar 06 2022 at 21:36):

Richard Feldman (Mar 06 2022 at 21:37):

but at least with Num you have to actually implement the entire range of numeric operations, so it's very clear that you're doing something discouraged if you're using it for your URL type

Richard Feldman (Mar 06 2022 at 21:37):

Richard Feldman (Mar 06 2022 at 21:38):

so I'd prefer it if the ergonomics for the use cases like units of measure, arbitrary-sized integers/fractions/ratios/etc, and so on, were good

Richard Feldman (Mar 06 2022 at 21:38):

but the ergonomics for overloading individual operators for non-numeric use cases were bad, so it would be actively discouraged to do that!

jan kili (Mar 06 2022 at 21:44):

Makes sense, so then nevermind about the un-bundling - this is an ability set that we definitely want.

jan kili (Mar 06 2022 at 21:46):

However, naming-wise, I think designing for verbs would be better than for nouns:

» 1
1 : a | a can NumberMath

Even though there isn't a great word for this, has Num sounds too much like it's a record with a number field.

jan kili (Mar 06 2022 at 21:47):

» 0x1
1 : a | a can IntegerMath

» 0.1
0.1 : a | a can FractionMath

Richard Feldman (Mar 06 2022 at 21:47):

jan kili (Mar 06 2022 at 21:48):

This may seem verbose for builtins, but it clearly communicates the concept of bundled abilities to new users

Richard Feldman (Mar 06 2022 at 21:48):

» 1
1 : Num *

Richard Feldman (Mar 06 2022 at 21:49):

Richard Feldman (Mar 06 2022 at 21:50):

Num a : a | a can DoMath

jan kili (Mar 06 2022 at 21:51):

(maybe unrelated, but would has Num also support encoding & decoding in it, in addition to supporting math operators?)

Richard Feldman (Mar 06 2022 at 21:51):

jan kili (Mar 06 2022 at 21:52):

I wonder if it would be useful to make ability sets/bundles distinct from abilities?

jan kili (Mar 06 2022 at 21:52):

jan kili (Mar 06 2022 at 21:53):

Derek Gustafson (Mar 06 2022 at 21:56):

Richard Feldman (Mar 06 2022 at 21:58):

Num.add : a, a -> a | a can DoMath

Richard Feldman (Mar 06 2022 at 21:59):

» 1 + 1
2 : a | a can DoMath

Richard Feldman (Mar 06 2022 at 21:59):

» 0.1 + 0.2
0.3 : a | a can DoFracMath

Richard Feldman (Mar 06 2022 at 21:59):

(it'd be FracMath or something like that, instead of Float, because Dec isn't floating-point!)

Richard Feldman (Mar 06 2022 at 22:00):

Num.add : a, a -> a | a has Num

» 1 + 1
2 : a | a has Num

» 0.1 + 0.2
0.3 : a | a has Frac

jan kili (Mar 06 2022 at 22:01):

Richard Feldman (Mar 06 2022 at 22:01):

jan kili (Mar 06 2022 at 22:03):

jan kili (Mar 06 2022 at 22:05):

Richard Feldman (Mar 06 2022 at 22:08):

I dunno, I have a hard time imagining myself demoing Roc to someone who's not already on board and seeing their face when they put 1 + 1 into the repl and it prints 2 : a | a can DoMath

Richard Feldman (Mar 06 2022 at 22:09):

Richard Feldman (Mar 06 2022 at 22:10):

Derek Gustafson (Mar 06 2022 at 22:11):

remmah (Mar 06 2022 at 22:15):

I've mostly been skimming this thread, but as someone new to the language, my brain parsed a has Num as "The thing known as a has a number", not "The thing known as a has the functionality of a number". Not sure if that's helpful or not ^^;

Derek Gustafson (Mar 06 2022 at 22:17):

@remmah To better understand your thoughts, what's your experience with other ML languages like elm or Haskell?

Richard Feldman (Mar 06 2022 at 22:17):

I'm familiar with somewhere in the neighborhood of 20 programming languages, and off the top of my head, I think 100% of them refer to 1 as some variation of a "number" as opposed to referring to it by what functions it supports :sweat_smile:

remmah (Mar 06 2022 at 22:18):

The past few years, I've been mostly looking at (and writing a very modest amount of) Elm and F#. In the years before that, I got started with programming via Objective-C, which is... quite a bit different! It was actually a newsletter from longtime objc teacher Aaron Hillegass that first introduced me to Erlang and the notion of functional languages.

jan kili (Mar 06 2022 at 22:18):

I don't know what the syntax/implementation would be, but something like this would be nice

# in REPL

» 1 + 1
2 : Num

» Num
Num : a | a can [ DoMath, Encode, Decode, Hash, Sort ]

Folkert de Vries (Mar 06 2022 at 22:19):

Richard Feldman (Mar 06 2022 at 22:19):

I like that direction if we could figure out how to make it work somehow :thinking:

Richard Feldman (Mar 06 2022 at 22:20):

as in, integers and fractions still work the way they do today in terms of type inference

Richard Feldman (Mar 06 2022 at 22:20):

Richard Feldman (Mar 06 2022 at 22:21):

I guess the question is: is it possible to have those still be the types, and have them refer to abilities somehow?

Derek Gustafson (Mar 06 2022 at 22:21):

I feel like this is exactly what you're proposing with Abilities: moving away from a classification, like number, and towards what functions it supports.

Richard Feldman (Mar 06 2022 at 22:24):

Richard Feldman (Mar 06 2022 at 22:25):

because then you could just write (add (div 2 3) 4) and everyone would be like "yeah got it, no sweat"

Richard Feldman (Mar 06 2022 at 22:26):

Richard Feldman (Mar 06 2022 at 22:27):

(this is also why I want to make it possible for people to use abilities to define the infix operators for numeric operations - otherwise there's this really unfortunately large downside to using units of measure, even though they're really useful for preventing errors!)

Richard Feldman (Mar 06 2022 at 22:27):

I guess what I'm saying is that numbers are a special case because of the notation we're all taught, and I think they're an important enough special case to accommodate

jan kili (Mar 06 2022 at 22:32):

# ROC CHANGELOG

## v4.0.0
March 6, 2032
- Remove the `Num` crutch, to smooth the learning curve
- ...

Derek Gustafson (Mar 06 2022 at 22:34):

jan kili (Mar 06 2022 at 22:36):

Are numbers the only special case(s)? Can we look to other fundamentals for pattern insights?

Derek Gustafson (Mar 06 2022 at 22:38):

elm has 4 typeclasses: number, comparable, appendable, compappend
This turns into 3 abilities: DoMath, Compare, Append

Derek Gustafson (Mar 06 2022 at 22:40):

We could also look at what Haskell does, but there are lots of typeclasses, and lots of them are higher kinded. Makes the analysis more difficult

Brendan Hansknecht (Mar 06 2022 at 22:44):

There was a lot of discussion about grouping, but I think that can get really complicated. What if I have a number that I want to use in embedded where division is not supported or too slow? So I essentially make a version of float that intentionally doesn't have division to insure it is never used? The grouping of just DoMath wouldn't allow that.

Brendan Hansknecht (Mar 06 2022 at 22:45):

Also if we do a grouping called Num what about complex numbers or matrices. They aren't really numbers but they share most of the properties.

Derek Gustafson (Mar 06 2022 at 22:48):

Derek Gustafson (Mar 06 2022 at 22:51):

Brendan Hansknecht (Mar 06 2022 at 22:51):

Yeah probably, though I thought some operations may ruin that.... Modulus maybe?

Brendan Hansknecht (Mar 06 2022 at 22:53):

Derek Gustafson (Mar 06 2022 at 22:53):

Brendan Hansknecht (Mar 06 2022 at 22:53):

Brendan Hansknecht (Mar 06 2022 at 22:54):

I just in general feel that there will be a number of cases where a large grouping will have pieces that a type wouldn't want to implement despite otherwise fitting.

jan kili (Mar 06 2022 at 22:55):

(tangential questions, should Iterate be a built-in ability, and does that have any implications for List * changing alongside Num *?)

Derek Gustafson (Mar 06 2022 at 23:12):

Jared Cone (Mar 06 2022 at 23:22):

FWIW I've never done any real work in a lispy language but I would still prefer the above syntax despite its unfamiliarity. I dislike having to do double-takes on math expressions to make sure things are ordered correctly. I wonder how many others are in the same boat?

Jared Cone (Mar 06 2022 at 23:40):

From the doc: Hopefully the name "abilities" will frame the feature as giving a type a new ability and nothing more I was thinking instead of "Type X has Ability Y", it's more like "Ability Y supports Type X". It's a minor mental flip, but I think important to emphasize that this is not declaring what types are, but rather what you can do with them.

Coming from c++ it is nice being able to write a template function that requires "there must be a function in scope called "Equal" that accepts these two parameters and returns a bool". Would abilities be able to support something similar?

Richard Feldman (Mar 07 2022 at 00:18):

from what I hear, it's the second most common reason people decline to give Lisp a real shot - the main one being all the )))s

Richard Feldman (Mar 07 2022 at 00:20):

also apparently it's also considered a drawback to Lisp fans - Paul Graham is among the staunchest/loudest Lisp advocates in the programming world, and he's said "I've used Lisp my whole programming life and I still don't find prefix math expressions natural."

jan kili (Mar 07 2022 at 01:53):

No matter their syntax, I'm looking forward to experimenting with abilities for domain modeling :) thanks for adding every language feature with great care

Richard Feldman (Mar 07 2022 at 02:35):

a | a can Calculate could be a reasonable alternative to can DoMath :thinking:

Richard Feldman (Mar 07 2022 at 02:52):

Richard Feldman (Mar 07 2022 at 02:53):

e.g. can Equate, Sort, Encode, Decode, Calculate - those all read well together to me, even if they are a bit longer!

Richard Feldman (Mar 07 2022 at 02:53):

and if we can do Num a instead of a | a can Whatever, that's sufficiently concise anyway

Derek Gustafson (Mar 07 2022 at 03:12):

Yorye Nathan (Mar 07 2022 at 03:29):

I love can! so many times in C# I have an interface called ICanCombobulate or whatever, and it never felt right to even have that interface. But saying about something that it literally can Combobulate is just :kissing_cat:

Yorye Nathan (Mar 07 2022 at 03:29):

Brendan Hansknecht (Mar 07 2022 at 05:10):

Really like the name Calculate, but in one sense it is super generic. Like a lot of things are calculations that aren't basic math operations. Also, I still think Calculate should be some sort of alias ability that points to Add, Subtract, etc.

Yorye Nathan (Mar 07 2022 at 05:23):

Yorye Nathan (Mar 07 2022 at 05:27):

Richard Feldman (Mar 07 2022 at 15:45):

a thought: in something like List.sort : List elem -> List elem | elem can Sort it's not really that the element itself can sort, but rather that it can be sorted. Same with Equate - it's more like isEq : val, val -> Bool | val can be Equated

Richard Feldman (Mar 07 2022 at 15:45):

Folkert de Vries (Mar 07 2022 at 15:46):

Richard Feldman (Mar 07 2022 at 15:46):

Folkert de Vries (Mar 07 2022 at 15:47):

Richard Feldman (Mar 07 2022 at 15:49):

or maybe Ordering - since you can also compare for other things (e.g. equality)

Richard Feldman (Mar 07 2022 at 15:49):

jan kili (Mar 07 2022 at 15:51):

val impl Equatable
elem impl Orderable

Richard Feldman (Mar 07 2022 at 15:53):

jan kili (Mar 07 2022 at 15:54):

foo supports Baring is such a wonderfully casual way of explaining abilities in an alternative way

Richard Feldman (Mar 07 2022 at 15:56):

yeah I like that even if you don't know what abilities are, I think your odds of understanding the type signature at a glance are reasonable

jan kili (Mar 07 2022 at 15:57):

Then the ability definition keyword could be something like entails or requires

Ordering requires
    isGreaterThan : ...
    isLessThan : ...

jan kili (Mar 07 2022 at 15:58):

Brian Carroll (Mar 07 2022 at 17:18):

These all sound really nice and intuitive. But numbers are going to mess up the English grammar again I think! Aaagh!

Richard Feldman (Mar 07 2022 at 17:34):

I think supports Calculating is reasonable - keep in mind that won't appear in the Num APIs outside of the definitions for Num, Int, and Frac themselves

Richard Feldman (Mar 07 2022 at 17:34):

Richard Feldman (Mar 07 2022 at 17:35):

Brendan Hansknecht (Mar 07 2022 at 18:57):

So Int and Frac would be like Num, but they would support more operations, correct? What would be those different operations and will there naming be really odd since "Calculating" is already defined.

For example, would it be Int a : Num a | a supports Modulus # and other int specific stuff?

Brendan Hansknecht (Mar 07 2022 at 18:58):

Or i guess: Int a: a | a supports Calculating, Modulus. That just seems weird since it is clearly saying that modulus isn't part of "Calculating"

Richard Feldman (Mar 07 2022 at 18:59):

Richard Feldman (Mar 07 2022 at 19:00):

where Frac a is defined the same way as Num a but with the additional supports of CalculatingFractions

Richard Feldman (Mar 07 2022 at 19:01):

I'm not sold on the upsides of being granular on operations (e.g. Modulus, Add, etc.) outweighing the downsides

Brendan Hansknecht (Mar 07 2022 at 19:01):

I think this points to my exact issue with a large group like Calculating. What exactly does it contain? How is it different from CalculatingInt. It is extremely non obvious when you see someone use Calculating vs use CalculatingInt.

Richard Feldman (Mar 07 2022 at 19:01):

Richard Feldman (Mar 07 2022 at 19:02):

or, put another way: literally when would you see someone using them? :big_smile:

Brendan Hansknecht (Mar 07 2022 at 19:03):

As a side question, can abilities be nested? As in CalculatingInt being defined as Calculating plus some extra operations.

Brendan Hansknecht (Mar 07 2022 at 19:05):

Matrices, vectors, complex numbers, potentially custom measurement types.
If no one is going to use Calculating, why do we want to add it? I would say that it shouldn't be defined at all if it isn't expected to be used in the language.

Richard Feldman (Mar 07 2022 at 19:06):

Richard Feldman (Mar 07 2022 at 19:07):

but that should be like 0.001% of the population of Roc programmers over time :big_smile:

Richard Feldman (Mar 07 2022 at 19:07):

so I think it's much more important that the way most people interact with it (Num and Int) is really intuitive

jan kili (Mar 07 2022 at 19:08):

Would you? Or would you just implement the ability's requirements in the module and code would automatically work?

Richard Feldman (Mar 07 2022 at 19:08):

Brendan Hansknecht (Mar 07 2022 at 19:09):

But if most people don't see the underlying Calculating ability, why does it matter if it is one Calculating ability instead of many specific abilities?

Richard Feldman (Mar 07 2022 at 19:09):

Richard Feldman (Mar 07 2022 at 19:10):

because + desugars to Num.add, which requires Num a, which requires more abilities than just Adding

Richard Feldman (Mar 07 2022 at 19:11):

so in order for Adding to be useful as a separate thing, it would need to be Num.add : a, a -> a | a supports Adding instead

Richard Feldman (Mar 07 2022 at 19:11):

Brendan Hansknecht (Mar 07 2022 at 19:12):

I guess I wasn't thinking about how this could break type signitures. Though arguably, the supports Adding type signiture is more correct due to taking the most narrow interface.

Brendan Hansknecht (Mar 07 2022 at 19:12):

Though even if that is the underlying type, can't we print out the type that we see at the call site?

Brendan Hansknecht (Mar 07 2022 at 19:13):

So if somone calls Num.add with a Num a we print it out as Num.add : Num a, Num a -> Num a in any related error messages?

Richard Feldman (Mar 07 2022 at 19:13):

Richard Feldman (Mar 07 2022 at 19:14):

separately, something I'd like to explore more specifically is the difference between matrices, vectors, and units of measure

Richard Feldman (Mar 07 2022 at 19:15):

Brendan Hansknecht (Mar 07 2022 at 19:15):

I think the big one will be supported operations especially when it comes to things like multiply vs dot product.

Richard Feldman (Mar 07 2022 at 19:15):

Richard Feldman (Mar 07 2022 at 19:16):

like a + (b * c) - z is a thing that comes up reasonably often in normal arithmetic

Richard Feldman (Mar 07 2022 at 19:16):

how often do complex nested infix operator expressions come up in vector and matrix math?

Richard Feldman (Mar 07 2022 at 19:17):

put another way, how actually bad is it if there's like a Vec2.mul or Vec2.dot and that's the way you do multiplication and dot products of vectors?

Brendan Hansknecht (Mar 07 2022 at 19:17):

Richard Feldman (Mar 07 2022 at 19:17):

and then the other question is: how bad is it if matrices and vectors need to implement operations that don't really make sense for them, like sqrt?

Richard Feldman (Mar 07 2022 at 19:18):

Brendan Hansknecht (Mar 07 2022 at 19:18):

For example if you look at a lot of ML papers, they will write the code in matrix math notation and that can be translated directly to the regular math notation in a programming language, just with the use of matricies instead of regular numbers.

Brendan Hansknecht (Mar 07 2022 at 19:20):

As for operations, some will be a pain to implement, others may not exactly be well defined, and the most likely case, some of the operations are just horridly slow if they ever get used.

Richard Feldman (Mar 07 2022 at 19:21):

Brendan Hansknecht (Mar 07 2022 at 19:21):

I think it wouln't be terrible to use Vec2.mul and things of that nature, but you would have to implement your own abilities if you wanted to make your own generic functions that do math on vectors and matricies and numbers, instead of just using the existing ones.

Brendan Hansknecht (Mar 07 2022 at 19:23):

Would have to make a wrapper type for num otherwise you wouldn't be able to add your own abilities to it.

Brendan Hansknecht (Mar 07 2022 at 19:25):

So like if I have a generic math function that just takes addition, multiplication, and sqrt. It would either require wrapping the num type with my custom abilities that my matrices already use, or it would require making a special version of the function for the num type.

Richard Feldman (Mar 07 2022 at 19:25):

Richard Feldman (Mar 07 2022 at 19:26):

why not have it implement Calculating so that *, -, +, etc. all work on it?

Brendan Hansknecht (Mar 07 2022 at 19:27):

That is a possibility, but I think that Calculating will likely support operations that don't make sense or can't be implemented on a matrix.

Richard Feldman (Mar 07 2022 at 19:28):

like I get that ideally I wouldn't need to implement operations that don't make sense on it

Brendan Hansknecht (Mar 07 2022 at 19:28):

What do you mean? If I can't implement one of the methods, how do I implement Calculating?

Richard Feldman (Mar 07 2022 at 19:28):

Richard Feldman (Mar 07 2022 at 19:29):

Brendan Hansknecht (Mar 07 2022 at 19:36):

Lets go with sqrt. It is only possible to calculate the square root of a NxN matrix. So for any other matrix, sqrt has no meaning.

Brendan Hansknecht (Mar 07 2022 at 19:37):

Most likely, sqrt would not use supports SquareRooting because it would require a special method that returns an error when the matrix isn't square

Derek Gustafson (Mar 07 2022 at 19:43):

All of the matrix operations require the dimensions make sense. I would be surprised if there was a way to type check that without doing the peano number type level shenanigans

Brendan Hansknecht (Mar 07 2022 at 19:59):

Haha. I guess I somehow didn't think about that. I guess Calculating and matrix may not work at all due to sizing errors.

Derek Gustafson (Mar 07 2022 at 20:08):

Richard Feldman (Mar 07 2022 at 20:10):

Richard Feldman (Mar 07 2022 at 20:11):

Richard Feldman (Mar 07 2022 at 20:12):

you could do something like Matrix.new3x3 : (some args go here) -> Matrix [ N3 ] [ N3 ]

Richard Feldman (Mar 07 2022 at 20:12):

Richard Feldman (Mar 07 2022 at 20:13):

Richard Feldman (Mar 07 2022 at 20:16):

Richard Feldman (Mar 07 2022 at 20:18):

Derek Gustafson (Mar 07 2022 at 20:18):

Richard Feldman (Mar 07 2022 at 20:19):

are there any matrix or vector operations where we'd want to support an infix operator, and also either of these is true?

Derek Gustafson (Mar 07 2022 at 20:21):

Derek Gustafson (Mar 07 2022 at 20:22):

I think an infix * would be nice...
But I think not doing that would probably be better.

Richard Feldman (Mar 07 2022 at 20:22):

Derek Gustafson (Mar 07 2022 at 20:22):

Richard Feldman (Mar 07 2022 at 20:23):

Derek Gustafson (Mar 07 2022 at 20:28):

Richard Feldman (Mar 07 2022 at 20:28):

Richard Feldman (Mar 07 2022 at 20:29):

Derek Gustafson (Mar 07 2022 at 20:30):

Richard Feldman (Mar 07 2022 at 20:31):

Richard Feldman (Mar 07 2022 at 20:33):

Richard Feldman (Mar 07 2022 at 20:34):

like another option is to have a Matrix.invalid type that works kinda like Infinity in floats

Richard Feldman (Mar 07 2022 at 20:35):

but now you have "poison value" semantics to deal with, defensive programming, etc...

Derek Gustafson (Mar 07 2022 at 20:36):

And you end up fighting the compiler: "I know the dimensions work! Just compile!"

Richard Feldman (Mar 07 2022 at 20:38):

Derek Gustafson (Mar 07 2022 at 20:38):

Richard Feldman (Mar 07 2022 at 20:38):

Richard Feldman (Mar 07 2022 at 20:39):

but it's still strictly more annoying compared to not having to deal with it :big_smile:

Derek Gustafson (Mar 07 2022 at 20:39):

Richard Feldman (Mar 07 2022 at 20:40):

Richard Feldman (Mar 07 2022 at 20:41):

I've generally classified overflows along with "out of memory" problems (e.g. making a List that's so big its length overflows)

Richard Feldman (Mar 07 2022 at 20:41):

in that it's probably not worth it in the general case to demand that people program defensively around it

Richard Feldman (Mar 07 2022 at 20:42):

whereas we do have division returning Result because I do think being defensive about division by 0 is a good default

Richard Feldman (Mar 07 2022 at 20:46):

(but I'm open to being wrong about that one...one nice thing about trying out Result is that if it's too annoying, it's easy enough to switch it to panic, but not so much the other way around)

Brendan Hansknecht (Mar 07 2022 at 20:46):

I guess with matrix the issue is that you would probably want to be able to pick when to panic and when to use result, but roc wouldn't give you control over that.

I guess with my current knowledge of roc, I would implement matrix as a result type itself that would enable nice chaining but still record and propogate errors. But that would also make it hard to track down size mismatches though. They would just have happened somewhere in the chain or operations.

Richard Feldman (Mar 07 2022 at 20:48):

Richard Feldman (Mar 07 2022 at 20:49):

potential idea to throw out there: I wonder if having a Matrix builtin could make sense 🤨

Richard Feldman (Mar 07 2022 at 20:49):

Derek Gustafson (Mar 07 2022 at 20:51):

My inclination is to default to "implement in Roc" and only move it to rust/llvm/zig when there's a clear reason

Richard Feldman (Mar 07 2022 at 20:51):

Jared Cone (Mar 07 2022 at 22:41):

Just curious, why can't operator overloading be a thing? Namespace the operator and the compiler figures out which definition to use based on the arguments? Spitballing, but something like

interface Matrix
...
+ = \a, b -> Matrix.add a b

interface Vector
...
+ = \a, b -> Vector.add a b

a = Vector.new 1 2 3
b = Vector.new 4 5 6
c = a + b # Compiler knows to use Vector.+ instead of Matrix.+

Richard Feldman (Mar 07 2022 at 22:45):

I generally agree with James Gosling's perspective when he decided to consciously exclude operator overloading from Java based on having seen how it was used in C++

Jared Cone (Mar 07 2022 at 22:48):

Gotcha, but in that case how is Matrix implementing the Calculate ability to let it work with the + operator different from overloading?

Richard Feldman (Mar 07 2022 at 22:49):

I think in the case of Matrix and infix operators, my thinking based on the discussion so far is that we shouldn't support that use case :big_smile:

Jared Cone (Mar 07 2022 at 22:49):

Richard Feldman (Mar 07 2022 at 22:50):

but with things like units of measure, fractions, and arbitrary-sized integers (I'm told there are use cases where i128's undecillions aren't big enough), I think it's still worth it to support!

Jared Cone (Mar 07 2022 at 22:57):

re: Ability shouldn't be used by most people - as soon as you give someone the "ability" to do something (pun intended), they will most certainly abuse it. However people also dislike it when the built-ins are allowed to do things that custom code cannot. Tough line to tread. It's interesting getting a glimpse of some decisions that may have crazy ramifications later :)

Richard Feldman (Mar 07 2022 at 23:07):

yeah I think in the case of abilities there are a relatively small number of disproportionately valuable use cases that justify all the times the feature will be (as I see it) misused

Richard Feldman (Mar 07 2022 at 23:08):

but I'll still try to minimize the damage by trying to encourage a culture where they're rarely used :big_smile:

Brian Carroll (Mar 08 2022 at 16:52):

Wow this conversation got really long really fast but I just thought of something...
supports Arithmetic actually sounds pretty good

Derek Gustafson (Mar 08 2022 at 16:53):

I like how it sounds. I think it supports the idea of abilities instead of classification.

Derek Gustafson (Mar 08 2022 at 16:54):

Folkert de Vries (Mar 08 2022 at 16:54):

Folkert de Vries (Mar 08 2022 at 16:55):

Derek Gustafson (Mar 08 2022 at 16:55):

Richard Feldman (Mar 08 2022 at 18:14):

yeah can Arithmetic wouldn't have worked, but supports Arithmetic sounds nice!

jan kili (Mar 08 2022 at 18:56):

Kevin Gillette (Mar 13 2022 at 17:05):

@Richard Feldman Abilities in Roc mentions tuples in the Default Abilities section. Is that worth rephrasing in terms of records to avoid confusion, since Roc does not have tuples?

Also, it seems like _almost everything_ in the language has equality defined (thus nearly all records and lists would also have equality defined)? I don't know what all the caveats are, but it would be quite powerful to have a language in which any value can be compared for equality with any other value of the same type, thus removing the need for an Eq ability.

Are functions the only exception to this? It seems that we could merely tag functions which are compared for equality with some internal identifier, and a function is semantically equal to another function if they both refer to the same source location and any values they close over (including other functions) are also equal.

Zeljko Nesic (Mar 13 2022 at 17:10):

You can make two functions on the fly, and there is no way to say that they are the same, without inspecting their underlying expression.

main =
  startFn = \ a , b , c -> (add (div b c) a)
  fnA = \ a , b -> startFn 42 a b
  fnB = \ x , z -> startFn 42 x z

  if fnA == fnB then
    launchMissles
  else
   drinkTea

But since equality check is done during runtime, we don't have a way to evaluate that unless we carry AST at the run time ...

Ayaz Hafiz (Mar 13 2022 at 17:21):

The intention is indeed to define structural equality automatically for every type that can support it. So there will be equality automatically derived for all records, tags, etc. There is still a value to an Eq ability for defining custom equality over opaque types; since those are exposed only by name and not by structure, you may want to define equality in a different way for those custom types.

I would be worried about trying to define equality for functions. It’s undecidable in general and I think there would be too many false negatives for it to be useful. Also, given the false negatives, defining equality for functions would break referential transparency - for example

f = \x->x
f == f

Brendan Hansknecht (Mar 13 2022 at 17:34):

I think custom equality is mostly important for data structures. A simple example is the unordered hash map I have been working on. Definitely don't want tombstones, random seed, or capacity to be part of equality checking.

Richard Feldman (Mar 13 2022 at 18:39):

Kevin Gillette (Mar 14 2022 at 00:49):

Presumably you'd be able to get it back out via iteration (although unless it's the _only_ function, you wouldn't be able to distinguish the functions from each other, except perhaps by evaluating them), but in any case, I see your point.

Brendan Hansknecht (Mar 15 2022 at 22:46):

Pulling this to the abilities chat because my question is specific to abilities.
Don't we want to enable users to add abilities to builtin types. For example the MyCoolPrettyStringifier ability. It turns a type to a string, but does more formatting and such based on a struct that is passed in. It would enable users to define the params they want and than this ability on their own types. The library would define it on all builtins.

Ayaz Hafiz (Mar 15 2022 at 23:45):

it’s tough to say, there are a lot of trade offs both ways. Rust had a long discussion about this regarding orphan rules (ghc (Haskell) has too, the specification admits orphans but ghc gives a warning for it bc it’s error prone). I’m going to read through that before trying to form an opinion

Brendan Hansknecht (Mar 15 2022 at 23:49):

Then you have go where interfaces are implicit. If a type happens to have the right methods, you don't even have to add the interface to the type. That being said, you can't add methods to a type defined in another package. So you wouldn't be able add a method to a builtin type.

Tommy Graves (Mar 15 2022 at 23:53):

Ayaz Hafiz (Mar 15 2022 at 23:57):

well the intention is for interface implementations to be implicit. so if there are conflicting implementations you can do little more than sigh and give up which is not good because then you can never import two modules that implement an ability for the same type different

Richard Feldman (Mar 16 2022 at 00:03):

Richard Feldman (Mar 16 2022 at 00:04):

so let's say I wrote an ability called DoStuff, and I publish it in a package called stuff.

Richard Feldman (Mar 16 2022 at 00:04):

also in that package, I define that the Foo type - from another package, foo - supports my DoStuff ability

Richard Feldman (Mar 16 2022 at 00:05):

Richard Feldman (Mar 16 2022 at 00:06):

now let's say I'm the author of the Foo type and I'm like "oh I want to add a DoStuff ability implementation to my Foo type, because I'm going to depend on DoStuff and do some cool stuff with it"

Richard Feldman (Mar 16 2022 at 00:06):

well, tough luck - I can't do any of that now, because the stuff package depends on my foo package, so if I even try to add a stuff dependency, we'll have a cyclic import

Richard Feldman (Mar 16 2022 at 00:06):

Richard Feldman (Mar 16 2022 at 00:07):

so I have to tell the stuff package author "hey, stop implementing DoStuff for Foo and drop my package as a dependency of yours, then publish a major release (because this would necessarily be a breaking change), so that I can then depend on your package and publish a release of my own"

Richard Feldman (Mar 16 2022 at 00:08):

so basically, in order to resolve this scenario, the two package authors have to get together and coordinate

Richard Feldman (Mar 16 2022 at 00:08):

now consider the other scenario, where implementing abilities for imported types isn't supported

Richard Feldman (Mar 16 2022 at 00:09):

I want Foo to support my DoStuff ability, so my only option is to coordinate with the author of the Foo package directly

Richard Feldman (Mar 16 2022 at 00:09):

now they may or may not want to do that, or they may implement it in a different way than I would have, which are arguments for supporting the feature

Richard Feldman (Mar 16 2022 at 00:10):

but when I look at these two scenarios, I definitely don't think it's good for the person who created the type to be unable to control what abilities it implements

Richard Feldman (Mar 16 2022 at 00:11):

and if they want to add a new ability, for them to be the person trying to convince someone else to do a breaking change of their package just to facilitate that

Richard Feldman (Mar 16 2022 at 00:11):

Richard Feldman (Mar 16 2022 at 00:12):

this is a problem with arbitrary orphan instances, e.g. "in my Foo module, which defines neither the Blah type nor the DoStuff ability, I am going to declare that the Blah type now supports the DoStuff ability"

Richard Feldman (Mar 16 2022 at 00:13):

there's a design where you say "you can only assign orphan instances in one of two places: either where the opaque type is defined, or where the ability is defined, and that's it"

this doesn't have the "multiple conflicting instances" problem because if the declaration site for the opaque type imports the ability's module in order to specify it, then that means the ability's module can no longer import the opaque type (in order to specify its own implementation) because that would be a cyclic import.

and that's true regardless of which one "wins" - they can't even both attempt to specify it without causing a cyclic import, meaning in that design "multiple conflicting instances" is impossible because cyclic imports are disallowed

Brendan Hansknecht (Mar 16 2022 at 00:13):

Or just add a wrapper type and call it a day because coordination is too much effort. I feel like that is much more likely.

Brendan Hansknecht (Mar 16 2022 at 00:15):

As a note, this is extra problematic for builtins. Adding a new ability to a builtin would need to be a compiler update. That is much more effort than anything you listed above.

Richard Feldman (Mar 16 2022 at 00:16):

so, to back up a step, I want to revisit the original motivation for Abilities in Roc

Brendan Hansknecht (Mar 16 2022 at 00:17):

I hope so, but it depends how generic encoding is. Would need to relook at the design.

Richard Feldman (Mar 16 2022 at 00:18):

the main motivations had to do with function equality (prior to abilities there was the "functionless constraint"), custom equality so people can make custom data structures, and custom numeric types like units of measure and Complex being able to use arithmetic operators. As a bonus, I think encoding and decoding are super promising, but the jury's still out on those.

Brendan Hansknecht (Mar 16 2022 at 00:19):

"function equality" -> equality applied to functions or the ability to override ==?

Richard Feldman (Mar 16 2022 at 00:19):

my main fear is that people use them to introduce a lot of complexity that doesn't justify its benefits; that the Roc package ecosystem becomes more needlessly complex than it would have been if Abilities weren't in the language

Richard Feldman (Mar 16 2022 at 00:19):

Richard Feldman (Mar 16 2022 at 00:20):

to be honest, I'm not 100% certain it's the right design to support defining custom Abilities in user space

Richard Feldman (Mar 16 2022 at 00:20):

I think it's probably more good than bad, overall, I definitely don't think it's a slam dunk

Richard Feldman (Mar 16 2022 at 00:20):

like Elm doesn't support that, and Elm has the nicest package ecosystem I've ever used

Richard Feldman (Mar 16 2022 at 00:21):

so to me, "let's improve the ergonomics of user-defined Abilities so that they can be reached for in more situations, and used more, and become more prevalent in the ecosystem" is by default a red flag to me

Richard Feldman (Mar 16 2022 at 00:22):

by default that's under the "things I'm worried about as downsides of having Abilities in the language" umbrella

Richard Feldman (Mar 16 2022 at 00:22):

more that I very much want to set a high bar for facilitating more user-defined Abilities

Richard Feldman (Mar 16 2022 at 00:23):

so I would want to be talking about specific use cases that we agree are good ideas that are worth encouraging

Richard Feldman (Mar 16 2022 at 00:23):

as opposed to general like "it would be good to support this," if that makes sense!

Brendan Hansknecht (Mar 16 2022 at 00:23):

Richard Feldman (Mar 16 2022 at 00:24):

maybe a better way of saying all that is "I'd like Abilities to be minimally powerful such that they achieve their goals and that their downsides don't materialize, and be very cautious about giving them more power than necessary because of what that might do to the ecosystem" :big_smile:

Richard Feldman (Mar 16 2022 at 00:25):

it's impossible to define a Monad or Functor ability, but Monoid can be defined because it doesn't require HKP

Richard Feldman (Mar 16 2022 at 00:26):

because of that, I think it is 100% safe to assume that someone will eventually publish a package with a Monoid ability, and close to 100% that they encourage people to use it

Richard Feldman (Mar 16 2022 at 00:26):

so, if that's true, is it good for the Monoid ability to be able to be retroactively applied to builtins?

Richard Feldman (Mar 16 2022 at 00:28):

Zeljko Nesic (Mar 16 2022 at 01:46):

Zeljko Nesic (Mar 16 2022 at 01:47):

Derek Gustafson (Mar 16 2022 at 02:21):

I'm not familiar with F#, other than ML in .net, but he really seems to be responding to higher kinded types.

Derek Gustafson (Mar 16 2022 at 02:23):

And I see what he means, other than I'm not sure how much category theory actually helps when doing type level Haskell

Brendan Hansknecht (Mar 16 2022 at 02:23):

It seems to suggest that F# will/partially already does have overloading of thing like +, -, etc.

Richard Feldman (Mar 20 2022 at 03:17):

interesting observation: if you could add abilities to builtins, then you could do something like this:

readBytes : PathLike * -> Task (List U8) (FileReadErr *)

and then call readBytes passing either an opaque Path or a Str, so you could call it with like "foo.txt" or myPath with myPath having the type Path.

this would be assuming PathLike a was a type alias for an a which supports an ability that has an a -> Path function. (Rust uses a similar pattern with traits, e.g. in functions like File::open)

Richard Feldman (Mar 20 2022 at 03:18):

that would be pretty convenient for opaque wrappers around primitive types - e.g. paths and URLs

Richard Feldman (Mar 20 2022 at 03:18):

because today there's tension between calling ergonomics and wanting to use opaque types to prevent things from getting mixed up

Richard Feldman (Mar 20 2022 at 03:19):

e.g. if you have Http.get take a Str then you can do Http.get "/foo" but can't have an opaque Url type that works with it

Richard Feldman (Mar 20 2022 at 03:19):

and if you have it take an opaque Url type, then you'd have to call it with something like Http.get (Url.from "/foo")

Richard Feldman (Mar 20 2022 at 03:19):

Richard Feldman (Mar 20 2022 at 03:23):

Richard Feldman (Mar 20 2022 at 03:25):

some languages would do this with union types but they have some massive downsides when it comes to type-checker performance, error message helpfulness, and (I think but am not sure) decidable principal type inference.

Richard Feldman (Mar 20 2022 at 03:26):

of course a simpler solution is to use a type alias like Url : Str and don't make it opaque at all

Richard Feldman (Mar 20 2022 at 03:27):

so then you miss out on some type safety, but the type is self-documenting and actually nicer (Http.get : Url -> ... instead of Http.get : UrlLike * -> ...), and of course you can just give it string literals.

Martin Stewart (Mar 20 2022 at 08:39):

In your Path and Url example, wouldn't converting Str to one of those return a Result since Str might not be a valid Path or a valid Url?

Richard Feldman (Mar 20 2022 at 13:11):

oh I don't think we'd want to eagerly validate either of those types in general.

Both of them get used in operations that can fail in a bunch of ways, only one of which is that the string is invalid.

So it's not like knowing it's valid up front gives you a really valuable guarantee you can rely on from then on. It would mean instead of Http.get having like 20 failure cases it would only have 19 because "malformed URL" had been handled earlier.

But considering that, wouldn't it be nicer to handle all 20 at the same time? If the URL comes from user input, I could see wanting to give them feedback earlier about it being invalid, but all that requires is a separate function to check it!

Kevin Gillette (Mar 20 2022 at 15:04):

It sounds reasonable to have URL validation happen by default at request time (as that's considerably more convenient in the common case).

That said, I believe it should still also be provided as a separate check (one which doesn't need to pass through a platform boundary) for applications that need it, for example to best-effort validate a URL before returning 202 Accepted (at which point performing the request as a means of validation is highly undesirable.

Other use cases for explicit URL validation are producing a list of URLs to store in a file for later; unit tests; and a program that accepts a list of URLs but needs to exit before calling any of them if any seem malformed.

Zeljko Nesic (Mar 20 2022 at 16:04):

How would we the go about the scenario where I have (eg.) ColorLike ability and I would like to add that ability to Str?

Martin Stewart (Mar 20 2022 at 16:10):

I think it would make sense to have a Http.getWithStr : Str -> ... and also have a Http.getWithUrl : Url -> ... since as you mentioned, using a Str directly is a common use case. But it should not be possible to create a Url directly from Str without getting a Result. If you skip validation then you end up with Results in all your helper functions, i.e. Url.getProtocol : Url -> Result Str [ InvalidUrl ]. Also as Kevin mentioned, in some cases you want to store the urls for later and get type safety around the fact that they really are urls.

Richard Feldman (Mar 20 2022 at 16:38):

Richard Feldman (Mar 20 2022 at 16:45):

sure - but at least in the case of URLs, I think almost all of the benefit there is just having an opaque type at all, so you can't (for example) accidentally use a Url instead of an Email or something like that. :big_smile:

Martin Stewart (Mar 20 2022 at 16:53):

I think there's a lot of value in being certain that when you see a Url, Path, Email, etc you know it is valid because the only way to create it is via a Url.fromString : Str -> Result Url [ InvalidUrl ]. That said, I agree opaque types are also useful for preventing mixing up Url data with Email data.

Edit: I just noticed you weren't speaking generally but only for Urls. I still think it's valuable to know for certain it's a valid Url but if it's maybe only Urls that can be created without validation then I don't think it's too big of an issue.

Richard Feldman (Mar 20 2022 at 17:02):

yeah, maybe the better question to focus on here is "let's assume you've decided you want to go from a Str directly to this opaque type Foo, what do we think of the FooLike *design? Seem good?"

Kevin Gillette (Mar 20 2022 at 18:03):

I'm not terribly convinced about FooLike if we mean it's an ability which Str and other types know about. imo, Str has absolutely no business knowing how to validate itself into a URL (and really, a Str module ideally shouldn't know how to encode "from" a Num, since that's a slippery slope in which the only way to avoid inversions of responsibility is by having Num and other modules know about Str representations, while Str knows about none of those other types).

If we had UrlLike, which is implemented by Url and Str, that seems like it is just inventing unnecessary complexity (akin to modeling geometry in OOP and debating whether a square is a subclass of rectangle, or the other way around). The language already has good tools for UrlLike, which is a regular function that returns a result (Url.fromString as others have mentioned). This does suggest to me that Richard's concern about user-definable abilities is well founded: perhaps we add it to the language as a way to self-host definitions, and as a syntax for documenting the builtin abilities, yet we don't actually enable app/module code to introduce their own abilities until we've figured out how to introduce good guardrails.

Kevin Gillette (Mar 20 2022 at 18:09):

I do like the notion of opaque types to guarantee correctness, such as Url. At that point though it's a judgement call as to whether Http.get takes a Str or a Url

Zeljko Nesic (Mar 20 2022 at 18:39):

The more I think about abilities, I get the sense that they should not be part of the language. They bring heck of a confusion in my head, not sure about the others.

There is some do be said about types adopting certain abilities, but maybe, that should just be the boilerplate code that editor can help with?

Kevin Gillette (Mar 20 2022 at 19:20):

I think some abilities are pretty straightforward in value and without major caveats (that I can see), such as Equating, Ordering.

Others I believe have great potential value, such as stringifying a value or encoding, but have caveats:

Then there are abilities that I see little value in (that detract away from the exercise of sound engineering), such as overly-abstracted abilities defined within an application solely for use by that same application (the Roc equivalent of duck-typing taken to the extreme of defining all coupling/integration points in terms of ducks/abilities).

Richard Feldman (Mar 20 2022 at 19:27):

here's an alternative idea, which could accomplish approximately the same thing.

So let's say there are no abilities involved, and we have Url.fromStr : Str -> Url. So then when calling Http.get : Url -> Task (List U8) [ HttpErr ]* I would do something like:

bytes <- Http.get (Url.fromStr "/blah")

    fromStr : Str -> a | a supports FromStr

then Url could implement the FromStr ability using the fromStr : Str -> Url function it's already exposing.

now we introduce one more trick, which is a unary prefix operator (like the ! and - unary prefix operators we have) which desugars to a call to the Str.fromStr : a -> Str | a supports FromStr function from the FromStr ability. Let's say the prefix operator is $ for the sake of argument (but could be lots of things).

bytes <- Http.get (Url.fromStr "/blah")

bytes <- Http.get (Str.fromStr "/blah")

bytes <- Http.get $"/blah"

Richard Feldman (Mar 20 2022 at 19:29):

Brian Carroll (Mar 20 2022 at 19:47):

@Zeljko Nesic I can empathise with your cautious feelings on Abilities, I'm feeling some of that too! I really like that Roc avoids unnecessary abstraction, and that's what we have to be careful of with Abilities!
The FromStr API here feels like it's on the wrong side of the line for me. I would rather the library author just picked either Str or Url, rather than abstracting over the two. If it needs to be wrapped in a tag, I'll happily take the extra keystrokes to explicitly wrap it in a tag. If it takes a string and I have a wrapped thingy, I'll happily do the "from string" unwrapping myself. This abstraction doesn't feel like it meets a high enough bar to be worth it.

Brendan Hansknecht (Mar 20 2022 at 23:13):

I am a little lost, why do want FooLike at all? Or to hide the Url.fromStr? A Url is not a Str. As such, explicit conversion is a feature of the language. If we want implicit conversion, just don't make it an opaque type. Also, that conversion should most likely require a Result, but if a library authors wants to group the url checking into other calls that is also fine.

What is the issue with writing bytes <- Http.get (Url.fromStr "/blah")? That seems like clean, accurate, and honestly not too long of Roc code.

Kevin Gillette (Mar 21 2022 at 00:47):

In practice, while I'd expect a ToStr ability to have a toStr : a -> Str | a supports ToStr implementing function, a FromStr would probably need to return a Result. Serializing to a string is safe and will generally only fail in out-of-memory conditions, but deserializing from a string (parsing) will fail, often, for reasons that are often not the programmer's fault.

Thus it seems a bit strange the signature (after de-generalizing any abilities) essentially become: Http.get (Result Url e) -> Task ...

If we did actually specify FromStr as Str -> a, then opaque types would be little more than type aliases, since there'd be a means to produce a Url (or any other implementing type) without doing any input validation (thus bypassing, at least afaict, the primary benefit of opaque types).

Richard Feldman (Mar 21 2022 at 02:13):

Richard Feldman (Mar 21 2022 at 02:14):

I think I should back up and explain why I think this is an interesting topic to explore

Richard Feldman (Mar 21 2022 at 02:14):

Tommy Graves (Mar 21 2022 at 02:15):

You can still do this by just requiring the values to be tagged, right? Http.get (Str "/foo") and Http.get (Url url)

Richard Feldman (Mar 21 2022 at 02:20):

so I think the 4 main strategies for representing a URL or file system path would be:

Richard Feldman (Mar 21 2022 at 02:25):

I agree that bytes <- Http.get (Url.fromStr "/foo") is fine, but the Url.fromStr call really isn't preventing any bugs in this code snippet. I'm specifying exactly what I want in a string literal and then giving that directly to the function - the opaque type's ability to give me a type mismatch could only really help me here I were giving it the wrong string.

For example, if I accidentally did Http.get username (maybe I should have written Http.get "/users/\(username)" or something) then having Http.get take an opaque type for its first argument could help me out, because Http.get username would be a type mismatch (unless somehow username had the type Url for some reason).

Richard Feldman (Mar 21 2022 at 02:26):

but in the common case where I'm specifying the URL using a string literal, the opaque type doesn't help, it's just an extra call to make that doesn't really add value

Richard Feldman (Mar 21 2022 at 02:27):

to be honest, I don't think Http.get taking a Str is particularly error-prone. Elm has it take a string and it's been totally fine...I can't recall ever getting a bug from that - perhaps in part because if I did pass it the wrong string, I'd find out the first time I tried it because the URL would almost certainly not point to a valid endpoint, and the request would fail. :laughing:

Brendan Hansknecht (Mar 21 2022 at 02:29):

On thing that is made me realize that may not matter too much for this case but could generally be nice.

Having separate Http.getStr and Http.getUrl means that you can tailor the return error union of each. As such, passing a url would remove certain errors from the return type. In this cause, I think that would be quite minor, but in other cases it might be more useful.

Richard Feldman (Mar 21 2022 at 02:30):

but I was thinking about how Rust has APIs like File::open which let you pass either an opaque Path type or something that can become one - in their example they could do either of these:

f = File::open("foo.txt")

f = File::open(Path::new("foo.txt"))

Richard Feldman (Mar 21 2022 at 02:31):

now personally, the amount of complexity I'm willing to pay for a little extra conciseness is very low

Richard Feldman (Mar 21 2022 at 02:31):

Brendan Hansknecht (Mar 21 2022 at 02:31):

I think that is more a comment about how Url.fromStr should return a result. This will likely make people want to construct them differently instead of locally for every call to get.

Richard Feldman (Mar 21 2022 at 02:31):

so I wanted to explore the idea of "what would it look like if approached this API design question the way the Rust stdlib does it?"

Brendan Hansknecht (Mar 21 2022 at 02:33):

Definitely an interesting premise. I feel I need to look at more concrete types, especially when thinking about the abilities that would be used. If someone can't add abilities to builtins, anything we add to support this would likely be kinda inflexible.

Richard Feldman (Mar 21 2022 at 02:36):

well I think the FromStr idea is extremely lightweight. Like let's say I decided to make Url opaque but not validated (like Rust's Path, which - independently from any of these considerations - is what I think is the best design, but I guess that's a separate topic!)

bytes <- Http.get (Url.fromStr "/foo")

the proposed FromStr idea is that literally I write one line of code differently: when defining my Url opaque type, I add this:

supports FromStr { from Str }

bytes <- Http.get $"/foo"

Richard Feldman (Mar 21 2022 at 02:38):

Richard Feldman (Mar 21 2022 at 02:40):

anyway, I'm not saying this is some super critical feature or anything, but I think it's interesting to think about the design space

Kevin Gillette (Mar 21 2022 at 02:41):

So $"/path/to/endpoint" sounds like it relates to point 3. Opaque types, even those which do not validate their input are at minimum a use of nominal typing in order to avoid "very expensive mistakes," yet any kind of sugaring that makes it trivial to convert a string into that opaque type mitigates much of that protection.

Consider if we applied this to filesystem paths. You might have: fromStr : Str -> Path, and have readFile : Path -> Task (List U8) [...]*. If I see code implementing some content management system that takes user input and resolves that into a filesystem path, then the following would stick out as pretty problematic:

readFile (Path.assumeSafe userInput)

(pretend userInput is "/etc/passwd" for example). However, the following would hide some of that unsafety:

readFile $userInput

(pretend Path.assumeSafe is the function used to implement the ability that $ desugars to for Path values).

Richard Feldman (Mar 21 2022 at 02:46):

Kevin Gillette (Mar 21 2022 at 02:56):

This could be addressed with the compiler permitting some things at compile time that cannot be done as easily at runtime. Rust does this with "constant functions" and macros and such.

Consider any function that returns a Result. Roc could perhaps permit an a -> Result b e to be used as if it were a -> b when a is some kind of literal (such as a string literal), thus evaluating the literal at compile time and returning a compilation error whenever the result represents a failure. Presumably this would use some special notation as to be less magical, and potentially this could be used with non-literals in select cases to balance convenience with readability (but certainly only if the value of expression can be evaluated fully at compile time).

In other words, Roc could parse your strings as Urls or well formed Paths or markdown documents or OpenGL shaders or whatever, and you'd find out at compile time rather than runtime: your code in such cases would be more succinct while still requiring you to handle errors gracefully when they can actually occur (for runtime errors that can only be produced by runtime inputs).

In the case of Http.get Url you do gain something in the common case of using a hardcoded URL/path, since the @ in a hypothetical @"https://x.com/path" may combine an ability like FromStr (though of the form Str -> Result a e), and then force evaluation and verification of the Result at compile time. It's still convenient, but actually provides correctness guarantees.

Kevin Gillette (Mar 21 2022 at 02:58):

In the hypothetical example, yeah. Or any situation in which user input: 1) can't be trusted, 2) hasn't been validated, and 3) in which a convenience desugaring into an Opaque type "conversion" would be dangerous from an application-level security/safety perspective.

Kevin Gillette (Mar 21 2022 at 03:01):

I think this is a fair point when talking about Http.get concretely. We are using the example to discuss the feature as a whole, right now, thus just because it's safe in that case doesn't mean it always is in all cases. So in terms of "Http.get the function" I agree; in terms of "Http.get the conceptual proxy debate for the reasonable extent of abilities," I'm not as sure.

Richard Feldman (Mar 21 2022 at 03:06):

I understand the idea here, but I don't know of a precedent in any language for a file path API where the only way to construct one from a string is to call a function named like assumeSafe, and any safety benefit here is coming from the name, not the types (since the conversion function in either case has exactly the same type).

I do think it's a good idea to try to reduce the likelihood that an untrusted user input will get passed to a dangerous function, but I think there are better ways to achieve that goal! :big_smile:

Kevin Gillette (Mar 21 2022 at 03:08):

If you could add abilities you do control to types you don't control, that strikes me as being a bit magical: people might be expecting the magic when working with your ability, but the user may need to get used to guessing about what a piece of code does ("known unknown"). If you can add abilities you don't control to types you don't control, I believe that's no different than monkey-patching: if a program uses an ability-consuming module, and you convince them to merely import your own module without using anything out of it (besides, say Zero = 0), the rest of their program might change, by compile-time side effect, as a result.

Richard Feldman (Mar 21 2022 at 03:09):

I definitely think this is a promising idea, but I think the way to do it is with something more like a linter rule. Any Roc expression that has only top-level dependencies (e.g. it doesn't depend in any way on user input) can be safely evaluated at compile time, and then we can have a linter rule that validates the output and tells us if it's valid.

as an application author, I benefit from being informed about the problem (ideally at build time rather than runtime); it doesn't necessarily matter to me whether I find out from a type-checker or a linter!

Richard Feldman (Mar 21 2022 at 03:10):

(Of note, the FromStr idea doesn't require this, although the previous FooLike * idea did.)

Brendan Hansknecht (Mar 21 2022 at 03:13):

This is what I mean by limited/inflexible. At this point, I don't think we should be able to add abilities to types that we don't own, but without it you only get the exact from of FromStr that is blessed by the standard library. It may not match what you actually want.

Brendan Hansknecht (Mar 21 2022 at 03:13):

Kevin Gillette (Mar 21 2022 at 03:14):

Is it explicit though? A casual reading will likely read $userName as "nothing interesting is going on here" (assuming userName is a Str and not some other opaque type, the compiler won't reject), and thus the $ is not sufficiently explicit to have prevented a bug (and the function it desugars to proved nothing, thus didn't prevent a bug either). Casually reading through the code, in the general case, I would not be able to tell what $ desugared into, and should be concerned, unless $ provides some correctness guarantees (because as defined, it's a way of opting out of the clarity that opaque type names/conversion functions provide, thus it's opting out of the safety that clarity provides).

Brendan Hansknecht (Mar 21 2022 at 03:15):

I think running it at compile time and unwrapping the result is much more useful and powerful. Can save on many runtime costs. A linter doesn't give you that.

Richard Feldman (Mar 21 2022 at 03:16):

I think it's safe to assume that in a hypothetical world where this feature existed, it would get used enough that people would know what the feature does :big_smile:

Brendan Hansknecht (Mar 21 2022 at 03:17):

Richard Feldman (Mar 21 2022 at 03:17):

Richard Feldman (Mar 21 2022 at 03:18):

Richard Feldman (Mar 21 2022 at 03:19):

if it were really beneficial to mandate calling fromStr, why would Elm choose to use String, and why would Rust go out of the way to introduce AsRef and then in the documentation use the string literal version?

Brendan Hansknecht (Mar 21 2022 at 03:20):

I think the issue is that this is generic for all types. But path and url are specific apis that know the tradeoff.

Brendan Hansknecht (Mar 21 2022 at 03:21):

Richard Feldman (Mar 21 2022 at 03:22):

well an ability like that wouldn't be auto-derived, so you'd have to opt into it for your particular opaque type

Richard Feldman (Mar 21 2022 at 03:23):

so whatever opaque type you're making, you'd have the option to make it more concise (or not)

Richard Feldman (Mar 21 2022 at 03:23):

Richard Feldman (Mar 21 2022 at 03:24):

let's say this feature doesn't exist, and instead of naming my function Url.fromStr, I just name it f, and I encourage everyone to import it unqualified

Richard Feldman (Mar 21 2022 at 03:24):

Richard Feldman (Mar 21 2022 at 03:25):

this is already possible today, and it's one space, one paren, and one ASCII character off from bytes <- Http.get $"/blah"

Richard Feldman (Mar 21 2022 at 03:26):

so if the concern is that people could use this to make dubious API decisions, well...they already can :big_smile:

Richard Feldman (Mar 21 2022 at 03:27):

so I think the more important question is not "can someone literally use this to make a mistake" but rather, to what extent does the feature make things more or less error-prone?

Richard Feldman (Mar 21 2022 at 03:28):

and that's actually part of what makes me interested in this idea in the first place: it creates an incentive to use opaque types more

Richard Feldman (Mar 21 2022 at 03:29):

personally I would (and in fact already have in an example!) go the type alias route of Path : Str using Roc's current feature set

Richard Feldman (Mar 21 2022 at 03:30):

because I come to the same conclusion Evan did with Elm's Http.get API: the number of bugs it will actually avoid, in practice, by having the type be opaque is so small it's not worth the mandated function call

Richard Feldman (Mar 21 2022 at 03:30):

in fact there's even a case to be made that Http.get (Url.fromStr "foo") is more error prone

Richard Feldman (Mar 21 2022 at 03:32):

because one of the ways humans can make errors is by misreading something, and it's harder to misread the relevant parts of Http.get "/foo" because there's a higher signal-to-noise ratio; the parts of that operation that are most likely to have an error are that I specified the wrong HTTP method (e.g. get vs head or something...more likely would be post vs put, I suppose), or that I specified the wrong URL

Richard Feldman (Mar 21 2022 at 03:33):

and for sure, "saving one function call makes it easier to see the relevant parts of the expression" (e.g. in a code review) is an extremely minor benefit

Richard Feldman (Mar 21 2022 at 03:33):

but I think the benefit of the opaque type here is so comparably extremely minor that I'm actually not sure which of the two is the most minor!

Richard Feldman (Mar 21 2022 at 03:34):

just use the opaque type everywhere and give it FromStr, easy decision tree to navigate

Richard Feldman (Mar 21 2022 at 03:35):

and sure, it would be specific to Str, but honestly I don't think these things are evenly distributed. There's a reason there's a meme of "stringly typed" - because it's so common to use raw strings for things instead of (equivalents of) opaque types

Richard Feldman (Mar 21 2022 at 03:37):

I realize I'm spending a lot of time defending this idea, but I swear I'm not really attached to it or anything :laughing:

Richard Feldman (Mar 21 2022 at 03:39):

really I just have a lot of thoughts about trade-offs around opaque types, type aliases, and raw types, and this is the first time they've happened to come up in relation to a specific idea, and I like talking about them :big_smile:

Richard Feldman (Mar 21 2022 at 03:39):

Ayaz Hafiz (Mar 21 2022 at 03:51):

Personally, I prefer the pure-alias approach, e.g. something like Path : Str. I agree with most points you raised Richard, in particular that if bugs with path operations are unlikely to be the most prominent bugs in a piece of code. And when they are, you are likely already aware of path operations that need to happen and deal with them out-of-band.

My anecdote here is the current place that I work at. Admittedly most things we do is something like bank Python but hey, everyone uses it and it works, so something must be going right. In most places where need some nominal wrapper (read: opaque type) around a built-in type like str, people use a structural alias like Path = str (in Python; in Roc of course it would be Path : Str). Why? Because it increases readability tremendously when you use type annotations, without also increasing the cognitive load of moving back and forth between strings and specialized wrappers for them. In practice the kinds of things we do with these types is not all that complicated so it doesn't make sense to try to express them at the type level (please forgive my reluctance to give specific examples), because it's not worth the engineering/cognitive effort to focus on bugs there - there is much more important logic in the broader context of the program, which is where we'd want time to be spent analyzing.

Kevin Gillette (Mar 21 2022 at 04:00):

Side note: I found it really hard to find comprehensive resources when I started trying to learn Haskell (and suffice it to say, web-searching for symbols, then and now, is difficult). It took me several months of intermittent learning/study before I came across the meaning of $, and the meaning was not at all what I had inferred-ish it to mean (I never really ended up using Haskell, but iirc, it's similar to Roc's |>). I'm not terribly convinced that enough people would know what $"abc" means in Roc if it's an "advanced" feature. Moreover, if $ were the symbol hypothetically used, I'd expect many to see $someVar and believe it to be a shell/perl style variable notation (rather than a unary operator applied to an expression), get confused when it sometimes works and sometimes doesn't (because as you've defined it, $ essentially is a way to adapt arbitrarily-shaped things into square shaped holes), and then ultimately some will leave because Roc seems too complex or magical ("I thought I figured out when dollar-variables are used and when non-dollar variables are used, but I guess I was wrong").

In a very practical sense, the approachability and readability of languages (and thus, as you say, their mainstream popularity) occurs in the wider context of the languages people are used to; since $xyz is likely read as "variable notation" and $"xyz" is possibly inferred as "interpolated string that happens not to contain any interpolations," we might as well use them for those purposes when possible (I know $ was just a hypothetical example, but hypothetical examples sometimes land as production language features :shrug:).

Using notation that is wholly unfamiliar to mainstream audiences is generally okay, since there's nothing for them to confuse it with (or rather, nothing for them to believe is familiar and known when it is actually quite unknown). But using notation that gives the false impression of familiarity, I suspect, will tend to limit adoption by mainstream audiences while doing nothing to diminish adoption by those already using languages in the same family (FP language adoption by functional programmers has generally never been the problem).

Kevin Gillette (Mar 21 2022 at 04:03):

I think the more telling question (to which I don't know the answer) is why Rust even introduced Path in that API when it, from your description, provides automatic conversion from string. A safety formality that you can trivially bypass (i.e. by trivializing the concern) isn't a formality at all.

Richard Feldman (Mar 21 2022 at 04:04):

sure - to be fair, I picked $ basically because it was the first thing that came to mind, not because I think it's necessarily optimal :big_smile:

Richard Feldman (Mar 21 2022 at 04:06):

Richard Feldman (Mar 21 2022 at 04:07):

so maybe you've got a Path value because you're doing various transformations on it, and then you end up wanting to pass it to File::open

Richard Feldman (Mar 21 2022 at 04:07):

if File::open takes a string, then you have to call like path.as_str() on it

Richard Feldman (Mar 21 2022 at 04:08):

(personally I'm fine with that, but I suspect avoiding that conversion call is at least part of the motivation there!)

Kevin Gillette (Mar 21 2022 at 04:09):

From the perspective of someone using the function which refers to the type which has the ability, the presence of the ability might go unnoticed, and thus the auto-derivation would go unnoticed. I could use someone else's API that is based on abilities that are not auto-derived (for safety), and if I just copy-paste the examples ("just put a dollar sign or at-sign or whatever in front of the argument, just because :tm:"), I could be making some blunder and not realize it.

I'm not worried about experts making this mistake: I'm concerned about beginners making the mistake. If something as widely used as Http.get uses such an ability, and a hard-to-google symbol is used for desugaring, and the beginner just wants to get something working, then it's a pretty good recipe for people to learn the feature by rote ("I just remember that I need to put this symbol in front of whatever I pass to Http.get") rather than understanding it and its implications.

Richard Feldman (Mar 21 2022 at 04:15):

Kevin Gillette (Mar 21 2022 at 04:19):

True! But f is the result of one module author's misguided design, whereas a fromStr desugaring symbol (in my mind, particularly one that only works on non-Result conversions) is a language-level blessed choice that is equally hard to discover the meaning of (via web search), that but will show up in many places, thus will have a statistical minimum level of misuse that presumably will exceed the impact of f (assuming f doesn't become the next left-pad).

+ is also a single symbol, but what it desugars into isn't really surprising, and even if you don't know that it is based on sugaring, you can still write code effectively because little or nothing about it is surprising. That's a great use of sugar. In cases where there's ambiguity of meaning, a lot of care should be taken, even if that means using a builtin named function (like parse) instead of an operator to disambiguate, for example.

Brendan Hansknecht (Mar 21 2022 at 04:20):

I think there is a very important difference here. If we add $ we are essentially saying that this is a good coding practice and expected to be used. In fact, it is so nice that we added syntactic sugar for it. I have a huge problem with that premise.

Kevin Gillette (Mar 21 2022 at 04:24):

And as you say, Http.get returns a Task which can fail, including if the URL is malformed, thus it still gets validated, albeit perhaps in a delayed way. Perhaps Http.get isn't really serving well as the example to discuss for when opaque types (and a potential FromStr) should be used: we could introduce FromStr and a desugaring syntax yet also choose to have Http.get accept Str instead of an opaque type.

Are there practical examples people know of where use of Str instead of opaque types creates a noticeably higher risk of bugs?

Brendan Hansknecht (Mar 21 2022 at 04:25):

Brendan Hansknecht (Mar 21 2022 at 04:27):

Also, $ makes me really feel like what is really wanted is something akin to saying 123km where km specifies the type, but for strings.

Brendan Hansknecht (Mar 21 2022 at 04:28):

Yes, but I think it in general promotes bad design that should be avoid. Sure it is good for some subset of types. but it will be very confusing for other types. Also, I am pretty sure you can use it to implement string decoding to any type.....which is really weird.

Kevin Gillette (Mar 21 2022 at 04:29):

I see. Path as a utility type is as much or more of a factor than Path being a "safety type." It's really that File::open takes a Path just because it's annoying to need to convert it to str (which is the most naturally expressed type for this operation in Rust), rather than str being a safety bypass for Path.

Brendan Hansknecht (Mar 21 2022 at 04:30):

Overall, I still think my biggest concern is that it I believe it should force returning a result of a type.

Richard Feldman (Mar 21 2022 at 04:35):

e.g. there should only be a way to get a Path in Rust after going through a Result

Brendan Hansknecht (Mar 21 2022 at 04:39):

So I think this is were specific api vs language feature are important to distinguish. I think that it may be fine for path to essentially consider an alias for string (like how rust uses it). The problem with $ is that it is not just for path. As mentioned by Kevin, path ends up getting checked by the actual Fille::open call. That is an api design. I am fine with individual apis making that choice. I think that $ and the convenience that it brings will promote too many apis making the wrong choice.

Richard Feldman (Mar 21 2022 at 04:41):

Richard Feldman (Mar 21 2022 at 04:42):

Brendan Hansknecht (Mar 21 2022 at 04:42):

adding $ to a type is very similar to allowing floats to have NaN. They both enable the type to encode invalid data that might cause many issue. It may be totally fine, but it is a huge decision with a lot of tradeoffs.

Richard Feldman (Mar 21 2022 at 04:45):

do you see the same concern as applying to the PathLike * approach from earlier?

Richard Feldman (Mar 21 2022 at 04:46):

that one just required being able to add your own custom ability to a builtin type

Kevin Gillette (Mar 21 2022 at 04:46):

iiuc, Rust has ways to panic on any expression returning a Result. I'm not sure I'd advocate for use of such a feature, because I believe there should be a clear distinction between "clean handling of errors that can and do happen at runtime" (network errors, dns resolution, bad utf8 from users, etc) and "programmer mistakes" (bugs). I'm in favor of a default policy of programs crashing when a bug has been detected. I'm open to certain types of programs crashing when no reasonable progress can be made (such as if the user of a CLI application refers to an option flag or subcommand that doesn't exist), but I do believe that programs should control their own output, and so am not in favor of ceding control of output and exit code to the language or compiler (or maybe even platform, depending) in exchange for a convenient way to crash based on a failed Result, i.e. Rust's ! suffix notation/methods.

Richard Feldman (Mar 21 2022 at 04:46):

Kevin Gillette (Mar 21 2022 at 04:50):

I'm pretty skeptical of PathLike (assuming it's an ability) or any XyzLike. I'll say this somewhat in jest, but if we let types implement abilities to let them act like other types (and indeed if we fully generalize by letting programmers introduce their own abilities of that style), then what's the point of the type system? Anything can be like anything, and when I introduce a module, I'll get feature requests to make my type work like ten other types so that everything can get Likes and interoperate without conversions.

Kevin Gillette (Mar 21 2022 at 05:03):

I don't know whether aliases-for-readability is a win. I haven't seen it used at scale, so I don't know. A language that introduces aliases casually to serve much the same purpose as parameter [variable] names, then it may be hard to notice novel types when they do appear, since Kilometers, Miles, Path, Url, UserId would easily outnumber data-structure types, for example (also, an "alias all the things" would lead to Kilometers and Miles being aliases rather than opaque types, which I believe is probably a mistake).

I would certainly greatly prefer Url.fromStr to return a Result, and provide some of mechanism, if it's really needed, to get around the inconvenience of that. If Http.get takes a Url opaque type, I'm perfectly happy (nay, appreciative of the opportunity) to handle that failure explicitly across multiple lines of control flow, or using Result convenience functions, or backpassing, or whatever. I won't be of the opinion that "if I can't succinctly say Http.get (something) then it must be bad." I don't care much about one-liners.

Kevin Gillette (Mar 21 2022 at 05:04):

I think I agree with the sentiment that I'd rather use aliases if we're just using opaque types as non-validating type system speed bumps but then providing a convenient way to drive over those speed bumps at high speed. I'd rather reserve opaque types as a tool that is clearly used to make guarantees about values, rather than merely as a means to document values (aliases can document values just as well, and are already low friction without the introduction of a new builtin/ability).

Brendan Hansknecht (Mar 21 2022 at 05:04):

Besides the concerns that have already been mentioned around enabling adding abilities to builtins, I would be a lot less concerned about this design.

Brendan Hansknecht (Mar 21 2022 at 05:05):

Richard Feldman (Mar 21 2022 at 21:00):

I just realized something interesting that abilities permit: a game developer (for example) could make a package that reimplements the Num primitives with exactly the same API except that instead of panicking on overflow, it would wrap by default

Richard Feldman (Mar 21 2022 at 21:01):

so whereas ordinarily if you do int1 + int2 and it overflows, you get a panic, if you're using int types from that library, you could still write that same expression (but the types would be like MyI32 instead of I32) and it would compile to a single machine instruction

Richard Feldman (Mar 21 2022 at 21:01):

which would wrap instead of panicking on overflow - which is probably what you want in e.g. games where performance is critical

Brendan Hansknecht (Mar 21 2022 at 21:37):

Brendan Hansknecht (Mar 21 2022 at 21:38):

Kevin Gillette (Mar 22 2022 at 02:48):

Do common architectures consistently provide wrapping semantics on integer overflow? I wasn't aware of that being a universal behavior (and if it's not, I wouldn't expect it to always be achievable with a single instruction.

Brendan Hansknecht (Mar 22 2022 at 02:50):

Yeah, that is how most (all that I know of) add instructions work on x86, arm, risc-v, mips, etc.

Brendan Hansknecht (Mar 22 2022 at 02:50):

Brendan Hansknecht (Mar 22 2022 at 02:51):

The reason is that they really compute a 65 bit number. The extra bit is just the overflow/carry bit.

Ayaz Hafiz (Apr 14 2022 at 19:14):

app "helloWorld"
    packages { pf: "c-platform" }
    imports []
    provides [ main ] to pf

SaysHello has
    hi : a -> Str | a has SaysHello

Abilities := {}

hi = \$Abilities _ -> "Hi from an ability!"

main = hi ($Abilities {})

❯ cargo run -- examples/hello-world/helloWorld.roc
    Finished dev [unoptimized + debuginfo] target(s) in 0.12s
     Running `target/debug/roc examples/hello-world/helloWorld.roc`
🔨 Rebuilding host... Done!
Hi from an ability!%

Ayaz Hafiz (Jun 13 2022 at 20:42):

If anyone would like to play around with abilities, after https://github.com/rtfeldman/roc/pull/3218 is landed, you should be in a pretty good spot to implement and use custom abilities for opaque types. We are working on adding auto-derived abilities for structural types next. Your use will help us discover bugs we missed :) There is an example of using the standard library Encoding ability in this PR as a basis.

Qqwy / Marten (Jun 17 2022 at 19:29):

@Richard Feldman I've read through the rationale document about abilities, and I have two notes:

Qqwy / Marten (Jun 17 2022 at 19:31):

Qqwy / Marten (Jun 17 2022 at 19:33):

Also, NaN is a curious case (as it is in many places) when implementing a Hash ability vs. implementing a Serialize ability for floating-point types.

Folkert de Vries (Jun 17 2022 at 19:35):

I'm going to make the claim that any rust project of decent complexity depends on serde

Folkert de Vries (Jun 17 2022 at 19:35):

Richard Feldman (Jun 17 2022 at 19:37):

Richard Feldman (Jun 17 2022 at 19:38):

as for orphan instances, I assume there will be some amount of demand for them, but I don't want to take it as a given that their benefits would outweigh the costs

Richard Feldman (Jun 17 2022 at 19:39):

Qqwy / Marten (Jun 17 2022 at 19:40):

It's the kind of thing where you definitely do not want it in a library, but sometimes might need to have it in your final application to make sure all dependencies you are using play nice together.

Qqwy / Marten (Jun 17 2022 at 19:41):

Richard Feldman (Jun 17 2022 at 19:42):

yeah so one downside of allowing it for applications but not packages is that it is no longer the case that you can take any chunk of code from your application and library-ize it

Richard Feldman (Jun 17 2022 at 19:42):

Richard Feldman (Jun 17 2022 at 19:43):

because it might require an orphan instance to work, and there'd be no possible workaround for that

Richard Feldman (Jun 17 2022 at 19:44):

so the alternative is "if you want these libraries to work together, their authors need to coordinate; one needs to depend on the other and define instances for it"

Richard Feldman (Jun 17 2022 at 19:45):

which also has downsides, but at least at face value seems like a better thing to encourage than orphan instances in libraries

Qqwy / Marten (Jun 17 2022 at 19:47):

Hashi364 (Jun 19 2022 at 16:50):

I'm not sure if it's worth talking about. Unison lang uses the name Ability to refer to an Algebraic Effect. So the first time I saw Abilities here I thought: "Nice, Roc has Algebraic Effects, eventually I'll take a look at it!".

I'm saying all this, because the name confused me for a moment and may confuse others.

Richard Feldman (Jun 19 2022 at 19:14):

yeah, there's really no standard name for any of these things unfortunately :sweat_smile:

Ayaz Hafiz (Jun 19 2022 at 20:12):

I certainly don't feel that "ability" is more synonymous to algebraic effects than to typeclasses

Hashi364 (Jun 19 2022 at 23:20):

jan kili (Jun 27 2022 at 07:01):

Hey, I heard that the keyword consensus changed from supports back to has? I personally think that the verb "have" is too vague here, but I'm curious to hear more.

jan kili (Jun 27 2022 at 07:05):

I'd like to re-nominate can, as it's more explanatory. Sure, it sometimes leads to weird English grammar in code (can Arithmetic), but so does has (has Monoid) and this also isn't a high-priority criteria for a lower-level/rarer language feature.

jan kili (Jun 27 2022 at 07:07):

(Hmm, though I can see an argument for leaving the word "can" available for app variable names...)

jan kili (Jun 27 2022 at 07:09):

If there are indeed many votes for has, then I'd also submit hasAbilities for consideration: Foo hasAbilities [Bar, Baz]

jan kili (Jun 27 2022 at 07:23):

However (other than keyword reservation nuances), I see switching from Foo has [Bar, Baz] to Foo can [Bar, Baz] as a pure upgrade.

jan kili (Jun 27 2022 at 07:27):

Final (sassier) note: when I think of "has" in programming, it's almost always either for data encapsulation or mutexes :stuck_out_tongue:

Qqwy / Marten (Jun 27 2022 at 11:05):

I am against can because it sort of necessitates naming the abilities things as verbs.
I have seen Ruby go off the deep end in a similar situation: Ruby mixin modules (Ruby's answer to multiple-inheritance, akin to 'interfaces' in other OOP languages) became to be known as 'concerns'. The convention became that the word is was used. This resulted in nearly all of these being named *-able. Things like Commentable, PasswordRecoverable, EqualityComparable, JsonConvertable etc.

Qqwy / Marten (Jun 27 2022 at 11:18):

The main problem being that the names were long, and that people really tried to shoehorn whatever into the '*-able' format

Qqwy / Marten (Jun 27 2022 at 11:19):

With can you have a similar situation: Abilities would end up with names like BeEncoded, BeCompared etc.

jan kili (Jun 27 2022 at 12:43):

Richard Feldman (Jun 27 2022 at 13:01):

I think of has as being short for "has the _____ ability" which works for whatever ability name you put in the blank :smiley:

Qqwy / Marten (Jun 27 2022 at 13:03):

jan kili (Jun 27 2022 at 13:21):

Would that same intuition extend positively to the more explicit Foo hasAbilities [Bar, Baz]?

jan kili (Jun 27 2022 at 13:35):

Since this keyword is intended to be rare and represents a unique concept among programming languages, I don't mind that level of verbosity

jan kili (Jun 27 2022 at 13:43):

Like with other syntax decisions (like opaque bools), we might want to avoid making concise/terse syntax for special cases. I believe that we're dissuading Roc app devs from using abilities for everyday tasks like domain modeling, but the simplicity of has makes it seem like a common pattern.

jan kili (Jun 27 2022 at 13:43):

Richard Feldman (Jun 27 2022 at 13:49):

oh it would actually come up often when defining opaque types - one thing that changed from the original design doc is realizing that automatically derived abilities should work differently for structural types and opaque types:

Richard Feldman (Jun 27 2022 at 13:52):

case in point: I'm currently working on a Path opaque type for a CLI platform, and it needs to have custom equality because it looks like this:

Path := [
    FromOperatingSystem : List U8,
    FromRocList : List U8,
    FromStr : Str,
]

...but all of those alternatives should be directly comparable to each other, even if the tags are different

Richard Feldman (Jun 27 2022 at 13:52):

but if this type were to get the default Eq automatically, it would be an incorrect implementation of equality

Richard Feldman (Jun 27 2022 at 13:53):

separately, if it were to get Encode and Decode automatically, then it would be implicitly exposing its internal implementation to everyone to rely on - and one of the main features of an opaque type is that its internal implementation is opaque! :big_smile:

Richard Feldman (Jun 27 2022 at 13:55):

also, if I forget to add Eq to an opaque type and I publish a package, someone can remind me and I can always add it later as a nonbreaking change. Whereas if I forget to remove Encode, I'm now exposing my internal implementation details by default - and if I later realize I'd been doing that and want to go back and remove Encode before people start relying on it, it's too late; it will have to be a breaking change, and also people might already be relying on it

Richard Feldman (Jun 27 2022 at 13:56):

Path := [
    FromOperatingSystem : List U8,
    FromRocList : List U8,
    FromStr : Str,
] has [Eq.{ isEq }, Ord.{ compare }, Hash.{ hash }, Encode.{ encode }, Decode.{ decode }]

Richard Feldman (Jun 27 2022 at 13:56):

jan kili (Jun 27 2022 at 14:00):

jan kili (Jun 27 2022 at 14:04):

Richard Feldman (Jun 27 2022 at 14:15):

Brendan Hansknecht (Jun 27 2022 at 14:18):

Ayaz Hafiz (Jun 27 2022 at 14:20):

One thing I really like about has is its symmetry in the language for various ability-related constructs. That is one keyword is reused in three contexts:

Hash has # to say what members an ability has
  hash : a -> U64 | a has Hash # to bind variables to abilities

Foo := { ... } has [Hash] # to say what abilities a type has

Brian Carroll (Jun 27 2022 at 19:43):

Hmm. To me the first of those three seems like a really different concept to the second and third. The second and third world work with implements.
I think I'd actually prefer the first one to have a different keyword.

Brendan Hansknecht (Jun 27 2022 at 22:14):

I think Brian is right and 2 keywords make more sense.
The second and third use are identical in meaning from a user perspective. The first is not.

Richard Feldman (Jun 27 2022 at 23:43):

Brendan Hansknecht (Jun 28 2022 at 00:59):

Brian Carroll (Jun 28 2022 at 05:08):

Hash is
  hash : a -> U64 | a implements Hash

Foo := { ... } implements [Hash]

Maybe it would be OK to reuse the type definition syntax for ability definition?

Hash a :=
  hash : a -> U64 | a implements Hash

Foo := { ... } implements [Hash]

Richard Feldman (Jun 28 2022 at 12:56):

yes, that's what I meant! Although I wasn't saying specifically implements for the rest of it, just is for the definition

Richard Feldman (Jun 28 2022 at 12:57):

I'd rather not reuse the opaque type definition syntax for abilities - I'd like them to look different in some way

Richard Feldman (Jun 28 2022 at 13:01):

really the only thing I prefer about has compared to implements is that it's a lot shorter :big_smile:

Richard Feldman (Jun 28 2022 at 13:02):

Qqwy / Marten (Jun 29 2022 at 16:10):

Qqwy / Marten (Jun 29 2022 at 16:13):

Say I want to create a new type such as Lazy a := @Lazy a, to allow explicit lazy ('deferred') evaluation.
You could envision Lazy.lazy : ({} -> a) -> Lazy a and Lazy.force : Lazy a -> a functions as the rest of the API.

Now we would like Lazy a to have many of the same abilities that a has.
For instance, we'd want it to have Display iff a has Display, Eq iff a has Eq, Num iff a has Num, etc.

Is this possible with the current proposal? If so, how? If not, do we want this? (Personally I am strongly in favor.)

Qqwy / Marten (Jun 29 2022 at 16:18):

For extra clarity: I am only talking about uni -kinded types (i.e., ability implementations constrained for a single type variable) here.
(For reasons listed and discussed earlier in multiple places, we explicitly do not want higher -kinded abilities. They run the risk of people building and using 'over-abstracted' Functor/ Monad etc. abilities)

Ayaz Hafiz (Jun 29 2022 at 16:35):

Ayaz Hafiz (Jun 29 2022 at 16:36):

Ayaz Hafiz (Jun 29 2022 at 16:37):

We don't yet support, for example, Lazy a := ... | a has Eq, a has Hash, to say that the type variable a is bound by both Eq and Hash

Ayaz Hafiz (Jun 29 2022 at 16:38):

However from the perspective of the compiler, we can easily support that, it's a simple extension of how type variables are bound to single abilities today

Brendan Hansknecht (Jun 29 2022 at 18:10):

I think the one issue here is that you want to say Lazy a := ... | Lazy has what a has. That way at each instance of lazy could support different things based on the a that it wraps.

Brendan Hansknecht (Jun 29 2022 at 18:12):

Otherwise you either make an overly strict lazy definition. All lazy objects must have eq and hash and .... Or you make get a factorial explosión problem. LazyWithDisplay, LazyWithEq, LazyWithDislayAndEq.

Ayaz Hafiz (Jun 29 2022 at 18:21):

Ayaz Hafiz (Jun 29 2022 at 18:22):

Foo has f : a -> a | a has Foo

Lazy a := {} -> a

force : Lazy a -> a

forceConstrFoo : Lazy a -> a | a has Foo
forceConstrFoo = \x -> force x

Ayaz Hafiz (Jun 29 2022 at 18:23):

here you constrain it per-instantiation-of-Lazy, rather than at the definition of Lazy

Brendan Hansknecht (Jun 29 2022 at 19:13):

Sure, but that doesn't give the Foo ability to Lazy. So I guess I worded what I wanted wrong.

My idea would be if a has Foo. Then Lazy a has Foo as well. That way you can just use Lazy a the same way you would use a.

Brendan Hansknecht (Jun 29 2022 at 19:15):

So instead of going Lazy a to a to use Foo, you would just go Lazy a to use Foo. Lazy would define a generic mapping such that it would now how to use abilities that a has.

Brendan Hansknecht (Jun 29 2022 at 19:16):

Hmm....I guess ultimately this would just lead to the user observing an implicit conversion from Lazy a to a. So maybe that is a bad thing. Though if I simply made Lazy require that a has Foo, this would still be doable.

Ayaz Hafiz (Jun 29 2022 at 19:17):

But you can only use Lazy a as Foo if you also know a has Foo, at which point you either need to explicitly type a as having Foo, or rely on conditional compilation if you want the "this implements Foo iff a implements Foo"

Brendan Hansknecht (Jun 29 2022 at 19:17):

Ayaz Hafiz (Jun 29 2022 at 19:19):

I think it's better to just thread the a has Foo type through where you need it. Otherwise you need some mechanism to say "here's what should happen if you try to use Lazy a as Foo but a doesn't have Foo", which IMO is too complicated

Brendan Hansknecht (Jun 29 2022 at 19:19):

So if I made a Lazy U16. I would like for it to just be able to do what a U16 can do. Like call lazyU16A + lazyU16B

Brendan Hansknecht (Jun 29 2022 at 19:20):

Brendan Hansknecht (Jun 29 2022 at 19:25):

I totally get that I could require Lazy a | a has Display and then implement Display on Lazy because a is guaranteed to have Display. The issue is when you want to support the fact that not all a in reality will have Display. Which means it is not possible to add Display to Lazy.

In other words, if you want to support a single type that doesn't support all of the abilities you want Lazy to have, you either have to make a custom variant like LazyWithDisplay or you need to accept that Lazy can't have certain abilities.

Ayaz Hafiz (Jun 29 2022 at 19:27):

Well Lazy is a bad example here, in reality with lazy you can't do much until you force the value anyway right. Like maybe a better example is Map k v and you want display for map if k and v have display

Ayaz Hafiz (Jun 29 2022 at 19:28):

displayMap : Map k v -> String | k has Display, v has Display

Brendan Hansknecht (Jun 29 2022 at 19:28):

Brendan Hansknecht (Jun 29 2022 at 19:31):

Just lose out on using map (with displayable key and value) like any other type with Display

Qqwy / Marten (Jun 30 2022 at 07:10):

Qqwy / Marten (Jun 30 2022 at 07:12):

The problem with ad-hoc functions like displayMap is that they do not compose.
If my type is a List (Map Str Str)), it will never internally be able to usedisplayMap, nor will it be able to use the ability directly.

Qqwy / Marten (Jun 30 2022 at 07:20):

Maybe Display is a bad example; Inspect or Encode would be better ones: You almost surely want the wrapper to have these abilities when the contained thing has these abilities.

Qqwy / Marten (Jun 30 2022 at 08:27):

It even is true for e.g. List a which should have Encode iff a has Encode.

Richard Feldman (Jun 30 2022 at 12:26):

Richard Feldman (Jun 30 2022 at 12:27):

like if I make AssocList k v, the k needs to have Eq in order for the data structure to work at all

Richard Feldman (Jun 30 2022 at 12:27):

however, v doesn't; there's no reason I couldn't store a function in there (keyed to something that has Eq), even though functions can't have Eq

Richard Feldman (Jun 30 2022 at 12:28):

but I'd still want to be able to do assocList1 == assocList2, especially for testing

Richard Feldman (Jun 30 2022 at 12:28):

Richard Feldman (Jun 30 2022 at 12:29):

Richard Feldman (Jun 30 2022 at 12:30):

which the current design doesn't support. I didn't think of this use case at the time! :big_smile:

Richard Feldman (Jun 30 2022 at 12:30):

Folkert de Vries (Jun 30 2022 at 12:31):

Richard Feldman (Jun 30 2022 at 12:32):

Folkert de Vries (Jun 30 2022 at 12:32):

Richard Feldman (Jun 30 2022 at 12:32):

Folkert de Vries (Jun 30 2022 at 12:32):

Richard Feldman (Jun 30 2022 at 12:39):

AssocList k v := { ... }
    | has [Eq { isEq }]

isEq : AssocList k v, AssocList k v -> Bool
    | k has Eq
    | v has Eq
isEq = \a, b -> ...

Richard Feldman (Jun 30 2022 at 12:40):

in other words, you add additional constraints to your particular implementation (here, isEq requires that k and v have Eq even though the definition of AssocList doesn't)

Richard Feldman (Jun 30 2022 at 12:41):

and that could imply that AssocList only gets Eq if k and v have Eq, because that's necessary in for its implementation to be valid

Richard Feldman (Jun 30 2022 at 12:42):

however, that doesn't work as nicely for abilities that require implementing multiple functions; what if an ability requires functions foo, bar, and baz, and you give them conflicting requirements?

Richard Feldman (Jun 30 2022 at 12:44):

another downside is that when you look at the declaration, you just see | has [Eq { isEq }] which makes it look like AssocList always has Eq, but that wouldn't be true; it would only have Eq under certain circumstances

Folkert de Vries (Jun 30 2022 at 12:45):

Folkert de Vries (Jun 30 2022 at 12:46):

Richard Feldman (Jun 30 2022 at 12:46):

Richard Feldman (Jun 30 2022 at 12:47):

although to be fair, deriving is already full of implicit behavior, so you already have to know exactly what it's doing to know what derive(Eq) means

Folkert de Vries (Jun 30 2022 at 12:49):

yes, also, in some cases the constraints added to type variables are not what you want

Folkert de Vries (Jun 30 2022 at 12:49):

Ayaz Hafiz (Jun 30 2022 at 12:51):

In our case the phantom types would not be a problem, we would check the derivability of the underlying type, where the phantom types are gone

Folkert de Vries (Jun 30 2022 at 12:53):

Qqwy / Marten (Jun 30 2022 at 12:59):

Qqwy / Marten (Jun 30 2022 at 13:00):

Richard Feldman (Jun 30 2022 at 13:06):

AssocList k v := { ... }
    has [
        Eq if k has [Eq], v has [Eq] { isEq },
        Ord if k has [Ord], v has [Ord] { compare },
        Hash if k has [Hash], v has [Hash] { hash },
        Encode if k has [Encode], v has [Encode] { toEncoder },
        Decode if k has [Decode], v has [Decode] { toDecoder },
        Display if k has [Display], v has [Display] { display },
        Inspect if k has [Inspect], v has [Inspect] { inspect },
    ]

Richard Feldman (Jun 30 2022 at 13:11):

AssocList k v := { ... }
    has [
        Eq { isEq } if k has [Eq], v has [Eq],
        Ord { compare } if k has [Ord], v has [Ord],
        ...etc
    ]

Richard Feldman (Jun 30 2022 at 13:11):

Qqwy / Marten (Jun 30 2022 at 13:17):

I don't really like that commas both separate constraints within an ability as well as the current and the next ability.

Richard Feldman (Jun 30 2022 at 13:17):

AssocList k v := { ... }
    has [
        Eq { isEq } | k has Eq | v has Eq,
        Ord { compare } | k has Ord | v has Ord,
        ...etc
    ]

Richard Feldman (Jun 30 2022 at 13:19):

AssocList k v := { ... }
    has [
        Eq { isEq }
            | k has Eq
            | v has Eq,
        Ord { compare }
            | k has Ord
            | v has Ord,
        ...etc
    ]

Nikita Mounier (Jun 30 2022 at 18:49):

This looks great! It reminds me a lot of Swift's conditional conformance syntax:

extension Array: Equatable where Element: Equatable {
    func ==(lhs: Self, rhs: Self) -> Bool {
        // ...
    }
}

Brendan Hansknecht (Jun 30 2022 at 19:15):

AssocList k v := { ... }
    has [
        Eq { isEq }
            | if k has Eq
            | if v has Eq,
        Ord { compare }
            | if k has Ord
            | if v has Ord,
        ...etc
    ]

Maybe that isn't quite right, but something else similar to point out it isn't required

jan kili (Jul 01 2022 at 02:36):

AssocList k v := { ... }
    has [
        Display { display },
        if k has Eq && v has Eq then
            Eq { isEq }
        else
            SomethingElse { idk },
        if k has Ord && v has Ord then
            Ord { compare }
        else
            Nevermind,
        ...etc
    ]

An else case seems like it might be useful sometimes, though I don't know how to gracefully "null"/"skip" when no need for it...

Martin Stewart (Jul 01 2022 at 07:18):

Is it possible to still do type inference if there are conditionals in the types?

Brian Carroll (Jul 01 2022 at 07:36):

I don't think there's anything you can do for the SomethingElse. If the AssocList has the Eq ability then you can write an expression like assocList1 == assocList2 and it will compile without errors. If it doesn't have the Eq ability then that expression doesn't compile.
It either does or it doesn't, there's not really any alternative. There's only one == operator and it operates on anything that has Eq. So there's nothing you can put in the else branch.

Joshua Warner (Oct 19 2022 at 02:44):

Lim has lim : {} -> a | a has Lim

... and I'm having a really hard time mentally parsing that. There are two hases, one :, one ->, and one | - and I don't have any basis for what the "binding order" is for those operators.

This feels like something where it'd be useful to somehow indicate that with syntax - or perhaps have the formatter insert parens to indicate grouping there.

Richard Feldman (Oct 19 2022 at 02:53):

Richard Feldman (Oct 19 2022 at 02:54):

Lim has lim : {} -> a | a supports Lim

Richard Feldman (Oct 19 2022 at 02:57):

Richard Feldman (Oct 19 2022 at 02:59):

Eq has
    isEq : a, a -> Bool
        | a supports Eq

Joshua Warner (Oct 19 2022 at 03:00):

Richard Feldman (Oct 19 2022 at 03:00):

a past syntax idea had where instead of | which also might make this more descriptive.

Richard Feldman (Oct 19 2022 at 03:00):

Joshua Warner (Oct 19 2022 at 03:01):

I think part of what threw me off is I didn't realize that Eq has ... is declaring an ability

Richard Feldman (Oct 19 2022 at 03:01):

Richard Feldman (Oct 19 2022 at 03:02):

like one possibility is to say ability Eq = but now ability has to be a reserved keyword

Richard Feldman (Oct 19 2022 at 03:02):

although perhaps it might not need to be, since lowercase Uppercase = isn't valid Roc right now :thinking:

Richard Feldman (Oct 19 2022 at 03:03):

so we could parse anything lowercase there without having it be a reserved keyword

Joshua Warner (Oct 19 2022 at 03:03):

Coming from the rust/python/go/etc land, I'd expect something more like Eq = ability ... or ability Eq = ... - yeah

jan kili (Oct 19 2022 at 03:04):

I've been unplugged from abilities design for a while, but I did enjoy a more verbose style like

Eq isImplementedBy
    isEq : a, a -> Bool
        | a implements Eq

which taps into the familiar language of interface/implement. However, keyword choice is key, as they're taken off the app developers menu of variable names.

jan kili (Oct 19 2022 at 03:05):

Lim entails/supports/requires/defines/means
    lim : {} -> a
        | a implements Lim
    otherMethodFoo : ...
    otherMethodBar : ...

Joshua Warner (Oct 19 2022 at 03:06):

Another thing that might give some hint of the meaning, is requiring that either (1) the functions are on separate lines, or (2) there are parens/square brackets around the function list - so Eq has (...) in this example

Richard Feldman (Oct 19 2022 at 03:06):

Richard Feldman (Oct 19 2022 at 03:07):

we use single line in examples and tests but in real world use I don't think that restriction would be annoying, and it does read better to me

Joshua Warner (Oct 19 2022 at 03:07):

Lim entails/supports/requires/defines/means as a general syntax feels weird to me. I think most other examples I've seen in roc use = for defining named things. Why not abilities?

Richard Feldman (Oct 19 2022 at 03:16):

ablility Lim = seems clear to me and doesn't require reserving a keyword unless I'm missing something

Richard Feldman (Oct 19 2022 at 03:20):

Joshua Warner (Oct 19 2022 at 03:20):

Some other random food for thought. There are interesting parallels between record types and ability definitions (notably, that I expect abilities to compile down to a record of function pointers - one per function). What about making that parallel clear in syntax - e.g. using {} around the functions?

Richard Feldman (Oct 19 2022 at 03:22):

the functions also get added to the surrounding scope - e.g. once you've defined isEq like above, you can then expose it like any other function, and it will have that name and type

Richard Feldman (Oct 19 2022 at 03:22):

Joshua Warner (Oct 19 2022 at 03:23):

Joshua Warner (Oct 19 2022 at 03:24):

In that context I think both the = suggestion above and the {} feel less useful

Joshua Warner (Oct 19 2022 at 03:25):

Joshua Warner (Oct 19 2022 at 03:26):

Anyway, I don't feel like I have strong intuition on what the "right" answer is here; probably a mix of things

Joshua Warner (Oct 19 2022 at 03:28):

I _do_ I think one of those hases in the above example should go away / turn into something else

Joshua Warner (Oct 19 2022 at 03:29):

I'm also slightly suspicious of the | - there are definitely other contexts where that clearly means where - but it's also pretty common for that to mean or - which could accidentally lead someone to believe they're dealing with a union type of some sort.

Joshua Warner (Oct 19 2022 at 03:34):

One of the nice things about having reserved words is that the compiler can more easily point out where you're going wrong. If someone thought the | meant or and bound more tightly than the -> operator, they might think the a has Lim part is a weird looking curried function call of some sort - but making the compiler be _clear_ that has has a special meaning, and that meaning is uniform in all contexts, can be good for education.

Joshua Warner (Oct 19 2022 at 03:34):

If a user sees some cases where a given string used as an identifier, and some where it's used as a keyword, I think that can add a lot to the confusion.

Joshua Warner (Oct 19 2022 at 03:34):

Joshua Warner (Oct 19 2022 at 03:36):

Where by "sensible" I mean "correct", in that the highlighter correctly distinguishes those cases and can highlight accordingly. That probably requires the highlighter to be more than a regex engine, which may not be supported in all contexts.

Brendan Hansknecht (Oct 19 2022 at 03:56):

As an aside, i think talking about ability definition is definitely more confusing that talking about ability implementation and use.

Dict k v := { something something internal impl } | k has Hash & Eq

And this feels ok to me. Though now that I am looking at it more a tad inconsistent with the last example:

LowLevelHash := { internal details } has [
        Hasher {
             addByte: definition or function variable etc,
             nextFunc: \x -> x,
        }
    ]

Joshua Warner (Oct 19 2022 at 04:02):

I'm enough of a roc noob that neither of those examples jump out to me as being an 'implementation' of an ability, versus the definition of an ability itself.

Joshua Warner (Oct 19 2022 at 04:03):

That's another good reason for sprinkling keywords around - when chosen correctly, they can give a hint of what's going on.

Joshua Warner (Oct 19 2022 at 04:03):

Brendan Hansknecht (Oct 19 2022 at 04:03):

To be fair, everyone is new to abilities because they are pretty new to the language.

Brendan Hansknecht (Oct 19 2022 at 04:03):

Joshua Warner (Oct 19 2022 at 04:04):

abilities / traits / type classes / protocols / interfaces are all highly-related concepts

Brendan Hansknecht (Oct 19 2022 at 04:04):

Joshua Warner (Oct 19 2022 at 04:04):

I think it's pretty safe to say that most people coming to roc will be familiar with one or more of those

Brendan Hansknecht (Oct 19 2022 at 04:06):

As a note, the first example is just some random data structure with a type variable k that is required to implement the ability Hash and Eq.

The second example is a data structure LowLevelHasher that implements the Hasher ability. That is how you define it's implementation of the ability.

jan kili (Oct 19 2022 at 04:10):

jan kili (Oct 19 2022 at 04:25):

Ayaz Hafiz (Oct 19 2022 at 14:14):

As a person who has probably done the most with abilities, the has syntax everywhere doesn't bother me at all. But I'm obviously very biased so that opinion may not be worthwhile.

Ayaz Hafiz (Oct 19 2022 at 14:15):

But there are things that in the near future will definitely be a confusing parse. For example, when hash-based Dict lands, we'll have something like

Dict k v := ... | k has Eq & Hash has [Encode, Decode, Eq, Hash]

Ayaz Hafiz (Oct 19 2022 at 14:15):

Dict k v := ...
    | k has Eq & Hash
    implements [Encode, Decode, Eq, Hash]

Ayaz Hafiz (Oct 19 2022 at 14:16):

Dict k v := ...
    | k has Eq & Hash
    implements [
        Encode { toEncoder },
        Decode { decoder },
        Eq,
        Hash { hash }
    ]

Ayaz Hafiz (Oct 19 2022 at 14:17):

It's worth saying that today, "has" is not a keyword proper, except for in type-annotation contexts - which just means you can't use it as a type variable. Something like implements would be similar I think

Ayaz Hafiz (Oct 19 2022 at 14:23):

Re the ability A = syntax Richard suggested - I wonder what you think about a syntax like A ::, which is a bit closer to the syntax for aliases and opaque types. Personally I can see arguments either way for making abilities seem closer or further away, as a concept, to aliases and opaque types, but just writing it out,

ability Hash =
    hash : hasher, a -> hasher
        | hasher supports Hasher, a supports Hash

Hash ::
    hash : hasher, a -> hasher
        | hasher supports Hasher, a supports Hash

ability Hash = is more communicative, but I wonder if it could be mistaken as a value by newcomers (since it starts with a lowercase identifier, and =is not used elsewhere in the type-level syntax)

jan kili (Oct 19 2022 at 14:25):

jan kili (Oct 19 2022 at 14:26):

Feels like a "super"type or "meta"type, and abities are basically a set of function types

Brendan Hansknecht (Oct 19 2022 at 14:26):

Joshua Warner (Oct 19 2022 at 14:26):

Syntax highlighting working uniformly(-enough) across all surfaces can help with this.

Joshua Warner (Oct 19 2022 at 14:28):

Anton (Oct 19 2022 at 14:29):

Anton (Oct 19 2022 at 14:30):

It may look like a lot of words now but I think it would be ok with syntax highlighting.

Brendan Hansknecht (Oct 19 2022 at 14:32):

that's pretty fair. If you were to just dim out those words or make them less bold, it would probably reduce noise and still be readable.

Ayaz Hafiz (Oct 19 2022 at 15:22):

Do you need to pronounce it? I think it's like opaque types and aliases - Dict k v := ... is "a definition of an opaque type", but := itself doesn't have/need a pronunciation

Anton (Oct 19 2022 at 15:39):

I'd say being able to pronounce it is an advantage for communication, with beginners especially.

Anton (Oct 19 2022 at 15:40):

Richard Feldman (Oct 19 2022 at 16:05):

if I were telling someone exactly what to write, I'd say "Foo colon colon" and if I were just talking coloquially I'd say "define an ability called Foo" - and I think I'd do the same with "blah colon list str" as opposed to "blah is a list of strings"

Anton (Oct 19 2022 at 16:25):

There is a benefit to having more similarity between the exact way and the colloquial way.

Richard Feldman (Oct 19 2022 at 16:27):

totally! I'm just thinking through how I'd actually do it if it came up in conversation :big_smile:

Ghislain (Nov 26 2022 at 00:27):

Is there a fresh documentation of the current implementation? I don't think I fully understand how this feature works. (the current tutorial section is empty)

Richard Feldman (Nov 26 2022 at 01:26):

not yet - I have it on my TODO list to write up some proper documentation for it, but there are some unresolved syntax questions around it that I'd like to resolve first

Richard Feldman (Nov 26 2022 at 01:26):

Richard Feldman (Nov 26 2022 at 01:27):

there are a lot of options, but the main point of consensus is that the current syntax is not what we should go with :big_smile:

Ayaz Hafiz (Nov 26 2022 at 01:44):

Bryce Miller (Apr 14 2023 at 17:58):

I haven’t been around much since last fall. I was recently wishing for a feature like Rust traits for an Elm side project. (Needed a set of non-comparable items). Needless to say, I’m STOKED that Abilities are a thing in Roc now. Or that they’re in progress at least.

Bryce Miller (Apr 14 2023 at 18:01):

Ok, I guess Abilities were in the works long before I joined Zulip, I just didn’t realize it. Or didn’t understand. Or something. Just wanted to cheer you on 🙌

Richard Feldman (Apr 14 2023 at 20:08):

they're fully implemented now! (Give or take a known bug here and there.) I haven't added them to the tutorial yet because we're planning to change the syntax (but that hasn't been a high priority recently)

Richard Feldman (Apr 14 2023 at 20:08):

I guess I should write that up for #contributing because someone might be interested in doing it :thinking:

Bryce Miller (Apr 14 2023 at 21:18):

Oh cool! I did notice discussion about the syntax. I wasn’t sure if you had settled on a new syntax or not.

Stream: ideas

Topic: Abilities

Richard Feldman (Oct 27 2021 at 14:35):

Matthias Beyer (Oct 27 2021 at 14:41):

Richard Feldman (Oct 27 2021 at 14:46):

Richard Feldman (Oct 27 2021 at 14:46):

Richard Feldman (Oct 27 2021 at 14:46):

Matthias Beyer (Oct 27 2021 at 15:28):

Lucas Rosa (Oct 27 2021 at 15:29):

Lucas Rosa (Oct 27 2021 at 15:29):

Lucas Rosa (Oct 27 2021 at 15:34):

Richard Feldman (Oct 27 2021 at 15:53):

Richard Feldman (Oct 27 2021 at 15:54):

Brendan Hansknecht (Oct 27 2021 at 16:00):

Matthias Beyer (Oct 27 2021 at 16:01):

Richard Feldman (Oct 27 2021 at 16:02):

Brendan Hansknecht (Oct 27 2021 at 16:11):

Richard Feldman (Oct 27 2021 at 16:13):

Lucas Rosa (Oct 27 2021 at 16:14):

Brendan Hansknecht (Oct 27 2021 at 16:18):

Brendan Hansknecht (Oct 27 2021 at 16:20):

Joseph Anthony Zullo (Oct 27 2021 at 22:52):

Zeljko Nesic (Oct 27 2021 at 23:44):

Martin Stewart (Oct 28 2021 at 07:05):

Tim Whiting (Oct 28 2021 at 14:13):

Richard Feldman (Oct 28 2021 at 14:32):

Brendan Hansknecht (Oct 28 2021 at 16:14):

Matthias Beyer (Oct 28 2021 at 18:22):

Matthias Beyer (Oct 28 2021 at 18:28):

Locria Cyber (Oct 28 2021 at 18:32):

Locria Cyber (Oct 28 2021 at 18:35):

Locria Cyber (Oct 28 2021 at 18:38):

Martin Stewart (Oct 28 2021 at 19:06):

Tim Whiting (Oct 28 2021 at 19:27):

Martin Stewart (Oct 28 2021 at 19:50):

Brendan Hansknecht (Oct 28 2021 at 19:55):

Brendan Hansknecht (Oct 28 2021 at 19:57):

Brendan Hansknecht (Oct 28 2021 at 20:00):

Nigel Thorne (Oct 29 2021 at 01:34):

Joseph Anthony Zullo (Oct 29 2021 at 03:39):

Joseph Anthony Zullo (Oct 29 2021 at 03:41):

Lucas Rosa (Oct 29 2021 at 04:08):

Zeljko Nesic (Oct 29 2021 at 11:15):

Richard Feldman (Oct 29 2021 at 12:11):

Richard Feldman (Oct 29 2021 at 12:12):

Folkert de Vries (Oct 29 2021 at 12:29):

Richard Feldman (Oct 29 2021 at 12:31):

Richard Feldman (Oct 29 2021 at 12:32):

Richard Feldman (Oct 29 2021 at 12:34):

Richard Feldman (Oct 29 2021 at 12:35):

Richard Feldman (Oct 29 2021 at 12:36):

Richard Feldman (Oct 29 2021 at 12:38):

Folkert de Vries (Oct 29 2021 at 12:40):

Richard Feldman (Oct 29 2021 at 13:30):

Richard Feldman (Oct 29 2021 at 13:39):

Richard Feldman (Oct 29 2021 at 13:45):

Richard Feldman (Oct 29 2021 at 13:45):

Richard Feldman (Oct 29 2021 at 13:56):

Richard Feldman (Oct 29 2021 at 13:56):

Richard Feldman (Oct 29 2021 at 14:22):

Richard Feldman (Oct 29 2021 at 14:31):

Brendan Hansknecht (Oct 29 2021 at 15:07):

Brendan Hansknecht (Oct 29 2021 at 15:13):

Richard Feldman (Oct 29 2021 at 15:32):

Richard Feldman (Oct 29 2021 at 15:32):

Richard Feldman (Oct 29 2021 at 15:34):

Brendan Hansknecht (Oct 29 2021 at 16:01):

Brendan Hansknecht (Oct 29 2021 at 16:03):

Zeljko Nesic (Oct 29 2021 at 16:21):

Lucas Rosa (Oct 29 2021 at 16:21):

Richard Feldman (Oct 29 2021 at 16:36):

Richard Feldman (Oct 29 2021 at 16:37):

Richard Feldman (Oct 29 2021 at 16:38):

Richard Feldman (Oct 29 2021 at 16:40):

Brendan Hansknecht (Oct 29 2021 at 20:18):

Folkert de Vries (Oct 29 2021 at 20:20):

Brendan Hansknecht (Oct 29 2021 at 20:23):

Richard Feldman (Oct 29 2021 at 22:00):

Martin Stewart (Oct 30 2021 at 16:58):

Brendan Hansknecht (Oct 30 2021 at 17:16):