Stream: ideas

Topic: Numeric infix ops

Brendan Hansknecht (Oct 17 2023 at 16:53):

I am putting this in a separate thread because it is my personal proposal. I think this is probably the simplest way to enable dynamic tensors with infix operators in userland. I believe that static tensors would require larger type system changes to be ergonomic. As such, I am not considering them for this proposal. Also, please ignore names, they can all be flexible and solidified later.

Base proposal

The base idea has been has already be touched upon. Enable infix math operators by adding abilities to the numeric functions.
Num.add will become a function of type a, a -> a where a has Arithmetic.
This would happen for all of the common math operations. They would all get added to the same ability.

In userland, anyone can define types that support Arithmetic and they will get access to a full set of infix operators related to that ability.

This is all that is needed to enable dynamically shaped tensors to work. It is agnostic to the underlying data storage technique. This also enables thing like complex numbers.

Addon 1: Extended Arithmetic

We may also want to enable more infix operators for types that want it. For example, not all types support remainder/modulus. So maybe some of the still common but not super common arithmetic operations could be put in a separate ability. I don't think this should be granular to the individual op (to avoid abuse and non-math use), but maybe we can have 2 or 3 abilities total.

Addon 2: Matrix Ops

If we want to give slightly nicer support specifically to matrices, I would advise adding a custom ability with infix operators for common matrix operations. Off of the top of my head, I can only think of matrix multiplication and division, but maybe there are more. Those operations would benefit from a totally separate infix operator leaving the default * and / for pervasive multiplication and division.

Addon 3: From Num

To deal with the case of mat * 2, we could explicitly add an ability. This ability will automatically be applied when when have a number and a matrix trying to be used with the same infix operator. It will essentially be used to force mat and 2 to have the same type, just like we would get when making matrices a builtin.

I am not 100% convinced on exactly how this should work, but probably could be used anytime we have an unconstrained number type and type that support from num. This probably should not work automatically in general with numbers that have a defined type already that is not a matrix. That should probably require an explicit broadcast request, but I am not really fully sure.

Richard Feldman (Oct 17 2023 at 17:04):

hm, I don't think the third one can work with the current type system but I might be missing something! What would be the types of the functions in question?

Brendan Hansknecht (Oct 17 2023 at 17:09):

So how I imagine it. We have Num.mul: a, a -> a where a has Arithmetic. Then we have the mat which is a Tensor F32. We have 2 which instead of being a Num a is an a where a has FromNumber . Tensor F32 has FromNumber. So the types merge and a is defined as a Tensor F32. We then automatically add in the call to Tensor.fromNumber to get a Tensor F32 from the 2.

Brendan Hansknecht (Oct 17 2023 at 17:09):

Something roughly like that

Brendan Hansknecht (Oct 17 2023 at 17:10):

That or on typechecking failure, check for FromNumber and instead of failing insert Tensor.fromNumber

Brendan Hansknecht (Oct 17 2023 at 17:12):

I do believe that fromNumber may require some compiler magic to work, but I am also fine with that if it is pretty simple and enables this. (though this is also an addon cause I don't think it is needed for the base proposal)

Richard Feldman (Oct 17 2023 at 17:23):

hm, but if 2 has that type, what's the type of something like Num.div which accepts fractions and not any number?

Richard Feldman (Oct 17 2023 at 17:24):

(we can open up this rabbit hole if desired, but I looked into what it would mean if we used abilities to represent integers, fractions, and numbers, and it had a lot of messy consequences)

Brendan Hansknecht (Oct 17 2023 at 17:28):

Let's not open the rabbit hole then. For addon 3, I have 2 other ideas then. Either:

1. Just don't do it at all. Tensor libraries would expose their own from number function and probably would just make it one letter to avoid messing up readability: mat * (m 2)
2. Add it, but make it explicit. It would be an explicit prefix function. Don't know what character, but something like mat * $2

Richard Feldman (Oct 17 2023 at 17:29):

both are interesting!

Richard Feldman (Oct 17 2023 at 17:30):

what I like best about $2 (which I'd prefer not to do) is that the idea itself makes it easier to try out the "just don't offer it and see if it's ok" strategy, because there's a known alternative if it doesn't work out

Richard Feldman (Oct 17 2023 at 17:32):

regarding infix ops just for matrices: what operators specifically would they be? I'd suggested ** previously for matrix multiplication, but // is already in use for integer division

Richard Feldman (Oct 17 2023 at 17:33):

also, it might be weird (but also might be fine) to have a builtin ability that's never implemented anywhere in the builtins - namely, the "is a matrix" ability for matrix infix ops

Richard Feldman (Oct 17 2023 at 17:43):

knowing that several different use cases want several different representations for dynamic matrices makes a strong case that this is the way to go if we want infix arithmetic ops to work with matrices

Brendan Hansknecht (Oct 17 2023 at 18:01):

Didn't think about // being integer division. I'm not sure what to use for matrices. Maybe others have ideas (or examples from other languages).

Richard Feldman (Oct 17 2023 at 18:05):

I'm also open to making integer division be something different (I just used that because it's what Elm and Python use). Main thing is that it's different from / because silently truncating integer division when you don't realize that's what is going to happen is a huge footgun

Richard Feldman (Oct 17 2023 at 18:07):

speaking of which, that's another tricky thing here: if we want Num.div to accept matrices while also accepting fractions but not accepting integers, what does its type signature become?

Brendan Hansknecht (Oct 17 2023 at 18:09):

I guess div should only work on Matrix (Fractional a) and not Matrix (Num a) due to these exact same issues with truncations...hmm

Brendan Hansknecht (Oct 17 2023 at 18:10):

Maybe it means division needs to be separate form other numberic abilities. Like Maybe it would fall into an extension that enables fraction operations? Though is division the only special fractional operation?

Richard Feldman (Oct 17 2023 at 18:18):

the same consideration would apply to any function involving either Frac or Int (which runs into the same issue)

Richard Feldman (Oct 17 2023 at 18:19):

any function that we want to work with matrices, that is

Richard Feldman (Oct 17 2023 at 18:43):

it might require opening up the rabbit hole after all haha...the only way I can think of to make it work is if int and frac come from abilities, e.g.

Matrix a := { rows : List (List a) }
    implements
        MatOps { mulMat, divMat, divMatInt },
        FracOps if a implements FracOps { div },
        IntOps if a implements IntOps { divInt }

Brendan Hansknecht (Oct 17 2023 at 18:45):

I thought this already implicitly worked. you define the div implementation and then you add a a where a has FracOps to it. I feel like we hit something similar with sets and it just worked, but maybe that was just in using mixed abilities and not implementing mixed abilities.

Richard Feldman (Oct 17 2023 at 18:46):

today it's div : Frac a, Frac a -> Frac a

Richard Feldman (Oct 17 2023 at 18:46):

so there's no ability constraint in there

Brendan Hansknecht (Oct 17 2023 at 18:47):

But today I could write Mat.div : Mat (Frac a), Mat (Frac a) -> Mat (Frac a) and Mat.divInt : Mat (Int a), Mat (Int a) -> Mat (Int a). Those function would just work.

Brendan Hansknecht (Oct 17 2023 at 18:47):

The question is if I could add them to an ability, I guess

Richard Feldman (Oct 17 2023 at 18:49):

yeah the problem is that - for example, if we want + to be overloadable, then that means the type of Num.add (which + desugars to) has to change from:

add : Num a, Num a -> Num a

to something like:

add : a, a -> a
    where a implements Add

(or whatever)

Richard Feldman (Oct 17 2023 at 18:49):

so the question is, if you want to do that same change but for Num.div, how do you do it in a way where it still works for matrices but also still restricts it to just fractions?

Richard Feldman (Oct 17 2023 at 18:50):

one option is to say div : a, a -> a where a implements FracDiv

Richard Feldman (Oct 17 2023 at 18:50):

and then you give that to all the fraction types but not all the integer types

Richard Feldman (Oct 17 2023 at 18:50):

(like, F32 gets it, but I32 doesn't get it)

Richard Feldman (Oct 17 2023 at 18:51):

that works, but it means functions that accept Frac a can no longer do division on them

Richard Feldman (Oct 17 2023 at 18:51):

because Frac a is a type alias for something that doesn't have that ability (unless maybe there's some way we can change it to do that?)

Richard Feldman (Oct 17 2023 at 18:52):

I'm actually not sure what's possible there; I guess that's an Ayaz question :big_smile:

Richard Feldman (Oct 17 2023 at 18:54):

as an aside, part of the rabbit hole of redefining Num a := a implements NumOps, IntOps, FracOps (for example) is that it means you have to actually define what those abilities are

Richard Feldman (Oct 17 2023 at 18:55):

so like for IntOps you have to list every primitive function that's necessary for something to be an integer

Richard Feldman (Oct 17 2023 at 18:58):

and then people can of course define their own integers, which on the one hand can be beneficial if someone wants to do their own arbitrary int (or whatever) implementation, but on the other hand as soon as people do that, if we ever need to add a new primitive int op (e.g. because CPUs introduce a new feature we want to support), now that's an unavoidable breaking language change for the whole ecosystem

Richard Feldman (Oct 17 2023 at 18:58):

bc the IntOps ability has to get a new member

Brendan Hansknecht (Oct 17 2023 at 19:07):

I think you are over complicating this. Conditional abilities already work today

Brendan Hansknecht (Oct 17 2023 at 19:09):

I think this does all we need and it already works today

app "helloWorld"
    packages { pf: "https://github.com/roc-lang/basic-cli/releases/download/0.5.0/Cufzl36_SnJ4QbOoEmiJ5dIpUxBvdB3NEySvuH82Wio.tar.br" }
    imports [pf.Stdout]
    provides [main] to pf


FracDiv implements
    div : a, a -> a where a implements FracDiv

IntDiv implements
    divInt : a, a -> a where a implements IntDiv


MyF32 := F32 implements [
    FracDiv {
        div: myF32Div
    }
]

myF32Div = \@MyF32 x, @MyF32 y -> @MyF32 (x / y)

MyI32 := I32 implements [
    IntDiv {
        divInt: myI32Div
    }
]

myI32Div = \@MyI32 x, @MyI32 y -> @MyI32 (x // y)


Vec a := List a implements [
    FracDiv {
        div: vecFracDiv
    },
    IntDiv {
        divInt: vecIntDiv
    }
]

vecFracDiv : Vec a, Vec a -> Vec a where a implements FracDiv
vecFracDiv = \@Vec x, @Vec y ->
    vecFracDivHelper = \out, i ->
        when (List.get x i, List.get y i) is
            (Ok a, Ok b) ->
                List.append out (div a b)
                |> vecFracDivHelper (i+1)
            _ ->
                out
    vecFracDivHelper [] 0
    |> @Vec

vecIntDiv : Vec a, Vec a -> Vec a where a implements IntDiv
vecIntDiv = \@Vec x, @Vec y ->
    vecIntDivHelper = \out, i ->
        when (List.get x i, List.get y i) is
            (Ok a, Ok b) ->
                List.append out (divInt a b)
                |> vecIntDivHelper (i+1)
            _ ->
                out
    vecIntDivHelper [] 0
    |> @Vec

main =
    @Vec a = divInt (@Vec (List.map [4, 5, 6] @MyI32)) (@Vec (List.map [1, 2, 3] @MyI32))
    @Vec b = div (@Vec (List.map [4, 5, 6] @MyF32)) (@Vec (List.map [1, 2, 3] @MyF32))

    aStr = a
            |> List.map \@MyI32 x -> x
            |> List.map Num.toStr
            |> Str.joinWith ", "
    bStr = b
            |> List.map \@MyF32 x -> x
            |> List.map Num.toStr
            |> Str.joinWith ", "

    Stdout.line "A: \(aStr)\nB: \(bStr)"

Brendan Hansknecht (Oct 17 2023 at 19:10):

If we can do this with wrappers, I think we can do this with builtin numbers just fine.

Brendan Hansknecht (Oct 17 2023 at 19:14):

Can we not define Frac a as something that enforces certain abilities? I mean all of the types with Frac a are forced to have all of the fractional ops. Kinda like how Dict k v guarantees that k has Hash.

Brendan Hansknecht (Oct 17 2023 at 19:15):

Also, yeah, definitely no suggesting this: Num a := a implements NumOps, IntOps, FracOps. Just want the subtypes of num to implement the correct ablities.

Richard Feldman (Oct 17 2023 at 19:16):

Brendan Hansknecht said:

Can we not define Frac a as something that enforces certain abilities? I mean all of the types with Frac a are forced to have all of the fractional ops. Kinda like how Dict k v guarantees that k has Hash.

I think so, but I'm not positive - @Ayaz Hafiz would know best

Brendan Hansknecht (Oct 17 2023 at 19:16):

but on the other hand as soon as people do that, if we ever need to add a new primitive int op (e.g. because CPUs introduce a new feature we want to support), now that's an unavoidable breaking language change for the whole ecosystem

Yeah, I think we should scope this to just a reasonable subset of math ops. Something that likely will stay static forever. If we add something new to integers, it would not affect this. It just wouldn't be accessible from matrices, complex numbers, and the like

Richard Feldman (Oct 17 2023 at 19:17):

it's different from Dict because Dict is an opaque type, whereas Frac is a type alias for the Num opaque type with particular type parameters

Richard Feldman (Oct 17 2023 at 19:18):

specifically, Frac a : Num (Fraction a)

Richard Feldman (Oct 17 2023 at 19:20):

so if Num were an ordinary opaque type, we'd have to do something like Num a := { ...whatever... } implements FracDiv if a implements FracDiv - and then we'd need Fraction a := { ...whatever else... } implements FracDiv

Richard Feldman (Oct 17 2023 at 19:21):

which I guess would actually work because you can't construct a Fraction in userspace anyway :big_smile:

Richard Feldman (Oct 17 2023 at 19:23):

so I guess that sounds like it would work?

Brendan Hansknecht (Oct 17 2023 at 19:23):

Roc does let this work:

FracVec a := Vec a where a implements FracDiv
    implements [
        FracDiv {
            div: fracVecDiv
        }
    ]

fracVecDiv = \@FracVec x, @FracVec y -> @FracVec (div x y)

# Do I need to specify the ability?
someDivFunc : FracVec a, FracVec a -> FracVec a
someDivFunc = \x, y -> div x y

Brendan Hansknecht (Oct 17 2023 at 19:25):

But yeah, types always get complex, don't they...

Richard Feldman (Oct 17 2023 at 19:26):

you can put the ability in the type alias

Richard Feldman (Oct 17 2023 at 19:26):

to me the bigger concern with the IntOps / FracOps design is what happens when you want to add a new one and it's a major breaking change

Brendan Hansknecht (Oct 17 2023 at 19:27):

I don't think the goal is to list out all IntOps or FracOps

Brendan Hansknecht (Oct 17 2023 at 19:27):

I think it should just be to put a reasonable list to give infix operators to more types

Brendan Hansknecht (Oct 17 2023 at 19:28):

So we can still add new ops to a regular integer without updating those abilities

Richard Feldman (Oct 17 2023 at 19:28):

yeah I agree, and that does seem in theory to be possible :thumbs_up:

Brendan Hansknecht (Oct 17 2023 at 19:28):

So maybe they need a different name, but something to keep them limited and focused if possible.

Brendan Hansknecht (Oct 17 2023 at 19:29):

I do agree that it is quite inconvenient that there are subtle differences (divsion being the main one cause technically fmod exists)

Richard Feldman (Oct 17 2023 at 19:30):

fortunately we recently decided that mod and rem shouldn't have infix ops after all :big_smile:

Richard Feldman (Oct 17 2023 at 19:32):

so the infix op abilities in question would be something like:

• DivFrac
• DivInt
• AddSubMulNeg

...is that it?

Brendan Hansknecht (Oct 17 2023 at 19:33):

I think so...oh, pow is infix as well

Richard Feldman (Oct 17 2023 at 19:35):

yeah that one has a frac and int version

Richard Feldman (Oct 17 2023 at 19:35):

so we should probably either give it two infix ops or zero, like we do with div

Richard Feldman (Oct 17 2023 at 19:35):

having a ^^ operator sounds kinda silly though :stuck_out_tongue:

Brendan Hansknecht (Oct 17 2023 at 19:36):

Why does pow need two version?

Brendan Hansknecht (Oct 17 2023 at 19:36):

semantically, I think they are the same, just different underlying impl, all numbers should support it

Brendan Hansknecht (Oct 17 2023 at 19:36):

I think that is something we should be able to hide from the users? though maybe I am missing something

Richard Feldman (Oct 17 2023 at 19:37):

Martin Stewart said:

I thought Num.powInt was separated because it needs to produce an err if you write Num.powInt 2 -1 (otherwise you have Elms problem where this function says it returns an int but actually it’s a float)

Brendan Hansknecht (Oct 17 2023 at 19:38):

I think it may just return zero or panic currently

Brendan Hansknecht (Oct 17 2023 at 19:39):

Or it just returns 1 currently....

Brendan Hansknecht (Oct 17 2023 at 19:39):

Also, I think the infix ^ is always pow and never powInt currently

Brendan Hansknecht (Oct 17 2023 at 19:39):

but these all may just be other inconsistencies rather than a comment on what we should design here

Richard Feldman (Oct 17 2023 at 19:42):

yeah

Brendan Hansknecht (Oct 17 2023 at 19:54):

Should powInt just a take a Nat as the second arg?

Richard Feldman (Oct 17 2023 at 19:57):

some other unsigned value I think, because we're going to get rid of Nat

Richard Feldman (Oct 17 2023 at 19:58):

I think we determined anything bigger than U8 would overflow anyway, right?

Johan Lövgren (Oct 17 2023 at 20:07):

Brendan Hansknecht said:

Didn't think about // being integer division. I'm not sure what to use for matrices. Maybe others have ideas (or examples from other languages).

Just a small comment following up on the notation. One thing to keep in mind is that there is both left and right matrix division. This is because matrix division is really multiplying with the inverse matrix, and this can be done from both left and right. I.e. we might say $A / B = A * B^{-1}$, and $B \backslash A = B^{-1} * A$. (It is when constructing the inverse of a matrix that you need to take fractions of numbers). And in fact when solving linear equations it is very common to multiply with the inverse from the left. Here's Julia's matrix division operators.

(Usually inverses are only defined for square matrices, but there are generalizations to non-square. Then only one of the divisions make sense, depending on the shapes of the matrices involved...)

Richard Feldman (Oct 17 2023 at 20:08):

heh, we actually could make a \\ operator

Richard Feldman (Oct 17 2023 at 20:08):

but not \ because that already means lambda

Richard Feldman (Oct 17 2023 at 20:09):

so if we wanted to have // and \\ for matrix division, that would be a nice symmetry (although then we'd need to find something else for integer division of course)

Richard Feldman (Oct 17 2023 at 20:11):

not to bikeshed too much on it, but since it's desugaring to Num.divTrunc, I could see an argument for /- or /_ or /!

Brendan Hansknecht (Oct 17 2023 at 20:14):

This is where full unicode is often awesome even though it doesn't makes sense to add here: /⌊

Brendan Hansknecht (Oct 17 2023 at 20:17):

Also look strange but makes a lot of sense /_

Brendan Hansknecht (Oct 17 2023 at 20:18):

Though also a bit weird cause _ is don't care.

Richard Feldman (Oct 17 2023 at 20:21):

yeah

Richard Feldman (Oct 17 2023 at 20:21):

definitely looks weird to me haha

Richard Feldman (Oct 17 2023 at 20:21):

could also do / for integer division and /. for frac division, but if either of them should get / it should be frac division :sweat_smile:

Richard Feldman (Oct 17 2023 at 20:22):

I guess you could argue /. makes sense for int division because it's truncating everything after the .

Richard Feldman (Oct 17 2023 at 20:22):

but OCaml programmers would be like :scream:

Richard Feldman (Oct 17 2023 at 20:26):

I'd say /- is the frontrunner to me, because truncation is kinda like doing accurate division and then subtracting away the part after the decimal point

Brendan Hansknecht (Oct 17 2023 at 20:28):

I also don't mind (/) or /~. You know /~ you divide, then you get lazy and give a roughly correct answer. it is ~7

Brendan Hansknecht (Oct 17 2023 at 20:29):

I guess that would be divRound not divTrunc

Richard Feldman (Oct 17 2023 at 20:31):

I strongly want to avoid 3-character infix operators

Richard Feldman (Oct 17 2023 at 20:31):

partly for aesthetic reasons but also then we can't use efficient SIMD operations to parse them

Richard Feldman (Oct 17 2023 at 20:31):

/~ is interesting!

Richard Feldman (Oct 17 2023 at 20:31):

or even ~/

Richard Feldman (Oct 17 2023 at 20:32):

anyway, I think we have enough viable options there that the matrix infix ops could reasonably be **, //, and \\

Richard Feldman (Oct 17 2023 at 20:33):

are there others people would want?

Richard Feldman (Oct 17 2023 at 20:49):

an important question to ask here is: where do we draw the line on how many (and which) infix ops to add to builtins for these use cases?

Richard Feldman (Oct 17 2023 at 20:49):

like how many does Julia have?

Brendan Hansknecht (Oct 17 2023 at 20:55):

Based on the doc here: https://docs.julialang.org/en/v1/stdlib/LinearAlgebra

Julia has *, /, \, and^ as special linalg ops. Also presumably the standard + and -, but it isn't listed there, or I missed it.

Richard Feldman (Oct 17 2023 at 20:56):

oh yeah, how would we distinguish between frac and int division with matrices?

Richard Feldman (Oct 17 2023 at 20:56):

...is it even useful to have int division of matrices?

Brendan Hansknecht (Oct 17 2023 at 20:58):

probably not useful. I think it is fine to leave that as explicit

Richard Feldman (Oct 17 2023 at 20:59):

to be honest, I now wonder the same thing about int division in general :thinking:

Brendan Hansknecht (Oct 17 2023 at 20:59):

Also, rest of the ops are here: https://docs.julialang.org/en/v1/base/math/

For math specifically, they have +, -, /, and //. They then also have multiple bitshifts and a few other things

Richard Feldman (Oct 17 2023 at 20:59):

like maybe you should need to call Num.divTrunc or Num.divCeil or whichever you want

Richard Feldman (Oct 17 2023 at 20:59):

yeah I assume nobody wants to do bitshifts on matrices

Brendan Hansknecht (Oct 17 2023 at 21:00):

That said, julia super embraces custom operators, you can just write ⊗(x,y) = kron(x,y) to get the new infix operator ⊗. They have a super long precedence table in that link...crazy

Brendan Hansknecht (Oct 17 2023 at 21:01):

Richard Feldman said:

like maybe you should need to call Num.divTrunc or Num.divCeil or whichever you want

I get the sentiment, but would be really inconvenient for a lot of things with indices.

Richard Feldman (Oct 17 2023 at 21:03):

fair

Richard Feldman (Oct 17 2023 at 22:53):

interestingly, I think in this design game devs might choose not to use the infix matrix multiplication because it requires dynamic

Richard Feldman (Oct 17 2023 at 22:54):

unless it's specifically like a 4x4 matrix

Richard Feldman (Oct 17 2023 at 22:54):

because for smaller matrices they might not want to pay for storing the dimensions dynamically

Richard Feldman (Oct 17 2023 at 22:54):

at runtime

Richard Feldman (Oct 17 2023 at 22:55):

and instead have like a Mat4x4 := (...nested F32 tuples...)

Richard Feldman (Oct 17 2023 at 22:55):

and maybe another for 4x3, 3x4, etc.

Brendan Hansknecht (Oct 17 2023 at 22:55):

They also can use infix with static size matrices, just with more limited shape possibilities.

Brendan Hansknecht (Oct 17 2023 at 22:55):

Have to be the same shape for anything they define so only works with square matrices

Richard Feldman (Oct 17 2023 at 22:56):

right, exactly

Richard Feldman (Oct 17 2023 at 22:58):

zooming out, I think an important question to ask at some point (maybe not yet, maybe when it's more concrete) is: assuming we do all the implementation work, will people actually use it?

Richard Feldman (Oct 17 2023 at 22:58):

or will they just use Python anyway? :sweat_smile:

Richard Feldman (Oct 17 2023 at 23:09):

because I've heard that (for example) the main barrier to Julia adoption is Python inertia seeming impossible to overcome, see for example Mojo's design decisions

Luke Boswell (Oct 17 2023 at 23:14):

I will definitely use this functionality, and be very happy that it exists.

Richard Feldman (Oct 17 2023 at 23:22):

sweet! For graphics?

Richard Feldman (Oct 17 2023 at 23:23):

(please don't say "for making a bunch of DSLs using all the arithmetic operators at once")

Brendan Hansknecht (Oct 17 2023 at 23:43):

or will they just use Python anyway? :sweat_smile:
the main barrier to Julia adoption is Python inertia seeming impossible to overcome, see for example Mojo's design decisions

I think we have to be careful when we say this. While this is true, Julia is has found itself a nice niche in part of the scientific computing community.

On top of that, Roc is not really trying to compete in this space. We just want to enable users that happen to overlap with this space to be happier. So I think our metric of success really shouldn't relate to this much at all.

Richard Feldman (Oct 18 2023 at 00:01):

sure, my main concern is ecosystem

Richard Feldman (Oct 18 2023 at 00:02):

like as I understand it, domains like scientific computing need significant library support for people to be able to do things in them effectively

Richard Feldman (Oct 18 2023 at 00:02):

and we won't have that unless there's a certain minimum level of interest

Richard Feldman (Oct 18 2023 at 00:03):

the specific thing I'm worried about is that we don't (and won't) have direct C FFI, so you can't just grab LAPACK off the FFI shelf unless you want to do it through a platform and have Tasks for math ops

Richard Feldman (Oct 18 2023 at 00:04):

so there has to be sufficient interest for people to actually implement (or at least port) a chunk of linear algebra libraries over, right? (I'm mainly worried about this because I don't know the domain, but I've heard things, and so I don't really have relevant intuition here about how things might go)

Richard Feldman (Oct 18 2023 at 00:06):

so a concrete outcome I would be worried about is that we make the infix ops happen, but still nobody is using Roc for scientific computing (for example) and then we're back to #ideas talking about how to get LAPACK access into Roc to unblock the scientific computing use case for a second time :sweat_smile:

Richard Feldman (Oct 18 2023 at 00:07):

(for context, I am currently leaning toward turning this idea into a proposal and accepting it, so I'm drifting into poking-holes mode)

Richard Feldman (Oct 18 2023 at 00:10):

past thread for context: https://roc.zulipchat.com/#narrow/stream/304641-ideas/topic/FFI

Brendan Hansknecht (Oct 18 2023 at 00:31):

The comment on LAPACK/BLAS/etc is quite pertinent. I hadn't thought of that. It does mean that everything probably has to be reimplemented in roc and will not be as performant....yeah, very good point

Richard Feldman (Oct 18 2023 at 00:31):

how much less performant would it be? :thinking:

Richard Feldman (Oct 18 2023 at 00:32):

I kind of assume we could get it to be equivalent performance, as long as we have simd (which separately we do want to have anyway)

Richard Feldman (Oct 18 2023 at 00:32):

is it just bounds checks?

Richard Feldman (Oct 18 2023 at 00:33):

because we support all the fastest arithmetic CPU ops

Richard Feldman (Oct 18 2023 at 00:34):

and if we also have SIMD on top of that, I'd assume the only thing any implementation could use where the perf couldn't be matched by pure Roc code would be something making use of memory unsafety, because we require bounds checks

Brendan Hansknecht (Oct 18 2023 at 00:45):

I think some of those libraries are hand optimized assembly and they have a number of clever representation tricks. So I just assumed no one would put in the work to fully reproduce their perf in roc. So more a limit in terms of what someone would do than what could be done was my main assumption.

Richard Feldman (Oct 18 2023 at 00:59):

ahh gotcha

Richard Feldman (Oct 18 2023 at 01:00):

yeah like trying to get LLVM to optimize it into assembly that's at least that fast

Luke Boswell (Oct 18 2023 at 02:34):

I could use this feature for graphics and drawing things. I would use this for a state space controller for e.g. a robot. I haven't got a burning desire to do scientific computing with Roc, but I definitely feel like it has the potential to mature to the point where you can do a lot, with all the benefits of using roc entails. If I was a professor teaching electrical engineering, I could use Roc to teach the mathematical principals, run simulations, etc - not to compete with Matlab or Python but I could see a future where it is possible and desirable to do these things. There are significant benefits with some of Roc's design choices that I think make a compelling argument to use it in many critical areas too. Not just performance, but things I really care about like maintainability, security, reliability, reproducability, verifiability, validation, testing, simulation, ease of use/learning, etc.

Declan Joseph Maguire (Oct 18 2023 at 06:38):

I'm halfway through reading this and something just hit me like lightning - why not just make matrices matrix-valued functions? I mean, if that doesn't completely wreck the type system. That's what a matrix literally is, a matrix valued operator. You wouldn't even need an infix, it'd just be a space between matrices. Likewise a tensor, acting on any other class of tensors of fixed shape, can be seen as a function into a different tensor space. I don't think it works as neatly there (a tensor is basically many functions at once depending on how they're used), and I'm not even confident about it working even constrained to matrices (let alone what it might do to nums!), but I needed to get it out of my head.

Declan Joseph Maguire (Oct 18 2023 at 06:43):

And yeah, I don't think everything should end up in the builtins, all those diagonals and tridiagonals and so on, I brought them up for largely the reason you've been discussing - where should the limits of the builtins end and the userspace begin? And what changes would be needed to let userspace tensors be ergonomic-ish? Bringing up einsum wasn't necessarily about bloating the requirements as capping the bloat - if you can make this work everything else probably can. @Richard Feldman said earlier he didn't have much experience with the area (nor do I in language design), so I thought I might info dump about it to give him enough clues to sniff out the right response.

Anton (Oct 18 2023 at 10:45):

Brendan Hansknecht said:

Richard Feldman said:

like maybe you should need to call Num.divTrunc or Num.divCeil or whichever you want

I get the sentiment, but would be really inconvenient for a lot of things with indices.

Can you give an example here @Brendan Hansknecht?

Anton (Oct 18 2023 at 10:50):

why not just make matrices matrix-valued functions? I mean, if that doesn't completely wreck the type system. That's what a matrix literally is, a matrix valued operator. You wouldn't even need an infix, it'd just be a space between matrices.

Can you explain this further @Declan Joseph Maguire?

Declan Joseph Maguire (Oct 18 2023 at 11:12):

I don't know if this would remotely work with the type system, but you could have as functions any matrices A: x -> y, B: y -> z, and thus B A: x -> z. The type variables are vector spaces, or just lists of nums. More generally, any tensor is characterised by the vector spaces it spans over, which itself has a lot of nuances to do with dual spaces but I don't think it needs us to go there. In any case, any tensor is simultaneously an object and a sort of "multifunction" able to embody multiple mappings, one for every binary partition of its vector spaces.

Declan Joseph Maguire (Oct 18 2023 at 11:13):

I could dig into the general tensor case, but I want to lay out the simpler matrix case just to see if this has legs.

Richard Feldman (Oct 18 2023 at 11:15):

Some more detail on why we can't support indexing syntax in general: suppose we had bracket syntax that desugared to List.get, e.g. myList[index] like Python has.

If we wanted to generalize that beyond List to userspace types, it wouldn't be implementable because the ability function would need to know not only the type of the collection (which it does know, so no problem there) but also the type of the element being returned (which it does not know, and which it couldn't know unless we introduced something like associated types to abilities, which I don't think we should).

So this same problem applies to any syntax you might choose, including functions. Actually functions have several additional problems, like how functions aren't comparable for equality, can't unify to userspace opaque types, and probably others I'm not thinking of right away. :big_smile:

Declan Joseph Maguire (Oct 18 2023 at 11:20):

Yeah, had a sense it'd be too good to be true. Sucks about the indices too, and maybe there's an argument to bake indices into tensors too (unless I misunderstood the depths of the issues preventing it), but we've got bigger fish to fry design-wise.

Richard Feldman (Oct 18 2023 at 11:43):

it can't work with the design in this thread

Declan Joseph Maguire (Oct 18 2023 at 11:54):

Ah well. I'm pretty excited about the rest of the thread though.

Brendan Hansknecht (Oct 18 2023 at 15:10):

Anton said:

Can you give an example here Brendan Hansknecht?

Super simple example, in a binary search you want to write m = l + (r - l) // 2.

Lakin Wecker (Oct 18 2023 at 15:15):

I would 100% use this for any teaching I do. If we have a reasonably efficient graphics output, I could see Roc like apps becoming very useful for teaching concepts like animation, where the predictable nature of an ergonomic statically typed functional language reap huge rewards.

Lakin Wecker (Oct 18 2023 at 15:17):

I haven't been following these conversations as much as I want. But I would use it for many things. I'm currently in s scenario where I have a bunch of shitty C++ code that attempts to follow reactive functional UI principles (AKA elm architecture) and it works pretty good, but it's still C++ so the errors are shite, and every once in while you blow your leg off with something incredibly difficult to spot that would be completely avoided in a language like roc. So I would switch to it for a bunch of the stuff I do for research and potentially even professionally

Lakin Wecker (Oct 18 2023 at 15:17):

Especially if the cross-compilations works as well and as easily as you intend for it to.

Lakin Wecker (Oct 18 2023 at 15:18):

Would other people use it? I have no idea. I've long ago learned to distrust my ability to predict how the average programmer will react. :stuck_out_tongue:

Anton (Oct 18 2023 at 15:34):

Super simple example, in a binary search you want to write m = l + (r - l) // 2.

I'd have to do a bunch of examples to get a better feel for it, but in this case, I do like this kind of formatting:

m =
    Num.divTrunc
       l + (r - l)
       2

Lakin Wecker (Oct 18 2023 at 15:39):

Luke Boswell said:

I could use this feature for graphics and drawing things ... If I was a professor teaching electrical engineering, I could use Roc to teach the mathematical principals, run simulations, etc - ... but things I really care about like maintainability, security, reliability, reproducability, verifiability, validation, testing, simulation, ease of use/learning, etc.

I wholeheartedly agree with all of this.

Brendan Hansknecht (Oct 18 2023 at 15:40):

@Anton Would be:

m = l + (Num.divTrunc (r - l) 2)

Anton (Oct 18 2023 at 15:49):

oops, haha, as I said, editor plugins to render math are the way to go :p

Brendan Hansknecht (Oct 18 2023 at 16:23):

I'm not sure I agree, mostly cause source is plain text. You will end up seeing code in plain text in many locations, and we want that to be readable. Sure, an editor plugin helps, but there will always being people using different editor, viewing code on github, etc.

Anton (Oct 18 2023 at 16:30):

Yeah, I've thought about that as well, it seems all possible solutions have significant shortcomings.

Kevin Gillette (Oct 20 2023 at 00:24):

what about [*], [/], etc, as the matrix operators? By bracketing them, they're visually distinguished as being related to matrices

Kevin Gillette (Oct 20 2023 at 00:25):

It's not clear if we need left vs right division when we could just reorder the operands

Richard Feldman (Oct 20 2023 at 00:45):

I'd like to avoid operators with more than 2 characters, partially for aesthetic reasons but also because 1-char and 2-char operators can be detected using a single SIMD classification operation, whereas 3+ have to use extra logic.

Last updated: Jun 16 2026 at 16:19 UTC