Stream: compiler development

Topic: zig compiler - sending opaque pointers to host

Richard Feldman (May 25 2025 at 16:59):

we currently have a special way to send polymorphic types: you declare a special concrete type in the application module (Model is the classic example of this) and then this ends up being an opaque pointer (void*) in the host

Richard Feldman (May 25 2025 at 17:00):

this solves the problem of needing to send things to the host whose size varies based on the application

Richard Feldman (May 25 2025 at 17:01):

but this was always intended to be a stopgap until we figured out a way to offer an API like what Elm does, which is based on normal type variables and has essentially no learning curve for users

Richard Feldman (May 25 2025 at 17:05):

I finally thought of a way to do this!

Richard Feldman (May 25 2025 at 17:06):

so the goal would be to have a platform which asks the application to provide something of this type:

main : {
    init: (() => model),
    update: (model, event -> (model, (() => event))),
    view: (model -> Elem(event)),
}

Richard Feldman (May 25 2025 at 17:08):

the platform would get this from the application, and then convert it to the following before sending it to the host:

main_for_host : {
    init: (() -> Box(model))
    update: (Box(model), Box(event) -> (Box(model), Box(() => Box(event)))),
    view: (Box(model) -> Elem(Box(event)))
}

Richard Feldman (May 25 2025 at 17:09):

the basic idea here is that all functions and type variables that get sent to the host must be wrapped in a Box, which gives them a statically knowable size and makes the ABI boundary easy to get right

Richard Feldman (May 25 2025 at 17:11):

the host can't statically know the type that's inside the box - e.g. when we generate the glue, the host will see it as essentially a Box<void> and it has no way to look inside it to see what's in there because the host knows nothing about its size or shape

Richard Feldman (May 25 2025 at 17:12):

but that's fine, because the point of these is that the host doesn't need to; the host obtains the value from Roc and then all it ever does with that value is pass it back to Roc later at the appropriate time

Richard Feldman (May 25 2025 at 17:12):

an obvious downside of this design is that it requires boxing at the boundary

Richard Feldman (May 25 2025 at 17:14):

I think (but don't know for sure) that we can optimize Box.map() to avoid unboxing and re-boxing, such that in update in there, we only end up allocating a single model in practice for the life of the program, and then updating that one in-place many times

Richard Feldman (May 25 2025 at 17:15):

event (named msg in Elm) would need boxing every time, but they should be very small allocations, so hopefully that's ok in practice

Richard Feldman (May 25 2025 at 17:17):

the functions themselves would be inside a Box, but since they'd be top-level functions with no captures, they wouldn't actually be on the heap; they'd basically be function pointers plus a refcount that's always hardcoded to the special "immortal" refcount

Richard Feldman (May 25 2025 at 17:18):

so assuming all that's true, it should result in:

• an ergonomic API for application authors
• a simple compilation model
• an amount of runtime overhead that is small enough that it's likely insignificant in practice

Richard Feldman (May 25 2025 at 17:20):

and then we don't need any new language features, we just have errors if you try to define a platform which sends functions to the host that contain type variables or functions that aren't wrapped in a Box

Richard Feldman (May 25 2025 at 17:22):

a nice thing about this design is that it means if the perf is a problem in practice, we haven't closed the door to making it more flexible in the future somehow (e.g. allowing unboxed type variables to be passed to the host) - although note that Folkert and I spent months trying to get that to work correctly and reliably, and ended up giving up.

it's not impossible, but it's so astronomically difficult that I'm very convinced we should not attempt it unless the vastly simpler "box at the boundary" design hits performance problems in practice

Richard Feldman (May 25 2025 at 17:23):

anyway, that seems like the way to go for the new compiler...any thoughts welcome on the subject!

Brendan Hansknecht (May 25 2025 at 17:38):

Yeah. I thought we already agreed on doing this. Specifically that everything going to the host must be concrete or behind a pointer. So List model should also be fine.

Brendan Hansknecht (May 25 2025 at 17:40):

I think (but don't know for sure) that we can optimize Box.map() to avoid unboxing and re-boxing, such that in update in there, we only end up allocating a single model in practice for the life of the program, and then updating that one in-place many times

Probably not quite without invasive transforms that would require deep analysis, but minimizing cost should definitely be doable. Specific making sure the write back is correct is the hard part. Cause you may make the new struct in a subfunction that still has access to the original struct. So inplace mutations would be problematic. But we definitely can reduce a couple of copies

Richard Feldman (May 25 2025 at 18:03):

we agreed on it for closures but I don't remember it in the context of type variables...but either way, seems like the way to go! :smile:

Richard Feldman (May 25 2025 at 21:37):

Brendan Hansknecht said:

I think (but don't know for sure) that we can optimize Box.map() to avoid unboxing and re-boxing, such that in update in there, we only end up allocating a single model in practice for the life of the program, and then updating that one in-place many times

Probably not quite without invasive transforms that would require deep analysis, but minimizing cost should definitely be doable. Specific making sure the write back is correct is the hard part. Cause you may make the new struct in a subfunction that still has access to the original struct. So inplace mutations would be problematic. But we definitely can reduce a couple of copies

yeah one way I think this could work well in practice is:

• we code gen a special version of Box.map for the specific case where it's given an a -> b function that doesn't change the input and output type
• in that version, if the refcount is 1, then we pass the closure a pointer to the struct which the closure can mutate in-place, and there is no write-back
• that reduces the hard part to "how do we detect that the struct on the stack inside the closure is unique?" - previously Morphic would have done this, but we could make use of the fact that the pointer we'll have will be to an address with a refcount right in front of it (bc it's pointing to the memory inside the Box) which means we can look at that refcount to determine uniqueness. This would require an extra Layout for like a "refcounted struct" but I believe it could work!

Brendan Hansknecht (May 26 2025 at 02:03):

Sure, but in most cases, you aren't guaranteed to finish reading the struct before trying to edit it.

In practice, I assume the function would be like this:

1. No matter the refcount, if the struct is large, pass by constant reference. So just pass a pointer to the data
2. read the data and create a new version on the stack
3. if the refcount is 1, write back to the original box.

Richard Feldman (May 26 2025 at 02:20):

hm, could be

Last updated: Jul 26 2025 at 12:14 UTC