tarkah (Jun 27 2024 at 17:28): tarkah (Jun 27 2024 at 17:28):I'm working on a platform to write iced applications in Roc. I've got a working MVP (see demo below), but there's a huge caveat that needs addressing before I can progress this further. I have an element type of Elem message where message is a user defined message that elements emit, which the platform passes in during update along w/ the user defined model to update it. Elements like checkbox or text input require a user defined closure, such as onToggle : Bool -> message or onInput : Str -> message.
The problem I'm facing is that the closure data stored on these fields _changes_ depending on the number of user defined elements / closures exist at compile time since the closure data seems to just be an index which references the appropriate underlying implementation. If I have >i8 number of closures defined, it becomes i16, etc. I'm able to somewhat deal w/ this by using the following glue, where I define closure_data to be up to 8 bytes, then use the relevant size function to slice it down appropriately before passing it into the function.
#[repr(C)]
#[derive(Clone, Debug)]
pub struct RocFunction_72 {
closure_data: [u8; 8],
}
impl RocFunction_72 {
pub fn force_thunk(mut self, arg0: bool) -> RocBox<ffi::c_void> {
extern "C" {
fn roc__mainForHost_3_size() -> i64;
fn roc__mainForHost_3_caller(
arg0: *const bool,
closure_data: *const u8,
output: *mut RocBox<ffi::c_void>,
);
}
let size = unsafe { roc__mainForHost_3_size() as usize };
let clousure_data = &mut self.closure_data[..size];
let mut output = core::mem::MaybeUninit::uninit();
unsafe {
roc__mainForHost_3_caller(&arg0, clousure_data.as_ptr(), output.as_mut_ptr());
output.assume_init()
}
}
}
This is obviously super hacky and doesn't work when the closure data isn't the last field of the element. Is there any way to force roc to output a stable size for closure data?
Richard Feldman (Jun 27 2024 at 19:26):in theory, wrapping it in a Box should do that, but in practice @Folkert de Vries and I ran into compiler bugs when we tried that, and we haven’t solved them yet
tarkah (Jun 27 2024 at 21:19):Richard Feldman said:
in theory, wrapping it in a
Boxshould do that, but in practice Folkert de Vries and I ran into compiler bugs when we tried that, and we haven’t solved them yet
So I just tried this and it seems to work, except maybe I hit the same compiler bug. The llvm-ir looks like it's doing everything correctly, but when I run the program, it seems to always invoke _one_ of the closures defined, regardless of which element I interact with. When I only have 2 elements / closures, it always invokes the first elements closure. When I have 3, the 2nd elements closure.
This seems promising though, I'll continue to dig into what's going on.
tarkah (Jun 27 2024 at 21:45):For example, here is where the closure data is boxed up in the llvm-ir. I have 2 closures defined, so it makes sense it's optimized to a bool representation.
%call = call fastcc ptr @Box_box_e6845638e158b704aa8395d259110713932beb5d7a34137f5739ba7e3dd198(i1 false), !dbg !200
..
%call21 = call fastcc ptr @Box_box_e6845638e158b704aa8395d259110713932beb5d7a34137f5739ba7e3dd198(i1 true), !dbg !200
Then how that closure data is handled:
define internal fastcc ptr @Types_59_bfa1d47a221bdaf089999196bed323c433d1a6b8c78ec612e6fa7b3e3d811(i1 %state, { ptr, {} } %"#arg_closure") !dbg !151 {
entry:
%result_value1 = alloca { [0 x i64], [3 x i64], i8, [7 x i8] }, align 8, !dbg !152
%result_value = alloca { [0 x i64], [3 x i64], i8, [7 x i8] }, align 8, !dbg !152
%struct_field_access_record_0 = extractvalue { ptr, {} } %"#arg_closure", 0, !dbg !152
%call = tail call fastcc i1 @Box_unbox_cb411178cb7686889a4ee0e4b4c57e63975186dc9f1448b79e94c2721a21a2(ptr %struct_field_access_record_0), !dbg !152
br i1 %call, label %then_block, label %else_block, !dbg !152
then_block: ; preds = %entry
call fastcc void @"#UserApp_25_68697e959be5e5da06cc73b6f998e193cbf2d9b22efd0355a3d37129951b"(i1 %state, ptr nonnull %result_value), !dbg !152
br label %joinpointcont, !dbg !152
else_block: ; preds = %entry
call fastcc void @"#UserApp_23_4e123451c288c52798d3df0fc84811d2d957f324242982575c70dfd6d338df"(i1 %state, ptr nonnull %result_value1), !dbg !152
br label %joinpointcont, !dbg !152
joinpointcont: ; preds = %else_block, %then_block
%joinpointarg = phi ptr [ %result_value, %then_block ], [ %result_value1, %else_block ], !dbg !152
%call2 = call fastcc ptr @Box_box_7f7e162ee4345c12acb2c8dddfd129c8c9ef562ecb31841cfff13d4789ffc2(ptr nonnull %joinpointarg), !dbg !152
ret ptr %call2, !dbg !152
}
This all looks correct :thinking:
Brendan Hansknecht (Jun 28 2024 at 03:18):Wait, are you trying to fit all closure in the same static size box?
Why not just allocated the space that the closure writes its data to at runtime? Could even pool them to avoid too many allocations. You just have to query the size once at startup technically.
Brendan Hansknecht (Jun 28 2024 at 03:19):The data will also include any captured variables which could hugely increase the size
Brendan Hansknecht (Jun 28 2024 at 03:20):Also, if you don't need your closure to be long lived, you can just grab stack space before loading and execution instead of allocating at all
tarkah (Jun 28 2024 at 19:37):@Brendan Hansknecht I'm not sure I follow 100%, but the closure data is part of the Elem I'm returning to the platform. Notice Checkbox { onToggle : Box (Bool -> message) }
Elem message : [
Column (List (Elem message)),
Checkbox { label : Str, isChecked : Bool, onToggle : Box (Bool -> message) },
..
]
I then return an element to the platform, where I've defined 2 checkboxes (so the closure data gets optimized to become a bool).
view : Model -> Elem Message
view = \model ->
Container {
content: Column [
Text "Roc + Iced <3",
Button {
onPress: IncrementCount,
content: Text "Pressed $(Num.toStr model.count) times",
},
Checkbox {
label: "Foo",
isChecked: model.fooEnabled,
onToggle: box FooToggled,
},
Checkbox {
label: "Bar",
isChecked: model.barEnabled,
onToggle: box BarToggled,
},
TextInput {
content: model.input,
width: Fixed 150,
onInput: box Input,
onSubmit: Submit,
},
],
width: Fill,
height: Fill,
centerX: Bool.true,
centerY: Bool.true,
}
However I need to box message before returning to the platform, so I have this map function:
map : Elem a, (a -> b) -> Elem b
map = \elem, mapper ->
when elem is
Checkbox { label, isChecked, onToggle } ->
Checkbox { label, isChecked, onToggle: box \state -> mapper ((unbox onToggle) state) }
.. main.roc ..
view : Box Model -> Elem (Box Message)
view = \model ->
program.view (unbox model)
|> map box
With that context in mind, I believe I've found the bug! It appears within my map function, unbox isn't getting called in the llvm-ir, it's simply copying the old box data pointer into the new box, so I'm left with a _nested_ box. When calling mainForHost_3_caller (the function for this closure), it doesn't know about the nested box (it must believe the unbox occurred) and tries to access the data pointer as bool, which is invalid.
To workaround this, I can do the following and my boxed closures now work. Notice I read outer box data, which is a pointer to the nested box, and use the nested box, which is the what the llvm-ir is expecting (a single box around the concrete closure data). However, this will probably fail if the user calls Elem.map as I assume the problem will compound.
#[repr(transparent)]
#[derive(Clone, Debug)]
pub struct RocFunction_72 {
closure_data: RocBox<ffi::c_void>,
}
impl RocFunction_72 {
pub fn force_thunk(&self, arg0: bool) -> RocBox<ffi::c_void> {
extern "C" {
fn roc__mainForHost_3_caller(
arg0: *const bool,
closure_data: *const RocBox<ffi::c_void>,
output: *mut RocBox<ffi::c_void>,
);
}
let mut output = core::mem::MaybeUninit::uninit();
unsafe {
let nested_box: RocBox<std::ffi::c_void> =
std::ptr::read(self.closure_data.deref() as *const _ as *const _);
roc__mainForHost_3_caller(&arg0, &nested_box, output.as_mut_ptr());
std::mem::forget(nested_box);
output.assume_init()
}
}
}
So it seems there's something fishy with calling unbox inside a closure and then box'ing that up. The relevant snippet from Elem.map:
onToggle: box \state -> mapper ((unbox onToggle) state)
Yeah, I missed that the closure data was deeply nested into the model. That is an important distinction.
That said, you theoretically could build a one off dynamically sized type to build out the list without any boxing.....
Brendan Hansknecht (Jun 28 2024 at 20:12):Honestly not sure if rust would be happy with that/what hoops you would need to jump through
Brendan Hansknecht (Jun 28 2024 at 20:12):Boxing is definitely easier (and honestly might be better due to how large a closure capture could become)
tarkah (Jun 29 2024 at 15:45):I've got this fully working now. The boxed closure wasn't an issue, nor was the llvm-ir incorrect, I just misunderstood what i needed to pass back to Roc.
I was passing in *const RocBox<ffi::c_void> / the boxed closure data stored on the returned Elem. However Roc expects to receive just the closure data when invoking the closure, not the boxed closure data. So instead I pass in *const ffi::c_void and all is well.
It seems everything is working, just a misunderstanding on my part. Between this and ensuring I properly manage refcounts, it's been quite a fun adventure to get everything working as intended. Thank you both for taking a look!
Last updated: Jul 05 2025 at 12:14 UTC