Oskar Hahn (Jan 12 2025 at 09:05): Oskar Hahn (Jan 12 2025 at 09:05):If you assume, that you have a platform, that could call effects in parallel, for example send to http-requests at the same time. How could you call this from roc?
Of cause, you could create an effect like doTwoHttpRequests : Request, Request -> (Response, Response), but I want to do it more generic, so it not just works on Request, but on all Effects that do IO, like reading a file and sending a Request at the same time.
I mean something similar to Cmd.batch in elm.
I don't think that List.forEach! can help here, since it would require Roc to start the greenthread instead of letting the platform handle it.
Before purity inference, Roc would have created lambdas for each effectful function and return them at once, then the platform could have called each lambda in parallel. But with purity inference, the platform gets called immediately. So the platform can not know, that [doStuff1!, doStuff2!, doStuff3!] |> List.forEach! is actually one call and each effect should be called in parallel but returned when all are finished.
Before purity inference, the idea was, that Roc returns tags and the platform handles them like a state machine. I can see, how parallel effects where possible in this world.
Is there a plan/story, how Roc can do parallel effects?
Oskar Hahn (Jan 12 2025 at 09:11):The documentation on List.forEach! has this example:
List.forEach! ["Alice", "Bob", "Charlie"] \name ->
createAccount! name
log! "Account created"
Could the current or a future version of Roc support something like List.forEachParallel! that gives all the effectful functions to the platform, so the platform can call them in parallel, if it supports it.
Sam Mohr (Jan 12 2025 at 09:14):The current plan:
Sam Mohr (Jan 12 2025 at 09:15):In short, instead of implementing state machines under the name of effect interpreters
Sam Mohr (Jan 12 2025 at 09:15):We just steal golang's green thread model
Sam Mohr (Jan 12 2025 at 09:15):And run stuff that way
Luke Boswell (Jan 12 2025 at 09:15): Sam Mohr (Jan 12 2025 at 09:17):It depends on the platform to do a lot of the implementation work, but then your List.for_each_parallel! would work pretty easily with a platform that supports such stackful coroutines
Oskar Hahn (Jan 12 2025 at 10:11):Thank you for linking the other thread. I missed that. I will ask my question there to keep the discussion in one place.
Brendan Hansknecht (Jan 12 2025 at 16:44):Yeah, I think the easier answer is that platforms will need to have some sort of threading solution (green or otherwise) that roc can interact with via effects
Brendan Hansknecht (Jan 12 2025 at 16:44):Also would be valid for a platform to expose something that takes a lists of commands and runs them in parallel under the hood
Oskar Hahn (Jan 15 2025 at 16:33):From @Brendan Hansknecht answer in the other thread I understand, that the platform has to return a lambda to the host, that the host can call in another thread.
I am trying, if this works in the current version of roc. I getting errors and I am not sure, if I am doing something wrong or if this is currently not possible.
The platform has the effect do_foo! : Box ({} => {}) => {}. It is called like Host.do_foo! Box.box(\_ -> Host.stdout_line!("in the box")). The platform also exposes this function to the host:
doer! : Box ({} => {}) => {}
doer! = \boxed_fn! ->
fn! = Box.unbox boxed_fn!
fn!({})
For the moment, there is no concurrency or threading involved. I just try to run the lambda with this doer! function. But I only get: panic: a Lambda Set is empty. Most likely there is a type error in your program.
I can see in the emitted llvm-ir, that Roc exports roc__doer_0_caller(ptr %0, ptr %1, ptr %2) and roc__doer_1_exposed(ptr %0). I tested it with both of them.
roc_doer_1_exposed calls the following function, which seems to always panics. The given argument does not seem to be relevant
@_str_literal_14773762520492002118 = private unnamed_addr constant [81 x i8] c"\00\00\00\00\00\00\00\00a Lambda Set is empty. Most likely there is a type error in your program.", align 8
define internal fastcc {} @"_doer!_4fe2c0cee861629d2ef04c3f725dba5813b563598f88e6fe57cefd4dd1a133"(ptr %"boxed_fn!") !dbg !51 {
entry:
%const_str_store = alloca %str.RocStr, align 8
%result_value = alloca {}, align 8
call fastcc void @Box_unbox_99e2ebbd98e8a2a4c7ed9bd71d205d9f7b5d7e7a9ddb68dab65f2ad1c2198b(ptr %"boxed_fn!", ptr %result_value), !dbg !52
store %str.RocStr { ptr getelementptr inbounds (i8, ptr @_str_literal_14773762520492002118, i64 8), i64 73, i64 73 }, ptr %const_str_store, align 8, !dbg !52
call void @roc_panic(ptr %const_str_store, i32 0), !dbg !52
unreachable, !dbg !52
}
roc__doer_0_caller calls the following function, that also always panics:
define internal fastcc {} @_48_f0adb8f180253d489b50ac5199522556362f583929ee5e65c919bd9ed2bc82f({} %"50", ptr %"#arg_closure") !dbg !141 {
entry:
%const_str_store = alloca %str.RocStr, align 8
store %str.RocStr { ptr getelementptr inbounds (i8, ptr @_str_literal_14773762520492002118, i64 8), i64 73, i64 73 }, ptr %const_str_store, align 8, !dbg !142
call void @roc_panic(ptr %const_str_store, i32 0), !dbg !142
unreachable, !dbg !142
}
It seems, that Roc generates code for doer!, that will always panic. Is this correct? Is there a way in today's Roc, that allows me to return a lambda and call it?
Brendan Hansknecht (Jan 15 2025 at 16:42):The short answer is that this isn't supported today.
The medium length answer is that I don't think boxing of closures and passing them back into roc works. You probably need to handle the closure unboxed and handle calling it directly from the platform. That said, it still wouldn't be necessarily safe to use in a multithreaded context. It might capture variables from the other thread. The variables do not have atomic refcounting and would potentially hit use after free or double free bugs.
Brendan Hansknecht (Jan 15 2025 at 16:43):I'm also not fully sure if effects generate all the necessary closure infrastructure. Directly exposed functions should, but not sure about effects.
Oskar Hahn (Jan 15 2025 at 16:48):Yea. I know about not having atomic refcouting. But wanted to see, what is currently possible, and how far I can go.
Without the boxing in doer!, the code does not compile. It crashes with
thread 'main' panicked at /home/ossi/.cargo/git/checkouts/inkwell-946411d814d2c9f8/89e06af/src/values/enums.rs:325:13:
Found StructValue(StructValue { struct_value: Value { name: "", address: 0x5bc65f74d1f8, is_const: false, is_null: false, is_undef: false, llvm_value: "{} %1", llvm_type: "{}" } }) but expected PointerValue variant
stack backtrace:
0: rust_begin_unwind
1: core::panicking::panic_fmt
2: inkwell::values::enums::BasicValueEnum::into_pointer_value
3: roc_gen_llvm::llvm::build::expose_function_to_host_help_c_abi
4: roc_gen_llvm::llvm::build::build_procedures_help
5: roc_gen_llvm::llvm::build::build_procedures
6: roc_build::program::gen_from_mono_module
7: roc_build::program::build_loaded_file
8: roc_build::program::build_file
9: roc_cli::build
10: roc::main
I thought, that the box is necessary, since all arguments need a known size.
Is there another way to call the lambda?
Brendan Hansknecht (Jan 15 2025 at 17:45):My guess is that we don't generate closure callers for effects. Theoretically you would remove doer!, unbox the lambda passed to do_foo! and roc would generate a closure callers for the lambda passed to foo, but I bet closure callers are only generated for closures return directly to the host.
Oskar Hahn (Jan 15 2025 at 17:57):Yes. I already tried this. There is no _caller function generated for the do_foo! effect.
Do you think there is a chance, I can convert the lamda to a go function and call it? I think this seems unlikely.
Or can I somehow manually implement the do_foo_caller function? Maybe in zig and then load the zig implementation as a shared library?
Brendan Hansknecht (Jan 15 2025 at 18:28):Yeah, I think it isn't supported then.
Last updated: Jul 05 2025 at 12:14 UTC