Stream: beginners

Topic: Does roc support asyncronous function calls into the host?


view this post on Zulip mainrs (Sep 06 2025 at 09:38):

I had the idea of creating a web crawler platform 2y ago. At that time, there was no way for crawlers to call into the host in parallel. This can be useful while processing the response of a website. The crawler could ask the host to parse and fetch elements according to css or xpath selectors. This can be done using a list and map, but the call would happen synchronous on the host side. It would receive every map call one after the other.

There was discussion about Task.map2, but I've read the changelogs and couldn't find map2 nor Task in general. It seems that it has been removed.

Would be great if someone with a little bit more knowledge than me could quickly chime in and answer this. At least I couldn't find any references to async/await.

view this post on Zulip Anton (Sep 06 2025 at 09:44):

@Richard Feldman

view this post on Zulip Richard Feldman (Sep 06 2025 at 11:42):

short answer is that this should be very ergonomic once the new compiler gets there! I have a WIP design doc for it. Can you remind me what some of the code would look like in a different language? I can translate it into what it'll look like in the upcoming design :smiley:

view this post on Zulip mainrs (Sep 07 2025 at 09:00):

Richard Feldman said:

[...]. Can you remind me what some of the code would look like in a different language? I can translate it into what it'll look like in the upcoming design :smiley:

I am looking for two types of parallel processing. One would be async/await for I/O-heavy operations like file reading, networking etc..

The other one would be parallel CPU intensive tasks, usually done on thread pools. In Rust, the first would be done using tokio, the second using rayon.

This is already possible in roc right now, but one has to sacrifice ergonomics for it. I can simply introduce platform-provided methods that take a list of arguments. And return a list of results. One can introduce custom types to bundle the arguments, but it makes working with the result types and arguments cumbersome in a functional way...

fn get_requests(urls: Vec<String>, bodies: Vec<Vec<u8>>,  headers: ....);
fn process_xpath(requests: Vec<Requests>, xpaths: Vec<String>, ....);

I think it would be nice to have the parallelism inside of roc itself instead of having to delegate it to the host.

view this post on Zulip Richard Feldman (Sep 07 2025 at 13:23):

some hosts can't support parallelism, but my goal is to make it easy to say "these can run in any order" so that it will run in parallel if the host supports it

view this post on Zulip mainrs (Sep 07 2025 at 14:15):

Richard Feldman said:

some hosts can't support parallelism, but my goal is to make it easy to say "these can run in any order" so that it will run in parallel if the host supports it

Thanks for letting me know! Are there any issues / resources I can keep an eye on to stay up-to-date? :)

view this post on Zulip Richard Feldman (Sep 07 2025 at 14:37):

I still need to finish writing it up, but I'll definitely post about it once it's done!


Last updated: Sep 09 2025 at 12:16 UTC