Stream: beginners

Topic: Understanding Effects

Luke Boswell (Oct 30 2022 at 08:33):

I’ve been playing with the Roc platforms and exploring examples a bit more. I have a few questions and would appreciate any assistance with some of my questions below. I think effects may be a new addition to Roc and just haven’t made their way into the docs yet. I’m probably just a bit early with these questions; but thought I’d share what I’ve discovered so far. I’m keen to help out with the docs if I can… still learning and trying to improve my understanding.

What does it mean for a .roc file to be hosted as in CLI’s Effect? It looks like this module declares functions to make them available in Roc; the function definition is then implemented in the host. The keywords generates and with instruct the compiler to generate these functions in the host, and after, map, always, forever, loop tell the compiler to generate these functions for the Effect, which makes them available to other Roc modules. What is the difference between a Task and an Effect? They seem very similar, though it looks like all of the Tasks are implemented using Effects. Would it be possible to not use Tasks at all?

hosted Effect
   exposes [
       Effect,
       after, map, always, forever, loop,
       fileReadBytes # pub extern "C" fn roc_fx_fileReadBytes(roc_path: &RocList<u8>) -> RocResult<RocList<u8>, ReadErr>
   ]
   imports [InternalHttp.{ Request, Response }, InternalFile, InternalDir]
   generates Effect with [after, map, always, forever, loop]

fileReadBytes : List U8 -> Effect (Result (List U8) InternalFile.ReadErr)

From crates/compiler/parse/src/header.rs the possible module types are;

• app is the main Roc module which defines the app, see tutorial
• hosted provides function types that are implemented in the host
• platform is the main Roc module which defines the host
• interface is a pure Roc module which defines types and functions, see tutorial

It looks like you can only provide a single symbol from a platform to the host, typically named mainForHost. This is why CLI takes main of type InternalProgram, converts it to an effect to provide for the host. Similarly in breakout programForHost is a record of functions which is provided for the host.

Brendan Hansknecht (Oct 30 2022 at 15:17):

Yeah, most of this sounds pretty accurate

Brendan Hansknecht (Oct 30 2022 at 15:18):

Also, a Task tends to just be an Effect of a Result

Brendan Hansknecht (Oct 30 2022 at 15:18):

So it is managing the potential error state. It is not required at all

Brendan Hansknecht (Oct 30 2022 at 15:18):

Though it is generally useful to wrap the raw effect type and give it a better api

Brendan Hansknecht (Oct 30 2022 at 15:23):

As an extra comment, effects are not strictly needed and may go away at some point in favor of just returning a tag union with a closure (basically telling what you want the platform to do and a continuation). A more direct form of defining an effect.

Brendan Hansknecht (Oct 30 2022 at 15:23):

You already have to do that if you want to build a platform cleanly on top of async rust.

Richard Feldman (Oct 30 2022 at 16:25):

yeah it doesn't work yet, primarily because glue needs some new features to make it realistic, but the thing you'd return to the host would probably look something like this: https://gist.github.com/rtfeldman/ef5ced61ca2aa2e24355b306bd685c54

Brendan Hansknecht (Oct 30 2022 at 16:35):

Yeah, currently it would have to be done manually. So it is not the best experience.

Luke Boswell (Nov 02 2022 at 01:24):

Does that mean we won't need the hosted module type? I'm having trouble wrapping my head around the concept with just a tag and closure tbh. I can wait until we have an example using glue. Does the closure capture any data?

Richard Feldman (Nov 02 2022 at 02:08):

yeah the hope is that this will make hosted unnecessary, at which point we can take it out of the language!

Richard Feldman (Nov 02 2022 at 02:08):

so the basic idea is that the host receives an Op value like:

    FileReadUtf8 Str (Result Str [NotFound, Malformed] -> Op),

Richard Feldman (Nov 02 2022 at 02:09):

it says "ok, I see that the user has requested a FileReadUtf8 operation, so I'll read a file as UTF-8 using the path Str in the tag"

Richard Feldman (Nov 02 2022 at 02:10):

then once it gets the contents of the file - or an error - it says "okay, now I can call that (Result Str [NotFound, Malformed] -> Op) function using the Result of what happened with the file (either Ok and the contents of the file or Err if the file was not found, malformed, etc.) and it will return back to me a new Op to run"

Richard Feldman (Nov 02 2022 at 02:10):

then it can repeat the loop with the fresh Op that it got

Richard Feldman (Nov 02 2022 at 02:10):

each of the Op tags contains both info about what to do as well as a closure like this for what to do next

Richard Feldman (Nov 02 2022 at 02:11):

and the Task type assembles these Op chains while providing the same exact Task API you see today (with Task.await, File.readUtf8, etc. all having the same types they do today)

Richard Feldman (Nov 02 2022 at 02:11):

so to the app author, the API looks the same as today, but to the host author, they receive this Op value instead of the hosted stuff

Richard Feldman (Nov 02 2022 at 02:12):

besides simplifying the language, there are 2 other benefits to the Op design:

Richard Feldman (Nov 02 2022 at 02:13):

one, it lets the host do async stuff with as little overhead as Rust's async keyword (which was designed to be so low-overhead that you couldn't do better if you were handwriting a C async system)

Richard Feldman (Nov 02 2022 at 02:14):

because you can just do "oh I got the FileReadUtf8 tag, I'll enqueue an async file I/O operation, and then register that when it's done I want to run the advance-to-the-next-state closure, and then I can move on to do other things with the CPU while waiting for the async I/O to finish"

Richard Feldman (Nov 02 2022 at 02:14):

the other benefit is testability: since this is an ordinary Roc data structure, you can also do a "simulation" on it where you test your effectful logic without actually running the effects - just by traversing the data structure and making assertions about which things happen when

Luke Boswell (Nov 02 2022 at 07:41):

Thank you @Richard Feldman that was very helpful. I really like the look of Effects.

I think I am comfortable with passing a task to program. They feel like asynchronous callbacks and seem to be for initiating actions from app -> host. I recall a similar discussion on the logging thread about Html.log which seemed really useful functionality for a host to be able to provide.

However, when I try and apply these ideas to the Action-State concept I'm having trouble making sense of it again. I'm not sure if this changes the onPress handler for the button? e.g.

button {
        label: text "+",
        onPress: \prev, _ -> Action.update (prev - 1),
    }

Does the onPress here become an Op maybe something like Update (\prev, _ -> prev - 1) (Result {} [] -> Op) chained into a Render (Result {} [] -> Op)? How does the click event flow back in to update the model and the elements?

When the host initially render the world, it can recurse down the tree, and when it comes across this button element it can save the closure. In the future when the host detects a click over the top of this button element it calls this closure and passes in the latest state for the button somehow, and then re-renders. I'm a bit confused about Elem.translate which seems to be a pure function in the Roc app, so the host can't see it, helps to do incremental re-renders. How can we render just the button and not the whole tree again? Or does this imply we build the whole tree, and then diff against the previous view tree?

I feel like I've veered off course here, but I'm not sure where it's all gone wrong. I'm sure it will all be very clear when these new features drop and examples are updated.

Effects seem a bit like Elm's Ports (Cmds and Subs), however I am guessing they are quite different. Ports were tags with a Msg while these include a tag with a Model. It would be great to have an ELI5 on if/how they are different at some point.

I can see there are papers and video that explains the science behind effects system, so I am off to do some deeper research. :smiley:

Brian Carroll (Nov 02 2022 at 08:40):

As far as I can tell from the Action State document, the Action.update just updates the state and does no effects. There is actually no mention of any effects anywhere, (other than rendering). So there is no equivalent of Cmd in that system yet.

I think we might need to return an Effect along with the Action or something.

I have been thinking about this too because I'm building a system like this right now!

@Richard Feldman is this right? Do you have an extension to the design in mind?

Luke Boswell (Nov 02 2022 at 08:49):

I have been following your work and am very interested! Looks very cool.

Richard Feldman (Nov 02 2022 at 10:23):

oh, I was actually thinking of having something like:

Action.run : state, Task (Action state) [] * -> Action state

Richard Feldman (Nov 02 2022 at 10:23):

this should serve the same role as ( model, Cmd msg ) in Elm

Luke Boswell (Nov 03 2022 at 07:34):

Ok, I think I’m starting to understand. I’ve added corrections to my last post below, and expanded a bit.

Side-effects - Lazy Rendering

First if we look at Elem.lazy which has a side effect to cache views. It is defined as lazy: state, (state -> Elem state) -> Elem state and expands in the tree of elements sent to the host in render like Lazy (Result (Cached state) [NotCached] -> Cached state). Note that this is just a tag with a function closure. This function closes over the state that is passed in at the time lazy is called and saves it so that it can be compared. A Lazy tag isn’t a static element that can be rendered, the host needs to check the cache first. The host sees the tag in this element and it knows to check its cache, and then pass the saved value to the callback function. If there was something in the cache it will be an Ok (Cached state) value, and if not it will be a Err NotCached value. This is then passed into the callback function and the output will be a Cached state. The host can then store this value in its cache; and use it to render the rest of the elements in the sub-tree. The implementation of lazy checks the state that is saved in Cached state is Bool.isEq to the state that was passed into the lazy function at the time of render.

Without Side-Effects - Render Elements

The function button { onPress : \prev, _ -> Action.update (prev + 1) } (Text "+") expands to the following Button { onPress : \prev, _ -> Update (prev + 1) } (Text “+”) which is passed to the host in the tree of elements from render. The host displays a button and registers a click handler. When a press is detected over this element; the host passes in the value of the latest state, and the PressEvent into the onPress function. The result will be an Update callback action. Note that this is just a tag with a callback function. The host knows to pass the current state into this callback which gives an updated state, and then uses this to re-render the element. If this was a Delegate callback action instead then the host knows to use the updated state to re-render the parent element instead.

Side-effects - Using Tasks

Instead of using Action.update we can use Action.run : state, Task (Action state) [] * -> Action state to queue tasks for the host on button press. For example consider the following task;

myTask = \prev ->
    if prev > 10 then
        _ <- showModal "You click too much!" |> Task.await
        Task.succeed (Action.Update 0) # reset the state to zero
    else
        Task.succeed (Action.Update prev + 1)

button { onPress : \prev, _ -> Action.run prev myTask } (Text "+")

This expands into Button {onPress: (Run callback)} (Text “+”), and closes the task into a callback for later. If the button is pressed the host knows to call the callback and pass in the current value of the state. In this case it checks if the value of the counter is above 10 and if so it shows a modal to let the user know and resets that value, otherwise it increments the state by one.

Things to investigate further

How does the host know which element to cache on lazy? Is there a reliable way to automatically assign IDs to Elem state values? or would some kind of path from parent, or maybe a hash be necessary?
How does the host keep track of states and functions across Elem.translate boundaries?

Brian Carroll (Nov 03 2022 at 10:00):

In examples/gui/platform/Elem.roc the type signature is
Lazy (Result { state, elem : Elem state } [NotCached] -> { state, elem : Elem state })

Brian Carroll (Nov 03 2022 at 10:05):

But it seems like it should contain not just a function but also a cached value. Maybe like this:
Lazy (Result { state, elem : Elem state } [NotCached]) (Result { state, elem : Elem state } [NotCached] -> { state, elem : Elem state })

Brian Carroll (Nov 03 2022 at 10:10):

Or with type aliases

Cached state : { state, Elem state }
MaybeCached state : Result (Cached state) [NotCached]
Lazy (MaybeCached state) (MaybeCached state -> Cached state)

This puts the cache directly into the element tree, which seems a sensible place for it. Maybe Richard had a different design in mind though.

Luke Boswell (Nov 03 2022 at 10:33):

Hmm, that doesn't seem right to me. I'm definitely not sure about my understanding here, but that would read to me like the app is supplying a MaybeCached state to the host. But the cached value would be in the host, and the Roc app would never see the caching. :man_shrugging:

Brian Carroll (Nov 03 2022 at 12:31):

Well I suppose there's more than one way to do things! You can probably make it work either way. There might be different design tradeoffs in native graphics compared to virtual DOM.
With the virtual DOM I'm working on, I'm putting as much of it as possible in Roc, including the diff logic. In that design, the host doesn't need to know the structure of the Element tree. Of course it needs to store the tree, but that's just a matter of remembering a pointer and returning the same pointer to Roc next time. That way, Roc knows the structure but the host doesn't care about it.
The host only needs to understand the output of the diff.

Brian Carroll (Nov 03 2022 at 12:53):

For graphics I guess the host is already translating the Element tree into some kind of bitmap or GPU data, so it needs to understand that structure anyway. And then it can store the cached values too. I'm not sure what it's using as the cache key though? Maybe the state itself?

Last updated: Jul 06 2025 at 12:14 UTC