Stream: ideas

Topic: module params

Richard Feldman (Aug 05 2023 at 03:03):

here's a proposal for how platform-agnostic effects (among other things) could work in Roc: https://docs.google.com/document/d/110MwQi7Dpo1Y69ECFXyyvDWzF4OYv1BLojIm08qDTvg/edit?usp=sharing

all feedback welcome!

Ajai Nelson (Aug 05 2023 at 07:31):

What's funny is that in some ways, this seems super similar to the ML-style "strong" modules we were just talking about here! (I guess this really proves that the blog post linked there didn't do a good job of conveying the benefits of ML-style modules. This is exactly the use case for them.) If I understand right, Menu.roc is very similar to an OCaml functor (which is basically a "function" that takes a module and returns a new module).

Are the types of echo and read inferred in Menu.roc? Would it be possible to specify them like this?

module {
    echo : Str -> Task {} [],
    read : Task Str [],
} -> [menu]

menu : Task {} []
menu =
    input <- read |> Task.await
    echo "You entered: \(input)"

Richard Feldman (Aug 05 2023 at 11:05):

we could allow type annotations on them, sure

Richard Feldman (Aug 05 2023 at 11:05):

but they could be inferred

Martin Stewart (Aug 05 2023 at 14:25):

I might be missing something but is this proposal just syntax sugar for having a package with a function in it that takes all the effects it needs as parameters and returns a record containing all the functions in that module?

In the proposal you have this example

module { echo, read } -> [menu]

menu : Task {} []
menu =
    input <- read |> Task.await
    echo "You entered: \(input)"

app [main] imports [echo, read] from platform "https://…"

import Menu { echo, read }

main =
    {} <- echo "Welcome!" |> Task.await
    {} <- Menu.menu |> Task.await

    echo "Bye!"

but could it not be instead be written like this?

module [ getFunctions ]

getFunctions =
    \{ echo, read } ->
        { menu :
                input <- read |> Task.await
                echo "You entered: \(input)"
        , # additional functions are added as additional fields
        }

app [main] imports [echo, read] from platform "https://…"

import Menu

menuFuncs = Menu.getFunctions { echo, read }

main =
    {} <- echo "Welcome!" |> Task.await
    {} <- menuFuncs.menu |> Task.await

    echo "Bye!"

Since it's just one big record with all the functions, it seems like it would only be slightly more inconvenient than this proposal but would have the advantage of already being possible and not requiring any new syntax.

Edit: Another disadvantage with the record approach is that, unless it's possible to add documentation for record fields, it's hard to document the package API

Brendan Hansknecht (Aug 05 2023 at 14:37):

I think that is essentially correct analysis. Though also, the custom version should also have a bit more perf because the functions are required to be done at compile time during import. So no runtime closures or anything of that nature.

Brendan Hansknecht (Aug 05 2023 at 14:41):

Also, I think this is clearer(simpler? More understandable? Not sure exactly what wording I want to use) than the record version.

Brendan Hansknecht (Aug 05 2023 at 14:42):

I like the proposal besides hosts.

Instead of working on more features for effects, can we push for fixing bugs and getting effect interpreters working. That way we can just move to what we want long term and remove all the extra effect functions.

Brendan Hansknecht (Aug 05 2023 at 14:45):

Oh, one other note, it doesn't guarantee you can see all effects used in a module at a glance. A user could still opt to pass an effect function directly into a function exposed by a module. So you are guaranteed the effect functions are always passed in at some point, but not necessarily on import.

Anton (Aug 05 2023 at 16:49):

Instead of working on more features for effects, can we push for fixing bugs and getting effect interpreters working. That way we can just move to what we want long term and remove all the extra effect functions.

That seems smart. If I understand correctly, effect interpreters would allow us to use the state machine approach as described in the document: "The package gives the application author a state machine of continuations representing the chaining logic. The application author writes an interpreter for this state machine using the Tasks it already has access to from its platform. "?

Brendan Hansknecht (Aug 05 2023 at 17:44):

Yes. That sounds correct

Richard Feldman (Aug 05 2023 at 17:45):

Folkert and I have been pairing on effect interpreters about once a week for coming up on a year - they're close, but it's a very long tail of "oops we gotta take a detour to address that before we can proceed"

Richard Feldman (Aug 05 2023 at 17:46):

really when we say effect interpreters the feature we're talking about is glue being able to generate the right thing for records of closures

Brendan Hansknecht (Aug 05 2023 at 17:48):

Hmm. I thought it would be a tag of data and continuations, but I guess records of functions hits the same issues. Also, I know we still have some type issues blocking this.

Richard Feldman (Aug 05 2023 at 17:48):

but yeah in general this proposal is more about figuring out the right direction to work toward than prioritization :big_smile:

Brendan Hansknecht (Aug 05 2023 at 17:48):

Yeah, if it is still a lot of work, changing this sooner sounds nicer.

Agus Zubiaga (Aug 05 2023 at 19:33):

Martin Stewart said:

I might be missing something but is this proposal just syntax sugar for having a package with a function in it that takes all the effects it needs as parameters and returns a record containing all the functions in that module?

I like this about it actually. Since it’s just another way to provide input to functions, it’s straightforward to reason about.

Agus Zubiaga (Aug 05 2023 at 19:38):

It’s similar to how backpassing gives us the ergonomics of do-notation in Haskell while keeping the type system simple

Chris Duncan (Aug 07 2023 at 15:57):

Is the three argument Task type being phased out? I can see how it wouldn't be relevant for these platform-agnostic effect modules, since the consumer of the module would determine what effect the module can use, but I can see the value of explicitly stating how an effect flows through the application through the type signature. If I have a bug around the Reading effect, I can safely ignore the part of my application that doesn't use that effect.

Richard Feldman (Aug 07 2023 at 16:01):

yeah so it turns out that because Task needs to be a builtin for certain concurrency primitives to work, we have to pick a number of parameters for it, and 2 seems better than 3 overall

Chris Duncan (Aug 07 2023 at 16:49):

Will there be a way to express through the type system the effects used? It could be a distinction between a safe Task and an unsafe Task.

I'm looking forward to seeing this concept coming to Typescript through the Effect library even though it's not enforced on the language level. I was looking forward to the same in Roc, but I see there's a constraint that prevents that.

Richard Feldman (Aug 07 2023 at 18:48):

we could, but currently I'm not convinced it's a net positive...what are the things you'd be excited about for having it at the type level in addition to a more customizable one at the value level? :thinking:

Chris Duncan (Aug 07 2023 at 23:32):

Just like we surface errors to the type level, I was excited to see the effects in the type level. As I mentioned, I could better understand how an effect flows through the application through the type signature.

Something that's not done now, but I could imagine in the future if we follow this path, is that when I'm testing effectful code, the type can tell me what stateless alternatives (mocks) are missing to cover. Just like how we handle every single error we accumulate in the application.

Agus Zubiaga (Nov 22 2023 at 15:18):

@Richard Feldman It looks like this proposal is going to happen. Do you want to wait until some of the other efforts are completed or could we start working on at least parts of this?

Richard Feldman (Nov 22 2023 at 15:18):

definitely can start anytime!

Richard Feldman (Nov 22 2023 at 15:19):

if anyone's interested in getting involved on that lmk and we can coordinate! :smiley:

Agus Zubiaga (Nov 22 2023 at 15:20):

Nice! I’m considering it because this unlocks so much and also there are a few bugs related to imports that we could fix while rewriting this part

Richard Feldman (Nov 22 2023 at 15:23):

great point! :100:

Richard Feldman (Nov 22 2023 at 15:24):

yeah lmk if you'd like to talk specifics!

Agus Zubiaga (Nov 22 2023 at 15:30):

Yeah, let's talk! I'll start familiarizing myself with what needs to change

Agus Zubiaga (Dec 01 2023 at 14:37):

I just realized the syntax for exposed names is different in the proposals:

Module Params:

import Menu { echo, read } as Mnu exposing [menu]

Module and Package Changes:

import [CodePoint, first] from CodePoint as Cp

Which one do we actually want to go with? I already implemented the latter but it'd be pretty easy to change.

Brendan Hansknecht (Dec 01 2023 at 15:31):

Iirc the module paramos change had a specific reason so probably that syntax.

Richard Feldman (Dec 01 2023 at 17:43):

yeah agreed, I remember we'd discussed it and decided to try going with the exposing keyword like Elm has

Agus Zubiaga (Dec 01 2023 at 17:45):

Ah ok, cool. Something I like about the exposing syntax in module params is that autocompletion can give you better suggestions because the module name comes first.

Agus Zubiaga (Dec 01 2023 at 17:47):

It also makes the multi-line formatting simpler

Richard Feldman (Dec 01 2023 at 17:55):

great points, I hadn't thought of those!

Eli Dowling (Dec 02 2023 at 00:08):

Big fan of this one, the js style syntax is such a pain in the butt to write. If you want good autocomplete you have to write it kind of outside in, which is pretty awkward.

Richard Feldman (Mar 24 2024 at 19:24):

I just realized a potentially interesting technique that module params unlock. I'm not sure if it's a good idea or not, but it's definitely possible!

Richard Feldman (Mar 24 2024 at 19:26):

the basic idea is that whenever you have a big record of info that gets passed around to basically every function in a module as a matter of code (these are often named Env), you can make that record be a module param and now you don't need to pass it around to every function because it's already in scope for all of them

Richard Feldman (Mar 24 2024 at 19:27):

using inline import, you can do this without even exposing the fact that you're doing this to callers

Richard Feldman (Mar 24 2024 at 19:29):

just make an internal module that takes the Env module param, and then have a separate (non-parameterized) API module where the public functions essentially just initialize Env, import the internal module (providing the Env they just initialized), and then call the appropriate function(s) from that module

Norbert Hajagos (Mar 24 2024 at 20:15):

Haha, for the coming up book, I am working on simulation and almost every business logic function has a world parameter. I wouldn't put it in the book, but still cool that I could do that.

Agus Zubiaga (Mar 24 2024 at 20:53):

I imagined something similar for env variables or secrets in a web server. The main app module loads them and puts them in a record, and then inline imports the implementation passing the record as module params.

Brendan Hansknecht (Mar 24 2024 at 21:04):

I think that only works if your env variable is constant. I think that is unlikely to be the case for large records most of the time. Though probably varies some. Like config can be static and constant, but env normally is mutable.

Richard Feldman (Mar 24 2024 at 21:19):

yeah it's definitely more for config than something that changes :thumbs_up:

Agus Zubiaga (Mar 24 2024 at 21:40):

True. It doesn’t work if the module’s functions return a new Env. That said, the params only need to be constant in the scope the import is introduced. So in a web server again, you can use it for values that are constants in the scope of a request, such as user session details.

Brendan Hansknecht (Mar 25 2024 at 00:03):

Woah, inline imports work that ways....intresting

Kasper Møller Andersen (May 20 2024 at 08:16):

I really like the proposal, and I've got a bit of feedback:

Coming from Elm, it's normal for us to have big files which mix a bunch of different concerns. For example, the file represents some page in an app, and it contains rendering, business logic, server communications, and more.

If something similar emerges in Roc, I would probably want to lock down all HTTP communication pretty hard, so it would be natural to create one or more modules whose only purpose are to expose the API for talking to the server for example. This way it would be really easy to control the use of HTTP essentially, and using the right sandboxing for it.

All of this is very good so far, but I think what's missing is: do I now need to check that every import of this API module uses the sandboxed HTTP client? Or could I do something like:

app [main] imports [stdio] from platform "https://…"

echo : Str -> Task {} []
echo = \str -> stdio.writeLine stdio.stdout str

reexport Foo { echo } as Foo

In other words, I only give my sandboxed effect to the module once, and then I expose that fully populated module to the rest of my application while making the un-populated module unavailable (in this case using shadowing, but could also be done without). This way I don't have to audit all imports of the module to see that everyone is importing it correctly.

Kasper Møller Andersen (May 20 2024 at 08:28):

Richard Feldman sagde:

the basic idea is that whenever you have a big record of info that gets passed around to basically every function in a module as a matter of code (these are often named Env), you can make that record be a module param and now you don't need to pass it around to every function because it's already in scope for all of them

Richard Feldman sagde:

using inline import, you can do this without even exposing the fact that you're doing this to callers

When you say this doesn't get exposed to callers, I'm assuming you mean that module A takes Env as a module param, and in your function foo in module B, you import A while providing an Env instance, and this is invisible to callers of B.foo?

Kasper Møller Andersen (May 20 2024 at 09:13):

I'm concerned about using module parameters to import non-effect parameters though, when the only argument for them is convenience. Coming back to Elm experience, many files can have a wide scope, and getting something like Env for the entire file potentially means many functions having access to it even though they don't need to. If _all_ functions in that module genuinely use Env, then it feels nice of course, but if the module exposes any functions which do not need it, you start requiring people provide Env to call them, even when they don't need it.

This can have different negative consequences:

• Testing functions which require Env, but don't use it, now becomes more cumbersome
• You can no longer reason about a function by looking only at its signature. For performing effects, the function must still return a Task, so even if the module imports an HTTP client, you can still tell when a function uses an effect at least (just not which one necessarily). But to see if a function uses Env as provided in a module param or not, you have to read the body of the function.
• I think this encourages viral spread for things like Env. Say you've got a module that takes Env as a param, and now you expose a new function bar to the module, but bar doesn't need Env. Do you refactor the entire module to now only take Env as function params, so bar doesn't get polluted? Probably not. As modules grow larger, refactoring them away from module params becomes harder, and it becomes easier to just live with having it, even if it's no longer the best option.
• It creates two different ways of passing arguments to a function. As a caller, I would be annoyed if I had to write something like

func = \env ->
    import Thing { env }
    Thing.otherFunc

to call such a function, instead of just

func = \env -> Thing.otherFunc env

In the same code base, you may also get both styles present, so you have to do the little song and dance of figuring out which style is used for the function you're about to call.

Anthony Bullard (Nov 30 2024 at 13:05):

I see a lot of code around this in the compiler, but I have never seen any documentation of it, or use in the wild. Is this still a valid feature in current Roc?

Anton (Nov 30 2024 at 13:14):

Yes, it's quite new. @Agus Zubiaga may know about real world usage examples

Oskar Hahn (Nov 30 2024 at 13:39):

The AoC template from Luke uses the feature.

Anton (Nov 30 2024 at 14:25):

For convenience; it's used here

Last updated: Jun 16 2026 at 16:19 UTC