Sam Mohr (Jan 07 2025 at 05:40): Sam Mohr (Jan 07 2025 at 05:40):We have simplified the syntax for app modules and interface modules, but we didn't get to the other module types yet. Currently the hosted module header is
hosted Host
exposes [...] imports [...]
In the style of interface modules, we should move to
hosted [...]
where the [...] is the "exposes" list, and we use import statements like every other module type
Sam Mohr (Jan 07 2025 at 05:42):We will have updated every module header except for platform modules, and I'm not sure how to reduce those
Sam Mohr (Jan 07 2025 at 05:42):platform "cli"
requires {} { main! : List Arg.Arg => Result {} [Exit I32 Str]_ }
exposes [...]
packages {}
imports []
provides [main_for_host!]
This is the current one for basic-cli with the exposes section elided
Sam Mohr (Jan 07 2025 at 05:45):We can remove the "cli" name, remove "imports" in favor of import statements, and combine the two "requires" sections, but it seems not reducible after that
Sam Mohr (Jan 07 2025 at 05:45):platform
requires { main! : List Arg.Arg => Result {} [Exit I32 Str]_ }
exposes [...]
packages {}
provides [main_for_host!]
I think removing the words would just get confusing, but maybe there's something I'm missing
Sam Mohr (Jan 07 2025 at 05:47):The one other option I can consider for that one is to put everything in a single record for visual consistency with all other module types only having one or two sets of fields:
platform {
requires: { main! : List Arg.Arg => Result {} [Exit I32 Str]_ },
exposes: [...],
packages: {},
provides: [main_for_host!]
}
Thoughts?
Oskar Hahn (Jan 07 2025 at 07:46):When you are changing this both headers, could you consider the problem, that it is not possible to import the requires symbols in any other module of the platform. Especially not in the hosted module. One consequence is, that it is impossible to have effects that have a Modelas argument or return type.
For example, it is impossible for an platform to implement an STM as discussed in this thread. It would need effects like
get_model_for_reading! : {} => Model or do_stuff_with_Model! : (Model => a) => a. But currently, it is impossible for the hosted module to import the Model-symbol.
So my question is, could/should the hosted module header also have something like an requires argument? Or should the requires feature be redesigned in a way, that solves that problem? (and therefore would need a different header)
Sam Mohr (Jan 07 2025 at 08:22):I can understand wanting there to be platform functions that can access the requires types, but I don't think it's viable for the hosted module to send Model to the host.
Sam Mohr (Jan 07 2025 at 08:23):FFI needs to send/receive statically-sized types, and Model changes depending on the user's code outside of the platform
Sam Mohr (Jan 07 2025 at 08:24):Currently, we compile the platform before even sending it to the user as a code archive via GH releases, so there's not a performant way to make a dynamically-sized value available to the host
Sam Mohr (Jan 07 2025 at 08:26):That said, we could make it available but not allow sending it over the FFI boundary, instead requiring the platform to serialize/deserialize to and from List U8
Sam Mohr (Jan 07 2025 at 08:27):But then we're promoting low-perf behavior, right?
Sam Mohr (Jan 07 2025 at 08:27):Would love to hear a lower-level expertised opinion on this
Oskar Hahn (Jan 07 2025 at 09:12):To send a Model to the host, you have to box it. Its the same thing you have to do, when you use the Model in the exported functions. So an effect would actually look like get_model_with_lock : {} => Box Model and afterwords there could be another effect like release_lock : [Commit (Box Model), Rollback] => {} . In this example, the host would not need to do anything with the Model, just return the pointer. And the Roc part of the platform could provide a function like call_with_model : (Model -> a) => a that internally calls this both functions. But this function would also need access the the requires part from the header.
(I think I highjacked your topic. If you think, that my posts are off topic, you could move them.)
Sam Mohr (Jan 07 2025 at 09:20):Nah, they're on topic enough. I don't expect much resistance to aligning the format of the hosted module header to the standard module header, and this discussion could help us align the hosted and platform headers
Sam Mohr (Jan 07 2025 at 09:23):So anyway, the Box approach makes sense, thanks for outlining that to me.
Sam Mohr (Jan 07 2025 at 09:26):I believe there was recent discussion surrounding the desire for type variables in requires types, but we can't do it if we want all types used in the program to be concrete for runtime
Sam Mohr (Jan 07 2025 at 09:27):As long as requires types like Model need to be concrete, then what you're suggesting makes sense
Sam Mohr (Jan 07 2025 at 09:28):There was a spitball idea maybe 6-8 months ago from Richard to use modules params in the platform module to define the host functions, rendering the hosted module type redundant
Sam Mohr (Jan 07 2025 at 09:28):Since module params may be going the way of the dodo, I don't think it's wise for us to assume we'll have them long-term
Sam Mohr (Jan 07 2025 at 09:31):But something is of intrigue there: for each platform, the following things need to be done only once:
requires types and values/functions that are passed to the host via the provides clauseprovides functions using the requires entitiesroc-json, weaver, etc.) for the platform Sam Mohr (Jan 07 2025 at 09:32):In theory, all of these could be done in one, big module, but that would probably lead to having a really big module for mature platforms
Sam Mohr (Jan 07 2025 at 09:32):Though as you point out, the current hosted module separation prevents platform authors from being able to access requires entities
Sam Mohr (Jan 07 2025 at 09:34):I think we should either:
platform and hosted into a single module type (leads to bigger modules, but simplifies Roc a bit)requires clause to hosted modules as well, and give errors when platform and hosted disagree on the contents of the requires clauses Oskar Hahn (Jan 07 2025 at 09:38):There is another problem, with a combined platform and hosted modules. It is currently impossible to import stuff from it. For an platform to export something to the app, it has to be inside another module, so it can be imported with import pf.Module. If the exported functions would live inside the platform-module, they could not be used be an app or any other Module provided by the platform.
Sam Mohr (Jan 07 2025 at 09:38):I'd vote for the first one because it seems simpler, and I'm not convinced it'll be that much of a problem for big platforms to have everything in a single file since you tend to only have a couple functions that glue requires to provides, and the FFI definitions are only type signatures, not function bodies
Sam Mohr (Jan 07 2025 at 09:39):you can import Stdout from the platform module
Sam Mohr (Jan 07 2025 at 09:39):Oh, I see
Sam Mohr (Jan 07 2025 at 09:50):A solution we could push is to combine the two modules and require the platform module to be named Platform.roc or something like that. And then everyone can always just import Platform exposing [...]
Oskar Hahn (Jan 07 2025 at 09:56):Maybe the way, stuff gets passed around between the app and the platform has to be rethinked :grinning:
If you do, another (off topic) issue is, how a platform-Module can define, that something is exported to the app or exported only to other modules inside the platform. In my previous example, the platform only wants to export call_with_model. But the hosted-module exports get_model_with_lock and release_lock. How can the platform make sure, that the app can not import this effects directly? I think, basic-cli and basic-webserver use Modules with an Internal-Prefix. But If I understand it correct, then an app could import the stuff from the Internal-Modules as well.
An example where this is a problem is the current version of the kingfisher platform. The hosted module exports save_event! . But the app is only allowed to use it in a write request as defined here. If an app just imports pf.Host and uses this functions, all guaranties of the platform are off.
Go solves the problem with an internal directory. Maybe Roc could forbid apps to import stuff from the hosted (or combined hosted and platform) module and also forbid apps import from Modules, when the Name starts with Internal.
Oskar Hahn (Jan 07 2025 at 10:00):Sam Mohr said:
A solution we could push is to combine the two modules and require the
platformmodule to be namedPlatform.rocor something like that. And then everyone can always justimport Platform exposing [...]
If this Platform.roc-Module could also export Model then my initial problem would be solved. And if Roc forbids apps to import stuff from the Platform-module, then I would be fully happy :innocent:
Brendan Hansknecht (Jan 07 2025 at 14:42):Note, we should be able to pass a Box a to and from the host just fine. The specific problem is that we need to link the a from two different effects. This is currently done through requires, but it doesn't have to be.
Brendan Hansknecht (Jan 07 2025 at 14:44):For example, it would be easier to deal with basic webserver if the Model as actually model. Then you wouldn't have to worry nexted about type variables and some of the typing conplxity
Brendan Hansknecht (Jan 07 2025 at 14:45):To do this, we just need a way to specify that model from init! and respond! are the same type. The same would apply to many effects.
Brendan Hansknecht (Jan 07 2025 at 14:45):I think this would be far nicer to support in general and would prefer it over requires syntax.
Brendan Hansknecht (Jan 07 2025 at 14:46):Caveat. This only removes the use of requires for types. It would still be needed to pass app functions to the platform unless the platform just imports the app and calls something.
Brendan Hansknecht (Jan 07 2025 at 15:05):Note, we can work around this for platform main functions, but not for effects. For basic webserver init! and respond!, we can instead use a record { init!, respond! } and use the record to link the type. It would be nicer to also link effect types.
Oskar Hahn (Jan 07 2025 at 16:19):Wow. I was fixated on the dogma that Model has to be pass to the platform. Your idea seems much simpler.
I don't understand how this could work on effects or other exported functions. How could the platform specify, that the argument from do_stuff_with_Model! : (model => a) => a is the same as the return value from the provided function init! : {} => model?
Brendan Hansknecht (Jan 07 2025 at 17:53):Yeah, I'm not sure how we would wire it up, but fundamentally it is just propagating a constraint. (Just have to make that clear to the user and platform author somehow)
Dumb idea, but it could be a simple as a ! suffix links type variables in the platform
So:
do_something_with_model! : (model! => a) => a
And
init! : {} => model!
Richard Feldman (Jan 07 2025 at 18:00):yeah I like the way Elm does it and would like for us to do something that too - where it's just normal type variables and that's it
Richard Feldman (Jan 07 2025 at 18:01):e.g.
init! : {} => (model, (model, Request => (model, Response)))
so in that type, init! returns both the initial model as well as the function which takes the current model and a request and returns the new model and a response
Richard Feldman (Jan 07 2025 at 18:02):although as discussed somewhere else, that would have race condition problems :sweat_smile:
Richard Feldman (Jan 07 2025 at 18:03):if handlers can run concurrently
Sam Mohr (Jan 07 2025 at 18:07):You could do something like this to allow async model updates:
init! : {} => {
model,
respond!: model, Request => (update, Response),
updateModel: model, update -> model,
}
Wow, I didn't even read that link
Richard Feldman (Jan 07 2025 at 19:08):yeah in Elm terminology it would be:
init! : () => {
model,
respond!: model, Request => (msg, Response),
update!: model, msg => model,
}
Does update need to be effecful?
Richard Feldman (Jan 07 2025 at 19:09):doesn't have to be, but you might want to do logging or something in there
Sam Mohr (Jan 07 2025 at 19:09):Sure
Richard Feldman (Jan 07 2025 at 19:09):it does have to be single-threaded
Richard Feldman (Jan 07 2025 at 19:09):as in, at most one update function is ever running at a time
Richard Feldman (Jan 07 2025 at 19:09):just going through the queue of messages
Sam Mohr (Jan 07 2025 at 19:10):The problem with this approach is you no longer guarantee seeing updates to model before the next request handling happens, but also that's just the case anyway with proper multithreading
Richard Feldman (Jan 07 2025 at 19:10):right
Sam Mohr (Jan 07 2025 at 19:10):I think this API is pretty good
Richard Feldman (Jan 07 2025 at 19:11):I definitely think it's good to use normal type variables for this
Sam Mohr (Jan 07 2025 at 19:11):So we still don't need to expose model to the host? Is there a use case that would make that desired?
Richard Feldman (Jan 07 2025 at 19:11):I'd love to get rid of the Model concept we have today
Richard Feldman (Jan 07 2025 at 19:12):actually it was originally a workaround
Richard Feldman (Jan 07 2025 at 19:12):because we didn't have sending closure captures to the host working, and that seemed like a big project
Richard Feldman (Jan 07 2025 at 19:12):so Folkert suggested adding this as a workaround to unblock the use cases :big_smile:
Sam Mohr (Jan 07 2025 at 19:12):That's one more section gone from the platform
Richard Feldman (Jan 07 2025 at 19:12):but it was never intended to be the long-term design
Richard Feldman (Jan 07 2025 at 19:17):the other thing is that I suspect this pattern of init! returning the request handler function is going to end up being what every webserver wants to do
Richard Feldman (Jan 07 2025 at 19:17):because it's such a convenient way to get environment variables etc. on startup
Richard Feldman (Jan 07 2025 at 19:18):and then you just close over them and that's it
Richard Feldman (Jan 07 2025 at 19:22):I'm actually not sure if today it might need to use Box
Richard Feldman (Jan 07 2025 at 19:23):in order to work around current compilation issues, not sure
Richard Feldman (Jan 07 2025 at 19:23):we could try it out on basic-webserver, see if it works as-is
Oskar Hahn (Jan 07 2025 at 19:57):I cannot see, how this solution would support effects that use model. You would need to define all effects in this init structure.
Richard Feldman (Jan 07 2025 at 20:18)::thinking: what effects use Model?
Sam Mohr (Jan 07 2025 at 20:24):Something like save_to_db! : Model => {} or load_from_db : {} => Model`
Sam Mohr (Jan 07 2025 at 20:24):Stuff for Kingfisher
Richard Feldman (Jan 07 2025 at 20:31):ah I see
Richard Feldman (Jan 07 2025 at 20:36):so a tradeoff there is that if you have to represent that using a value that you pass around - e.g. something like:
save_to_db! : Db model, model => ()
load_from_db! : Db model => model
...then it's less convenient in that you have an extra value to pass around, but on the other hand, without that, you wouldn't have a way to simulate database state during tests. You'd have to always the real platform functions
Richard Feldman (Jan 07 2025 at 20:36):as opposed to being able to construct a fake Db and pass it around
Brendan Hansknecht (Jan 07 2025 at 21:31):Richard Feldman said:
yeah in Elm terminology it would be:
init! : () => { model, respond!: model, Request => (msg, Response), update!: model, msg => model, }
What is the benefit of that over just exposing a record of functions to the host?
Richard Feldman (Jan 07 2025 at 21:48):you can read env vars (and similar) and use them in any of the returned functions
Brendan Hansknecht (Jan 07 2025 at 22:12):You can also do that by storing them in model
Brendan Hansknecht (Jan 07 2025 at 22:13):But I guess this would mean that you could avoid storing queries in the model. They are constant after creation. That would simplify all the error managing.
Richard Feldman (Jan 07 2025 at 22:45):yeah exactly
Brendan Hansknecht (Jan 07 2025 at 23:37):I guess that may be a reasonable api once repeatedly using closures with the platform just works.
Oskar Hahn (Jan 08 2025 at 05:56):Richard Feldman said:
so a tradeoff there is that if you have to represent that using a value that you pass around - e.g. something like:
save_to_db! : Db model, model => () load_from_db! : Db model => model
I don't get it. What is Db model? Is this something Init has to return?
Last updated: Jul 05 2025 at 12:14 UTC