Richard Feldman (Jul 15 2023 at 21:02): Richard Feldman (Jul 15 2023 at 21:02):I've been working on some glue improvements, specifically having it generate glue for the user-defined effects in hosted modules - I have a draft PR for this, but it's giving:
thread 'main' panicked at 'unspecialized lambda sets left over during resolution: LambdaSet([] + (<475>FlexAble(fmt, [
Encode.EncoderFormatting]):Encode.record:1), ^<473>), UlsOfVar(VecMap { keys: [], values: [] })', crates/compiler/mono/src/layout.rs:1980:17
this is for $ cargo run -- glue crates/glue/src/RustGlue.roc glue-out examples/cli/cli-platform/main.roc on that branch
Richard Feldman (Jul 15 2023 at 21:04):I'm assuming this is because of a combination of this change and this line (I added the .chain)
Richard Feldman (Jul 15 2023 at 21:04):those variables have an interaction with lambda sets a bit later on
Richard Feldman (Jul 15 2023 at 21:05):but I'm guessing maybe they need to be registered in some other way?
Ayaz Hafiz (Jul 15 2023 at 23:33):Is this coming from when you try to generate the glue for a host-exposed lambda set? If so, this should be fine - it just means the lambda set is polymorphic, like the other polymorphic HELSs. I would probably add a flag to the layout generator to drop those unspecialized lambda sets for glue generation (or always drop them, but if that road is taken, let's add a debug flag that checks against this, since the assert is intended to catch bugs in specialization)
Richard Feldman (Jul 15 2023 at 23:47):oh hm, I hadn't even thought about where it was coming from
Richard Feldman (Jul 15 2023 at 23:47):I just tried it on platform-cli in examples/ - I didn't even look at what it was trying to operate on
Richard Feldman (Jul 15 2023 at 23:48):but everything in hosted should need to be monomorphic, right?
Richard Feldman (Jul 15 2023 at 23:48):(I don't think we have a check for that yet though)
Richard Feldman (Jul 15 2023 at 23:48):and if so, then shouldn't the lambda sets no longer be polymorphic?
Ayaz Hafiz (Jul 15 2023 at 23:48):they should be monomorphic in the surface types, but the lambda sets can almost never be monomorphic
Richard Feldman (Jul 15 2023 at 23:49):gotcha
Ayaz Hafiz (Jul 15 2023 at 23:49):hence the need for the host-exposed lambda set wrapper functions
Ayaz Hafiz (Jul 15 2023 at 23:51):one case is if the host-exposed main is something like
mainForHost : {} -> (Int -> Int)
The lambda set of the Int -> Int is (very likely) only provided per-app, we don't know what lambda set to reference there when generating the glue
Ayaz Hafiz (Jul 15 2023 at 23:51):Folkert's numbering scheme should generate the right lambda set wrapper
Ayaz Hafiz (Jul 15 2023 at 23:51):we just need to tell the layout generator that it's okay for them to be unresolved as polymorphic for the purposes of glue
Richard Feldman (Jul 15 2023 at 23:58):hmmm ok, if I drop it (just disable that debug_assert! for now), I hit this unreachable! due to the builtin (!) Alias in question having a name of Decode.0 and a repr of LayoutRepr::Struct([]) :face_with_raised_eyebrow:
Ayaz Hafiz (Jul 16 2023 at 00:00):that is Decode.DecodeError. seems like that pattern match should be expanded?
Richard Feldman (Jul 16 2023 at 00:28):hm, I don't follow :sweat_smile:
Richard Feldman (Jul 16 2023 at 00:29):oh as in like:
LayoutRepr::Struct { .. } if *name == Symbol::DECODE_DECODE_ERROR => {
ha, looks like we do have some amount of checking for this :joy:
thread 'main' panicked at 'not yet implemented: TODO give a nice error message for a non-concrete type being passed to the host', crates/glue/src/types.rs:1386:13
Richard Feldman (Jul 16 2023 at 00:45):guess it's time to improve that error message!
Richard Feldman (Jul 16 2023 at 01:10)::thinking: I think probably the best way to go about that is to add an extra check after we've just solved all the types for a hosted module, while we still have all the decls in scope: traverse each of them, and if there are any unbound type variables in them, change the Decl's type to Erroneous and report it
Richard Feldman (Jul 16 2023 at 01:24):hm, ok I added a text fixture with a couple of effects: https://github.com/roc-lang/roc/blob/acb1a5ac04077c4abdffb203bb79b8c8ccd393f6/crates/glue/tests/fixtures/basic-effects/Effect.roc
now I'm running into this, though:
Richard Feldman (Jul 16 2023 at 01:25):interestingly, it's only for the getLine : Effect Str though
Richard Feldman (Jul 16 2023 at 01:25):if I delete that one, putLine : Str -> Effect {} works
Richard Feldman (Jul 16 2023 at 01:28):I would have guessed that it would be for the one with the actual function type, but I guess it's the wrapper?
Ayaz Hafiz (Jul 16 2023 at 02:57):what is the backtrace? I guess that could happen if you’re asking to generate a RawFunctionLayout for a host exposed lambda sets
Richard Feldman (Jul 16 2023 at 19:36):update: putLine : Str -> Effect {} works completes, but generates the wrong type :laughing:
Richard Feldman (Jul 16 2023 at 19:36):this is interesting...the Content of the type when I write it down is:
[crates/compiler/load_internal/src/file.rs:2436] symbol =
Effect.putLine
[crates/compiler/load_internal/src/file.rs:2473] f = "Func([<1792>Apply(Str.Str, []),], <261=1801>LambdaSet([Effect.putLine, ], ^<1800>), <1793>Opaque(Effect.Effect, [257, 256], <259>Func([<1>EmptyRecord,], <249=69>LambdaSet([Effect.IdentId(20)<244>Apply(Str.Str, []) , ], ^<1796>), <257>EmptyRecord)))"
Richard Feldman (Jul 16 2023 at 19:36):that looks correct, because the signature in Effect.roc is:
putLine : Str -> Effect {}
however, the generated glue is:
HostedFn {
name: "putLine",
arg_types: &[],
ret_type: R1,
}
#[derive(Clone, Debug, PartialEq, PartialOrd, )]
#[repr(C)]
pub struct R1 {
pub rest: roc_std::RocList<u8>,
pub result: roc_std::RocResult<f64, ()>,
}
so it thinks it's a thunk whose return value is what we generate for Decode on a record or tuple
Richard Feldman (Jul 16 2023 at 21:38):ok, the plot thickens:
Content in solved_subs, it's the above Func as expectedVariable (and Symbol) later on in env.subs in glue, it's a Record instead of a Func :face_with_raised_eyebrow: Richard Feldman (Jul 16 2023 at 21:39):so one guess at what's happening:
Symbol that's being recorded is global, so it's correctVariable that's being recorded is relative to the Effect module (and that module's Subs), whereas when I go to look it up later in env.subs, it's a different Subs (the one for the root module being used in glue), so the Variable is just referring to something totally different Richard Feldman (Jul 16 2023 at 21:41):if that is indeed what's happening, I guess a potential solution is to store the Symbol only and see if I can look up the appropriate Variable to use in the root module later?
Richard Feldman (Jul 16 2023 at 21:42):hm, but I don't have an easy way in glue to go from Symbol to Variable in the root module :thinking:
Richard Feldman (Jul 16 2023 at 21:49):like we have exposed_vars_by_symbol: Vec<(Symbol, Variable)> but these aren't exposed (at least not to the end user)
Richard Feldman (Jul 16 2023 at 21:51):but it must be possible to look these up somehow, because mono has to do it in order to generate the externs for linking
Richard Feldman (Jul 16 2023 at 21:51):like those need both the symbol and the appropriate Variable (or at least the solved type) at some point
Richard Feldman (Jul 16 2023 at 21:51):anyone know how that works and how we might do the same thing for glue?
Ayaz Hafiz (Jul 16 2023 at 21:53):why is the solved_subs different from the env.subs in glue?
Richard Feldman (Jul 16 2023 at 21:53):it's solved_subs for the hosted module
Richard Feldman (Jul 16 2023 at 21:53):actually wait, is that true? :thinking:
Richard Feldman (Jul 16 2023 at 21:53):or is solved_subs always for the root module
Richard Feldman (Jul 16 2023 at 21:54):yeah no it's for the hosted module
Ayaz Hafiz (Jul 16 2023 at 21:54):each module gets its own subs (and eventually a solved subs), so it depends where it's being referenced
Richard Feldman (Jul 16 2023 at 21:54):I'm printing it out for Msg::SolvedTypes
Richard Feldman (Jul 16 2023 at 21:54):so yeah it's the per-module one
Ayaz Hafiz (Jul 16 2023 at 21:55):got it. why is glue not using that subs too then? I would assume you want to generate glue from the hosted module?
Richard Feldman (Jul 16 2023 at 21:58):yeah so the algorithm I'm using right now (which has the aforementioned issue) is:
hosted module, we add its ModuleId to state.hosted_ids for later (this part is fine)Msg::SolvedTypes for that module, we add all of the symbols in its decls to state.hosted_vars_by_symbol: MutMap<Symbol, Variable> - this is where we add the correct Symbol, but the Variable we have access to at this point is for the hosted module's Subs and not the root module's Subsstate.hosted_vars_by_symbol makes it into LoadedModule, which glue then uses - but although it has the correct Symbols, it has the wrong Variables Ayaz Hafiz (Jul 16 2023 at 22:00):i see. I would add another subs, let’s call it H for now, alongside this hosted_vars_by_symbol object, and export the module’s variable from the module subs into H, and then work on H in glue
Richard Feldman (Jul 16 2023 at 22:00):cool, I like it! :thumbs_up:
Ayaz Hafiz (Jul 16 2023 at 22:01): Richard Feldman (Jul 16 2023 at 22:06):yeah but when would I call that? :sweat_smile:
Ayaz Hafiz (Jul 16 2023 at 22:08):can you do it during the time you add the mapping into state.hosted_vars_by_symbol?
Richard Feldman (Jul 16 2023 at 22:09):I think that would cause a race condition because root_subs: Option<Subs> might not be set yet
Richard Feldman (Jul 16 2023 at 22:10):but actually it turns out module_cache has solved_subs for each module
Ayaz Hafiz (Jul 16 2023 at 22:14):yeah
Richard Feldman (Jul 16 2023 at 22:17):ok I think things will go best if I use export_variable_to - do I understand correctly that the returned CopiedImport's variable field is the new Variable in the other Subs's variable space?
Ayaz Hafiz (Jul 16 2023 at 22:21):yes
Richard Feldman (Jul 16 2023 at 22:22):oh, it's on StorageSubs - I haven't worked with that before, what's the difference between it and Subs?
Ayaz Hafiz (Jul 16 2023 at 22:29):they are basically the same, StorageSubs is a wrapper of Subs IIRC. it just has some extra methods bc it’s intended for cross-module storage
Richard Feldman (Jul 16 2023 at 23:05):ok, module_cache.typechecked entries get removed from the cache too soon for my purposes
Richard Feldman (Jul 16 2023 at 23:06):one potentially interesting option I just thought of: if root_subs: Option<Subs> is None when I want to add the hosted variables to it, just initialize it to empty and copy in the variables
Richard Feldman (Jul 16 2023 at 23:06):then later, when starting to work on the root module, use that as the starting point if it's set
Richard Feldman (Jul 16 2023 at 23:07):that way, whichever one gets to it first doesn't matter
Richard Feldman (Jul 16 2023 at 23:36):at that point might as well drop the Option I suppose, and always initialize it
Richard Feldman (Jul 17 2023 at 01:26):ok, got all that working, but now this is blowing up because it has a return type of a FlexVar somehow :thinking:
Richard Feldman (Jul 17 2023 at 01:27):Adding function type ret_type_id for RocFunction_1540...
[crates/glue/src/types.rs:1323] &ret_layout = InLayout(VOID)
[crates/glue/src/types.rs:1324] types.get_type(lambda_set_type_id) = Unsized
[crates/glue/src/types.rs:1360] subs.get_content_without_compacting(var) = FlexVar(
None,
)
thread 'main' panicked at 'not yet implemented: TODO give a nice error message for a non-concrete type being passed to the host', crates/glue/src/types.rs:1365:13
Last updated: Jul 06 2025 at 12:14 UTC