Luke Boswell (Jan 21 2025 at 01:57): Luke Boswell (Jan 21 2025 at 01:57):Richard Feldman said:
Luke Boswell said:
I'm definitely keen to pickup the WASM platform stuff again. I'd still like to build a playground in the browser.
it would be super rad to have this before redoing all the tutorials for static dispatch and everything, so they could be built around being interactive! :smiley:
We've discussed a few times the idea of a roc interpreter. It's been raised as something we'll need to evaluate things at comptime. I was wondering if we could piggyback off this effort to help us get to a working WASM playground sooner, and also start building towards the end goal of a having a fully capable interpreter.
My thoughts are along the line of bundling the things we need like roc_parse, roc_can, and roc_interpreter into a WASM module that we could load into the browser and throw it roc source code, parse, canonicalize, and then evaluate expressions using the IR. Maybe it's just something really basic to start with, like simple expressions -- but that would be helpful for the tutorial experience.
I hope this is a simple architecture for using in the browser -- we aren't generating a new WASM module on each compilation and then loading that separately in the browser VM to run it. That is the approach taken in the web REPL. I've tried setting up something similar, but I think there may be a bug in gen-wasm or I've had issues wiring things up to link against a pre-built platform (even just a basic mainForHost : Str -> Str).
@Ayaz Hafiz would you have any thoughts on this?
Ayaz Hafiz (Jan 21 2025 at 02:13):i think this would be so sick
Ayaz Hafiz (Jan 21 2025 at 02:14):honestly, you could even do this over the AST. forego type checking, etc. unless it's needed to show errors
Ayaz Hafiz (Jan 21 2025 at 02:14):should be pretty simple to implement a basic one that is sufficient for small programs, which most browser apps (eg tutorials, small code snippets) will be
Ayaz Hafiz (Jan 21 2025 at 02:14):just a tree-walking interpreter
Luke Boswell (Jan 21 2025 at 02:18):The thing I'm not sure about is the scope of something like this.
Like where would it start -- from Parser AST? Can IR?
How would the evaluated expressions be represented? is that also just the same IR? would we need to serialise that (to json?) to feed back to JS. etc...
If someone could help guide me with these things, I think I could plug away at it.
Luke Boswell (Jan 21 2025 at 02:24):Or if anyone else is interested in building this then I'd be happy for that too. I'm just keen to see us have a nice learning/onboarding experience for people discovering roc for the first time.
Richard Feldman (Jan 21 2025 at 02:39):I figure can IR would be easier since otherwise the interpreter needs to reimplement name resolution :big_smile:
Richard Feldman (Jan 21 2025 at 02:39):(not that that's hard, but if we already have code that does it, might as well use it, right?)
Joshua Warner (Jan 21 2025 at 02:48):One other thought that crossed my mind is if we can make sure the semantics of roc are simple enough that it can be interpreted without type checking, and the interpreter is solid (and simple) enough, that can serve as a "source of truth" in compiler fuzzing - e.g. running a program in the interpreter and then the compiler, and if they differ spitting out an error.
Brendan Hansknecht (Jan 21 2025 at 03:02):I think a big question is what are our actual goals. If we want this to be a robust interpreter that eventually can be used for as an alternative to the dev backend, that has significantly different tradeoffs than just doing constant folding. Dealing with packages and especially dealing with platform ffi are likely to be the most complex parts. If we want to support this eventually path I think a much larger scoping is required. If we just want something that can constant fold and run pure roc code, something simpler is valid
Luke Boswell (Jan 21 2025 at 03:04):It sounds like something simple would have benefits for the tutorial and also for fuzzing.
Luke Boswell (Jan 21 2025 at 03:04):the experience of having something really simple might give us an idea of it it's worthwhile building the more complicated (and capable) thing.
Brendan Hansknecht (Jan 21 2025 at 03:05):For the tutorial we can just run the compiler in the browser with the wasm backend. So that doesn't feel like much of a specific gain
Brendan Hansknecht (Jan 21 2025 at 03:05):Also, how does it help fuzzing?
Luke Boswell (Jan 21 2025 at 03:07):I may be misunderstanding @Joshua Warner's comment above
Brendan Hansknecht (Jan 21 2025 at 03:07):Also, for the tutorial, I assume we want to teach users about effects eventually. So likely we want a solution that can call effects. Which a simple interpreter likely can not. This is why I think it is really important to scope out the goal
Brendan Hansknecht (Jan 21 2025 at 03:09):Oh, missed joshua's comment. Sure, that is fine. Though generating valid and interesting programs is pretty rough for fuzzers. We also would need the entire stack to never crash before an e2e fuzzer is possible. But long term could be nice.
Joshua Warner (Jan 21 2025 at 03:09):Also, how does it help fuzzing?
Interpreters are (often) much simpler to write than compilers, and thus often easier to prove bug-free. We can use that property to assume the interpreter is "correct", and so any time the compiler returns a different result, it must be wrong.
Joshua Warner (Jan 21 2025 at 03:10):We also would need the entire stack to never crash before an e2e fuzzer is possible.
Yep, definitely a more long-term thing
Brendan Hansknecht (Jan 21 2025 at 03:11):Yeah, sounds like an interesting long term goal/playground.
Joshua Warner (Jan 21 2025 at 03:11):FWIW I don't think the interpreter needs to be that fancy to use in fuzzing. So long as the interpreter bails out on inputs it can't run, in a way we can detect and ignore in the fuzzing infra, it can work.
Luke Boswell (Jan 21 2025 at 03:11):If the interpreter is simple... and we control the WASM program that is running it in the browser, could we just hardcode some effects in for the purpose of a tutorial. Like if it sees a node Apply "some_effect!" it can simulate it?
Joshua Warner (Jan 21 2025 at 03:12):Perhaps I'm unrealistically optimistic, but I think we can probably get platforms + effects running in the interpreter
Brendan Hansknecht (Jan 21 2025 at 03:14):I think so, but that probably requires the interpreter to run on mono (due to needing to know exact types). It also means the interpreter has to be able to form arbitrary dynamic ffi correctly (totally possible with libffi). Lastly means that a platform needs to be able to link to the interpreter and call the standard x_for_host function and for that to launch the interpreter into the correct function (probably the hardest part of the above).
Brendan Hansknecht (Jan 21 2025 at 03:14):That's said, I think this would be a much much more compelling and useful interpreter than something just made for constant folding.
Ayaz Hafiz (Jan 21 2025 at 03:14):i wouldn't complicate this. if there's something that can be done in a couple days and is a material benefit, i think that's a huge win even if it needs to be thrown out later, because the reality is there are few resources to build something sophisticated in the near term. a good example is the language server. I hacked something together in a couple days and @Eli Dowling did some work on it too, it's very far from a production LS and has limits inhereted from other parts of the compiler, but i think it's clear it's a net win.
Joshua Warner (Jan 21 2025 at 03:15):Yeah, I would definitely start with the simple thing
Joshua Warner (Jan 21 2025 at 03:16):(just an interpreter on can output)
Brendan Hansknecht (Jan 21 2025 at 03:16):Fair enough. I think it would be trivial to write code to walk mono and run it. Would be essentially the same thing as the dev backend, but actual execution instead of generating assembly.
Joshua Warner (Jan 21 2025 at 03:16)::man_shrugging: That works too I suppose. I don't feel super strongly
Luke Boswell (Jan 21 2025 at 03:17):My thoughts on the platform... if we have all of the crates good to go. A platform author can use those to provide a platform-specific interpreter using these crates. They could write a program that parses, canonicalises, and then use the interpreter to evaluate. Maybe the interpreter returns when it sees a node it doesn't recognise, and the platform author who is writing this custom interpreter and fill in the details of how to simulate (or run) that effect.
Luke Boswell (Jan 21 2025 at 03:18):At least this was how I imagined building a WASM playground / tutorial.
Brendan Hansknecht (Jan 21 2025 at 03:19):I honestly think that wouldn't be very useful in most cases. The boon of roc is that you can compile it can get good perf. Being stuck with either interpreter roc or compiled roc sounds pretty crappy. (But useful in a few very specific use cases to just have the interpreter version)
Luke Boswell (Jan 21 2025 at 03:19):Maybe it makes sense to have multiple interpreters one day?
Brendan Hansknecht (Jan 21 2025 at 03:20):Hopefully they would all be one interpreter in the long run, just with more features.
Richard Feldman (Jan 21 2025 at 03:20):I actually think it's important that if we're using the interpreter for actually running programs, we introduce intentional limitations on it
Richard Feldman (Jan 21 2025 at 03:21):that is, if canonicalization or type-checking says it needs to be a crash because of a type mismatch or naming error, we respect that and always crash
Brendan Hansknecht (Jan 21 2025 at 03:21):?
Richard Feldman (Jan 21 2025 at 03:21):even if at runtime it might work
Joshua Warner (Jan 21 2025 at 03:21):Yeah I'd agree with that
Ayaz Hafiz (Jan 21 2025 at 03:21):i genuinely don't think it'll be a big deal in most cases especially for the limited scope presented here. if you're writing a high perf game or low alloc web server, use the compiler version. if you want to run things in the browser or a simple script an interpreter will be fine
Brendan Hansknecht (Jan 21 2025 at 03:21):I assume we would basically run roc check, save the ast, and then interpret the ast
Brendan Hansknecht (Jan 21 2025 at 03:22):Sounds easy enough to just do
Brendan Hansknecht (Jan 21 2025 at 03:23):Ayaz Hafiz said:
simple script an interpreter will be fine
This only works if you figure out platform ffi. So I think it will really just be special cases for completely pure things.
Luke Boswell (Jan 21 2025 at 03:23):One thing that was a challenge for my previous playground attempts with WASM, is the file system isn't abstracted. I guess we might need to resolve that to use roc check
Richard Feldman (Jan 21 2025 at 03:23):yeah I mention the type mismatch thing because you do have to go out of your way to crash on type mismatches :big_smile:
Richard Feldman (Jan 21 2025 at 03:23):like you have to record that the type mismatch happened and then actually have the interpreter to do a different thing from what it would normally do
Joshua Warner (Jan 21 2025 at 03:23):One of my crazy-one-day ideas is to make a framework reminiscent of pypy or (even more closely) graal from Java-land - where we'd write an interpreter (for roc, in roc) and that automagically generates a reasonably-fast compiler
Joshua Warner (Jan 21 2025 at 03:24):By which I mean, interpreters aren't _always_ dog-slow
Joshua Warner (Jan 21 2025 at 03:25):Anywho; my main point is just that I don't want to shy away from having an interpreter just on performance grounds
Brendan Hansknecht (Jan 21 2025 at 03:26):Assuming we can make it work, I would argue that an interpreter makes more sense than the dev backend. But the platform architecture may make that complex to essentially impossible to actually make work in practice.
Brendan Hansknecht (Jan 21 2025 at 03:28):Cause we would still have to emit something that would link to a precompiled platform and that is seamlessly swaps with the llvm optimized assembly. (Yet launches the interpreter)
Richard Feldman (Jan 21 2025 at 03:29):right, it can work for an online playground
Richard Feldman (Jan 21 2025 at 03:29):but if you want to use it for like dev builds of plugins in games or editors, then now you have to emit and link the entire interpreter into the host binary :sweat_smile:
Joshua Warner (Jan 21 2025 at 03:30):Right - but that has to be done once per platform, instead of every build
Brendan Hansknecht (Jan 21 2025 at 03:30):Exactly
Luke Boswell (Jan 21 2025 at 03:33):In my head, the interpreter lives within the host -- so there's no need for any linking. The host is driving the interpreter to parse, and evaluate the roc app etc and can do what it likes with effects.
Luke Boswell (Jan 21 2025 at 03:34):But I have very little experience with how these things typically work. I'm just throwing ideas around to try and understand what it might look like for roc.
Luke Boswell (Jan 21 2025 at 03:38):From the sound of things interpreters are normally standalone programs that run things. In my head the roc interpreter is more like a library a platform author/host uses.
Luke Boswell (Jan 21 2025 at 03:43):Ok -- I just repeated what was said above. Cool. :smiley:
Luke Boswell (Jan 21 2025 at 03:49):Luke Boswell said:
One thing that was a challenge for my previous playground attempts with WASM, is the file system isn't abstracted. I guess we might need to resolve that to use
roc check
As a workaround for this or temporary step to validate the interpreter idea -- I wonder if something like this could get us there https://runno.dev/wasi
Richard Feldman (Jan 21 2025 at 03:57):I think Brian may have looked into that
Richard Feldman (Jan 21 2025 at 03:57):for the web repl
Richard Feldman (Jan 21 2025 at 03:57):not sure though!
Brendan Hansknecht (Jan 21 2025 at 04:01):In my head, the interpreter lives within the host -- so there's no need for any linking
Yeah, this is how it works for lua. Sadly, this leaves user in a split world, they either support roc the interpreter or roc the compiler. I guess you could support both with compilation flags, but it is less nice overall to do so. In a perfect world, the platform just supports Roc. The user than gets interpreted roc for dev builds and compiled roc for release builds.
Richard Feldman (Jan 21 2025 at 04:13):oh I thought the idea being discussed there was:
Brendan Hansknecht (Jan 21 2025 at 04:21):That is what I think we should try and do. Hopefully doing so seamlessly for the platform.
Luke Boswell (Jan 21 2025 at 04:22):I got something working using runno... effectively runs roc check on a module.
Luke Boswell (Jan 21 2025 at 04:25):If you want to try it out... load these files into https://runno.dev/wasi
Luke Boswell (Jan 21 2025 at 04:26):This is all it is
use std::path::PathBuf;
fn main() {
println!("RUNNING FROM WASI");
match roc_app_wasm(|s| println!("{}", s)) {
Ok(()) => {
println!("Successfully compiled app.roc");
}
Err(err) => {
println!("Error compiling app.roc: {}", err);
}
}
}
pub fn roc_app_wasm(log: fn(&str) -> ()) -> Result<(), String> {
let roc_cache = std::path::PathBuf::from(".cache").join("roc");
let arena = &bumpalo::Bump::new();
let opt_main_path = None;
let roc_cache_dir = roc_packaging::cache::RocCacheDir::Persistent(roc_cache.as_path());
let load_config = roc_load::LoadConfig {
target: roc_target::Target::Wasm32,
function_kind: roc_load::FunctionKind::LambdaSet,
threading: roc_load::Threading::Single,
render: roc_reporting::report::RenderTarget::Generic,
palette: roc_reporting::report::DEFAULT_PALETTE,
exec_mode: roc_load::ExecutionMode::Check,
};
log("STARTING TO LOAD AND MONOMORPHIZE");
let loaded = roc_load::load_and_typecheck(
arena,
PathBuf::from("./app.roc"),
opt_main_path,
roc_cache_dir,
load_config,
);
match loaded {
Ok(_module) => {
log("COMPLETED LOAD AND TYPECHECK");
}
Err(loading_problem) => {
log(format!("ERROR DURING LOAD AND TYPECHECK {:?}", loading_problem).as_str());
}
}
Ok(())
}
Screenshot 2025-01-21 at 15.26.50.png
Brendan Hansknecht (Jan 21 2025 at 04:34):Nice. Just need a can interpreter for the base case to work
Luke Boswell (Jan 21 2025 at 04:34):I tried an app module type first, but runno doesn't support loading files from URL's. But it looks like everything is happy using relative file paths.
Luke Boswell (Jan 21 2025 at 04:42):load_and_typecheck gives me one of these... I assume this contains enough information to do some interpreting.
#[derive(Debug)]
pub struct LoadedModule {
pub module_id: ModuleId,
pub filename: PathBuf,
pub interns: Interns,
pub solved: Solved<Subs>,
pub can_problems: MutMap<ModuleId, Vec<roc_problem::can::Problem>>,
pub type_problems: MutMap<ModuleId, Vec<TypeError>>,
pub declarations_by_id: MutMap<ModuleId, Declarations>,
pub exposed_to_host: MutMap<Symbol, Variable>,
pub dep_idents: IdentIdsByModule,
pub exposed_aliases: MutMap<Symbol, Alias>,
pub exposed_modules: Vec<ModuleId>,
pub exposed_values: Vec<Symbol>,
pub exposed_types_storage: ExposedTypesStorageSubs,
pub resolved_implementations: ResolvedImplementations,
pub sources: MutMap<ModuleId, (PathBuf, Box<str>)>,
pub timings: MutMap<ModuleId, ModuleTiming>,
pub docs_by_module: VecMap<ModuleId, ModuleDocumentation>,
pub abilities_store: AbilitiesStore,
pub typechecked: MutMap<ModuleId, CheckedModule>,
pub imports: MutMap<ModuleId, MutSet<ModuleId>>,
pub exposed_imports: MutMap<ModuleId, MutMap<Symbol, Region>>,
pub exposes: MutMap<ModuleId, Vec<(Symbol, Variable)>>,
}
Where does the IR live in here?
Sam Mohr (Jan 21 2025 at 04:51):I'm DM'ing with Luke about this!
Luke Boswell (Jan 21 2025 at 05:09):I've put my demo up on a repo in case anyone wants to try it themselves.
https://github.com/lukewilliamboswell/roc-playground-experiment
Luke Boswell (Jan 21 2025 at 05:09):I changed it around a little so it's easy to run locally... for experimenting with an interpreter
Luke Boswell (Jan 21 2025 at 06:35):I didn't realise, it's already a pretty cool runner for different languages. On the front page they support Python https://runno.dev
Luke Boswell (Jan 21 2025 at 06:35):And they've got issues for supporting different languages https://github.com/taybenlor/runno/issues?q=is%3Aissue%20state%3Aopen%20label%3A%22new%20language%22
Luke Boswell (Jan 21 2025 at 07:47):I know it's only small steps... but with my experiment https://github.com/lukewilliamboswell/roc-playground-experiment
I've set it up so it now loads multiple files, an app.roc and a platform.roc.
It then walks the IR, starting from main_for_host! and prints out information about each node.
Last updated: Jun 16 2026 at 16:19 UTC