Oskar Hahn (Feb 25 2024 at 08:45): Oskar Hahn (Feb 25 2024 at 08:45):Before the update to zig 0.11 is was possible, to export functions in zig to wasm. Since then, only the function _start is exported.
So in the past, it was possible to write something like this in zig:
export fn foobar(some: u32, args: u32) u32 {
[...]
roc__mainForHost_1_exposed_generic(&result, &argument);
[...]
return result_ptr;
}
Now, you can only use _start, which does not accepts any arguments.
The reason is, that zig does not automatically exports functions to wasm since 0.11: https://ziglang.org/download/0.11.0/release-notes.html#WebAssembly
To solve this, the line "-rdynamic", has to be added here: https://github.com/roc-lang/roc/blob/2681d81de7b5aeb2e56b15efe9a89dc12dbf7a59/crates/compiler/build/src/link.rs#L1234
Would you accept a PR that adds this line? Without it, I don't know how to use roc in a wasm context.
Brendan Hansknecht (Feb 25 2024 at 16:09):Won't -rdynamic export everything breaking all dead code elimination?
Oskar Hahn (Feb 25 2024 at 16:54):I don't know. The real solution will be the platform build script. But until this works, I don't see an alternative to rdynamic
Brendan Hansknecht (Feb 25 2024 at 17:57):Sounds like it is equivalent to what they used to do. Even though it blocks dce, it fixes other issues for now... So go for it
Oskar Hahn (Feb 26 2024 at 14:43):Thank you. Here is the PR: https://github.com/roc-lang/roc/pull/6542
Brian Carroll (Feb 28 2024 at 07:30):I see this differently. I don't think this breaks anything,-rdynamic only prevents DCE on things that are exported, which is desired behaviour. It don't think it will "export everything", just the stuff that the platform deliberately chose to export. Is that wrong?
Brian Carroll (Feb 28 2024 at 07:47):Is this referring to things exported from the builtins?
Brian Carroll (Feb 28 2024 at 07:50):All Wasm modules are dynamic libraries. There isn't really a concept of an executable because you always dynamically link to a host and you can only do IO by calling imported functions. There is no way to do a "syscall" without anybody knowing, like in a native binary. So -rdynamic makes sense for it.
Brian Carroll (Feb 28 2024 at 07:57):Our Wasm dev backend does not have a problem with DCE in this situation because it knows the Roc concept of host and app. So it can just expose the things the host wants to expose and doesn't keep any builtins it doesn't need. It's trickier to express that to wasm-ld though. I think you have to give it a list of the things you want to expose. So the platform build script solves it.
Oskar Hahn (Feb 28 2024 at 08:16):I am currently not on my development pc, so I can not tell you the exact list of exported symbols, but there are more, then needed. You can see it, by calling console.log(wasm.instance.exports). For example, it contains all the roc__mainForHost_1_exposed, roc__mainForHost_1_exposed_generic and all other roc_functions, that should only be visible to zig and not to the wasm-host.
This can be solved by the platform build scripts, as soon, as they are possible.
Oskar Hahn (Feb 28 2024 at 08:29):Another thing, that is currently not nice is, that the wasm-host has to export some wasi-functions. For example, the wasm-platform-switching example does currently not work, because javascript has to provide the function random_get. This seems not necessary. It works, if you provide a dummy function, that does nothing (random_get: () => {}).
It would be nice, if it would be possible to build a wasm module, that does not require the wasi(nterface). We discussed this before without a solution. I don't think, this will change until the platform build scripts are supported.
Brian Carroll (Feb 28 2024 at 10:31):Right I see, thanks.
Have you tried using the --dev option with Roc?
It will give you unoptimised code but should solve this linking problem I think.
Oskar Hahn (Feb 28 2024 at 10:36):Since the PR #6542 is merged, there is no linking problem any more. I have not tested the --dev option. I did not think, that this would have made a difference. I can test this, when I am back home.
Brian Carroll (Feb 28 2024 at 10:55):Great, please let me know if it works.
It uses a custom linker that I wrote, that knows about Roc concepts. So it should expose the right things I hope.
Oskar Hahn (Feb 28 2024 at 13:58):@Brian Carroll it does not work. It creates a .wasm file, but if I try to run it in the browser, I get the error message CompileError: wasm validation error: at offset 58727: expected number of function body bytes.
If I call wasm2wat main.wasm, I get the error message: 000e567: error: unable to read u32 leb128: function body size.
This happens on my wasm-project, but also on the platform-switching-example
$ cd example/platform-switching
$ roc build --target=wasm32 --dev rocLovesWebAssembly.roc
🔨 Rebuilding platform...
0 errors and 0 warnings found in 1018 ms while successfully building:
rocLovesWebAssembly.wasm
$ wasm2wat rocLovesWebAssembly.wasm
000bd24: error: unable to read u32 leb128: function body size
Do you know, when it worked? I am trying to bisec it, but it also fails for version before the zig 0.11 update. Maybe I am doing something wrong?
Hristo (Feb 28 2024 at 15:37):Yeah, this has been tricky for me too to narrow down when exactly the rocLovesWebAssembly.roc example used to be working most recently.
It doesn't seem to cater nicely to a git bisect search, and I tried a linear search (across the entire git history) in steps of 100 commits. What makes it non-trivial it seems is the fact that the examples have been moved around, renamed etc (which isn't an issue in itself, of course).
I think the authors of the example would be able to much more easily pinpoint where the discrepancy might've occurred (if it's indeed an actual discrepancy, and not a misunderstanding of mine how the platform rebuilding should be run; I have been successfully able to rebuild other platforms - just mentioning for completeness).
Hristo (Mar 04 2024 at 14:30):For completeness, I'd like to confirm that I've been able to successfully (re)build the Wasm32 platform.
I wasn't able to get it to work with the release version of Roc, however. I suppose with a little bit of further digging around the codebase, it should be possible to correctly indicate where the Wasi library lives and run int this way. My attempts involved specifying WASI_LIBC_PATH to the build command, but that didn't quite cut it.
Up until this point, I was getting:
% roc build --target=wasm32 examples/platform-switching/rocLovesWebAssembly.roc
🔨 Rebuilding platform...
An internal compiler expectation was broken.
This is definitely a compiler bug.
Please file an issue here: https://github.com/roc-lang/roc/issues/new/choose
thread 'main' panicked at 'cannot find `wasi-libc.a`', crates/compiler/build/src/link.rs:134:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Then I realised I could attempt via the locally built version of Roc (following the instructions from the official repo), and this actually solved the issue because the building process had generated ./target/release/build/wasi_libc_sys-.../out/wasi-libc.a.
The successful platform (re)build run looks like as follows now (please, note that ./target/release/roc isn't the same executable as roc):
% ./target/release/roc build --target=wasm32 examples/platform-switching/rocLovesWebAssembly.roc
🔨 Rebuilding platform...
0 errors and 0 warnings found in 2000 ms
while successfully building:
examples/platform-switching/rocLovesWebAssembly.wasm
The --dev option also works analogously.
Hopefully, this could help someone else as well. Again, with a little bit of spare time for investigation (regarding how the WASI_LIBC_PATH should be set), it probably shouldn't be necessary to build Roc locally from source, in order to get this functionality working.
Hristo (Mar 04 2024 at 14:33):@Oskar Hahn, perhaps you could double-check if a locally built-from-source Roc could solve your Wasm/Wasi issue (unless you're already using a locally built one). The latter might have to do with your local environment, and if the local roc works, this experiment would be a confirmation that it's the Wasm/Wasi environment that's the culprit in this particular context.
My apologies if I've misunderstood the context/use-case, and have been actually providing effectively useless input.
Brendan Hansknecht (Mar 04 2024 at 15:32):Quite strange that we need wasi at all. We should look into removing that dependency. Actual roc code should be free of all dependencies except those exposed by the platform
Hristo (Mar 04 2024 at 15:42):@Brendan Hansknecht, I hope I've understood the context correctly here.
In general I do agree with what you've said. However, my understanding is this is in the context of example code, which has to do with platforms, and we can't have a Wasm-platform example code, without this dependency sorted.
Perhaps, it's a matter of better documenting how the example could be run?
Brian Carroll (Mar 04 2024 at 15:46):Roc code should never need WASI
Wasm platforms designed for non-browser use cases may use WASI if they want to
Wasm platforms designed for browser-only use cases should not need WASI because WASI is a library for running Wasm in non-browser contexts.
I believe our example is meant to run in the browser so if it is relying on WASI then that is a bug.
Hristo (Mar 04 2024 at 16:01):Thanks for clarifying @Brian Carroll! I think I understand now better what @Brendan Hansknecht meant as well!
When I get a chance, I'll try to reproduce the issue in a container for the purpose of confirming whether I consistently get the same error message. crates/compiler/build/src/link.rs is definitely not an examples-only file.
It seems this commit might be relevant.
Oskar Hahn (Mar 05 2024 at 07:52):Brian Carroll said:
Roc code should never need WASI
Wasm platforms designed for non-browser use cases may use WASI if they want to
Wasm platforms designed for browser-only use cases should not need WASI because WASI is a library for running Wasm in non-browser contexts.
I believe our example is meant to run in the browser so if it is relying on WASI then that is a bug.
Currently, Roc could needs WASI. The roc compiler uses the wasm32-wasi target, to build the binary:
https://github.com/roc-lang/roc/blob/09e813166720a41af28bcae36b5c978cab6d183c/crates/compiler/build/src/link.rs#L302
and/or
I would really like, if roc would use the wasm32-freestanding target. But this would mean, that it would be impossible to build a wasi module, or that roc would need two different targets (--target=wasm32, --target=wasi).
When zig is compiled with wasm32-wasi, the wasm-module needs some wasi functions to start. For example random_get or _start.
Oskar Hahn (Mar 05 2024 at 07:57):Hristo said:
Oskar Hahn, perhaps you could double-check if a locally built-from-source Roc could solve your Wasm/Wasi issue (unless you're already using a locally built one). The latter might have to do with your local environment, and if the local
rocworks, this experiment would be a confirmation that it's the Wasm/Wasi environment that's the culprit in this particular context.
I also build roc from source. It was possible for me to build the wasm-module with the --dev argument, but the build .wasm file was broken. I was not able to run it in the browser or analyze it with wasm2wat. Without the --dev argument, it works for me. Are you able to run the wasm-module with the --dev flag? Here is the description who to run it
Hristo (Mar 05 2024 at 08:49):Thanks @Oskar Hahn! Now I'm able to understand the use-case better!
Hmm.. with and without --dev, I'm getting this in the browser console, and no "Hello, World" message rendered on the page:
Uncaught (in promise) CompileError: wasm validation error: at offset 48420: expected number of function body bytes
roc_web_platform_run http://localhost:8080/host.js:42
<anonymous> http://localhost:8080/:7
Without --dev, it should work. Make sure to clear the browser cache after recompiling the module. Sometimes it helped for me to restart the python webserver, but not all the time. I have not invested this further.
Hristo (Mar 05 2024 at 10:10):Hmm .. I'm getting a different error (without --dev). I somehow think also the calling location matters, but I'll need to perform further tests later on, in order to be able to provide some constructive examples. This happens both in Chrome and Firefox (with a slight difference of the messages) - tried both to confirm that it's not due to browser cache. Also, for the sake of sanity checking (just in case), I've been restarting the server every time, upon recompilation:
Uncaught (in promise) LinkError: WebAssembly.instantiate(): Import #2 module="wasi_snapshot_preview1" function="random_get": function import requires a callable
Oskar Hahn said:
For example does currently not work, because javascript has to provide the function
random_get. This seems not necessary. It works, if you provide a dummy function, that does nothing (random_get: () => {}).
You have to add the function random_get to the JavaScript file.
Here is an example: https://github.com/roc-lang/roc/blob/09e813166720a41af28bcae36b5c978cab6d183c/examples/nodejs-interop/wasm/hello.js#L28
But I think an empty function does also work.
Brian Carroll (Mar 05 2024 at 10:54):currently Roc could need WASI
Ok here's my argument.
Roc is a pure functional language that cannot do I/O.
We link to a libc that does all of its I/O through WASI syscalls. But you can't write Roc code that does I/O so it can't possibly use those parts of libc.
Zig is not a pure functional language so it cannot provide these language level guarantees. The only way is to have a separate target for freestanding Wasm.
Brian Carroll (Mar 05 2024 at 10:56):I don't understand where the random_get is coming from. I suspect the platform but not sure why it would use it.
Brian Carroll (Mar 05 2024 at 10:58):Oh I just noticed something you said that it does come from Zig.
Brian Carroll (Mar 05 2024 at 11:01):So the only reason to add a new target to Roc would be to fix this strange Zig behaviour where it does the random_get.
Brian Carroll (Mar 05 2024 at 11:04):That doesn't seem like the right move for Roc as a project if it's Zig specific. But I wonder if they're doing this to meet some part of the WASI spec.
Oskar Hahn (Mar 05 2024 at 11:05):Brian Carroll said:
So the only reason to add a new target to Roc would be to fix this strange Zig behaviour where it does the random_get.
Yes. And the other functions: proc_exit, fd_write and _start.
Brian Carroll (Mar 05 2024 at 11:09):Hmm ok very good point!
Those are more understandable. Any host language will generate proc_exit and _start if you tell it you want WASI. And fd_write is often present for debug printing.
This is a stronger case for adding a freestanding target.
Brian Carroll (Mar 05 2024 at 11:15):It would mean changing the Rust enum of targets and fixing up any Rust errors. Then using it to modify the linker commands. Pretty straightforward to implement.
Brian Carroll (Mar 05 2024 at 11:25):@Richard Feldman I would be interested in your take.
The summary of the above is that any host language build script should have 2 separate targets for browser Wasm and WASI. But currently that build script is inside the Roc compiler and we only have one target (which is implicitly always WASI)
When developing for browser, people have to define dummy JS functions to stub out WASI stuff. It is confusing.
It has been suggested to add a new target, which would only behave differently inside link.rs.
In the future if we remove link.rs and move host build scripts to the platform developer, we could remove that target from Roc.
Richard Feldman (Mar 05 2024 at 11:49):that makes sense, although I'd prefer if we instead made progress on removing link.rs if possible :big_smile:
Brian Carroll (Mar 05 2024 at 11:51):Yeah!
What's needed for that? Make a bunch of separate build scripts for each platform we have?
Brian Carroll (Mar 05 2024 at 11:52):I guess a good first step would be to extract the existing build commands.
Brian Carroll (Mar 05 2024 at 11:53):I added an environment variable recently to dump all the commands. So if you set that flag and compile, it will print out your build commands and you can put them in a bash script or makefile
Richard Feldman (Mar 05 2024 at 11:57):nice!
Richard Feldman (Mar 05 2024 at 12:00):yeah basically the goal would be that:
"🔨 Rebuilding platform..." message Brian Carroll (Mar 05 2024 at 12:14):OK so we should start recommending that to people who are having build problems.
ROC_PRINT_BUILD_COMMANDS=1 roc build my-program.roc to see the commands that Roc is using to build your host language code.roc build my-program.roc --precompiled-host=<path> and change the commands as you see fit. Brian Carroll (Mar 05 2024 at 12:15):And if anyone feels like doing this for the existing examples and basic-cli then that would be a very welcome contribution!
Brian Carroll (Mar 05 2024 at 12:16):Oh I should mention that ROC_PRINT_BUILD_COMMANDS=1 will only work with a development build of the Roc compiler.
Oskar Hahn (Mar 05 2024 at 16:47):This sounds amazing, but I was not able to build a non-wasi wasm object with that.
There were different problems. One of them might be this zig issue: https://github.com/ziglang/zig/issues/15005 It seems, that when zig 0.11 builds C-code to wasm, it always adds the "wasi_snapshot_preview1" "random_get". Roc-code probably counts as C-code. So until this is fixed in zig (and the milestone is set to zig 0.14), it will not be possible to build a wasm-file with roc, that does not need wasi_snapshot_preview1.
When I changed wasm32-wasi to wasm32-freestanding in link.rs, then I was able to build a wasm module, that did not require proc_exit, fd_write or _start (only random_get).
I was not able to get the same result with ROC_PRINT_BUILD_COMMANDS and --precompiled-platform (you write --precompiled-host=<path>, but you probably mean --precompiled-platform).
The reason is, that you not only have to change the "build"-target, but also the "link"-target. See the difference in line 268 and line 1209 in link.rs.
After I change line 1225 in link.rs from build-exe to build-lib and line 1234 to wasm32-freestanding-musl and also added the argument -dynamic, I got a wasm-module, that only depends on random_get.
Another problem with ROC_PRINT_BUILD_COMMANDS is, that the build command contains a local path to glue.zig. In my case --mod glue::/home/ossi/src/roc/crates/compiler/builtins/bitcode/src/glue.zig. So it is not possible to add this in a build.rs in a git-repo.
In conclusion, I don't think, that the ROC_PRINT_BUILD_COMMANDS is a solution for a freestanding wasm-module.
Brian Carroll (Mar 05 2024 at 17:53):What solution could ever possibly exist that does not have this problem?
Brian Carroll (Mar 05 2024 at 17:57):This seems to be an input into the build so I suppose it has to be configured somehow.
Brian Carroll (Mar 05 2024 at 17:59):That flag was not meant to be a finished solution, it was a starting point to modify.
Anton (Mar 05 2024 at 18:26):We do include crates/compiler/builtins/bitcode/src/glue.zig in our nightly releases, so the build script could retrieve it from there.
Oskar Hahn (Mar 05 2024 at 21:36):I am sorry, if my last comment sounded negative. My English is not so good and it is hard for me to express myself. I really liked to look at the build command from ROC_PRINT_BUILD_COMMANDS. I played around with it for more then an hour. I learned so much.
Brian Carroll (Mar 05 2024 at 22:11):No problem! Glad you found it useful.
Luke Boswell (Mar 05 2024 at 23:01):Richard Feldman said:
yeah basically the goal would be that:
- we always build applications from a precompiled host, so no more
"🔨 Rebuilding platform..."message- this means that the individual platform needs to have a script which does the relevant parts of "rebuilding platform" for its host
- the goal of the platform build script should be outputting a Roc precompiled host file
We have an issue for tracking this #6414 if anyone is interested.
Brendan Hansknecht (Mar 05 2024 at 23:12):Related issue: #6037
Brendan Hansknecht (Mar 05 2024 at 23:12):Might be duplicates to some extent
Luke Boswell (Mar 06 2024 at 10:46):Yeah looks like a duplicate. I dont mind closing, just want to check we have all the things covered. I wont be able to look at it in any detail for a couple of days.
Last updated: Jul 05 2025 at 12:14 UTC