Luke Boswell (Oct 08 2023 at 07:45): Luke Boswell (Oct 08 2023 at 07:45):@Brendan Hansknecht mentioned that surgical linking will be useful for the graphics platform I have been tinkering with.
I don't 100% understand how that all works, but my mental model is that we generate an executable that expects to dynamically link Roc in. We can then distribute that with the platform, in roc build --bundle I assume, and then when Roc builds the app the surgical linker can insert the app parts and fix it up so that it works without needing a dynamic library loaded. To do this I think it would also be desirable for platforms to be specify how they are built. I.e. offload the building to build.zig instead of doing that in the Roc cli build pipeline.
I don't really have a question in here, but just wanted to share.
Brian Carroll (Oct 08 2023 at 08:20):Yes that's how it works! Including the bit about platforms being able to specify their own build.
There is some minimal information the Roc build system needs to know, such as how to run the Zig compiler, what target OS and CPU to ask Zig to build for, etc. That remains true when you have a build.zig.
Luke Boswell (Oct 09 2023 at 08:12):Actually, I think I've just realised I can modify my Roc app, rebuild just the roc app using roc build --lib and the "platform" part (which is actually now an full executable) doesn't need to be recompiled because it will just dynamically link in the new Roc app (which was compiled to a dylib).
I guess this is kind of obvious regarding how dynamic libraries work... but I wonder if it would make sense for Roc to be able to distribute the platform as a prebuilt executable which is expecting to dynamically link in a roc app, and use this this from a URL package. So, when I run roc run it can see that the platform is already built into an app, and then builds roc into a dynamic library and uses that. Does this make sense? or am I talking crazy?
Folkert de Vries (Oct 09 2023 at 11:34):one tricky thing is that we have two-way communication: the roc app provides symbols to the host (e.g. mainForHost) , but also requires symbols from it (e.g. roc_alloc). That has been hard to get to work in the past (but should technically be possible I think?)
Brendan Hansknecht (Oct 09 2023 at 15:53):theoretically linkers have flags that should just fix this (--export-symbols on linux, --keep-symbols no mac, -rdynamic in general for rust code)...sadly sometimes code is removed before we even get to the linker
Luke Boswell (Oct 26 2023 at 03:23):I've been looking at how to make a build.zig that generates the expected files for a roc package. I am not sure what the .rh file is specifically. From crates/compiler/build/src/program.rs I can see that it is used with the surgical linker, and my understanding is that this is an executable that roc will copy and then the linker will modify in place?
To make a satisfactory .rh file manually, should I provide a dummy implementation separately for e.g. roc__mainForHost_1_exposed_generic and build an executable?
% ls -al ~/.cache/roc/packages/github.com/roc-lang/basic-cli/releases/download/0.5.0/Cufzl36_SnJ4QbOoEmiJ5dIpUxBvdB3NEySvuH82Wio/
.. removed
-rw-r--r-- 1 luke staff 202238528 13 Aug 00:46 linux-arm64.o
-rw-r--r-- 1 luke staff 195862808 13 Aug 00:46 linux-x64.o
-rw-r--r-- 1 luke staff 109044440 13 Aug 00:46 linux-x64.rh
-rw-r--r-- 1 luke staff 37414704 13 Aug 00:46 macos-arm64.o
-rw-r--r-- 1 luke staff 43776984 13 Aug 00:46 macos-x64.o
-rw-r--r-- 1 luke staff 2300 13 Aug 00:46 metadata_linux-x64.rm
Also what is the metadata_linux-x64.rm file? I'm also unsure what that file is or how to generate it.
Luke Boswell (Oct 26 2023 at 03:28):I should be able to modify roc cli to use zig build for rebuilding a zig platform. I'm also hoping to have a build.zig that can cross compiles to generate the various artifacts required for a bundle.
Brendan Hansknecht (Oct 26 2023 at 04:24):Yeah, both of those files are outputs of the surgical linker preprocessing step
Luke Boswell (Oct 26 2023 at 05:20):Right, so I have to build them on the target operating system if I want to include these in the bundle. It looks like roc re-builds the platform, then immediately pre-processes the host. So to clarify, do I need to include these files in the bundle? I think I may have gone down a rabbit hole, and this isn't the issue with my zig cross-compiled platform.
Brendan Hansknecht (Oct 26 2023 at 12:26):Theoretically the code could run on any platform
Brendan Hansknecht (Oct 26 2023 at 12:27):But not sure where all it is pipelined / exposed
Anton (Oct 27 2023 at 09:30):The .rm and .rh files need to be in the bundle.
Luke Boswell (Nov 03 2023 at 01:34):Can someone please have a look at https://github.com/lukewilliamboswell/basic-graphics for me? I think we are really close to having a platform that people can use to make graphics, but the surgical linking issue prevents that for linux. I'm not sure how to generate the .rm and .rh files.
Also we don't have roc cli building the platform using build.zig so I need another step for users to build the platform i.e. bash build.sh. This means that you have to use roc run --prebuilt-platform when running locally.
Maybe a better experience here would be to see the prebuilt platform and then default to use that? The issue with this is that when you make a change to a platform for development, you would have to tell roc somehow to rebuild the platform. So, I guess the best option is to always rebuilt the platform.
Brendan Hansknecht (Nov 03 2023 at 01:38):Does your platform link with roc or consume it as a shared library?
Luke Boswell (Nov 03 2023 at 01:39):The platform is a static library lib.a file.
Luke Boswell (Nov 03 2023 at 01:39):Roc cli links that in using ld for legacy linker I think.
Brendan Hansknecht (Nov 03 2023 at 01:40):For the surgical linker it also has to be able to build as an executable that consumes roc as a shared library.
Luke Boswell (Nov 03 2023 at 01:41):The other issue (which I believe is minor) is that I rename the .a files to .o and it seems to link fine. Should roc use .a archives instead or be able to accept both?
Brendan Hansknecht (Nov 03 2023 at 01:41):Roc should accept both probably.
Luke Boswell (Nov 03 2023 at 01:44):Maybe if I could provide a dummy implementation for the extern fn roc__mainForHost_1_exposed_generic(*RocStr, *RocStr) void; then I build it as an executable. But I'm not sure what happens next. Do I use roc cli somehow to preprocess it for surgical linker?
Brendan Hansknecht (Nov 03 2023 at 02:12):So roc can generate a dummy lib for you that will provide extern fn roc__mainForHost_1_exposed_generic(*RocStr, *RocStr) void; and friends.
Brendan Hansknecht (Nov 03 2023 at 02:12):then you would link against that
Brendan Hansknecht (Nov 03 2023 at 02:12):I am not sure the state of doing this from not linux
Brendan Hansknecht (Nov 03 2023 at 02:18):roc gen-stub-lib then link against the generated lib
Brendan Hansknecht (Nov 03 2023 at 02:18):This is something roc link.rs normally does under the hood automatically.
Brendan Hansknecht (Nov 03 2023 at 02:20):I think we may need to expose some way to call the surgical linker preprocessor on any platform if we want to enable building the .rm and .rh files from non-linux
Brendan Hansknecht (Nov 03 2023 at 02:20):Should just be a matter of pipelining and exposing a command
Brendan Hansknecht (Nov 03 2023 at 02:21):With that, I think you could drive generating all build artifacts from a build.zig file.
Luke Boswell (Apr 28 2024 at 04:16):@Brendan Hansknecht can you please help me understand how we go from a platform to the preprocessed files required for surgical linking?
My understanding is;
roc gen-stub-lib examples/app.roc to stubbed library of the application, the app can be anthing, it doesn't matter, e.g. just returns Task.ok {} and does nothing.roc preprocess-host standalone-platform-binary on that binary to produce the .rh and .rm files used by the surgical linker.rh and .rm alongside the platform/main.roc so they get packaged with roc build --bundle .tar.brI think the problem I am currently seeing is unrelated to this, though I'm not sure. Whenever I run gen-stub-lib or preprocess-host I get a linker error.
$ roc gen-stub-lib examples/hello-world.roc
thread 'main' panicked at crates/linker/src/generate_dylib/macho.rs:90:27:
Failed to link dummy shared library - stderr of the `ld` command was:
ld: Missing -platform_version option
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
$ roc preprocess-host examples/hello-world.roc
thread 'main' panicked at crates/linker/src/generate_dylib/macho.rs:90:27:
Failed to link dummy shared library - stderr of the `ld` command was:
ld: Missing -platform_version option
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Lol, almost immediately after posting that I figured out I'm missing the target roc gen-stub-lib --target linux-x64 examples/hello-world.roc wors :tada:
Luke Boswell (Apr 28 2024 at 04:46):So this confirms we have a bug in the command for building a dylib for macos, or it's not yet implemented.
Brendan Hansknecht (Apr 28 2024 at 17:31):As a note, gen-stub-lib is techincailly not need and we should maybe remove it. Instead you can do roc build --lib someExampleApp.roc Then link the platform against that and contiue with the rest of the process. Just important that the app is called libapp when linked to the platform.
Richard Feldman (Apr 28 2024 at 17:33):Brendan Hansknecht said:
we should maybe remove it
I like that idea - seems like it would remove a footgun since there's a better alternative way to do it! :big_smile:
Oskar Hahn (Apr 28 2024 at 18:03):I would like, if you could remove gen-stub-lib. There are currently two ways to do this step (with build --libor with gen-stub-lib). This was confusing for me.
When you remove it, you probably also have to changeroc preprocess-host. It is currently calling gen-stub-lib internally. This is a problem. It recreates the libapp.so-file. So currently, you have to remember to call roc build --lib after roc preprocess-host, or you have a wrong libapp.so file in your platform.
Last updated: Jul 06 2025 at 12:14 UTC