Oskar Hahn (Jan 31 2024 at 17:07): Oskar Hahn (Jan 31 2024 at 17:07):The current way, I build roc with a go-platform is by first building roc in a roc.o file, and afterwards call go build.
This has some problems. For example, it is not possible to publish the go-platform separately from the roc-application, since you always have to build the go code last.
It would be nice to do it the other way around: First build the go code and afterwards link it together with roc.
My only knowledge about linking is, that it combines binaries. But I don't know how it is done or what the requirements are. Until now, I only used languages, where linking was done automatically.
When I naively call roc build main.roc (with the go-platoform), I get an error like: failed to open file "dynhost".
If I create this file with go build -o dynhost, and then run roc build again, I get: Shared lib not found as a dependency of the executable.
Does anyone know, how this is possible or what terms I have to google?
Brendan Hansknecht (Jan 31 2024 at 17:09):So only linux has the surgical linker currently, but it is the long term solution that Roc is betting on.
Brendan Hansknecht (Jan 31 2024 at 17:10):For your executable, go build -o dynhost is probably correct, but the app has to be changed slightly.
Brendan Hansknecht (Jan 31 2024 at 17:10):The go application has to be changed to dynamically link to libapp.so which would be a shared library that roc generates. Instead of statically linking to a .o file.
Brendan Hansknecht (Jan 31 2024 at 17:11):You can generate an empty library to link against with roc gen-stub-lib
Brendan Hansknecht (Jan 31 2024 at 17:11):Or you can use a fully functional shared library with roc build --lib
Brendan Hansknecht (Jan 31 2024 at 17:12):Once you have go attempting to dynamically load and depend on the roc app, the surgical linker can do the rest.
Oskar Hahn (Jan 31 2024 at 17:28):This sounds good. I only need support for linux.
When I call roc gen-stub-lib example/main.roc (which is an example application), it creates a libapp.so. If I use this in go (#cgo LDFLAGS: ./libapp.so -ldl), then I get the errors like /tmp/go-build/roc.cgo2.c:83:(.text+0x50): undefined reference to roc__mainForHost_0_caller'`. It seems, that the roc function are not included.
But there is no error for roc__mainForHost_1_exposed_generic.
When I use roc build --lib example/main.roc, it creates a file libapp.so.1.0. When I use this file in go, I get the error: invalid flag in #cgo LDFLAGS: ./libapp.so.1.0.
When I rename libapp.so.1.0 to libapp.so, and use it with go build -o dynhost, then dynhost is created. But when I call roc build example/main.roc, I get the same error as in the beginning: Shared lib not found as a dependency of the executable.
Oskar Hahn (Jan 31 2024 at 17:31):By the way, here is the code, but it is not presentable yet :grinning: https://github.com/normanjaeckel/Meier/tree/main/platform
Brendan Hansknecht (Jan 31 2024 at 17:31):
undefined reference to roc__mainForHost_0_caller
The symbol names did change at one point. Maybe that didn't get update for stub lib generation.
Brendan Hansknecht (Jan 31 2024 at 17:35):I call roc build example/main.roc, I get the same error as in the beginning: Shared lib not found as a dependency of the executable.
Two more pieces. The host needs to be preprocessed...something like this:
roc preprocess-host examples/main.roc
If building locally, you need to use --prebuilt-platform currently. If you bundle it and upload it to a url, that flag is the default. So it should just work.
Oskar Hahn (Jan 31 2024 at 17:39):roc preprocess-host example/main.roc also returns the error Shared lib not found as a dependency of the executable.
roc build example/main.roc --prebuilt-platform returns something like I was expecting this file to exist: example/../linux-x64.rh which makes sense, when the first command failed.
Brendan Hansknecht (Jan 31 2024 at 17:39):Are you sure that go actually linked to the shared library?
Oskar Hahn (Jan 31 2024 at 17:40):I have no idea what the term shared libary means or how I tell go to do something like that :big_smile: I will google this.
Brendan Hansknecht (Jan 31 2024 at 17:42):Try:
#cgo LDFLAGS: -L. -lapp
Basically -L says search for libraries in the specified directory in this case ., but it should be whatever directory libapp.so is in.
-l say link to a library with some name. In this case app is the name.
Oskar Hahn (Jan 31 2024 at 17:47):This returns:
/usr/lib/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: cannot find -lapp: No such file or directory
collect2: error: ld returned 1 exit status
It returns a similar error for -llibapp or -libapp.so
Brendan Hansknecht (Jan 31 2024 at 17:48):That probably means the -L. is wrong. The current directory according to go probably doesn't include libapp.so.
Oskar Hahn (Jan 31 2024 at 17:49):Ahh, I had the file in pwd. When I move the libapp.so file in the same directory, as the .go file, then I get a different error
Brendan Hansknecht (Jan 31 2024 at 17:50):I don't have much time to look into this now, I but I can mess around with go linking on linux tonight if you don't figure it out before then. Figure out what actually is needed.
Oskar Hahn (Jan 31 2024 at 17:50):Now I get the errors, that roc__mainForHost_0_caller are not included.
Oskar Hahn (Jan 31 2024 at 17:50):Brendan Hansknecht said:
I don't have much time to look into this now, I but I can mess around with go linking on linux tonight if you don't figure it out before then. Figure out what actually is needed.
That would be very nice. I inform you, if I find something out
Oskar Hahn (Jan 31 2024 at 17:51):Thank you so far.
Brendan Hansknecht (Jan 31 2024 at 17:56):I guess make sure you have the real library. It definitely should be defined in that. Should be in the stub lib too, but I would be a lot less surprised if that has a bug.
Oskar Hahn (Jan 31 2024 at 18:02):I thould, I did use roc build --lib, but probably was wrong. So here is, what I am currently doing after removing all the artifacts.
roc build --lib --output roc/libapp.so example/main.roc
mv roc/libapp.so.1.0 roc/libapp.so
go build -o dynhost
roc preprocess-host example/main.roc
The last command returns Shared lib not found as a dependency of the executable.
Oskar Hahn (Jan 31 2024 at 18:03):In the go file, I use #cgo LDFLAGS: -L. -lapp
Brendan Hansknecht (Jan 31 2024 at 18:11):Can you run ldd on the executable. I think that prints dynamic dependencies
Oskar Hahn (Jan 31 2024 at 18:12):ldd dynhost
linux-vdso.so.1 (0x00007ffff23e3000)
roc/libapp.so.1 => not found
libresolv.so.2 => /usr/lib/libresolv.so.2 (0x00007f75eee81000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007f75eec9f000)
/lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f75eeebd000)
When I rename roc/libapp.so to roc/libapp.so.1, then it returns:
ldd dynhost
linux-vdso.so.1 (0x00007ffe605e0000)
roc/libapp.so.1 (0x00007f01a7a92000)
libresolv.so.2 => /usr/lib/libresolv.so.2 (0x00007f01a7a58000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007f01a7876000)
libm.so.6 => /usr/lib/libm.so.6 (0x00007f01a7789000)
libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f01a7764000)
/lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f01a7a9d000)
But roc preprocess-host example/main.roc still returns Shared lib not found as a dependency of the executable
Brendan Hansknecht (Jan 31 2024 at 18:13):Ah....our checks are too similar....
Brendan Hansknecht (Jan 31 2024 at 18:14):I think the surgical linker just checks for the name starting with libapp
Brendan Hansknecht (Jan 31 2024 at 18:14):So not smart enough to remove the directory before it.
Oskar Hahn (Jan 31 2024 at 18:43):Next thing I tried was changing the LDFlags to: LDFLAGS: -L.. -lapp
This produces a binary with:
ldd dynhost
linux-vdso.so.1 (0x00007ffcb1d9a000)
libapp.so.1 => not found
libresolv.so.2 => /usr/lib/libresolv.so.2 (0x00007f1da79e0000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007f1da77fe000)
/lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f1da7a1c000)
But I don't know why it can not find libapp.so.1. I copied the file from libapp.so to libapp.so.1 as before, but this time, it can not find it
Oskar Hahn (Jan 31 2024 at 18:59):For some reason, libapp.so.1 was not executable. After setting the bit, I now get
ldd dynhost
linux-vdso.so.1 (0x00007ffd4434b000)
libapp.so.1 => ./libapp.so.1 (0x00007f7b11c6f000)
libresolv.so.2 => /usr/lib/libresolv.so.2 (0x00007f7b11c35000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007f7b1141e000)
libm.so.6 => /usr/lib/libm.so.6 (0x00007f7b11b48000)
libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f7b11b23000)
/usr/lib64/ld-linux-x86-64.so.2 (0x00007f7b11c7a000)
But roc preprocess-host main.roc still returns Shared lib not found as a dependency of the executable
Brendan Hansknecht (Jan 31 2024 at 19:16):Sounds like our check may literally be for an exact match. This should be really easy to fix. Just a tiny bit of extra string processing in rust.
Brendan Hansknecht (Jan 31 2024 at 19:17):If you want to look into it. Probably search libapp.so in the linker folder.
Oskar Hahn (Jan 31 2024 at 19:18):I am already on it. It does not help, that I never wrote rust before :big_smile: but I was already able to find out, how to debug print a value :tada:
Oskar Hahn (Jan 31 2024 at 19:20):The problem is not the roc-prefix, but the .1suffix
Oskar Hahn (Jan 31 2024 at 19:22):The value is Some("libapp.so.1") but the expected value is Some("libapp.so")
Oskar Hahn (Jan 31 2024 at 19:27):Do you know, why go adds the suffix?
Oskar Hahn (Jan 31 2024 at 19:44):Ok. I am two steps farther. I change the line liked above to:
if Path::new(c_str).file_name().unwrap_or(OsStr::new("")).to_string_lossy().starts_with("libapp.so") {
Now, roc preprocess-host example/main.roc creates the .rh and .rm files. And roc build example/main.roc --prebuilt-platformsuccessfully builds something. But when I call example/main, I get
Segmentation fault (core dumped)
I repeated all the steps after removing all artifacts:
roc build --lib --output roc/libapp.so example/main.roc
mv roc/libapp.so.1.0 roc/libapp.so
cp roc/libapp.so roc/libapp.so.1
go build -o dynhost
roc preprocess-host example/main.roc
roc build example/main.roc --prebuilt-platform
./example/main
Now, it returns: ./example/main: error while loading shared libraries: unexpected PLT reloc type 0x00
Oskar Hahn (Jan 31 2024 at 20:14):When I change go build -o dynhost to go build -o dynhost -buildmode=c-share, then I get the Segmentation fault (core dumped) error. I don't know which error is closer to a correct result
Oskar Hahn (Jan 31 2024 at 22:46):I tried the same with the example GoPlatform since it is much simpler. But I get the same error.
I also used patchelf --replace-needed libapp.so.1 libapp.so dynhost instead of patching roc.
And I looked at the dynhost generated by a zig-platform. There it also says: libapp.so => not found. So this does not seem to be the problem.
But in the end, I always run into Segmentation fault (core dumped) and I don't know how to debug farther.
Brendan Hansknecht (Jan 31 2024 at 22:53):It may just be that the go runtime and all it's threading fun is doing something the surgical linker doesn't handle correctly
Brendan Hansknecht (Jan 31 2024 at 22:54):I'll try to take a look when I have time
Last updated: Jul 05 2025 at 12:14 UTC