Stream: compiler development

Topic: linker error cross-compiling to Linux

Richard Feldman (Sep 28 2023 at 18:09):

I'm running this command on an ARM mac, cross-compiling to ARM Linux...it works fine if I cross-compile to x86_64, but...

zig cc --target=aarch64-linux-gnu -o /Users/foo/main.node /Users/foo/main.o /Users/foo/node-to-roc.c -I/Users/foo/.volta/tools/image/node/16.16.0/include/node -fPIC -pthread -Wall -Wextra -Wendif-labels -W -Wno-unused-parameter -fno-omit-frame-pointer -lc -lm -lpthread -ldl -lutil -lrt -shared

...when I change nothing other than the --target to be aarch64 instead of x64, I get:

LLD Link... ld.lld: error: relocation R_AARCH64_MOVW_UABS_G0_NC cannot be used against local symbol; recompile with -fPIC
>>> defined in /Users/foo/main.o
>>> referenced by builtins-aarch64
>>>               /Users/foo/main.o:(roc_builtins.str.str_split)

(a bunch more of these, lots of the same error in different str builtins, like str_split here, graphemes, etc) - anyone know why this might be?

Richard Feldman (Sep 28 2023 at 18:12):

it seems to be saying that main.o is somehow not compiled with PIC, but:

• we always specify PIC as far as I can tell, when outputting from LLVM certainly (which is what we're using for the backend here)
• even if we weren't, why would it succeed on x64 but fail on aarch64? I also can't find any cases where we specify a different relocation mode when building for x64 vs aarch64

Brendan Hansknecht (Sep 28 2023 at 18:49):

iirc, the issue isn't pic, it is the linker not having been updated to support that relocation type or something like that

Richard Feldman (Sep 28 2023 at 20:42):

@Brendan Hansknecht hm, the linker as in lld?

Brendan Hansknecht (Sep 28 2023 at 20:43):

yes

Richard Feldman (Sep 28 2023 at 20:43):

hrm

Richard Feldman (Sep 28 2023 at 20:43):

I'm fairly sure this worked a few months ago - I wonder if it's some new zig builtins we added since then?

Richard Feldman (Sep 28 2023 at 20:44):

could that possibly make sense?

Brendan Hansknecht (Sep 28 2023 at 20:44):

Yeah, that would totally make sense.

Brendan Hansknecht (Sep 28 2023 at 20:44):

It is a rare relocation type.

Brendan Hansknecht (Sep 28 2023 at 20:44):

Probably weren't generating it before.

Richard Feldman (Sep 28 2023 at 20:44):

interesting! Is there a way we could change them to not encounter this?

Brendan Hansknecht (Sep 28 2023 at 20:46):

hmm, based on this PR, that relocation theoretically got added to lld in 2016: https://reviews.llvm.org/D24403?id=70987

Brendan Hansknecht (Sep 28 2023 at 20:48):

can you test with ld to see if you get the same behavior?

Richard Feldman (Sep 28 2023 at 20:49):

is there a way to tell zig to use ld instead of lld?

Richard Feldman (Sep 28 2023 at 20:52):

nm chatGPT told me - trying it now!

Brendan Hansknecht (Sep 28 2023 at 20:53):

Also, you could try changing our builtin build.zig files to set force_pic

Richard Feldman (Sep 28 2023 at 20:59):

yeah ld didn't fix it, I'll try that

Richard Feldman (Sep 28 2023 at 21:15):

also didn't fix it, unfortunately

Richard Feldman (Sep 28 2023 at 21:15):

also I noticed mainForHost is one of the ones giving the error, so maybe it's not just builtins after all :thinking:

Brendan Hansknecht (Sep 28 2023 at 21:21):

I wonder if we are generating llvm ir that can't be pic on aarch64 and llvm is just giving up. Is pic just a suggestion like the calling convention?

Richard Feldman (Sep 28 2023 at 22:59):

oof, I hope not

Richard Feldman (Sep 28 2023 at 22:59):

what's weird is, I definitely had cross-compiling to arm64 working a few months ago

Richard Feldman (Sep 28 2023 at 22:59):

so I'm not sure what could have changed

Richard Feldman (Sep 28 2023 at 23:00):

also this is the last thing blocking getting TypeScript calling Roc into production at Vendr (finally!), so I really hope it doesn't turn into a big rabbit hole :laughing:

Anton (Sep 29 2023 at 08:42):

Bisecting to the commit where this broke seems like a good next step.

Last updated: Jul 06 2025 at 12:14 UTC