Stream: contributing

Topic: improving NixOS platform download error

dank (Jan 26 2023 at 10:43):

Heyy, I want to make my first contribution. I went through the good-first-issues and saw https://github.com/roc-lang/roc/issues/4751 which I think is fitting (this is even the one of the first issues I ran into as a NixOS user lol).

So my main question is about error reporting; I see a lot of stuff in the reporting crate but not sure- should I use some custom error or just eprintln my error and exit process with code 1?

And also as a sanity check question- my plan is to check if cfg unix, if it is then check if NixOS is in uname -a, if so exit with <maybe custom roc error>, is that ok?

Richard Feldman (Jan 26 2023 at 13:17):

hm I think that issue may be obsolete, since we figured out a way to make them work (at least in theory - haven't applied it yet!)

Richard Feldman (Jan 26 2023 at 13:18):

but @dank maybe you'd like to give the fix a shot? The idea is to have the basic-cli platform statically link musl instead of dynamically linking glibc on Linux, which should make it work on Nix

dank (Jan 26 2023 at 13:43):

Ah yes I would. I hope I wont get in over my head but I'll give it a shot :+1:

dank (Jan 26 2023 at 13:58):

@Richard Feldman wyt about using fhs (https://nixos.org/manual/nixpkgs/stable/#sec-fhs-environments)?

Richard Feldman (Jan 26 2023 at 15:14):

so ideally I'd like to have Roc work the same way on NixOS as it does on other Linux distros, without needing Roc itself to be coupled to nix

Richard Feldman (Jan 26 2023 at 15:15):

as far as I can tell, the best way to achieve that is to have build-cli (and other platforms that want to work on NixOS) use musl instead of glibc!

dank (Jan 26 2023 at 15:23):

yes tbh you're right it is cleaner

dank (Jan 26 2023 at 15:31):

So the technical process would be adding musl as a cargo.toml dep on compiler build crate/ making it a target in https://github.com/roc-lang/roc/blob/main/crates/compiler/build/src/link.rs ? (still finding my way in the codebase)

Richard Feldman (Jan 26 2023 at 21:48):

it should be doable completely in the https://github.com/roc-lang/basic-cli repo - and yeah something to do with the cargo.toml in there, but I personally don't have any experience setting up a Rust project to use musl, so I don't know exactly what the steps are! (I believe @Ayaz Hafiz has done it before and might have tips?)

Ayaz Hafiz (Jan 26 2023 at 22:08):

You have to ask rust to please compile against a musl target (e.g. x86_64-unknown-linux-musl). I don't know the right incantations nix needs in order to do so, afaik basic-cli is build with nix. But yeah if you want to build a musl x86-64 linux binary, it's enough to get on a x86 linux box and do

cargo build --target x86_64-unknown-linux-musl

You need to do it at the rustc level because you need the musl standard library linked in

dank (Jan 26 2023 at 22:19):

Ayaz Hafiz said:

You have to ask rust to please compile against a musl target (e.g. x86_64-unknown-linux-musl). I don't know the right incantations nix needs in order to do so, afaik basic-cli is build with nix. But yeah if you want to build a musl x86-64 linux binary, it's enough to get on a x86 linux box and do

cargo build --target x86_64-unknown-linux-musl

You need to do it at the rustc level because you need the musl standard library linked in

yes and I believe you need to first rustup target add x86_64-unknown-linux-musl
i saw a way to do that in cargo.toml but it's in unstable : \ https://doc.rust-lang.org/cargo/reference/unstable.html#per-package-target

Richard Feldman (Jan 27 2023 at 01:03):

we can make basic-cli use nightly if that's the only way to get musl!

Anton (Jan 27 2023 at 08:31):

I believe it should be enough to set the target in basic-cli/src/rust-toolchain.toml.
Exactly like we set the targets value in the roc repo here.

dank (Jan 27 2023 at 16:43):

Anton said:

I believe it should be enough to set the target in basic-cli/src/rust-toolchain.toml.
Exactly like we set the targets value in the roc repo here.

Thanks. I'm trying to test it. But first can u explain a bit on how you build your tar.br?
compressed with brotli right? but how did you build it from the repo?

Anton (Jan 27 2023 at 16:46):

Sure :)
You can find the full build process in this file.

Richard Feldman (Jan 27 2023 at 16:50):

specifically roc build --bundle=.tar.br is what creates it - https://github.com/roc-lang/roc/blob/99050956d353aa544a97912bd2f664ab19c9ff98/.github/workflows/basic_cli_build_release.yml#L121 - and everything before that is just getting all the files in the same directory so roc build can find them!

Richard Feldman (Jan 27 2023 at 16:51):

you want to use roc build because it also takes care of hashing the .tar.br file at the end, and that hash has to be the filename of the .tar.br file in the URL or else roc won't download it

Richard Feldman (Jan 27 2023 at 16:52):

(it's a hash of the contents of the file and is used as an integrity check to make sure the file hasn't been tampered with in case the URL later gets compromised)

dank (Jan 27 2023 at 16:54):

thanks!!

Anton (Jan 27 2023 at 17:02):

For quick initial testing you probably want to avoid going through the whole bundling process.
Using the basic-cli platform through the path like here should work.
I also think we'll need to make a change somewhere in the roc codebase so that the musl target is used to compile the platform, I'm going to check that after dinner.

Anton (Jan 27 2023 at 18:14):

So, for testing I think that adding .cargo/config to basic-cli/src should be all that remains, the contents of the file should be:

[build]
target = "x86_64-unknown-linux-musl"

This will require no changes to the roc codebase.

dank (Jan 27 2023 at 23:15):

Anton said:

For quick initial testing you probably want to avoid going through the whole bundling process.
Using the basic-cli platform through the path like here should work.
I also think we'll need to make a change somewhere in the roc codebase so that the musl target is used to compile the platform, I'm going to check that after dinner.

so I think it's alright and working locally
but I just can't get the ci tests working
is the build script (ci/build_basic_cli.sh) not working atm or am I doing something dumb
failing with some path caused issues it seems. maybe something's different cz it's a fork

Anton (Jan 28 2023 at 11:02):

I'll take a look today

Anton (Jan 28 2023 at 15:43):

@dank, this modified build_basic_cli.sh should work.

dank (Jan 28 2023 at 16:18):

Anton said:

@dank, this modified build_basic_cli.sh should work.

thanks looks good ill test

dank (Jan 28 2023 at 16:36):

@Anton are you sure that [self-hosted, macOS, ARM64] is valid? i suspect not and that's why it hangs now looking for a runner
(first 2 jobs went great tho)

Anton (Jan 28 2023 at 16:48):

That runner is busy with a job of mine, it will probably start on yours after it's done.

dank (Jan 28 2023 at 16:50):

ohh alright

Anton (Jan 28 2023 at 16:50):

Because it is a self-hosted server it can only run on a PR on roc-lang/roc though, where are you running the tests now?

dank (Jan 28 2023 at 17:48):

Anton said:

Because it is a self-hosted server it can only run on a PR on roc-lang/roc though, where are you running the tests now?

hm i changed it to on: [workflow_dispatch] and ran on my roc fork actions

dank (Jan 28 2023 at 17:49):

can it not just upload it as a release on my fork

Anton (Jan 28 2023 at 18:17):

If you have an apple silicon device, you can set it up as an action-runner for your repo. But it'll probably be faster for me to test it. Should I test the main branch on https://github.com/dankeyy/basic-cli?

dank (Jan 28 2023 at 18:19):

Anton said:

If you have an apple silicon device, you can set it up as an action-runner for your repo. But it'll probably be faster for me to test it. Should I test the main branch on https://github.com/dankeyy/basic-cli?

yea thanks

Anton (Jan 28 2023 at 19:07):

The test started in https://github.com/roc-lang/roc/pull/4980

dank (Jan 28 2023 at 20:02):

ok so bad news that did not work
sure no change is required from the roc side?

dank (Jan 28 2023 at 20:05):

cc @Anton

dank (Jan 28 2023 at 20:08):

ohhh I fucked it up lol I put it in the second src
fixed it, you can run again sry anton

Anton (Jan 29 2023 at 08:53):

No problem, I started a new run

Anton (Jan 29 2023 at 08:57):

There's a failure on macos aarch64, It's trying to use the x86_64-unknown-linux-musl target there too, so we'll have to detect when we are on x86_64 linux and only then apply the target. No need to rush to fix this though, feel free to get to it when you like :)

dank (Jan 29 2023 at 11:06):

Anton said:

There's a failure on macos aarch64, It's trying to use the x86_64-unknown-linux-musl target there too, so we'll have to detect when we are on x86_64 linux and only then apply the target. No need to rush to fix this though, feel free to get to it when you like :)

ah right. so for aarch can't we use aarch64-unknown-linux-musl in some form of conditional targeting?
unless you really want it to target musl only on x86

dank (Jan 29 2023 at 11:48):

looking into it now, the x86 failed too, with
https://github.com/roc-lang/roc/actions/runs/4035991075/jobs/6938275625#step:4:385
not sure why's that :thinking:

Anton (Jan 30 2023 at 08:44):

ah right. so for aarch can't we use aarch64-unknown-linux-musl in some form of conditional targeting?

That actually does sound like a good idea, we don't do CI tests yet on aarch64 linux but it would be good to have that ready.

Anton (Jan 30 2023 at 08:50):

I believe the error on linux x86 is because musl-gcc is not installed on the ci machine, I'll take care of that

dank (Jan 30 2023 at 22:32):

Anton said:

I believe the error on linux x86 is because musl-gcc is not installed on the ci machine, I'll take care of that

alright. anything else u want me to look into?
(i can take on other issues)

Anton (Jan 31 2023 at 09:07):

Cool :) , #4751 is still an issue that can come up, I've edited the issue to improve it. Would you be interested in tackling it?

dank (Jan 31 2023 at 10:01):

Anton said:

Cool :) , #4751 is still an issue that can come up, I've edited the issue to improve it. Would you be interested in tackling it?

uh sure if that's what you want. but at this point we can just solve it, no?
isn't the only thing missing musl for aarch64 which we can target aswell?

Anton (Jan 31 2023 at 10:27):

but at this point we can just solve it, no?

Even after we solve it, this case can still come up. We shouldn't force the platform to be dynamically linked when bundle-ing the tar.br because it is necessary for some graphical platforms. Eventually we want to include a flake in the tar.br as well for this case, but due to the comparatively small userbase of NixOS I don't want to force this on platform authors either.

dank (Jan 31 2023 at 18:48):

Got you. on it, will pr soon

dank (Jan 31 2023 at 19:09):

hold up. how come basic_cli's linux-x86_64.o doesnt contain dynamic sections?

❯ readelf -d /home/dankey/.cache/roc/packages/github.com/roc-lang/basic-cli/releases/download/0.2.0/8tCohJeXMBUnjo_zdMq0jSaqdYoCWJkWazBd4wa8cQU/linux-x86_64.o

There is no dynamic section in this file.

dank (Jan 31 2023 at 19:46):

so wait what does this mean
https://gist.githubusercontent.com/dankeyy/34e25e2f5f2cadf858ee7579bc393b6d/raw/9fe946e51a42a6ed117aa1e6618768c02e83fdc4/traceback.txt

Brendan Hansknecht (Jan 31 2023 at 19:55):

Can you try just building (roc build ...) instead of building and running?

Brendan Hansknecht (Jan 31 2023 at 19:55):

Might give a better error here.

dank (Jan 31 2023 at 20:01):

Brendan Hansknecht said:

Can you try just building (roc build ...) instead of building and running?

quite the opposite

❯ RUST_BACKTRACE=FULL ~/dev/roc/target/debug/roc build

There is no dynamic section in this file.

0 errors and 0 warnings found in 585 ms while successfully building:

    hello

well makes sense it's just not observed until runtime

this is dev which is way less descriptive

❯ RUST_BACKTRACE=FULL ~/dev/roc/target/debug/roc dev

There is no dynamic section in this file.

Error Os { code: 2, kind: NotFound, message: "No such file or directory" }

Brian Carroll (Jan 31 2023 at 20:03):

You're missing the .roc file argument

dank (Jan 31 2023 at 20:03):

(The "There is no dynamic section in this file." i printed myself ofc)

dank (Jan 31 2023 at 20:03):

Brian Carroll said:

You're missing the .roc file argument

it's main.roc by default tho

dank (Jan 31 2023 at 20:04):

which i do have (that says hello world)

Brendan Hansknecht (Jan 31 2023 at 20:44):

So the build succeeded, what do you get from ./hello?

dank (Jan 31 2023 at 20:51):

Brendan Hansknecht said:

So the build succeeded, what do you get from ./hello?

exec: Failed to execute process '/home/dankey/testing/roctests/hello': The file exists and is executable. Check the interpreter or linker?

Brendan Hansknecht (Jan 31 2023 at 21:10):

Interesting. I guess for completeness. What do you get with roc --linker=legacy dev?

dank (Jan 31 2023 at 21:15):

same as without the flag Error Os { code: 2, kind: NotFound, message: "No such file or directory" }

dank (Jan 31 2023 at 21:16):

this is ldd output btw

❯ ldd hello
        linux-vdso.so.1 (0x00007ffe00768000)
        libgcc_s.so.1 => /nix/store/9xfad3b5z4y00mzmk2wnn4900q0qmxns-glibc-2.35-224/lib/libgcc_s.so.1 (0x00007fb45281a000)
        librt.so.1 => /nix/store/9xfad3b5z4y00mzmk2wnn4900q0qmxns-glibc-2.35-224/lib/librt.so.1 (0x00007fb452815000)
        libpthread.so.0 => /nix/store/9xfad3b5z4y00mzmk2wnn4900q0qmxns-glibc-2.35-224/lib/libpthread.so.0 (0x00007fb452810000)
        libm.so.6 => /nix/store/9xfad3b5z4y00mzmk2wnn4900q0qmxns-glibc-2.35-224/lib/libm.so.6 (0x00007fb452730000)
        libdl.so.2 => /nix/store/9xfad3b5z4y00mzmk2wnn4900q0qmxns-glibc-2.35-224/lib/libdl.so.2 (0x00007fb45272b000)
        libc.so.6 => /nix/store/9xfad3b5z4y00mzmk2wnn4900q0qmxns-glibc-2.35-224/lib/libc.so.6 (0x00007fb452520000)
        /lib64/ld-linux-x86-64.so.2 => /nix/store/9xfad3b5z4y00mzmk2wnn4900q0qmxns-glibc-2.35-224/lib64/ld-linux-x86-64.so.2 (0x00007fb453ae6000)

dank (Jan 31 2023 at 21:43):

so anyways back to the point
https://github.com/roc-lang/roc/issues/4751
the object file could have no dynamic section and still be linked dynamically cause it would just contains arbitrary symbols that should be resolved at linkage time right

so firstly readelf -d wouldnt suffice right
and second i still dont really understand that messy panic

dank (Jan 31 2023 at 21:46):

cc @Anton

Anton (Feb 01 2023 at 11:20):

I've looked into it and we can use ldd on the rh1 file instead:

ldd linux-x86_64.rh*
ldd: warning: you do not have execution permission for `./linux-x86_64.rh1'
        statically linked

So we'll want to look for statically linked in the output of that command.

Note the rh* to support rh1, rh2...

Anton (Feb 01 2023 at 11:22):

@dank what command generated that panic?

dank (Feb 01 2023 at 11:57):

Anton said:

dank what command generated that panic?

running roc run on a main.roc with a dynamically linked platform
doesnt really matter now. it is bc of the dynamic linking. just weird how non descriptive some of those panics gets

anyways yea so I'll just do ldd on rh1 file :+1:

dank (Feb 01 2023 at 20:02):

PR's up

Anton (Feb 02 2023 at 09:02):

Thanks :) , I'll review tomorrow!

Last updated: Jul 05 2025 at 12:14 UTC