Ian McLerran (Jan 16 2025 at 15:29): Ian McLerran (Jan 16 2025 at 15:29):Was going to update basic-cli to use Result.map_ok but ran into problems building. Does anyone know anything about this:
"Not enough free space between end of load commands and start of first section" when preprocessing surgical host.
INFO: Preprocessing surgical host ...
An internal compiler expectation was broken.
This is definitely a compiler bug.
Please file an issue here: <https://github.com/roc-lang/roc/issues/new/choose>
Not enough free space between end of load commands and start of first section
Consider recompiling the host with -headerpad <size> linker flag
Location: crates/linker/src/macho.rs:658:9
Program exited with error:
(ErrPreprocessingSurgicalBinary (CmdStatusErr (Other "Non-zero exit code: 1")))
Tip: If you do not want to exit on this error, use `Result.map_err` to handle the error. Docs for `Result.map_err`: <https://www.roc-lang.org/builtins/Result#map_err>
@Jakub Konka @Brendan Hansknecht
Jakub Konka (Jan 16 2025 at 17:58):That looks like my bad, will have a look shortly.
Jakub Konka (Jan 16 2025 at 17:58):On that topic, is preprocessing run in legacy linking mode too?
Brendan Hansknecht (Jan 16 2025 at 18:18):Oh, this is probably a case where we have a generic build command in basic CLI that attempts to preprocess even though Mac is experimental. So may be a case that we need to update the basic CLI build script
Jakub Konka (Jan 16 2025 at 18:24):Ah, yes, it looks like that's it
Jakub Konka (Jan 16 2025 at 18:29):@Ian McLerran do you have a branch I could test things on?
Ian McLerran (Jan 16 2025 at 19:04):Yep, here ya go: https://github.com/imclerran/basic-cli/tree/result-map-ok
Jakub Konka (Jan 16 2025 at 19:41):Hmm, but the error is different - it says that the module doesn't expose map_ok. Very confusing.
Ian McLerran (Jan 16 2025 at 19:50):Do you have the latest from roc main?
Jakub Konka (Jan 16 2025 at 19:53):Mhm, right, with a local build of Roc it builds fine, but I still don't see the linker error. Anyhow, this made me realise a few things and a few wrong assumptions about the build process, so for the time being I am inclined to convert the error into a warning until I figure out a way forward.
Jakub Konka (Jan 16 2025 at 19:56):@Brendan Hansknecht the change https://github.com/roc-lang/roc/pull/7524 As you will notice in the PR description I completely misunderstood who controls how the host is actually built which is crucial for applying the -headerpad <size> linker flag. Since it falls outside the Roc compiler, it may be difficult to enforce, so nerfing the error into a warning seems like a good temporary solution until we figure out if shifting in offsets (and memory) is viable. More in the PR description. Happy to discuss this further!
Ian McLerran (Jan 16 2025 at 19:56):Hmm.. thats funny you don't see the linker error.. also on MacOS/Aarch64?
Jakub Konka (Jan 16 2025 at 19:58):Yep, but do you use Nix?
Jakub Konka (Jan 16 2025 at 19:58):I have a sneaky suspicion you don't and thus you are using a different linker to me :D
Ian McLerran (Jan 16 2025 at 19:58):For building roc only... (not basic-cli)
Ian McLerran (Jan 16 2025 at 19:58):That could be!
Jakub Konka (Jan 16 2025 at 20:01):Right, yeah, so here's what happens. If you use Nix on a mac, you will use a Nix-maintained version of the Apple legacy linker which by default includes far more load commands and thus there is plenty of padding available for the surgical linker to re-use. If you are not using Nix, then you are using Apple's new linker (aka ld_prime) which aggressively compresses dynamic loader info, uses fixup chains, etc., which implies fewer load commands, which implies that the padding available between the end of the load commands and the start of the __text section is minimal.
Jakub Konka (Jan 16 2025 at 20:04):If shifting things in file/memory by the surgical linker will turn out to be difficult, another solution will be to tell the linker via Cargo flags/config to reserve more space between the load command headers and the __text section. But for now, my PR should silence the error since it's not that important anyhow since MachO surgical linker is still experimental. The good news is though that I have a much better understanding who controls who in the build process here :grinning:
Brendan Hansknecht (Jan 16 2025 at 21:12):Jakub Konka said:
Brendan Hansknecht the change https://github.com/roc-lang/roc/pull/7524 As you will notice in the PR description I completely misunderstood who controls how the host is actually built which is crucial for applying the
-headerpad <size>linker flag. Since it falls outside the Roc compiler, it may be difficult to enforce, so nerfing the error into a warning seems like a good temporary solution until we figure out if shifting in offsets (and memory) is viable. More in the PR description. Happy to discuss this further!
To be fair, I think it is reasonable for each platform author to set the flag. Most roc developers will just consume a pre built platform and won't have to interact with that
Jakub Konka (Jan 16 2025 at 21:43):With that in mind, maybe I could try setting that up in basic-cli and re-reverting the error? What do you think?
Jakub Konka (Jan 16 2025 at 22:49):Alright, here's an alternative solution that perhaps is the desired interaction between the host (platform) authors, and Roc app authors. Namely, https://github.com/roc-lang/roc/pull/7524 reverts back to throwing an error if insufficient space has been reserved but prints a much more informative error message that currently looks as follows:
INFO: Preprocessing surgical host ...
target/release/host
An internal compiler expectation was broken.
This is definitely a compiler bug.
Please file an issue here: <https://github.com/roc-lang/roc/issues/new/choose>
Not enough free space between end of load commands and start of first section in the host.
Consider recompiling the host with "-Wl,-headerpad,0x8000" linker flag.
Location: crates/linker/src/macho.rs:662:9
Program exited with error:
(ErrPreprocessingSurgicalBinary (CmdStatusErr (Other "Non-zero exit code: 1")))
Tip: If you do not want to exit on this error, use `Result.map_err` to handle the error. Docs for `Result.map_err`: <https://www.roc-lang.org/builtins/Result#map_err>
As you can see, we explicitly tell the user to have the platform rebuilt with a flag -Wl,-headerpad,0x8000.
Jakub Konka (Jan 16 2025 at 22:50):Next, https://github.com/roc-lang/basic-cli/pull/313 showcases how this can be accomplished for a Rust host.
Brendan Hansknecht (Jan 17 2025 at 02:53):Yeah, this is great. I think that is the expected interaction!
Last updated: Jul 06 2025 at 12:14 UTC