Stream: beginners

Topic: Platform linking

Luke Boswell (Oct 28 2022 at 06:41):

I have been playing around with the compiler to improve my understanding of how Roc platforms work. I thought I might take some notes and share what I have so far for anyone else on a similar journey. I would appreciate any corrections to the below, or additions.

What is the pattern for how Roc functions are named in the generated files?

Roc Platform Linking

roc build main.roc --debug with a debug build of the Roc compiler will compile the application into a dynamic library at main.o.

If there is a prebuilt platform available it will be located at platform/host.o. If not then the Roc binary will do something to generate it.

The surgical linker will then dynamically link the aplpication main.o into the platform host.o to generate an exectuable.

Inspecting generated object files

Using objdump -t main.o you can see the symbol tables for the application e.g.

main.o: file format mach-o arm64

SYMBOL TABLE:
0000000000000000 l     F __TEXT,__text ltmp0
0000000000000000 l     F __TEXT,__text _roc_builtins.utils.allocate_with_refcount
// collapsed for brevity
00000000000001a4 g     F __TEXT,__text _roc__renderForHost_1_exposed
0000000000000184 g     F __TEXT,__text _roc__renderForHost_1_exposed_generic
00000000000001b4 g     F __TEXT,__text _roc__renderForHost_size

And objdump -t platform/host.o | grep "roc_" to the symbol table for the host, which gives something like the following;

// collapsed for brevity, this can be a very large number of symbols
000000000007ce68 g     F __TEXT,__text _roc_alloc
000000000007ced4 g     F __TEXT,__text _roc_dealloc
000000000007cfe8 g     F __TEXT,__text _roc_memcpy
000000000007d024 g     F __TEXT,__text _roc_memset
000000000007cf00 g     F __TEXT,__text _roc_panic
000000000007ce98 g     F __TEXT,__text _roc_realloc
0000000000000000         *UND* _roc__renderForHost_1_exposed

Rust FFI

For now you can generate Rust bindings for most Roc types using the roc glue platform/main.roc platform/glue.rs command.

Roc functions will be exposed in the symbol table. Take for example a function in the Effect.roc file setCwd : List U8 -> Effect (Result {} {}) function. It requires the following rust function to be defined.

#[no_mangle]
pub extern "C" fn roc_fx_setCwd(roc_path: &RocList<u8>) -> RocResult<(), ()> {
    match std::env::set_current_dir(path_from_roc_path(roc_path)) {
        Ok(()) => RocResult::ok(()),
        Err(_) => RocResult::err(()),
    }
}

Luke Boswell (Oct 28 2022 at 06:51):

Another question... should roc glue generate rust type for builtins like Str etc?

Brian Carroll (Oct 28 2022 at 08:15):

We already have Rust definitions of Str etc since long before glue existed. See roc_std crate.

Luke Boswell (Oct 28 2022 at 09:45):

Thank you, I'm importing the dependency with the below in Cargo.toml, which works well.

[dependencies]
libc = "0.2"
roc_std = { git = "https://github.com/roc-lang/roc/" }

Richard Feldman (Oct 28 2022 at 12:27):

although I'd actually like the Rust glue to generate roc_std automatically for the particular project - long-term I think the best experience will be "all I need to do is run roc glue for my target language and it generates me everything I need for my bindings to work, no further dependencies to install or keep in sync"

Brendan Hansknecht (Oct 28 2022 at 14:00):

Your description of roc linking is not surgical linking. That is legacy linking. Also, i am a bit confused by main.o. unless something changed recently, the application file we generate should be created in a temporary folder and called app.o

EDIT: maybe --debug causes main.o to be generated?

Brendan Hansknecht (Oct 28 2022 at 14:04):

What is the pattern for how Roc functions are named in the generated files?

roc + __ + name of function + specialization number + exposed + optional generic. Generic version is a work around to complex c abi that instead of returning a type directly, take a pointer to the type as an out param. Theoretically shouldn't be needed.

Brendan Hansknecht (Oct 28 2022 at 14:04):

Also, specialization number might be slightly wrong wording.

Brendan Hansknecht (Oct 28 2022 at 14:06):

Also, each of these is paired with a size function because the size of the return type may not be know until the roc app is compiled, so we have to expose it to the platform

Brendan Hansknecht (Oct 28 2022 at 14:08):

Also, the main thing--debugdoes is run our llvm ir though a tool called debug ir that gives us debug info based on the llvm ir.

Luke Boswell (Oct 31 2022 at 07:06):

I've had a go at trying to visualise the flow of the files through to application binary. I'm pretty sure this isn't quite right, but would appreciate any corrections. I thought @Anton had filed an issue to describe common Roc terms, and thought something like this might be helpful.
Building-a-Roc-Application.png

Anton (Oct 31 2022 at 08:28):

Awesome initiative @Luke Boswell :heart_eyes: !
I'd love to generate a view like this from the code so it stays up-to-date and can show specific files.
I'm going to make an issue for that, we can probably log things to a file in debug builds and perhaps write a roc cli program to convert it to a graphviz image

btw Roc terms issue was #913.

Luke Boswell (Oct 31 2022 at 09:06):

What do you think of the content? Is it reasonably close?

Anton (Oct 31 2022 at 09:40):

Yeah, I'm not very familiar with the details of the compiler pipeline so I'm not sure there but the rest looks good.
We do call cargo in the code which does call rustc but perhaps best to rename that one to cargo.

Anton (Oct 31 2022 at 09:59):

New graph logging issue

Brian Carroll (Oct 31 2022 at 15:22):

The compiler pipeline is a good high level picture.
The app and host are joined together by a linker. You could draw a box for that if you like. We can use our own --linker=surgical or an external one --linker=legacy. I think it's fine without it though. Whatever you think.

Anton (Oct 31 2022 at 15:26):

I just happened to come across the fact that we use both cargo and rustc.

Luke Boswell (Nov 01 2022 at 19:51):

Is it worth adding this somewhere more permanent? maybe the rust docs README?

Folkert de Vries (Nov 01 2022 at 20:17):

I think in practice we only use cargo

Luke Boswell (Nov 01 2022 at 21:01):

So that block should say cargo then? I wasnt sure what the name os the rust compiler was. Thanks

Brian Carroll (Nov 02 2022 at 00:42):

That's right, the block should say cargo.

Context: rustc is the Rust compiler and cargo is a build tool that sits on top of it. Most Rust users only ever use cargo. rustc is considered very low level. Roc calls cargo practically all the time too. I never knew it called rustc directly before the comment above. For this high level diagram we can ignore that, and only mention cargo.

Last updated: Jul 06 2025 at 12:14 UTC