Brendan Hansknecht (Feb 08 2025 at 05:21): Brendan Hansknecht (Feb 08 2025 at 05:21):For the new zig compiler it sounds like we are hoping to:
From what I can tell, if we use the C++ api, we are able to fully interact with llvm and lld. Here is how zig calls lld for example: https://github.com/ziglang/zig/blob/a8af36ab10562e58ba237fa3dcc582228197402b/src/zig_llvm.cpp#L528-L541
This is where zig does actually llvm codegen: https://github.com/ziglang/zig/blob/4de2b1ea65e6b54cedfe56268a8bf8e9446addb0/src/codegen/llvm.zig
If I understand correctly, they end up directly generating a bc file to not have to deal with the fact the llvm abi constantly changes and it is really annoying (at my work we update llvm weekly to avoid the pain of large breaking changes). This is technically slower, but probably worth it. inkwell updates have always been a painpoint of the rust compiler. Directly generating bc files probably will alleveate that pain (it also makes it easier to distribute a version of roc that is only the interpreter, like for wasm)
From a quick scan, their llvm bitcode writer looks to be mostly a standalone library: https://github.com/ziglang/zig/tree/4de2b1ea65e6b54cedfe56268a8bf8e9446addb0/src/codegen/llvm
As such, I would assume there is probably a way we could fully share that code with zig instead of duplicating it.
One think I am not sure of is how we want to actually deal with building/packaging llvm. If we want to enable cross compilation of the entire compiler, we probably could statically build llvm with cmake (still might not be fun to cross compile, but should be possible). On top of that, this would mean that roc could be nearly fully standalone just needing to run zig build. The downside of this is that compiling roc requires compiling llvm (luckily that only has to be done once every six months to a year when we update).
If I understand how the actual zig compiler builds (which I might not, just skimmed source some), they support both finding the library locally and compiling from source using cmake. So we also technically have that option. I think that either way we would be compiling llvm in as a static library (that is the standard for llvm).
This likely will take a bit to get right (especially in terms of build scripts and the right factoring to share with the actual zig compiler). This is just my preliminary analysis to get the ball rolling
@Andrew Kelley any tips, expected pitfalls, or general things we should be aware of around wrangling llvm and potentially sharing code with the actual zig compiler?
Andrew Kelley (Feb 08 2025 at 06:41):Brendan Hansknecht said:
If I understand correctly, they end up directly generating a bc file to not have to deal with the fact the llvm abi constantly changes and it is really annoying (at my work we update llvm weekly to avoid the pain of large breaking changes).
to clarify, my primary motivation is https://github.com/ziglang/zig/issues/16270 - in other words it is a step towards complete elimination of all llvm libraries from the zig executable. Second reason is that it's possible to do incremental compilation with .bc files while it's not with the IRBuilder API. Your quoted point also stands, however.
Andrew Kelley (Feb 08 2025 at 06:44):Brendan Hansknecht said:
As such, I would assume there is probably a way we could fully share that code with zig instead of duplicating it.
happy to extract this into a proper package that exposes a module so you can depend on it via the build system if you wish, and we can independently maintain it together in a separate repo from ziglang/zig
Andrew Kelley (Feb 08 2025 at 06:47):Brendan Hansknecht said:
One think I am not sure of is how we want to actually deal with building/packaging llvm.
I believe you are conceptualizing the use case that we solve with https://github.com/ziglang/zig-bootstrap
This is how we produce upstream builds that are hosted on ziglang.org/download
However you'll still want to support a traditional way of building against host libraries, with host C toolchain, for system package maintainers to use (i.e. debian, homebrew, etc)
Andrew Kelley (Feb 08 2025 at 06:55):for creating cross-compiled upstream-provided builds of roc, including linking against llvm, you might consider maintaining a fork of zig-bootstrap.
it's really conceptually simple - literally the only thing that isn't a copy paste of zig, llvm, lld, or clang, is a 200 line shell script that has no branching logic. I recommend to give it a read, it's quite straightforward.
you should be able to get that going today with even just the tokenizer that you have. I don't think you need to solve any problems to do that
Andrew Kelley (Feb 08 2025 at 06:57):literally just
-cd "$ROOTDIR/zig"
+cd "$ROOTDIR/roc"
Man those cmake commands have too many config options, but yes, wow, that is super simple.
Brendan Hansknecht (Feb 08 2025 at 07:07):The more I learn about zig the more I realize that it (or related projects) really do solve all tooling problems.
Brendan Hansknecht (Feb 08 2025 at 07:09):Andrew Kelley said:
happy to extract this into a proper package that exposes a module so you can depend on it via the build system if you wish, and we can independently maintain it together in a separate repo from ziglang/zig
That would be awesome!
Brendan Hansknecht (Feb 08 2025 at 21:59):Not that this has actual llvm in it (cause roc doesn't import llvm yet), but just testing bootstrapping and cross compiling.
For anyone on aarch64 mac or x86_64 linux, please run these. If they work, just give a :check: response, if not, share the issue.
roc-aarch64-macos-none-baseline
roc-x86_64-linux-musl-baseline
Brendan Hansknecht (Feb 08 2025 at 22:07):Next will probably embedded lld into roc and test things.
Last updated: Jul 06 2025 at 12:14 UTC