Scott Campbell (Oct 01 2026 at 08:20): Scott Campbell (Oct 01 2026 at 08:20):Was intended to be a replacement for build.zig build script.
When I ported build.zig to basic-cli for godot-roc, the main thing that was missing was concurrency / parallel tasks & caching for some parts of my CI pipeline.
This is an attempt at parallel tasks.
https://github.com/scottc/roc-build
"0.0.1-pre-alpha-test1"
app [main!] {
pf: platform "https://github.com/scottc/roc-build/releases/download/0.0.1-pre-alpha-test1/8jZuyEFpCc7ep6yu2iXBT4cAYoxZdjTk5kxShUCMXqgx.tar.zst",
roc: "nightly-2026-09-27-a3ce7f1",
}
A trivial example:
app [main!] {
pf: platform "https://github.com/scottc/roc-build/releases/download/0.0.1-pre-alpha-test1/8jZuyEFpCc7ep6yu2iXBT4cAYoxZdjTk5kxShUCMXqgx.tar.zst",
roc: "nightly-2026-09-27-a3ce7f1",
}
import pf.Build
import pf.Log
main! : List(Str) => Try({}, [Exit(I32)])
main! = |_args| {
Log.info!("Hello World.")
pwd_id = 0
pwd = Build.cmd({
id: pwd_id,
depends_on: [],
inputs: [],
outputs: [],
program: "pwd",
args: [],
description: "Print working directory",
cwd: "",
env: [],
})
graph = Build.graph([
pwd,
])
match Build.run!(graph) {
Ok({}) => {
Log.info!("all tasks finished")
Ok({})
}
Err(BuildFailed(msg)) => {
Log.error!(msg)
Err(Exit(1))
}
}
}
roc run examples/pwd.roc
info: Hello World.
info: wave: 1 tasks in parallel
info: start Print working directory (id=0)
info: $ /nix/store/3qgy8q2j64v2m9jy3a5jmssacbblhd4r-coreutils-9.11/bin/pwd
/home/anon/Projects/roc-build
info: done Print working directory
info: all tasks finished
More complex examples in the examples directory.
https://github.com/scottc/roc-build/tree/master/examples
And yes, it builds itself!
roc run examples/bundle_self.roc
info: Build host, build & bundle platform.
info: wave: 2 tasks in parallel
info: start Install platform dependencies (id=1)
info: start Build Zig Host (id=2)
info: $ /nix/store/3qgy8q2j64v2m9jy3a5jmssacbblhd4r-coreutils-9.11/bin/cp
info: -a
info: vendor/targets/x64musl/.
info: platform/targets/x64musl/
info: $ /nix/store/83qs3ksyjclr149vpxbaz1z18wvi9rms-zig-0.16.0/bin/zig
info: build-lib
info: -fPIC
info: -OReleaseSafe
info: -target
info: x86_64-linux-musl
info: -lc
info: host/host.zig
info: -femit-bin=platform/targets/x64musl/libhost.a
info: done Install platform dependencies
info: done Build Zig Host
info: wave: 1 tasks in parallel
info: start Roc Bundle (id=3)
info: $ /nix/store/krc90wzmv2mkax5hp8zrpi2y5phnqsvx-roc-nightly-2026-09-27-a3ce7f1/bin/roc
info: bundle
info: platform/main.roc
info: platform/Build.roc
info: platform/Host.roc
info: platform/Log.roc
info: platform/targets/x64musl/crt1.o
info: platform/targets/x64musl/libc.a
info: platform/targets/x64musl/libcompiler_rt.a
info: platform/targets/x64musl/libzigc.a
info: platform/targets/x64musl/libhost.a
Created: cxyAQTomLcUFHczRwdmGxKT12EB2tpdgownuhxhJfGu.tar.zst
Compressed size: 1708119 bytes
Uncompressed size: 7427024 bytes
Compression ratio: 4.35:1
Time: 55 ms
info: done Roc Bundle
info: all tasks finished
It currently has a wave scheduler... I'll want to replace it with eager scheduler.
And add optional hashing & caching support, per task for tasks that don't implement their caching mechanisms etc.
Github actions intergration.
More targets etc.
And the api isn't finalised, I can't imagine too many breaking changes that'll cause too many headaches. It should be flexible enough for many/most configurations.
Anton (Oct 01 2026 at 10:38):Very cool @Scott Campbell!
Scott Campbell (Oct 12 2026 at 07:41):Ok, so I've been dog-fooding the platform for awhile now, and I've barely made any changes to it so far.
An experience report / TODO list:
Scott Campbell (Oct 12 2026 at 07:41):Pain point:
Wave scheduler has been fast enough, where it's not a DX pain point, for my project. The CPU utilisation for zig linker can be annoying on my local machine, makes laptop thermals & fan go vvvVVRRRRRRRR. And the flake dependencies are the biggest offenders for github actions CI build times.
Suggested solution:
Optimise & trim the dev shell CI dependencies, cache the entire flake dependencies as one big tarball of binaries? And only update the flake.lock once per week. Aggressively cache and avoid building zig hosts at all costs, but it's kind of needed to implement & test the project, so just test against 1-single-cached x64musl target for development, maybe we can make the generated glue more cache friendly? smaller, more granular files? And avoid nuking the glue-out dir, and add glue-out cache.
Scott Campbell (Oct 12 2026 at 07:42):Pain point:
Just executing utility commands can get you very far, but sometimes a bit ugly and not fun to maintain or use.
Suggested solution:
So the worst offenders, I'll probably add some helper functions to the platform host.
Scott Campbell (Oct 12 2026 at 07:42):Pain point:
Manually threading task ids as task dependencies, is awful DX.
Suggested solution:
I'll make the dependency field, just take a List(Task) instead of List(U32)... and let the host sort out auto allocated task ids if needed, and any manual ids or descriptions, will be purely optional & cosmetic.
Scott Campbell (Oct 12 2026 at 07:42):Pain point:
The manual input / output file lists, are awful DX. This was intended to be used for hashing & caching. Which I haven't implemented yet.
Suggested solution:
The host should automatically determine the list of input & output files, and their hashes. I'll need to provide some utility functions to replace calling utility programs like cp, mv for copying & moving files... cp is an opaquebox, unless there is some verbose mode, I think just using linux syscalls under the hood is the way to go, and a nice basic-cli like API etc. We need a build graph, so basic-cli doesn't work for this.
Scott Campbell (Oct 12 2026 at 07:42):Pain point:
Sometimes the LLM would write infinite recursion / memory leak, eat all 32gb of my system memory, and then the linux kernal would start choosing victim processes to kill. Not a fun time, having browser, text editor, dev environment etc. be repetitively SIG_KILL'd.
Additionally godot-roc, generates a ~200_000 line platform/main.roc, with matching C-FFIs etc. With a build matrix of ~10 CpuArch-OS targets x ~4 engines. So that's 40 zig builds in parallel, each linking a massive list of C-FFIs. So this could quite easily consume all the resources on a machine.
Suggested solution:
The host scheduler, should have resource budgets per task, and per graph. Undecided on the API for that. Task { ram_budget: [Gb(U32), InfiniteBits] }, Graph { ram_budget: [Gb(U32), InfiniteBits] }. And then we can schedule as many tasks that we can fit in parallel, within the overall graph budget.
I'll need to figure out a way to monitor, constrain or SIG_KILL these.
Scott Campbell (Oct 12 2026 at 07:42):Pain point:
The build script is ~1700 lines long.
Solution:
Probably worth breaking types, tasks, and helper functions out into their own separate modules, and just keep the main function & declarative data, and a brief overview of how the top level graph, and which tasks are in it. A ~350 line budget.
Scott Campbell (Oct 12 2026 at 07:42):Paint point:
roc & zig constraints, limiting where I can place output files. The platform needs the target output files in the platform subdirectory; platform-out/targets-out. Same with host/glue-out. While I can place files here, it just feels not quite as clean as it could be. I just wanted all these to be some top level directories: /docs-out, /glue-out, /platform-out (I need to generate exposed API & host C-FFIs), /bundle-out, /dump-out (godot api), /smoke-out (app smoke tests), /targets-out
Solution:
Maybe symlinks could help, maybe a top level /out directory could be nice. I doubt I can convince both zig & roc to lift the directory constraints just for my use-case. Moving files works as a workaround. I can maybe kind of make some of this go away, if I just treat /host as /host-out, then I can treat the entire host as just "it's all generated host glue". (Also seeing the roc% and zig% bars change towards roc/purple on the github project makes me happy, 100% roc FTW).
Scott Campbell (Oct 12 2026 at 07:43):Pain point:
roc nightly has some missing errors, instead you'll get a mystery segfault or a crash or something sometimes instead. And it's because the code is doing something invalid & silly, hoping one would get a nice type error instead.
Solution:
Help make bug reports & PRs, update to latest nightly or be lazy and just wait & hope until someone else solves them.
Scott Campbell (Oct 12 2026 at 07:43):Pain point:
Fetching & vendoring ZigGlue.roc to match nightly version by hand is tedious.
Solution:
Add a helper script for this, can use the short SHA7 hash.
Luke Boswell (Oct 12 2026 at 07:45):I'm working on very similar problems right now
Luke Boswell (Oct 12 2026 at 07:47):I've been iterating on these -- and I'm currently testing out shared roc utilities which are released as small binaries and keep CI runners fast.
https://lukewilliamboswell.github.io/roc-automation/
https://lukewilliamboswell.github.io/roc-blueprint/
I'm also spiking out more script like capabilities for Blueprint.roc
Luke Boswell (Oct 12 2026 at 07:48):I've been trying to work on Blueprint to help @blu with ideas for Kai and also to help me maintain all the repos I've got floating around
Luke Boswell (Oct 12 2026 at 07:49):Nix is great, but the cost in CI is pretty big
Luke Boswell (Oct 12 2026 at 07:50):I welcome any ideas or experience reports :grinning_face_with_smiling_eyes:
Anton (Oct 12 2026 at 11:36):To avoid nix build times on the Roc side, make sure to use https://github.com/roc-lang/roc-overlay
Scott Campbell (Oct 12 2026 at 11:54):Yup, already using roc-overlay.
I'm pulling in roc, zig.
godot, godot4.5, redot, rex, emscripten, godot-export-templates (includes all export targets, includes various sdks etc.)... a lot of heavy project specific dependencies.
I'm trying porting my flake to @Luke Boswell's blueprint tool, it's nicer then using a flake, before I had everything as 1 single devShell with all dependencies.
Scott Campbell (Oct 12 2026 at 11:54):config = [
Name("gdextension-roc-platform-generator"),
Overlay("roc", "github:roc-lang/roc-overlay"),
Overlay("redot", "path:./flakes/redot-nix"),
Overlay("rex", "path:./flakes/rex-nix"),
Environment("dev", []), # ... todo
Environment(
"dump-bindings",
[
Overlays(["redot", "rex"]),
Tools([
# engines
"godot",
"godotPackages_4_3.godot", # TODO: gd 4.0?
"godotPackages_4_5.godot",
"redot",
"rex",
]),
]
),
Environment(
"ci",
[
Overlays(["roc"]),
Command("roc", "rocpkgs.nightly-2026-09-27-a3ce7f1"),
Tools([
# compilers
# roc, via overlay...
"zig",
"godot",
]),
],
),
Environment("nightly-release", []),
Environment("stable-release", []),
Environment(
"web-export",
[
Overlays(["roc"]),
Command("roc", "rocpkgs.nightly-2026-09-27-a3ce7f1"),
Command("roc-stable", "rocpkgs.nightly-2026-09-27-a3ce7f1"),
Tools([
"godot",
"godot-export-templates-bin",
"emscripten",
#"zig", # [Optional] we use zig's "zig wasm-ld" for testing if objects are linkable.
#"wabt", # [Optional] we use wasm-objdump to inspect.
]),
],
),
Shell("default", [Use("dev")]),
# depends on dump bindings, dev & web export...
Task("ci", [Use("ci"), Run(["scripts/ci.roc", "--", "--baseline=godot", "--engines=none", "--targets=muslx64", "--tests=all", "-Doptimize=Debug"])]),
#Task("check", [Use("dev"), Run(["scripts/check_all.roc"])]),
#Task("bundle", [Use("dev"), Run(["scripts/build_bundle.roc"])]),
#Task("test-bundle", [Use("dev"), Run(["scripts/test_bundle.roc"])]),
#Task("package-examples", [Use("dev"), Run(["scripts/package_examples.roc"])]),
#Task("package-docs", [Use("dev"), Run(["scripts/package_docs.roc"])]),
#Task("test-published", [Use("dev"), Run(["scripts/test_published.roc"])]),
]
I think i'd want a function that takes some inputs, and produces the environments and such:
--baseline=godot4.1
--engines=godot4.7,redot26,rex0
--targets=wasm32,nintendo,x64musl
Scott Campbell (Oct 12 2026 at 12:27):main = |args| {
config = args_to_config(args)
config
}
Last updated: Oct 15 2026 at 17:56 UTC