Stream: compiler development

Topic: Excessive compiler rebuilds


view this post on Zulip Trevor Settles (Mar 08 2024 at 03:08):

When working on the compiler, has anyone else experienced more rebuilds than necessary? I'm using VSCode on an Intel Mac. If I run cargo check, save a file (without changes), then rerun cargo check it recompiles/rechecks a bunch of crates. It seems like its mostly roc specific crates. This takes 15-30 seconds each time, and really slows me down. I found some cargo docs that my be related, but I'm not sure. Anyone run into this, or know how to get cargo to only rebuild crates that it needs to?

view this post on Zulip Brendan Hansknecht (Mar 08 2024 at 03:48):

That sounds quite believable. We have a big stack of crates that definitely aren't the most well separated. We also have some heavy macros and some build commands external to cargo.

view this post on Zulip Brendan Hansknecht (Mar 08 2024 at 03:49):

I can repro the same thing

view this post on Zulip Brendan Hansknecht (Mar 08 2024 at 03:51):

I guess rust does the caching based on time stamp. So saving again causes issues.

view this post on Zulip Brendan Hansknecht (Mar 08 2024 at 03:52):

You can use --timings to get a time breakdown which may be helpful. Unwinding what is going on.

view this post on Zulip Brendan Hansknecht (Mar 08 2024 at 03:53):

But it may fall into the fundamentally we have too much of a web of crates and need to reachitecture dependencies to avoid these large costs.

view this post on Zulip Brendan Hansknecht (Mar 08 2024 at 03:54):

The roc load build script seems to be specifically high cost on my machine (I think it caches the builtins). So any change to the compiler might change what the builtins compile to which leads to recompiling and that takes a while.

view this post on Zulip Brendan Hansknecht (Mar 08 2024 at 03:55):

It also looks to be a specific bottleneck where things run in parallel before and after, but it runs alone. Not sure of an actual fix, but seems like a decent part of the pain point.

view this post on Zulip Brendan Hansknecht (Mar 08 2024 at 03:56):

Can you share the timings report you specifically hit, I would be curious to see if it matches or if something else also contributes.

view this post on Zulip Trevor Settles (Mar 08 2024 at 04:13):

I guess, good to see I'm not the only crazy one, haha. I haven't looked at these timing reports before, but it strikes me as odd that there's a bunch of crates in the targets section. I'd expect that only one crate would be there. In this case I ran cargo test --lib roc_parse --timings.
cargo-timing-20240308T041112Z.html

view this post on Zulip Trevor Settles (Mar 08 2024 at 04:24):

In the chart roc_repl_expect is the crate that takes the longest, but (in a perfect world) that wouldn't even need to compile when running tests for the roc_parse crate. It seems like cargo isn't able to:

  1. Detect what crates have changed
  2. Effectively build a dependency tree (either because roc_parse does actually depend on roc_repl_expect, or something else is wonky)

view this post on Zulip Brendan Hansknecht (Mar 08 2024 at 04:29):

Will need to dig in later (away from computer), but my guess is that in the tests we use the repl for simplicity. So instead of just testing the parser, it is testing the full compiler. Thus the toms of extra dependencies pulled on

view this post on Zulip Trevor Settles (Mar 08 2024 at 05:10):

No worries. Another annoying thing is, with this issue I've got to wait for it twice. Once for the LSP which does a cargo check, then I can actually run the command I want

view this post on Zulip Folkert de Vries (Mar 08 2024 at 08:50):

I think cargo test -p roc_parse should do better?

view this post on Zulip Folkert de Vries (Mar 08 2024 at 08:51):

the repl depends on the parser, so the parser cannot depend on the repl

view this post on Zulip Brendan Hansknecht (Mar 09 2024 at 01:27):

cargo test --lib roc_parse

This says: Build all libraries in the roc repo for testing. Then filter the tests to execute down to tests that contain the phrase roc_parse.

So yeah, should be fixed with folkert's command cargo test -p roc_parse. Which says only test the roc_parse package.

view this post on Zulip Trevor Settles (Mar 10 2024 at 01:04):

Ah, the --lib vs -p change is a nice improvement on that front. I might just turn off the LSP while working so I'm not slowed down there...

view this post on Zulip Brendan Hansknecht (Mar 10 2024 at 02:22):

I know that Ayaz's sets the language server to its own build target. That way it can build while you also build for tests or whatever else. Means more building overall but no pauses


Last updated: Jul 06 2025 at 12:14 UTC