Worklog (Draft PRs and coordination) · contributing

Just a note on the above, I'll be touching huge swaths of check/parse/AST.zig, a good bit of check/parse/NodeStore.zig, and a bit of check/parse/Parser.zig. If you could check in with me about changes there that will be coming in within the next week, it would be helpful to avoid a very painful merge conflict.

I do aim to merge the most impactful stuff to AST and NodeStore as soon as I have tests passing

Richard Feldman (Jun 29 2025 at 21:50):

Luke Boswell (Jun 29 2025 at 22:10):

Luke Boswell (Jun 29 2025 at 22:17):

I'm wanting to finish my crusade to add a single unit test per Can NodeStore add/get variant to verify they roundtrip correctly.

Kiryl Dziamura (Jun 30 2025 at 09:58):

I'm working on SingleQuote tokenization and possibly parsing @Anthony Bullard it will likely interfere with you changes. although the conflicts shouldn't be very big

Kiryl Dziamura (Jun 30 2025 at 09:59):

Also, I plan to make tokenizer.zig use Regions in relevant places in a separate pr

Anthony Bullard (Jun 30 2025 at 11:43):

@Kiryl Dziamura I don't think there is work to do here...Maybe I'm wrong. But the main thing that needs to be done is adding a node for it in the Parser and actually parsing it.

Anthony Bullard (Jun 30 2025 at 11:44):

Kiryl Dziamura (Jun 30 2025 at 11:44):

Anthony Bullard (Jun 30 2025 at 11:46):

Jared Ramirez (Jun 30 2025 at 22:29):

Planning on starting type checking on match statements once Luke's draft PR lands

Luke Boswell (Jun 30 2025 at 22:38):

I may get the Can for match (at least with current supported syntax) done by your time tomorrow. Somewhat tempted to take a detour and cleanup the snapshot mess I'm feeling like I've contributed to

Kiryl Dziamura (Jul 01 2025 at 10:20):

started working on tokenize.zig cleanup related to offsets (the goal is to use regions if they don't introduce any penalty):
https://github.com/roc-lang/roc/pull/7917

Brendan Hansknecht (Jul 01 2025 at 15:17):

Generally question about the above PR. Why are we starting lengths in the tokenizer at all?

Brendan Hansknecht (Jul 01 2025 at 15:18):

Brendan Hansknecht (Jul 01 2025 at 15:19):

So why not only store the offset and leave regions to later parts of the stack. Even later parts of the stack likely could just reference two token indices and then just request the start and end from the two tokens.

Brendan Hansknecht (Jul 01 2025 at 15:20):

Kiryl Dziamura (Jul 01 2025 at 15:30):

Kiryl Dziamura (Jul 01 2025 at 16:27):

Also, whitespaces are skipped, I think without end offset we'd have to collect all gaps as tokens

Jared Ramirez (Jul 01 2025 at 20:28):

Luke Boswell (Jul 01 2025 at 21:09):

@Jared Ramirez are you building on top of my branch? Does it look ok to merge as is?

Jared Ramirez (Jul 01 2025 at 21:15):

Jared Ramirez (Jul 01 2025 at 21:16):

I've been mostly reading about exhaustiveness checking so far, haven't written any code yet. But was planning on starting with error messages until your branch is merged!

Luke Boswell (Jul 01 2025 at 22:33):

#7919 -- I've started adding a roc package. I'm thinking about and exploring how we will do multi file snapshots. Also wanting more realistic examples to test our parser/can implementation against to understand what areas need work still.

My current line of thinking is we make a directory that represents the root, with the .roc files in it exactly as they would be.

Until we get multiple module things up and running, I might have the snapshot tool just pickup these .roc files and treat them as independent file type snapshots and generate a .md for each...

Luke Boswell (Jul 02 2025 at 00:21):

Richard Feldman (Jul 02 2025 at 02:29):

#7923 - draft of making effectful functions work (and not use a type variable anymore!) along with some type inference fixes and other miscellaneous improvements

Luke Boswell (Jul 02 2025 at 12:15):

I've started refactoring CIR a little... https://github.com/roc-lang/roc/pull/7925. Basically pulling the obvious parts into separate files (Expressions, Statements, Patterns, TypeAnnotations etc), adding doc comments and examples. etc.

I've been researching how to implement new features like nominal types, and figure I may as well clean up and document everything as I learn more. It's easier for me to understand how things are wired together when it's organised, and hopefully helps the next guy that comes along too.

I can keep this in Draft and rebase it until @Richard Feldman and @Jared Ramirez land those two PRs sometime tomorrow (my time) I assume, I'm not tracking anyone else working in CIR right now.

Luke Boswell (Jul 02 2025 at 23:30):

I've rebased this CIR refactor on main. I'll continue poking at it for a few hours. Please avoid making PR's that touch CIR unless your basing off this branch.

Jared Ramirez (Jul 03 2025 at 15:52):

I'm taking a look at type checking binops. Are all of the following, minus and, or, pipe_forward and null_coalesce going use static dispatch?

/// Binary operators available in Roc.
pub const Op = enum {
    add, // +
    sub, // -
    mul, // *
    div, // /
    rem, // %
    lt, // <
    gt, // >
    le, // <=
    ge, // >=
    eq, // ==
    ne, // !=
    pow, // ^
    div_trunc, // //
    @"and", // and
    @"or", // or
    pipe_forward, // |>
    null_coalesce, // ?
};

Richard Feldman (Jul 03 2025 at 16:07):

pipe_forward should be -> instead of |> (and prob could use a different name)

Richard Feldman (Jul 03 2025 at 16:07):

Richard Feldman (Jul 03 2025 at 16:08):

null_coalesce also doesn't use static dispatch, and should be renamed to something like return_err

Richard Feldman (Jul 03 2025 at 16:09):

answer = my_result?

answer = match my_result {
    Ok(val) => val
    Err(err) => return Err(err)
}

Richard Feldman (Jul 03 2025 at 16:10):

or maybe postfix_question_mark if we want to name it based on how it looks :smile:

Richard Feldman (Jul 03 2025 at 16:11):

btw I definitely want to make it so that if we get type mismatches with these, we report them using the binop names (since that's what was in the source code) rather than the functions they effectively desugar to!

Richard Feldman (Jul 03 2025 at 16:11):

like if I write + or ? in my code, I should see + or ? in the error message (and not just in the source code snippet, but in the words too!)

Jared Ramirez (Jul 03 2025 at 16:26):

okay cool, since no desugaring has happened by type checking, nice error messages should be straightforward here

Jared Ramirez (Jul 03 2025 at 16:28):

~~then ? shouldn't actually be a binop, since it's really just a suffix? Maybe I should remove it?~~

Richard Feldman (Jul 03 2025 at 16:28):

Jared Ramirez (Jul 03 2025 at 16:28):

Maybe this null_coalesce was ported from the older ?? or optional record fields?

Richard Feldman (Jul 03 2025 at 16:28):

Jared Ramirez (Jul 03 2025 at 16:29):

Richard Feldman (Jul 03 2025 at 16:29):

answer = my_result ?? 5

answer = match my_result {
    Ok(val) => val
    Err(_) => 5
}

Richard Feldman (Jul 03 2025 at 16:30):

the distinction matters if you want to use it with things that affect control flow, like ?? return 5 or ?? crash "blah"

Richard Feldman (Jul 03 2025 at 16:30):

whereas if it just desugared to actual .with_default, those use cases wouldn't work

Luke Boswell (Jul 04 2025 at 00:57):

Before I can do that we need to support parsing multiple UpperIdent separated by Comma tokens, and also a package prefix, e.g. json.Core.Utf8.Encoder might be an example of the Encoder type declaration (presumably nominal but could be an alias) in the Core/Utf8.roc module inside the json package.

Luke Boswell (Jul 04 2025 at 07:12):

I think it's looking ok, but I'm still working through the diffs in the snapshots and picking up minor bugs that I'm fixing. I've ran out of time today to finish this... I should be able to pick it up again Sat evening.

Luke Boswell (Jul 04 2025 at 07:14):

@Anthony Bullard there are changes in here in the Parser and AST/NodeStore to support parsing of qualified types and values in packages/module chains.

Kiryl Dziamura (Jul 04 2025 at 09:25):

Brendan Hansknecht (Jul 04 2025 at 22:52):

I was looking at switching our build script back to a proper check step. On top of that, making check cover all executables and tests, but then having smaller steps for faster checking. The most relevant probably being check-test which would likely be a solid default for most work. One thing that still annoys me with how zig/zls currently do checking is that if you have multiple check targets that include the same source, you get a ton of repeated error mesages. So for every executable, we get another copy of each error message. Theoretically the fix for this is to use more modules. That way the executables just import our "main" module and the "main" module runs check once on all the files it contains. I haven't tried factoring this way yet, but I want to make some of this nicer overall (though it would require a refactoring of imports).

Brendan Hansknecht (Jul 04 2025 at 22:59):

Not actually sure this is the right way to go, just tinkering and trying to match what zls suggests.

Luke Boswell (Jul 05 2025 at 08:30):

I feel like adding support for qualified Idents, and Nominal Types (including recursive) has been a mighty big yak.

Luke Boswell (Jul 05 2025 at 08:31):

I've gotten to the home stretch a few times now, and then noticed a bug which has taken me down another rabbit hole.

Richard Feldman (Jul 05 2025 at 12:53):

Jared Ramirez (Jul 05 2025 at 16:12):

Thinking a next mini-milestone for me might be: after nominal type checking work, I might create a pared-down version of the bool.roc built-in file. Then canonicalize and type check that, and pass those types into the actual user roc file to to use the built-in Bool nominal type in type checking in things like if conditions.

Richard Feldman (Jul 05 2025 at 18:06):

Richard Feldman (Jul 05 2025 at 18:07):

if you wanted to get advanced, you could do the same with List.roc and implement it as a Cons list (just for now, of course)

Richard Feldman (Jul 05 2025 at 18:10):

:alert: PSA! :alert: as of https://github.com/roc-lang/roc/pull/7949 we now have a new EXPECTED section of snapshots, which lists the PROBLEMS we expect to see in the rest of the snapshot (just the ALL CAPS name of the problem and the source region, not the entire error message)

the goal here is just that if we accidentally cause regressions in our snapshots, this will let us know! (If we actually make fixes and reduce the reported PROBLEMS, then we can update the snapshot with a revised EXPECTED section.) And since it doesn't verify the entire contents of the snapshot, just the errors reported, it shouldn't give us false positives for things like refactors that change ident numbers etc.

Brendan Hansknecht (Jul 05 2025 at 18:12):

Wow, that PR hits more files than I expected...we have way more snapshots than I realized.

Richard Feldman (Jul 05 2025 at 18:12):

in the future we could maybe also do something with like verifying expected types of things

Richard Feldman (Jul 05 2025 at 18:12):

Richard Feldman (Jul 05 2025 at 18:13):

(as in, unintentional ones, e.g. because we haven't implemented things yet, or in some cases because they're using old syntax)

Luke Boswell (Jul 06 2025 at 07:52):

I've been talking with @Joshua Warner about the snapshots. I've started reviewing all of the old-syntax ones and I'm going through and deleting any that I think are not helpful, and updating the ones I think are useful.

I still plan on implementing Can for where, but just taking a quick detour to clean up that snapshots a little. I got a bit carried away just converting them all using a script, and didn't take the time to properly review them.

Richard Feldman (Jul 06 2025 at 13:35):

I appreciate that! now that we have EXPECTED in there, would also be great to review those

Richard Feldman (Jul 06 2025 at 13:35):

right now I set it so that all the EXPECTED fields just line up with the current PROBLEMS

Richard Feldman (Jul 06 2025 at 13:35):

Richard Feldman (Jul 06 2025 at 13:38):

e.g. I've already seen quite a few where the snapshot seems to be trying to verify one thing, but actually it gets derailed by a syntax error (which is reported in PROBLEMS, at least!) which causes it to not actually be type-checked (or whatever thing it's actually trying to test)

Richard Feldman (Jul 07 2025 at 01:54):

I'm gonna start working on canonicalizing and type-checking other modules (e.g. imports, exposing, etc.)

Richard Feldman (Jul 07 2025 at 01:54):

cc @Jared Ramirez since I saw you had a commit involving canonicalizeHeaderExposes yesterday! :smile:

Jared Ramirez (Jul 07 2025 at 05:37):

Luke Boswell (Jul 07 2025 at 22:24):

If we're happy with where clause for now. I'm thinking of digging into the issues from our realistic example src/snapshots/plume_package/Color.md. It looks like we need to investigate;

Jared Ramirez (Jul 07 2025 at 22:37):

Luke Boswell (Jul 08 2025 at 00:45):

// FROM
pub fn addExpr(store: *NodeStore, expr: CIR.Expr) CIR.Expr.Idx
// TO
pub fn addExpr(store: *NodeStore, expr: CIR.Expr) std.mem.Allocator.Error!CIR.Expr.Idx

Brendan Hansknecht (Jul 08 2025 at 02:38):

Why? I thought OOMs were deemed unrecoverable errors. Mostly curious cause if we want this, we probably should be consistent across the compiler.

Luke Boswell (Jul 08 2025 at 02:39):

Luke Boswell (Jul 08 2025 at 02:40):

I'm not sure where. But @Jared Ramirez and I have been slowly converting to use the zig errors like this.

Luke Boswell (Jul 08 2025 at 02:41):

Richard Feldman (Jul 08 2025 at 02:44):

yeah, that plus also I think there may be an increasing number of cases where we're only using allocation for Problems

Richard Feldman (Jul 08 2025 at 02:45):

and in those cases I think it would be interesting to have Problems record an "OOM Problem" and stop recording further problems

Richard Feldman (Jul 08 2025 at 02:46):

which would mean that pushing to Problems could stop having allocation errors, and then it would be more useful to see which things are allocating and which ones aren't

Richard Feldman (Jul 08 2025 at 02:48):

separately, I also like the idea of actually knowing what will happen in resource-constrained environments, e.g. we could offer a CLI flag for constraining how much memory the interpreter can use when evaluating compile-time constants, we could have a better UX in wasm (e.g. in the playground) if we want to limit total memory usage in the browser and be able to more gracefully handle OOMs etc.

Luke Boswell (Jul 08 2025 at 05:32):

I thought to myself ... it should be nice and easy to implement Can for these guys;

dbg x
crash "msg"
expect 1 == 1
return x

Luke Boswell (Jul 08 2025 at 10:39):

Starting looking at de-structuring record sub-patterns, going on another tangent with record update syntax, as patterns, and pattern alternatives. :sweat_smile:

Luke Boswell (Jul 09 2025 at 03:01):

Currently we have a unit test that round-trips a whole module, and we added a verification test for every snapshot that serialises and deserialises from memory and compares the SExpr matches.

Next steps are to cleanup some of the serialisation logic (type.Store looks particularly hairy), and wire up hashes and filesystem parts into roc check.

Jared Ramirez (Jul 09 2025 at 04:00):

Lmk how I can help with types store! I'm also working on some changes in there related to aliases and nominal types, should hopefully MR tomorrow or thursday

Jared Ramirez (Jul 09 2025 at 04:01):

I also would like to reduce the number of backing arrays in types.Store too, which may be relevant

Luke Boswell (Jul 09 2025 at 04:07):

I'll see if I can fix the valgrind issue and maybe we merge as is, and fixup the types.Store backing/serialization in a follow up

Luke Boswell (Jul 09 2025 at 04:49):

It's taking a little while, I can't run valgrind on my mac so doing what I can and then throwing at CI to test.

Luke Boswell (Jul 09 2025 at 04:53):

Turns out the cleanup I had in mind is just the remedy I think for our valgrind issues :smiley:

Luke Boswell (Jul 09 2025 at 06:53):

Today I have been down the alignment rabbit hole -- I do not yet know if I will emerge victorious

Luke Boswell (Jul 10 2025 at 01:23):

I'm poking at the module caching again, there are some obvious improvements we should make. I just hacked something together to get it working yesterday, but would like to clean it up and simplify it a little today.

Luke Boswell (Jul 11 2025 at 05:48):

I've experimented with using RNG to provide integers (deterministically) which I think significantly reduces boilerplate.

// example using random helpers to reduce visual noise in the tests
try headers.append(AST.Header{
    .app = .{
        .packages = rand_idx(AST.Collection.Idx),
        .platform_idx = rand_idx(AST.RecordField.Idx),
        .provides = rand_idx(AST.Collection.Idx),
        .region = rand_region(),
    },
});

I'm reasonably confident this will help with the Parser NodeStore refactor, as we're not touching the internal Node representation, just using the AST representation.

Jared Ramirez (Jul 11 2025 at 15:11):

Jared Ramirez (Jul 11 2025 at 23:37):

^ FYI taking a detour to refactor Can to ensure that CIR nodes<-> type vars always match 1 to 1

Luke Boswell (Jul 13 2025 at 10:59):

I'm probably going on a crusade tomorrow to eradicate exitOnOom, and anything else that is OS specific deeper in our compiler than the cli.

This is in preparation for WASM -- so we can avoid a more painful extraction later.

JRI98 (Jul 13 2025 at 11:06):

Hey everyone. From my side, I started chipping away at the parse fuzz crashes that show up in https://roc-lang.github.io/roc-compiler-fuzz. The codebase is a treat to work with :100:
I am also planning on configuring a machine to run the fuzzing myself.

Brendan Hansknecht (Jul 13 2025 at 12:49):

Brendan Hansknecht (Jul 13 2025 at 12:51):

Awesome. Just make sure you are actually getting fuzzer coverage info when you do so. I know that afl++ and zig is a bit finicky.

Luke Boswell (Jul 13 2025 at 22:45):

std.io.getStdErr().writer().print(format, args) catch unreachable;
if (tracy.enable) {
    tracy.waitForShutdown() catch unreachable;
}
std.process.exit(1);

Brendan Hansknecht (Jul 13 2025 at 23:48):

Luke Boswell (Jul 14 2025 at 01:46):

exitOnOom is invasive... this is going to be a mammoth PR that basically touches everything.

Luke Boswell (Jul 14 2025 at 01:48):

I tried to do a little bit or think about gradual migration... I'm currently going with the rip-the-bandaid off approach.

Luke Boswell (Jul 14 2025 at 01:49):

In a way it's easier this way, because I can quickly scan through code and see what needs to be updated. Anything that allocates should return std.mem.Allocator.Error!... etc

Luke Boswell (Jul 14 2025 at 01:59):

Thanks, I need morale support... this is a very mechanical change. Thank the Zig gods for making a fast compiler :pray:

Brendan Hansknecht (Jul 14 2025 at 02:44):

Luke Boswell (Jul 14 2025 at 02:47):

Luke Boswell (Jul 14 2025 at 02:54):

Also -- I wouldn't trust an LLM with this kind of refactor, how would you ever review the changes with any confidence... there is just waaay too many.

Brendan Hansknecht (Jul 14 2025 at 03:12):

hmm. I haven't tested claude on this type of refactor, but it tends to do a great job with mechanical simple error driven things. Especially once you set up a base example.

Kiryl Dziamura (Jul 14 2025 at 09:20):

going to work on parsing utf8 escape seq in strings and single quote (\u(...))

Richard Feldman (Jul 14 2025 at 11:01):

nice! I think we ended up using curly braces instead of parens for those in the old compiler (we should keep whatever syntax it uses)

Luke Boswell (Jul 15 2025 at 13:03):

Luke Boswell (Jul 15 2025 at 13:04):

This is why people shouldn't let me touch any CSS -- I have the same problems with PPT slides in case anyone was wondering.

Kiryl Dziamura (Jul 15 2025 at 13:18):

I used to love hacking css so much that my over-engineered nonsense would end up on codepen's front page. but then I looked over the fence and found a healthy lifestyle without alcohol. now I can’t look at any css snippet without feeling pain. I even started thinking html tables were enough. just give me a11y. I don’t care about all your fancy dynamic layouts, animations, or endlessly improvable design systems. argh.

Jared Ramirez (Jul 16 2025 at 03:18):

Working on refactoring types store and type snapshots store to do less allocations

Jared Ramirez (Jul 16 2025 at 17:26):

Working on consolidating canonicalizeTypeAnno and canonicalizeTypeAnnoToTypeVar in Can

Kiryl Dziamura (Jul 17 2025 at 12:05):

Moving single quote contents validation from can to tokenize: https://github.com/roc-lang/roc/pull/8064/files
@Richard Feldman you mentioned you wanted to parse actual values values during tokenization, could you please clarify how you thought it to be implemented? create a new collection in module and put parsed numbers there in a similar to interned logic way?

Richard Feldman (Jul 17 2025 at 13:39):

Richard Feldman (Jul 17 2025 at 13:40):

(as in, add new entries to the extra union for in-memory numbers like u8, i8, etc.)

Kiryl Dziamura (Jul 17 2025 at 13:41):

Richard Feldman (Jul 17 2025 at 13:42):

Kiryl Dziamura (Jul 17 2025 at 13:42):

Richard Feldman (Jul 17 2025 at 13:42):

Richard Feldman (Jul 17 2025 at 13:43):

yeah I could actually see an argument for just storing 64-bit and smaller inline, and then if it's bigger than 64 bits, just store the full string

Richard Feldman (Jul 17 2025 at 13:43):

Kiryl Dziamura (Jul 17 2025 at 13:45):

actually, do we really need to parse them straight away exactly for that reason? we don't know if the number is custom ahead of time. when we got there, we'd need to have normalized num literals rather than parsed numbers

Kiryl Dziamura (Jul 17 2025 at 13:50):

I mean, we likely need to parse them during tokenization as a normalized num literal (NaN | +-Infinity | { value: +-digits, exponent: +-digits })

Kiryl Dziamura (Jul 17 2025 at 13:52):

we don't have it right now, but I think it means we need to convert the digits to actual number at canonicalization anyway, right?

Richard Feldman (Jul 17 2025 at 15:42):

the plan is to never support infinity/-infinity/Nan literals; they are special constants exposed by the relevant modules

Richard Feldman (Jul 17 2025 at 15:44):

fair point about not knowing if the number is custom ahead of time, but 99.99% of the time they will be not custom and will fit in 64 bits, so I think we should optimize for that case and convert back if it turns out we were wrong :smile:

Kiryl Dziamura (Jul 17 2025 at 15:52):

ok, so we want to get as much as we can from module file contents while it's in cache because later we want to touch heap anyway and it's better if this heap is optimized for the future cache rather than having random access to the module file contents via regions?

I'm asking because I'm learning along the way. I'm not experienced in system level development so my questions may be... simple :smile:

Luke Boswell (Jul 17 2025 at 23:54):

I'm looking at the build system with the goal to re-use modules for our different targets and also to help our generated docs be useful.

Luke Boswell (Jul 18 2025 at 03:48):

Refactoring to use modules and then setting up the tests to run on those modules has uncovered a heap of lurking bugs :bug:

Luke Boswell (Jul 18 2025 at 18:52):

Luke Boswell (Jul 20 2025 at 08:03):

Luke Boswell (Jul 20 2025 at 08:04):

I have the simplest of expressions working so far (|x| x + 1)(5) :smiley: -- but working on padding that out into something more complete

Luke Boswell (Jul 20 2025 at 10:40):

Taking a slight detour implementing unary_minus -x ... this Yak was just asking for a shave

Luke Boswell (Jul 20 2025 at 11:07):

Luke Boswell (Jul 22 2025 at 22:06):

I'm still working on interpreter Eval... I'm learning as I go and also getting distracted building debug tooling. :smiley:

Anton (Jul 23 2025 at 08:10):

Brendan Hansknecht (Jul 23 2025 at 13:45):

Brendan Hansknecht (Jul 23 2025 at 13:46):

That honestly would probably actually be a good thing if the debug tooling was good

Luke Boswell (Jul 24 2025 at 13:08):

I'll definitely go through it myself again a few times. But if anyone else has time I would appreciate any thoughts.

Luke Boswell (Jul 25 2025 at 07:40):

I've made some progress on the interpreter (I think) ... haven't got all my tests passing yet.

I also need to do a rebase on main now Richard's mammoth PR removing CIR landed.

Luke Boswell (Jul 25 2025 at 20:13):

Luke Boswell (Jul 28 2025 at 00:21):

I'm planning on continuing with my refactors to use proper Zig modules. Next I'm looking at Can and CheckTypes I think. It's not necessarily untangling the web of dependencies yet, but makes it much easier to refactor in future and also speed up compilation I think.

I'm aware that the playground is a little broken in release builds -- I have plans to refactor it and remove the custom build pipeline and instead have the playground, snapshots, roc check etc all using the same coordinate_simple pipeline. I think this will be much more DRY and maintainable.

Along with the playground changes I'd like to setup an e2e harness to run in CI and exercise our playground interface. Now the actual web app itself is in a separate repository it will be good to avoid regressions across that boundary.

Luke Boswell (Jul 28 2025 at 00:25):

After all this, I'd love to get the REPL online... upgrade our CLI to use https://github.com/rockorager/libvaxis and plug the REPL in etc with our reporting system to render reports etc.

The key thing I've been thinking about here is making a nice abstraction for that that support both the playground REPL, and the cli REPL frontends.

Jared Ramirez (Jul 28 2025 at 02:48):

When are you planning starting on this refactor? I have some stuff in progress in can/type checking that I should wrap up this week

Luke Boswell (Jul 28 2025 at 03:04):

I've almost finished the "check" module just now... about to make a PR for it. I can leave it in draft for you to look at

Luke Boswell (Jul 28 2025 at 03:09):

Luke Boswell (Jul 28 2025 at 03:11):

When I've done these migrations I've found things that were slipping through and not running with zig build test. For example this PR currently fails with this issue... which I was just looking into.

$ zig build test
test
└─ run test
   └─ zig test Debug native 3 errors
src/collections/safe_list.zig:587:17: error: Cannot serialize non-POD type problem.Problem
                @compileError("Cannot serialize non-POD type " ++ @typeName(T));
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/collections/safe_list.zig:649:62: error: enum 'meta.FieldEnum(multi_array_list.MultiArrayList(problem.Problem).Elem__struct_41809)' has no tag with value '2'
                    const field_ptr = slice.items(@as(Field, @enumFromInt(i))).ptr;
                                                             ^~~~~~~~~~~~~~~
/Users/luke/zig-macos-aarch64-0.14.0/lib/std/meta.zig:544:12: note: enum declared here
    return @Type(.{
           ^~~~~
src/check/snapshot.zig:391:36: error: expected type '[]const snapshot.SnapshotTag', found 'multi_array_list.MultiArrayList(snapshot.SnapshotTag).Slice'
        return self.tags.sliceRange(range);
               ~~~~~~~~~~~~~~~~~~~~^~~~~~~
/Users/luke/zig-macos-aarch64-0.14.0/lib/std/multi_array_list.zig:70:27: note: struct declared here
        pub const Slice = struct {
                          ^~~~~~
src/check/snapshot.zig:390:78: note: function return type declared here
    pub fn getTagsSlice(self: *const Self, range: SnapshotTagSafeList.Range) []const SnapshotTag {
                                                                             ^~~~~~~~~~~~~~~~~~~
error: the following command failed with 3 compilation errors:

Jared Ramirez (Jul 28 2025 at 04:28):

Nice, no go ahead and I'll rebase based from your changes when they land. I don't wanna hold you up!

Isaac Van Doren (Jul 28 2025 at 13:15):

Do we want to pay the price of adding this as a dependency? I'm not necessarily opposed to doing so, but it's worth considering

Anton (Jul 28 2025 at 13:45):

Yeah, we discussed it during the meetup, we think it's too tedious to build that all ourselves. Maybe we could vendor it and cut it down a bit, I don't think we need zigimg for example.

Brendan Hansknecht (Jul 28 2025 at 14:43):

I'm obviously out of the loop for roc currently, but just curious around the why of this? Like what is the advantage of making build.zig a lot more complicated and tracking all the dependencies inside of it instead of leaving the compiler as a single module. Or like a few executables and then a single library module.

Brendan Hansknecht (Jul 28 2025 at 14:45):

Also, I don't think modules would change compilation speed at all. At least not based on my understanding of how zig does compilation. (Zig always puts everything in a single compilation unit)

Luke Boswell (Jul 28 2025 at 22:18):

I probably misunderstood your message here, but this is why I started refactoring to use modules.

Brendan Hansknecht (Jul 29 2025 at 00:46):

I only quickly skimmed your PR earlier, but it look like you were introducing a bunch of modules.

I would not add a check and base and collections and etc module. I would make 1 singular lib module.

Brendan Hansknecht (Jul 29 2025 at 00:46):

Luke Boswell (Jul 29 2025 at 00:58):

Luke Boswell (Jul 29 2025 at 23:52):

@Jared Ramirez @JRI98 -- I'm tracking your PR's but planning on merging the cleanup in https://github.com/roc-lang/roc/pull/8124 first. That restores a significant number of our unit tests.

Just need to find some time to dig into the CI failures... it looks like some memory leaks need to be addressed.

Luke Boswell (Jul 30 2025 at 11:49):

I've been iterating on that, polishing and refactoring along with the playground as I go.

I hoped I would have a PR ready by now, but I've discovered that our current approach to generating HTML from the AST and CIR involves a lot of recursion and easily stack overflows on a WASM VM.

So taking a slight tangent exploring a alternate approach instead. The general idea is we could stream the IR by writing to our buffer as we go and reduce recursion by using a stack.

Luke Boswell (Jul 30 2025 at 11:52):

I added a new CI workflow for this, if you wanted to see what the output looks like it just ran and failed with the issue I'm currently digging into Re HTML

Luke Boswell (Jul 30 2025 at 12:38):

Gemini basically one-shotted the stack based HTML generator... I had to fixup some compile errors (converting a bunch of AST helpers to take a *const where they don't need to mutate).

Happy to report it works nicely within our WASM VM and doesn't stack overflow... no idea if the HTML looks any good yet. But that's a start. :smiley:

Luke Boswell (Jul 31 2025 at 00:57):

Our CI uncovered a bug in a transitive dependency specific to ARM... so seeing if I can patch that. I'll make a PR to upstream if it passes everything.

Luke Boswell (Jul 31 2025 at 01:07):

Luke Boswell (Jul 31 2025 at 01:14):

I'll update the actual TypeScript web app and confirm it's all working correctly before changing that Ready For Review.

Luke Boswell (Jul 31 2025 at 07:26):

Luke Boswell (Aug 01 2025 at 01:51):

Internal debate I'm having with myself... implement more of the interpreter, or wire the REPL into the playground :shrug:

Jared Ramirez (Aug 01 2025 at 02:29):

I’m continuing to work on type checking odds and ends, particularly around annotations and rigid vars!

Richard Feldman (Aug 01 2025 at 02:31):

I'm working on getting a dev build of an interpreter working on an actual host (single-module only for now, but gotta start somewhere!)

Luke Boswell (Aug 01 2025 at 02:36):

Ok, you convinced me... I'm going to flesh out support for the interpreter, we'll need that so the dev builds are useful :smiley:

Luke Boswell (Aug 01 2025 at 02:36):

(1u8, 2i8, 3u16, 4i16, 5u32, 6i32, 7u64, 8i64, 9u128, 10i128, 11.0f32, 12.0f64, 13.0dec)

Luke Boswell (Aug 01 2025 at 03:02):

(1u8, 2i8, 3u16, 4i16, 5u32, 6i32, 7u64, 8i64, 9u128, 10i128, 11.0f32, 12.0f64, 13.0dec, 0xE, 0xf, 0x20, 0b10001, 0b1_0010, 19, 20.0, 21_000, 22_000_000, 0.0, -0.1, 2e4, 3E2, -0.2e-2)

Luke Boswell (Aug 01 2025 at 07:28):

(True, False, Bool.True, Bool.False, !True, !False, True and False, !True or !True)

Luke Boswell (Aug 03 2025 at 12:02):

Good progress on evaluating strings. Initial implementation seems to be working ok.

Tomorrow I plan on testing it more thoroughly and ensuring it handles errors properly.

I'd like to add a new test expectation that asserts we evaluate to the correct RocStr value.

I'm not 100% sure this is the correct approach or anything... in hindsight I feel like I should have started with RocList. I'm trying to understand how to properly manage heap allocated things in the interpreter.

Richard Feldman (Aug 03 2025 at 12:12):

I think string was a better choice; List needs the interpreter to be able to do polymorphism, which is a rabbit hole :smile:

Richard Feldman (Aug 04 2025 at 02:20):

I could really use some help getting tests for that running on CI in Windows etc. with our actual LLD :sweat_smile:

Richard Feldman (Aug 04 2025 at 02:58):

oh yeah, quick benchmarks on the file in that demo vs python: (release build of roc)

Benchmark 1: ./zig-out/bin/roc test_direct.roc
  Time (mean ± σ):      26.9 ms ±   6.9 ms    [User: 5.0 ms, System: 10.1 ms]
  Range (min … max):    24.2 ms …  59.2 ms    47 runs

Benchmark 2: python3 test_direct.py
  Time (mean ± σ):      21.3 ms ±   1.5 ms    [User: 14.6 ms, System: 4.3 ms]
  Range (min … max):    19.2 ms …  27.8 ms    95 runs

Richard Feldman (Aug 04 2025 at 02:59):

I have some ideas on how we can get that to be faster than Python...we're almost there already! :grinning_face_with_smiling_eyes:

Anton (Aug 04 2025 at 09:21):

if (builtin.os.tag == .windows) {
        return;
}

Luke Boswell (Aug 04 2025 at 10:00):

I plan on looking at it also. I've been up to my eyeballs in this part of the codebase today. I think I've got two test platforms setup with CI, and linking using our LLD implementation.

Luke Boswell (Aug 04 2025 at 10:01):

Luke Boswell (Aug 04 2025 at 10:57):

Luke Boswell (Aug 04 2025 at 11:04):

I'm going to disable them. Tomorrow I can boot up my Windows machine and give it some love

Luke Boswell (Aug 04 2025 at 11:16):

Luke Boswell (Aug 04 2025 at 12:39):

I've merged my PR... if anyone wants to try it out I left a README explaining how to run the two test platforms/examples

Brendan Hansknecht (Aug 04 2025 at 20:14):

I'm so excited for some of the direct integration we get with the zig version of the compiler

Luke Boswell (Aug 04 2025 at 22:21):

I'm going to setup my Windows machine and finish the dev interpreter implementation.

Jared Ramirez (Aug 08 2025 at 18:20):

working on can/type checking for statements (statement annotations, inheriting forall type vars from parent annotation, etc)

Luke Boswell (Aug 08 2025 at 23:11):

Anthony Bullard (Aug 09 2025 at 02:45):

that's funny. i was actually playing with the repl this morning and got some stuff working

Anthony Bullard (Aug 09 2025 at 02:46):

Definitely implemented the loop but I sure do get a lot of crashes from eval :rolling_on_the_floor_laughing:

Luke Boswell (Aug 09 2025 at 06:34):

https://github.com/roc-lang/roc/pull/8175
I got the reassigment of defs thing working. I don't think this is a long term design, maybe more of a hack. I just wanted something working to play with the interpreter.

My approach is keeping track of definitions in a hashmap, then when you evaluation something concatenate all the defs and the expression together into a block expression and evaluate that.

Luke Boswell (Aug 09 2025 at 06:36):

Luke Boswell (Aug 09 2025 at 09:07):

Luke Boswell (Aug 09 2025 at 10:08):

I think I've got the new REPL wired through everything... we have REPL snapshots that support redefinitions (the intermediate steps in the snapshot are linked), the e2e WASM test harness also includes a number of roundtrip tests for the REPL features in the playground (sending JSON messages etc).

Tomorrow I'll try and upgrade the Playground web app to include a switch to try the experimental REPL mode.

It'll be rough for a while probably as the actual interpreter itself is fairly primitive. But at least we know it definitely will work in WASM once we finish implementing it

Anthony Bullard (Aug 09 2025 at 11:04):

Anthony Bullard (Aug 09 2025 at 11:05):

I've been adding it to the CLI. Mainly to explore the capabilities of the interpreter such as they are

Anthony Bullard (Aug 09 2025 at 12:05):

Luke Boswell (Aug 11 2025 at 22:31):

Thank you @Anthony Bullard -- I've had to take a major detour and refactor some things to sort out our modules. I'll come back to this once that work is completed.

Luke Boswell (Aug 12 2025 at 11:40):

FYI all, we just merged a big PR that reorganised things. It was such a big job there are some test files with slabs of code commented out, but we have all of our e2e tests operational and happy.

If you have any questions or need help merging WIP changes please let me know. Mostly just shuffling things into a module structure without any cyclic dependencies, so hopefully shouldnt be really big conflicts.

Anthony Bullard (Aug 12 2025 at 13:48):

Luke Boswell (Aug 14 2025 at 05:04):

I've got the basics wired up in this PR, but feel like a gorilla working with this CSS, just wondering if anyone is interested in helping?

Anton (Aug 15 2025 at 09:29):

Anton (Aug 15 2025 at 17:53):

Anton (Aug 16 2025 at 18:17):

Anton (Aug 16 2025 at 18:18):

Turns out tokio uses a smaller stack size than the OS default for the threads it creates.

Stream: contributing

Topic: Worklog (Draft PRs and coordination)

Anthony Bullard (Jun 29 2025 at 16:39):

Jared Ramirez (Jun 29 2025 at 20:16):

Anthony Bullard (Jun 29 2025 at 21:26):

Richard Feldman (Jun 29 2025 at 21:50):

Luke Boswell (Jun 29 2025 at 22:10):

Luke Boswell (Jun 29 2025 at 22:17):

Kiryl Dziamura (Jun 30 2025 at 09:58):

Kiryl Dziamura (Jun 30 2025 at 09:59):

Anthony Bullard (Jun 30 2025 at 11:43):

Anthony Bullard (Jun 30 2025 at 11:44):

Kiryl Dziamura (Jun 30 2025 at 11:44):

Anthony Bullard (Jun 30 2025 at 11:46):

Jared Ramirez (Jun 30 2025 at 22:29):

Luke Boswell (Jun 30 2025 at 22:38):

Kiryl Dziamura (Jul 01 2025 at 10:20):

Brendan Hansknecht (Jul 01 2025 at 15:17):

Brendan Hansknecht (Jul 01 2025 at 15:18):

Brendan Hansknecht (Jul 01 2025 at 15:19):

Brendan Hansknecht (Jul 01 2025 at 15:20):

Kiryl Dziamura (Jul 01 2025 at 15:30):

Kiryl Dziamura (Jul 01 2025 at 16:27):

Jared Ramirez (Jul 01 2025 at 20:28):

Luke Boswell (Jul 01 2025 at 21:09):

Jared Ramirez (Jul 01 2025 at 21:15):

Jared Ramirez (Jul 01 2025 at 21:16):

Luke Boswell (Jul 01 2025 at 22:33):

Luke Boswell (Jul 02 2025 at 00:21):

Richard Feldman (Jul 02 2025 at 02:29):

Luke Boswell (Jul 02 2025 at 12:15):

Luke Boswell (Jul 02 2025 at 23:30):

Jared Ramirez (Jul 03 2025 at 15:52):

Richard Feldman (Jul 03 2025 at 16:07):

Richard Feldman (Jul 03 2025 at 16:07):

Richard Feldman (Jul 03 2025 at 16:07):

Richard Feldman (Jul 03 2025 at 16:07):

Richard Feldman (Jul 03 2025 at 16:08):

Richard Feldman (Jul 03 2025 at 16:08):

Richard Feldman (Jul 03 2025 at 16:09):

Richard Feldman (Jul 03 2025 at 16:10):

Richard Feldman (Jul 03 2025 at 16:11):

Richard Feldman (Jul 03 2025 at 16:11):

Jared Ramirez (Jul 03 2025 at 16:26):

Jared Ramirez (Jul 03 2025 at 16:28):

Richard Feldman (Jul 03 2025 at 16:28):

Jared Ramirez (Jul 03 2025 at 16:28):

Richard Feldman (Jul 03 2025 at 16:28):

Richard Feldman (Jul 03 2025 at 16:28):

Jared Ramirez (Jul 03 2025 at 16:29):

Richard Feldman (Jul 03 2025 at 16:29):

Richard Feldman (Jul 03 2025 at 16:30):

Richard Feldman (Jul 03 2025 at 16:30):

Luke Boswell (Jul 04 2025 at 00:57):

Luke Boswell (Jul 04 2025 at 07:12):

Luke Boswell (Jul 04 2025 at 07:14):

Kiryl Dziamura (Jul 04 2025 at 09:25):

Brendan Hansknecht (Jul 04 2025 at 22:52):

Brendan Hansknecht (Jul 04 2025 at 22:59):

Luke Boswell (Jul 05 2025 at 08:30):

Luke Boswell (Jul 05 2025 at 08:31):

Richard Feldman (Jul 05 2025 at 12:53):

Jared Ramirez (Jul 05 2025 at 16:12):

Richard Feldman (Jul 05 2025 at 18:06):

Richard Feldman (Jul 05 2025 at 18:07):

Richard Feldman (Jul 05 2025 at 18:10):

Brendan Hansknecht (Jul 05 2025 at 18:12):

Richard Feldman (Jul 05 2025 at 18:12):

Richard Feldman (Jul 05 2025 at 18:12):

Richard Feldman (Jul 05 2025 at 18:12):

Richard Feldman (Jul 05 2025 at 18:13):

Luke Boswell (Jul 06 2025 at 07:52):

Richard Feldman (Jul 06 2025 at 13:35):

Richard Feldman (Jul 06 2025 at 13:35):

Richard Feldman (Jul 06 2025 at 13:35):

Richard Feldman (Jul 06 2025 at 13:38):

Richard Feldman (Jul 07 2025 at 01:54):

Richard Feldman (Jul 07 2025 at 01:54):

Jared Ramirez (Jul 07 2025 at 05:37):

Luke Boswell (Jul 07 2025 at 22:24):