casual conversation · compiler development

and the inference breaks down over higher-rank lifetimes so you just need to explicitly type it as

|something: Something<'ctx>| env.builder.build_whatever(something)

Ayaz Hafiz (Jul 07 2023 at 16:16):

Folkert de Vries (Jul 09 2023 at 16:28):

Folkert de Vries (Jul 10 2023 at 19:18):

fyi we just upgraded main to rust 1.67. upgrading should be seamless, but you may see some longer build times

Folkert de Vries (Jul 10 2023 at 19:18):

Ayaz Hafiz (Jul 12 2023 at 21:22):

Luke Boswell (Jul 12 2023 at 22:33):

Random question, is the intent to use this instead of lambda sets? I.e. this helps remove a complex area of the compiler that was responsible for bugs? Just not entirely understanding why here. I don't need to know, but it looks cool and I'm interested.

Ayaz Hafiz (Jul 12 2023 at 23:22):

it's an alternative to compiling functions to lambda sets but it won't get rid of them. We need type-erasure (for values and functions) in order to support Map2, so it will at least be used for that. But we could also eventually use it for dev builds, since type-erased functions are faster to compiler, or for roc check, since lambda sets are not important for type checking.

Richard Feldman (Jul 12 2023 at 23:23):

Ayaz Hafiz (Jul 15 2023 at 20:28):

would be cool to consider this. we do this already in the llvm backend but it could be generalzied

Richard Feldman (Jul 18 2023 at 18:16):

Ayaz Hafiz (Jul 18 2023 at 18:18):

Andrew Kelley (Jul 18 2023 at 20:59):

Richard Feldman (Jul 18 2023 at 21:00):

Andrew Kelley (Jul 18 2023 at 21:03):

when people complain about this, I usually say something to the effect of, "they're good lines, Brent"

Andrew Kelley (Jul 18 2023 at 21:04):

Pearce Keesling (Jul 18 2023 at 21:26):

That GitHub link makes the mobile app crash so I am gonna guess it's quite large :sweat_smile:

Ayaz Hafiz (Jul 18 2023 at 21:51):

Qqwy / Marten (Jul 18 2023 at 22:06):

:laughter_tears:
Makes me long for FORTH a little. For example, the infamous IOCCC submission buzzard.2's ~60 lines of C source code is essentially all you need to bootstrap a language runtime.

Folkert de Vries (Jul 21 2023 at 12:20):

Folkert de Vries (Jul 21 2023 at 12:21):

no idea what the status of the C api is actually, and what the chances are of that being accepted, but long term we will absolutely need guaranteed tail calls

Richard Feldman (Jul 24 2023 at 19:54):

Brendan Hansknecht (Jul 24 2023 at 20:32):

Brendan Hansknecht (Jul 24 2023 at 20:34):

More power to the compiler, probably at the cost of more hardware register contention. But compilers are almost certainly good enough and code complex enough that this will be a gain.

Ayaz Hafiz (Jul 24 2023 at 21:17):

that they're all caller saved is super sweet. hopefully will make compilers go faster too, less time for the register allocator to run

Folkert de Vries (Jul 27 2023 at 22:32):

Folkert de Vries (Aug 05 2023 at 16:44):

    // globally cache the temporary directory
    static TEMP_DIR: OnceLock<tempfile::TempDir> = OnceLock::new();
    let temp_dir = TEMP_DIR.get_or_init(|| tempfile::tempdir().unwrap());

it's not always the most ergonomic approach (and LazyLock, the stdlib name for this thing, is an unstable feature, so may make it in at some point) but in many cases this works quite well and does not require any dependencies

Folkert de Vries (Aug 05 2023 at 16:45):

Folkert de Vries (Aug 07 2023 at 15:09):

not holding my breath for quick progress there, but it's good that someone is working on it

Folkert de Vries (Aug 09 2023 at 14:21):

does anyone know if there is a technical reason for allocating from high to low addresses? It appears quite common, but I'd like to know if that is just a custom or that there are good technical reasons for it

Ayaz Hafiz (Aug 09 2023 at 14:36):

do you mean on the stack or the heap? on the heap maybe just because of memory regions?

Richard Feldman (Aug 09 2023 at 14:38):

Folkert de Vries (Aug 09 2023 at 14:38):

Richard Feldman (Aug 09 2023 at 14:39):

like the stack and the heap can't both use the same strategy, unless one of them is set up to start right where the other one ends, but to do that you'd need to know the exact size of the stack

Richard Feldman (Aug 09 2023 at 14:39):

which I think maybe requires a syscall or something, and/or may not always be supported on all OSes?

Folkert de Vries (Aug 09 2023 at 14:39):

well yes when you have some other region growing from the other side you need to make a choice

Folkert de Vries (Aug 09 2023 at 14:40):

but what if you didn't? is there still a reason to perfer starting at the end and growing down?

Brendan Hansknecht (Aug 09 2023 at 15:05):

Folkert de Vries (Aug 09 2023 at 15:07):

as in, the thing I could copy-paste went high-to-low, so that is just what it does now

Richard Feldman (Aug 11 2023 at 05:23):

looks like making valgrind work on Apple Silicon is...really difficult! :astonished:

Folkert de Vries (Aug 18 2023 at 14:23):

Ayaz Hafiz (Aug 19 2023 at 06:05):

ShalokShalom (Aug 20 2023 at 09:42):

How does Roc's reference counting with cycles? I am wondering, since I am fascinated with ORC, the way Nim did it. Does Roc do it at all at the given moment? Thanks for your answers :)

Ayaz Hafiz (Aug 20 2023 at 14:29):

Roc does not deal with cycles, because cycles are impossible to introduce in an immutable language like Roc, since to create a cycle you must create then mutate a value.

Folkert de Vries (Aug 20 2023 at 14:30):

small correction: immutable and strict language. Laziness in haskell can also create cycles

Richard Feldman (Aug 23 2023 at 13:26):

Richard Feldman (Aug 23 2023 at 13:27):

Brendan Hansknecht (Aug 23 2023 at 14:20):

Richard Feldman (Aug 23 2023 at 14:21):

I think the last time I wrote a whole C++ program was over 20 years ago :sweat_smile:

Pit Capitain (Aug 24 2023 at 12:49):

We realize that we're getting older when we look at our children... and at our old code :wink:

Zeljko Nesic (Aug 24 2023 at 18:02):

Richard Feldman (Aug 24 2023 at 18:22):

I wish I still had that RPG I wrote in c++ I wrote all those years ago...I'm sure I could spot some undefined behavior in it today :laughing:

Richard Feldman (Aug 24 2023 at 23:15):

Richard Feldman (Aug 26 2023 at 18:49):

Luke Boswell (Aug 31 2023 at 07:29):

PS C:\Users\bosyl\Documents\GitHub\roc> .\target\release\roc.exe run ..\basic-cli\examples\hello-world.roc
🔨 Rebuilding platform...
warning: ignoring debug info with an invalid version (0) in app
Hello, World!

Luke Boswell (Aug 31 2023 at 07:35):

Luke Boswell (Aug 31 2023 at 07:43):

I packaged basic-cli for Windows and uploaded the binary in the Github release. Not sure if this is a bad idea to share like this...

PS C:\Users\bosyl\Documents\GitHub\basic-cli> ..\roc\target\release\roc.exe run .\examples\hello-world.roc
Downloading https://github.com/lukewilliamboswell/basic-cli/releases/download/0.5.0/Dt_L3PF3VAAxtLzxXZ-g9nBp0Gzo6tbTevSep1SdDeQ.tar.br
    into C:\Users\bosyl\AppData\Roaming\Roc\packages

warning: ignoring debug info with an invalid version (0) in app
Hello, World!

Luke Boswell (Aug 31 2023 at 07:45):

app "hello-world"
    packages { pf: "https://github.com/lukewilliamboswell/basic-cli/releases/download/0.5.0/Dt_L3PF3VAAxtLzxXZ-g9nBp0Gzo6tbTevSep1SdDeQ.tar.br" }
    imports [pf.Stdout, pf.Task.{ Task }]
    provides [main] to pf

main : Task {} I32
main =
    Stdout.line "Hello, World!"

Luke Boswell (Aug 31 2023 at 08:01):

There are still a lot of issues here, other examples I've tried will build and run but not print anything... but at least this is a start. :smiley:

Fábio Beirão (Aug 31 2023 at 13:13):

I don't think the exe is code signed, so that can be a hurdle with SmartScreen :thinking:

Fábio Beirão (Aug 31 2023 at 13:15):

What I did is I used the Windows Sandbox to get a clean windows version, downloaded the Exe. Went to the properties and clicked Unblock (due to it being downloaded from the internet) and then just double clicked the exe file.

Fábio Beirão (Aug 31 2023 at 13:17):

@Luke Boswell let me know if you need help with testing Windows stuff. I have Windows 10 on one machine and Windows 11 on another.

Luke Boswell (Aug 31 2023 at 21:06):

Richard Feldman (Sep 04 2023 at 03:36):

this is wild - I can't believe that nibble mask works, those numbers look so random! But it totally does - I tried it out.

Richard Feldman (Sep 04 2023 at 14:05):

Folkert de Vries (Sep 04 2023 at 14:15):

I'm confused whether this is specific to the linux kernel, or more general for "programs running on linux"

Richard Feldman (Sep 04 2023 at 17:07):

Luke Boswell (Sep 05 2023 at 10:13):

I watched a few YouTube tutorials on interactive rebasing, and cherry-picking and I have to admit... I wish I had done that sooner! So simple, and would have saved me a lot of time. Learnt a few nice tricks along the way. :thumbs_up: :sweat_smile:

Fábio Beirão (Sep 05 2023 at 10:15):

This is something that when I started using Fork (https://git-fork.com/) just became so much clearer to me. Unfortunately doesn't have a linux port, only Windows and Mac

Luke Boswell (Sep 05 2023 at 10:51):

I'm just wondering, should the zig tests pass on MacOS? I get the below errors which seem strange to me thread 11673296 panic: incorrect alignment

Aarch64ZigTestError

192-168-1-105:bitcode luke$ pwd
/Users/luke/Documents/GitHub/roc/crates/compiler/builtins/bitcode
192-168-1-105:bitcode luke$ zig build test
Test [4/153] dec.test "fromStr: empty"... thread 11673296 panic: incorrect alignment
/Users/luke/Documents/GitHub/roc/crates/compiler/builtins/bitcode/src/libc/musl/memcpy.zig:41:32: 0x104f850b3 in libc.musl.memcpy.fallback_memcpy (test)
var s4 = @alignCast(4, s);
^
/Users/luke/Documents/GitHub/roc/crates/compiler/builtins/bitcode/src/libc.zig:31:31: 0x104fc4e83 in libc.memcpy (test)
return musl.memcpy(dest, src, len);
^
/Users/luke/Documents/GitHub/roc/crates/compiler/builtins/bitcode/src/str.zig:56:9: 0x104fb6ed3 in str.RocStr.init (test)
@memcpy(result.asU8ptrMut(), bytes_ptr, length);
^
/Users/luke/Documents/GitHub/roc/crates/compiler/builtins/bitcode/src/dec.zig:755:30: 0x104f8674b in dec.test "fromStr: empty" (test)
var roc_str = RocStr.init("", 0);
^
/nix/store/1jsz44m30pxqzll6s641cxsibzbpvjmx-zig-0.9.1/lib/zig/std/special/test_runner.zig:80:28: 0x104fc6f8f in std.special.main (test)
} else test_fn.func();
^
/nix/store/1jsz44m30pxqzll6s641cxsibzbpvjmx-zig-0.9.1/lib/zig/std/start.zig:551:22: 0x1050289b3 in std.start.callMain (test)
root.main();
^
/nix/store/1jsz44m30pxqzll6s641cxsibzbpvjmx-zig-0.9.1/lib/zig/std/start.zig:495:12: 0x104fc8a43 in std.start.callMainWithArgs (test)
return @call(.{ .modifier = .always_inline }, callMain, .{});
^
/nix/store/1jsz44m30pxqzll6s641cxsibzbpvjmx-zig-0.9.1/lib/zig/std/start.zig:460:12: 0x104fc896f in std.start.main (test)
return @call(.{ .modifier = .always_inline }, callMainWithArgs, .{ @intCast(usize, c_argc), c_argv, envp });
^
???:?:?: 0x1a1ebff27 in ??? (???)
???:?:?: 0xab71ffffffffffff in ??? (???)
error: the following test command crashed:
/Users/luke/Documents/GitHub/roc/crates/compiler/builtins/bitcode/zig-cache/o/00a242709486f6fe0044878fa17186a4/test /nix/store/1jsz44m30pxqzll6s641cxsibzbpvjmx-zig-0.9.1/bin/zig
The following command exited with error code 1:
/nix/store/1jsz44m30pxqzll6s641cxsibzbpvjmx-zig-0.9.1/bin/zig test /Users/luke/Documents/GitHub/roc/crates/compiler/builtins/bitcode/src/main.zig -lc --cache-dir /Users/luke/Documents/GitHub/roc/crates/compiler/builtins/bitcode/zig-cache --global-cache-dir /Users/luke/.cache/zig --name test
error: the following build command failed with exit code 1:
/Users/luke/Documents/GitHub/roc/crates/compiler/builtins/bitcode/zig-cache/o/1d46ff7b08d826ee45715195b67b4b10/build /nix/store/1jsz44m30pxqzll6s641cxsibzbpvjmx-zig-0.9.1/bin/zig /Users/luke/Documents/GitHub/roc/crates/compiler/builtins/bitcode /Users/luke/Documents/GitHub/roc/crates/compiler/builtins/bitcode/zig-cache /Users/luke/.cache/zig test

Luke Boswell (Sep 05 2023 at 10:52):

Folkert de Vries (Sep 05 2023 at 10:59):

Richard Feldman (Sep 06 2023 at 11:07):

Anton (Sep 06 2023 at 11:16):

Brian Carroll (Sep 06 2023 at 12:06):

Brian Carroll (Sep 06 2023 at 12:08):

We increase the alignment on many heap data structures because of the refcount. Like Str bytes for example.

Anton (Sep 06 2023 at 17:46):

The github web vscode editor (press '.' key on a repo's page) works well now. In the past I always had issues with commit signing but that seems to have been fixed :) Handy for when you quickly want to make small changes to multiple files.

Brendan Hansknecht (Sep 06 2023 at 17:49):

Zeljko Nesic (Sep 06 2023 at 17:56):

Folkert de Vries (Sep 14 2023 at 12:23):

Dec is now the default type for fractional values. This may cause breakage in some cases. Also I suspect there are still a bunch of missing functions for Dec that are used in practice. The quick fix is to explicitly make literals of the f64 type, like 3.14f64

Brendan Hansknecht (Sep 14 2023 at 16:57):

@Folkert de Vries If you aren't looking into Dec sin/cos impl, I may look into it. Sounds like an interesting side project. I have always wondered how they are implemented in software.

Folkert de Vries (Sep 14 2023 at 17:05):

Richard Feldman (Sep 14 2023 at 17:53):

Richard Feldman (Sep 15 2023 at 11:10):

Richard Feldman (Sep 15 2023 at 11:11):

Folkert de Vries (Sep 15 2023 at 16:29):

     Summary [  64.943s] 144 tests run: 19 passed, 125 failed, 709 skipped

Folkert de Vries (Sep 15 2023 at 16:30):

also the rust library we use has very limited options for macho relocations. it's not clear whether that is fundamental or whether they've just not implemented it yet

Folkert de Vries (Sep 15 2023 at 17:27):

also the dev backend repl just got merged, so on x86 linux, the repl should now be much faster!

Folkert de Vries (Sep 16 2023 at 12:25):

@Luke Boswell and I got it to work on macos too now, and fixed some other issues. Now there is some bug with branches that went unnoticed on x86 and then we'll have almost all of gen_num working already. (still some float stuff and 128-bit stuff to do)

Richard Feldman (Sep 16 2023 at 17:53):

Brian Carroll (Sep 16 2023 at 19:23):

Richard Feldman (Sep 16 2023 at 19:52):

Brendan Hansknecht (Sep 16 2023 at 20:53):

Luke Boswell (Sep 17 2023 at 06:21):

I've generated an object file from a test using e.g. ROC_DEV_WRITE_OBJ=1 cargo nextest-gen-dev add_checked_dec and then built this into an executable using

zig build-exe /var/folders/48/39th9k0n0wdcj18k3yhm_g5c0000gn/T/app.o crates/compiler/builtins/bitcode/builtins-aarch64.ll something.zig

. Note the ROC_DEV_WRITE_OBJ flags roc to write out the object file and print the location of the file.

But when I run step through the file from test_main it seems to never get into the #UserApp_main_482528569654279254. Instead it branches off at 0x100000804 <+52>: b.ne 0x100000850 and then returns successfully without calling the actual test code I'm interested in I think.

(lldb) target create "something"
Current executable set to '/Users/luke/Documents/GitHub/roc/something' (arm64).
(lldb) b test_main
Breakpoint 1: where = something`test_main, address = 0x00000001000007d0
(lldb) r
Process 12932 launched: '/Users/luke/Documents/GitHub/roc/something' (arm64)
Process 12932 stopped
* thread #1, stop reason = breakpoint 1.1
    frame #0: 0x00000001000007d0 something`test_main
something`test_main:
->  0x1000007d0 <+0>:  sub    sp, sp, #0x60             ; =0x60
    0x1000007d4 <+4>:  str    x30, [sp, #0x58]
    0x1000007d8 <+8>:  str    x29, [sp, #0x50]
    0x1000007dc <+12>: add    x29, sp, #0x50            ; =0x50
(lldb) disassemble --frame
something`test_main:
->  0x1000007d0 <+0>:   sub    sp, sp, #0x60             ; =0x60
    0x1000007d4 <+4>:   str    x30, [sp, #0x58]
    0x1000007d8 <+8>:   str    x29, [sp, #0x50]
    0x1000007dc <+12>:  add    x29, sp, #0x50            ; =0x50
    0x1000007e0 <+16>:  ldr    x17, [x17]
    0x1000007e4 <+20>:  stur   x8, [x29, #-0x8]
    0x1000007e8 <+24>:  stur   x17, [x29, #-0x10]
    0x1000007ec <+28>:  ldur   x0, [x29, #-0x10]
    0x1000007f0 <+32>:  bl     0x100000770               ; roc_setjmp
    0x1000007f4 <+36>:  stur   x0, [x29, #-0x20]
    0x1000007f8 <+40>:  stur   x1, [x29, #-0x18]
    0x1000007fc <+44>:  ldur   x17, [x29, #-0x20]
    0x100000800 <+48>:  cmp    x17, #0x0                 ; =0x0
    0x100000804 <+52>:  b.ne   0x100000850               ; <+128>
    0x100000808 <+56>:  stur   x17, [x29, #-0x10]
    0x10000080c <+60>:  bl     0x100000920               ; #UserApp_main_482528569654279254

const std = @import("std");

const RocCallResult = extern struct {
    tag: u64,
    error_msg: u64,
    value: bool,
};

extern fn test_main() callconv(.C) RocCallResult;

pub fn main() u8 {
    const value = test_main();

    std.debug.print("done {}\n", .{value.value});

    return 0;
}

Luke Boswell (Sep 17 2023 at 06:22):

#[test]
#[cfg(any(feature = "gen-llvm", feature = "gen-wasm", feature = "gen-dev"))]
fn add_checked_dec() {
    assert_evals_to!(
        indoc!(
            r#"
            Num.addChecked 2.0dec 4.0dec == Ok 6.0dec
            "#
        ),
        true,
        bool
    );
}

Luke Boswell (Sep 17 2023 at 06:59):

I'm pretty convinced this is my set-up and nothing to do with the test failing, tried the same method on a good test and had the same issue. :sad:

Folkert de Vries (Sep 17 2023 at 07:54):

on aarch at the moment we don't use all of the rocresult stuff. instead I call/break on

extern fn roc__main_1_exposed() callconv(.C) u128;

Folkert de Vries (Sep 17 2023 at 07:56):

the issue here is something with how the return value is passed. it is too big to fit into registers, and we allocate stack space wrong: it overwrites the stored frame pointer and link register. that's how far I got last night

Folkert de Vries (Sep 17 2023 at 14:31):

Folkert de Vries (Sep 19 2023 at 08:49):

weird thing: if you use the github UI to rebase a PR then the commits are unverified

Folkert de Vries (Sep 19 2023 at 08:49):

but if you e.g. adjust the readme from the ui then the commit is verified (at least it used to be)

Brendan Hansknecht (Sep 19 2023 at 14:45):

Yeah, I noticed that. No idea why. Can rebase a second time locally and that should verify the commits, but kinda defeats the purpose of the GitHub you button.

Luke Boswell (Sep 19 2023 at 20:44):

@Richard Feldman and @Anton re Windows, I think there is still a couple of issues which are causing most of the test failures. I suspect the issue is in strings and lists. It's a little slow making progress with the tests running slowly due to issues with caching on windows.

Anton (Sep 20 2023 at 08:59):

Luke Boswell (Sep 20 2023 at 09:20):

Locally, when you run tests on windows it looks like zig has to rebuild the builtins and link the app every time

Folkert de Vries (Sep 20 2023 at 13:52):

Luke Boswell (Sep 21 2023 at 23:13):

For a test which returns RocResult::ok(RocDec::from(6)) what would the equivalent zig type be? I'm having trouble tracking down something related in our builtins. I.e.

const RocCallResult = extern struct {
    tag: u64,
    error_msg: u64,
    value: RocResultOrSomething??,
};

Luke Boswell (Sep 21 2023 at 23:34):

const RocCallResult = extern struct {
    tag: u64,
    error_msg: u64,
    value: RocResult,
};

const RocDec = extern struct {
    num: i128,

    pub fn addWithOverflow(self: RocDec, other: RocDec) WithOverflow(RocDec) {
        var answer: i128 = undefined;
        const overflowed = @addWithOverflow(i128, self.num, other.num, &answer);

        return .{ .value = RocDec{ .num = answer }, .has_overflowed = overflowed };
    }
};

const RocResultTag = enum(u8) {
    RocErr = 0,
    RocOk = 1,
};

const RocResultPayload = extern union {
    ok: RocDec,
    err: u8,
};

const RocResult = extern struct {
    payload: RocResultPayload,
    tag: RocResultTag,
};

Brendan Hansknecht (Sep 21 2023 at 23:36):

Richard Feldman (Sep 25 2023 at 01:08):

I have encountered a load-bearing dbg! in my Rust code - the program does something differently depending on whether I do a dbg!(list.len()) :sweat_smile:

Brendan Hansknecht (Sep 25 2023 at 01:21):

Richard Feldman (Sep 25 2023 at 01:21):

relatedly: anyone who has valgrind set up want to pair on a basic-webserevr bug sometime? :big_smile:

Richard Feldman (Sep 25 2023 at 01:22):

I'm also seeing fun stuff like if I remove an if Bool.false then things work differently, so some UB somewhere seems likely to blame

Anton (Sep 25 2023 at 18:53):

= note: import resolution is stuck, try simplifying macro imports

Anton (Sep 25 2023 at 18:57):

This was during the upgrade to rust 1.72. I'm done for today, if anyone else wants to take a stab at fixing the import resolution error, by my guest :)
https://github.com/roc-lang/roc/pull/5856

Anton (Sep 25 2023 at 19:01):

Brendan Hansknecht (Sep 25 2023 at 19:31):

Just a note, clippy seems to be messing up the readability of a lot of the indoc tests.

Anton (Sep 26 2023 at 09:32):

I don't know how to fix the import resolution so I'm going to upgrade to rust 1.71.1 first and not let clippy autofix things :p

Richard Feldman (Sep 27 2023 at 14:00):

Ayaz Hafiz (Sep 28 2023 at 03:25):

Rasheed Starlet (Sep 28 2023 at 10:00):

I am keeping my eyes peeled waiting for when it is time for writing the debugger.
@Richard Feldman when will that be?

Richard Feldman (Sep 28 2023 at 10:40):

no concrete plans yet, but there's nothing blocking it as far as I know! Is it something you're interested in working on?

Rasheed Starlet (Sep 28 2023 at 12:08):

Yes but I don’t have the technical know how to take on that task yet, It would take me some time to level up before.

Notification Bot (Sep 28 2023 at 12:09):

Brendan Hansknecht (Sep 28 2023 at 14:48):

I think we would first need good debug info generation before we would be interested in our own debugger

Brendan Hansknecht (Sep 28 2023 at 14:49):

Richard Feldman (Sep 29 2023 at 12:02):

I'm doing a Software Unscripted episode with Matt Godbolt and he said "we should absolutely get Roc on Godbolt"

Richard Feldman (Sep 29 2023 at 12:08):

also he tried the nightly on a fresh Ubuntu install and when he put "hello world" into the repl and got a panic about missing a dynamic library

Anton (Sep 29 2023 at 12:16):

Hmm, we do test every nightly's repl on ubuntu, but this is github's workflow runner ubuntu so its dependencies are different. I happen to have a clean ubuntu vm so I'll try to reproduce.

Anton (Sep 29 2023 at 12:26):

I was able to reproduce it, I'll make an issue for the error and another one to test nightlies on all kinds of docker containers to mimic fresh installs.

Richard Feldman (Sep 29 2023 at 13:19):

awesome! I relayed this to him (we just finished recording) and he asked if there's something he can quickly apt-get install to fix it :big_smile:

Anton (Sep 29 2023 at 13:40):

Anton (Sep 29 2023 at 13:43):

The amount of time that would be saved if the "No such file or directory" actually told you what file or dir it was looking for... :p

Richard Feldman (Sep 29 2023 at 13:46):

one of my coworkers who has been helping me with getting Roc incorporated into certain parts of the build just started trying it out, and DM'd me a bunch of questions for things he ran into as a beginner, followed by:

Richard Feldman (Sep 29 2023 at 13:49):

Rasheed Starlet (Sep 29 2023 at 13:54):

Anton (Sep 29 2023 at 14:50):

I'm not sure what part of this script fixes the issue but it does :p I'll look into a less bloated fix later.

wget https://apt.llvm.org/llvm.sh
chmod +x llvm.sh
sudo ./llvm.sh 13

Folkert de Vries (Sep 29 2023 at 14:52):

it might be missing certain features? what features are enabled for their official builds seems very random

Folkert de Vries (Sep 29 2023 at 17:25):

why is cargo slow, you ask? well maybe because they just casually perform many small allocations in a recent PR?

it's just really weird to read that code?! it's so easy to remove most of those allocations

Richard Feldman (Sep 29 2023 at 17:34):

Thanks! I'm surprised llvm isn't statically compiled into the roc binary though :face_with_raised_eyebrow:

Richard Feldman (Sep 29 2023 at 17:34):

also surprising: I believe he's on x64, and ran into this in the repl...but the latest Linux x64 nightly should use the dev backend in the repl, not LLVM! :thinking:

Brendan Hansknecht (Sep 29 2023 at 17:58):

Though maybe somehow it failed to the load the dev backend generated shared library for the repl.

Brendan Hansknecht (Sep 29 2023 at 17:59):

Also, yeah, llvm can't be dynamic. Otherwise we would see way way more versioning issues, right?

Richard Feldman (Sep 29 2023 at 17:59):

Anton (Sep 29 2023 at 18:24):

Anton (Sep 29 2023 at 18:26):

Brendan Hansknecht (Sep 29 2023 at 21:45):

Anton (Sep 30 2023 at 08:58):

Anton (Sep 30 2023 at 18:27):

sudo apt install binutils

Anton (Sep 30 2023 at 18:36):

Luke Boswell (Oct 06 2023 at 21:50):

Does anyone know if it should be possible to import URL packages in the platform main file just like an app? I haven't been able to get it working. I'm getting errors like

thread 'main' panicked at '[Qualified("json", json.Core)] not in {} ', crates/compiler/load_internal/src/file.rs:2211:25
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Richard Feldman (Oct 07 2023 at 00:39):

it should be, but I remember there's some bug with it from way back :sweat_smile:

Richard Feldman (Oct 07 2023 at 00:39):

Luke Boswell (Oct 08 2023 at 07:20):

Anton (Oct 09 2023 at 08:58):

Fascinating. It would be cool to have Victor Taelin on Software Unscripted @Richard Feldman

Luke Boswell (Oct 10 2023 at 10:15):

What does the alignment mean in this function on a RocList pub fn decref(self: RocList, alignment: u32) void?

Brendan Hansknecht (Oct 10 2023 at 14:59):

Alignment is a memory loading concept. In aggregate data, it tends to be the size of the largest thing in the aggregate.

Brendan Hansknecht (Oct 10 2023 at 15:08):

Folkert de Vries (Oct 10 2023 at 15:36):

Richard Feldman (Oct 11 2023 at 18:25):

Folkert de Vries (Oct 11 2023 at 21:33):

Luke Boswell (Oct 14 2023 at 03:46):

Trying to compile from main and I think there may be something up with the nix configuration. I haven't seen this before.

192-168-1-105:roc luke$ nix develop
192-168-1-105:roc luke$ cargo build --release --locked
   Compiling roc_repl_wasm v0.0.1 (/Users/luke/Documents/GitHub/roc/crates/repl_wasm)
error: failed to run custom build command for `roc_repl_wasm v0.0.1 (/Users/luke/Documents/GitHub/roc/crates/repl_wasm)`

Caused by:
  process didn't exit successfully: `/Users/luke/Documents/GitHub/roc/target/release/build/roc_repl_wasm-ba13b2d4af2101b5/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-changed=build.rs
  cargo:rerun-if-changed=src/repl_platform.c

  --- stderr
  thread 'main' panicked at 'Output {
      status: ExitStatus(
          unix_wait_status(
              256,
          ),
      ),
      stdout: "",
      stderr: "wasm-ld: error: Unknown attribute kind (86) (Producer: 'LLVM16.0.6' Reader: 'LLVM 13.0.1')\n",
  }', crates/repl_wasm/build.rs:48:5
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Brendan Hansknecht (Oct 14 2023 at 04:12):

Looks like you are still pulling in new zig from when you used the updated llvm branch?

Brendan Hansknecht (Oct 14 2023 at 04:13):

That or a cached file that was built when you were on an updated branch....something along these lines

Luke Boswell (Oct 14 2023 at 05:28):

Ayaz Hafiz (Oct 19 2023 at 01:36):

module;
#include<evil.h>
export module I_AM_A_DEFINE_GOOD_LUCK_FINDING_OUT_WHERE_I_COME_FROM;

Ayaz Hafiz (Oct 19 2023 at 14:25):

Clever how the surface syntax almost immediately desugars to an MLIR. The slides on JIT architecture are super interesting too, especially the idea regarding shipping packages with bytecode for eval

Brendan Hansknecht (Oct 19 2023 at 14:36):

I thought their ideas of taking control of more llvm passes and then running llvm in parallel on a per function level is quite intriguing

Brendan Hansknecht (Oct 19 2023 at 14:37):

Richard Feldman (Oct 21 2023 at 00:15):

Ayaz Hafiz (Oct 21 2023 at 00:22):

Richard Feldman (Oct 21 2023 at 12:37):

Richard Feldman (Oct 21 2023 at 12:46):

Luke Boswell (Oct 22 2023 at 10:27):

Is there anyone online who has an apple silicon Mac and is able to test something for me? I have a graphical "hello world" in this gist and am interested to know if it works for others or if there are issues. It should be a prebuilt-package so just work, as you need the latest zig to re-build the platform. Next step is to figure out what I need to cross-compile to various platforms so I can pre-build those object/archive files too.

Luke Boswell (Oct 22 2023 at 10:28):

Oh, it should spit out a zigimg.png file in the local directory where you run it from using roc run test.roc

Ayaz Hafiz (Oct 22 2023 at 12:43):

Ayaz Hafiz (Oct 22 2023 at 12:55):

Ayaz Hafiz (Oct 23 2023 at 13:43):

Folkert de Vries (Oct 23 2023 at 13:44):

that trick to also store the color of an RBTree in the pointer (because you only need one bit for the node type) is cool

Folkert de Vries (Oct 23 2023 at 13:45):

you want very flexible niches on the one hand, on the other hand we need to generate code (and maybe also debuginfo) for that and that is hard

Ayaz Hafiz (Oct 23 2023 at 13:52):

Ayaz Hafiz (Oct 23 2023 at 13:53):

I feel like there must be some fundamental theorem that shows optimal-niche-finding is np-complete. it feels the same as instruction selection.

Brendan Hansknecht (Oct 23 2023 at 17:26):

Luke Boswell (Oct 26 2023 at 00:11):

I've got another build of basic-graphics gist of example here; I think I have managed to include in the bundle a build for for the following targets:

~~Just wondering if anyone could test this for me? and let me know if you have any issues.~~

I would like to clean up the API a fair bit, but hoping I have all the parts together and working now.

Ayaz Hafiz (Oct 26 2023 at 00:12):

Also, the first CI job appears done in the details page, but the status check didn’t update. @Anton any ideas?

Joshua Warner (Oct 26 2023 at 03:22):

If folks have thoughts on whether we should go for the xcode-select --print-path approach instead, that'd be useful.

Richard Feldman (Nov 03 2023 at 17:47):

lots of sweet stuff in here, but at the end he notes that you can make 32 bit pointers that Just Work on a 64-bit system, which immediately makes sense, but which had never actually occurred to me! https://www.youtube.com/watch?v=H8THRznXxpQ

Brian Carroll (Nov 03 2023 at 18:05):

Oh I watched that video a few weeks ago but never got as far as that 32-bit trick! Will rewatch it. I've been taking his course since the start, enjoying it.

Brian Carroll (Nov 03 2023 at 18:20):

Wow yeah so memory addresses are virtual and you can force them to be under 2^32 for your process as long as you don't actually use more than 4GB. Crazy.
So you could shrink all pointers in your data structures from 64 to 32 and fit more stuff in cache.
I never heard of this before and suddenly I think it's weird that more people don't do it!

Brendan Hansknecht (Nov 07 2023 at 16:35):

Do you guys know that rust has not small string optimization at all? It also can't ever have small string optimization due to how it defines its apis. Vec guarentees that it is always heap allocated. And String guaretees it can always seamlessly convert to a Vec without copying any data. As such String must also be heap allocated.

Brendan Hansknecht (Nov 07 2023 at 16:42):

Brendan Hansknecht (Nov 07 2023 at 16:44):

Make me wonder how our perf would be affect if we dropped SSO and instead only had seamless slices. That said, &str should be strictly higher perf than seamless slices, so maybe that wouldn't map to Roc in the same way.

Brendan Hansknecht (Nov 07 2023 at 16:45):

Cause &str is in the type system and avoids runtime checks. It also doesn't have to deal with any sort of refcount.

Brendan Hansknecht (Nov 07 2023 at 16:45):

Luke Boswell (Nov 08 2023 at 00:14):

Brendan Hansknecht (Nov 08 2023 at 00:25):

Anton (Nov 08 2023 at 09:17):

Ayaz Hafiz (Nov 08 2023 at 18:01):

Semi-big development update: we're going to halt development on lambda sets for now, in favor of getting boxed closures to work across the board, so that we can unblock development on effect interpreters.

Richard Feldman (Nov 08 2023 at 18:06):

this means closures will be heap-allocated, which unfortunately means worse runtime performance. However, this is actually something we want to try to speed up dev builds in the long term anyway (that is, heap allocating closures), so switching to having them all work that way will give us more data points about the runtime cost in practice as well as some idea of the effect on compile times

Brian Carroll (Nov 08 2023 at 18:53):

I bet that was a tough call, sounds wise to move things forward.
Does the boxed closure contain a function pointer or a tag union or both?

Richard Feldman (Nov 08 2023 at 19:17):

Richard Feldman (Nov 08 2023 at 19:18):

maybe we could keep the current functionality behind a flag like --experimental-closures or something, to make it easier to compare perf impact and compile times with and without

Brian Carroll (Nov 08 2023 at 19:22):

Brian Carroll (Nov 08 2023 at 19:24):

Richard Feldman (Nov 08 2023 at 19:33):

Brendan Hansknecht (Nov 08 2023 at 19:45):

Ayaz Hafiz (Nov 08 2023 at 19:46):

Actually, it’s a bit more efficient than that. instead of boxing the pointer, it’s stored as a fat pointer. A boxed closure is a three-word record - one for the pointer, one for the closure data (or NULL), one for the ref counter function (or NULL). I have more details in https://www.notion.so/rwx/Type-Erasure-a3ed13ef1305422eba00dbda026e52b3?pvs=4

Brendan Hansknecht (Nov 08 2023 at 19:46):

Also, just curious, why this over still keeping our static dispatch with a switch statement, but with just boxing the data. I thought mostly the data was the issue, not the function pionters.

Ayaz Hafiz (Nov 08 2023 at 19:48):

Brendan Hansknecht (Nov 08 2023 at 19:48):

Brendan Hansknecht (Nov 08 2023 at 19:51):

Also, for the refcounter function, is it a function that could be called by the host to increment the refcount of all closure captures such that the host can call the same closure multiple times?

Ayaz Hafiz (Nov 08 2023 at 19:53):

I think it could be, but I haven't fully thought that through. The host would need to manage the lifetime though

Ayaz Hafiz (Nov 09 2023 at 05:19):

Folkert de Vries (Nov 11 2023 at 01:35):

Ayaz Hafiz (Nov 11 2023 at 06:23):

Ayaz Hafiz (Nov 11 2023 at 06:28):

Ayaz Hafiz (Nov 11 2023 at 06:30):

this is actually really surprising to me, especially with the heavy multi-pass LLVM backend. i guess part of it might be explained by Rust trait selection/method selection being pretty involved. But it’s something we should be mindful of-right now all inference in our compiler is single pass, intentionally, and making it multi pass could have effects like this

Brendan Hansknecht (Nov 11 2023 at 09:16):

It just sounds like they manage to run more llvm threads at the same time cause other stuff finishes sooner, right?

Brendan Hansknecht (Nov 11 2023 at 09:16):

Ayaz Hafiz (Nov 11 2023 at 15:35):

i mean in principle the unit of parallelism between the front end and backend should be the same though

Brendan Hansknecht (Nov 11 2023 at 16:55):

Pretty sure this is why it is faster 8 instead of 1 thread generating llvm ir at the boundary of frontend and backend

Richard Feldman (Nov 12 2023 at 11:46):

Richard Feldman (Nov 12 2023 at 11:48):

(to be fair, we also don't have a solution for URL-based packages to the problem of "the domain expired or got taken over and now I don't have a good way to notify people about where the new package lives)

Richard Feldman (Nov 12 2023 at 11:49):

but we do have a good answer to the security problems, and not just "any new owner of your package's URL has limited scope for being malicious" or "in theory they can't publish a new version (with malicious code) and have it automatically picked up by current users, because existing users will stick with the current version until they specifically update (new users of the package are not so lucky)"

Folkert de Vries (Nov 14 2023 at 08:33):

Richard Feldman (Nov 14 2023 at 12:37):

really interesting real-world comparison of monomorphization and dictionary passing for parametric polymorphism

Luke Boswell (Nov 15 2023 at 01:44):

Luke Boswell (Nov 15 2023 at 01:45):

My work around is to delete the broken parts, but it's really fickle and is breaking CI for the webserver. I haven't figure out a good way to resolve for all the architectures

Brendan Hansknecht (Nov 15 2023 at 01:45):

Luke Boswell (Nov 15 2023 at 01:45):

I'm generating using roc glue ../roc/crates/glue/src/RustGlue.roc platform/glue2 platform/main-command-glue.roc

Luke Boswell (Nov 15 2023 at 01:45):

Brendan Hansknecht (Nov 15 2023 at 01:51):

Luke Boswell (Nov 15 2023 at 01:54):

Brendan Hansknecht (Nov 15 2023 at 01:55):

Luke Boswell (Nov 15 2023 at 01:55):

If you regenerate the glue using this command roc glue ../roc/crates/glue/src/RustGlue.roc platform/glue2 platform/main-command-glue.roc then in glue2 folder rust analyser should complain

Brendan Hansknecht (Nov 15 2023 at 01:55):

Brendan Hansknecht (Nov 15 2023 at 02:01):

Also, what is the difference between glue and command_glue? command_glue doesn't look like it is fully generated with regular glue commands?

Brendan Hansknecht (Nov 15 2023 at 02:01):

Luke Boswell (Nov 15 2023 at 02:02):

Yeah, so that is another workaround. If you run glue, it doesn't generate any of the types for the Command module.

Brendan Hansknecht (Nov 15 2023 at 02:03):

Luke Boswell (Nov 15 2023 at 02:03):

I think it is because the mainForHost is mainForHost : Request -> Task Response [] and neither Request or Response include anything to do with Command module

Luke Boswell (Nov 15 2023 at 02:05):

Brendan Hansknecht (Nov 15 2023 at 02:16):

Luke Boswell (Nov 15 2023 at 02:22):

Luke Boswell (Nov 15 2023 at 02:24):

Richard Feldman (Nov 15 2023 at 20:22):

Ayaz Hafiz (Nov 16 2023 at 01:24):

Ayaz Hafiz (Nov 16 2023 at 01:43):

Anybody else able to load it? Im getting open /var/lib/docker/tmp/docker-import-143820044/repositories: no such file or directory

Ayaz Hafiz (Nov 16 2023 at 01:45):

Luke Boswell (Nov 16 2023 at 03:31):

What tooling do we need to compile a platform? Specifically if someone new was wanting to work with basic-webserver do they only need rust's cargo and a roc nightly?

Brendan Hansknecht (Nov 16 2023 at 06:21):

Folkert de Vries (Nov 21 2023 at 09:59):

memory allocation of 97165916604719789682: bytes failed

Anton (Nov 24 2023 at 18:55):

Today I learned you can also use > to quote on zulip, just like on github.
No need for the triple backticks with quote

Luke Boswell (Nov 24 2023 at 21:15):

Are there any plans to roll roc_fn into glue? I found that a really useful tool for basic-webserver. Would it be a good idea to copy it across into basic-cli too?

Brendan Hansknecht (Nov 24 2023 at 21:23):

Oh, was that written manually and not generated by glue? I thought that was part of folkerts new glue work.

Folkert de Vries (Nov 24 2023 at 21:26):

Anton (Nov 25 2023 at 10:39):

I found the solution to #6088 using the cursor.so editor, I described the problem, told it where to start looking and it found the bug :)
Definitely an easy bug but I'm still impressed.

Luke Boswell (Nov 25 2023 at 11:44):

@Folkert de Vries and I have been investigating issues with slow tests on Windows and have posted a question on the Zig discord. Basically we have identified that linking a dynamic library using zig build-lib is much slower than we expected -- 1,793ms vs 81ms.

Folkert de Vries (Nov 25 2023 at 23:41):

Luke Boswell (Nov 25 2023 at 23:47):

Folkert de Vries (Nov 26 2023 at 00:06):

Folkert de Vries (Nov 26 2023 at 00:07):

Richard Feldman (Nov 26 2023 at 00:08):

Brendan Hansknecht (Nov 27 2023 at 18:14):

Brendan Hansknecht (Nov 27 2023 at 18:54):

Folkert de Vries (Nov 28 2023 at 22:49):

Folkert de Vries (Nov 28 2023 at 22:50):

though I tried it on some other code and ... I'm confused and certainly from a code size standpoint can do better myself

Brendan Hansknecht (Dec 02 2023 at 02:37):

Brendan Hansknecht (Dec 02 2023 at 03:04):

Brendan Hansknecht (Dec 06 2023 at 16:47):

The backlog of things I want to work on keeps growing. I think I'm gonna need a bigger whiteboard to track this:
tasks.jpg

Brendan Hansknecht (Dec 06 2023 at 18:44):

Luke Boswell (Dec 12 2023 at 09:52):

Does anyone know what dynhost.pdb is, and where I might find it? I have generated a binary for Windows using --profiling and am running it in the debugger. Without using profiling I haven't seen this file requested before. I can still disassemble and step though the binary without it, but I'm guess this has more information related to the source.

Folkert de Vries (Dec 12 2023 at 09:55):

Luke Boswell (Dec 12 2023 at 09:56):

Folkert de Vries (Dec 12 2023 at 09:58):

we don't have a way to do it I think. it should be the same one as for the host?

Folkert de Vries (Dec 12 2023 at 09:58):

in other words, when the host gets compiled, it should spit out this file for itself too (zig/rust/ect should). maybe you can just rename it and use that?

Brendan Hansknecht (Dec 21 2023 at 18:34):

Just saw a link to the inko programming language. Interesting idea on memory management. All values have one owner. When the owner goes out of scope it is dropped. Default is move semantics when assigning to a new variable and such (to keep all values having exactly one owner). You can make as many references to a value as you want (mutable and immutable). If an object is freed and references are used later (maybe if they exist at all), it will crash.

So doubling down on everything has exactly one owner like you usually get in rust via ownership and borrowing, but without complex tracking. No need to rc or GC. Some minor tracking to crash at runtime for use after free.

Brendan Hansknecht (Dec 21 2023 at 18:36):

Trying to be like rust but a lot less complex and faster compiler times, but for that tradeoff, they lose some compile time guarantees and add runtime crashes. And then also a lot less runtime overhead than languages with rc or GC....not a tradeoff I would make, bit interesting to see

Folkert de Vries (Dec 21 2023 at 19:13):

I met its author at the rust meetup here, but we did not have a lot if time to discuss details . Would be interesting sometime

Folkert de Vries (Dec 21 2023 at 19:13):

Anton (Dec 29 2023 at 19:25):

Today I learned about Rust's sanitizer. If miri helped us find bugs, this may be useful as well.

Folkert de Vries (Dec 29 2023 at 20:13):

miri can't really handle programs that aren't written to be run with miri; it is very limited

Folkert de Vries (Dec 29 2023 at 20:13):

Folkert de Vries (Dec 29 2023 at 20:14):

Richard Feldman (Dec 29 2023 at 20:16):

Anton (Dec 30 2023 at 09:45):

Well, yes, I linked to the sanitizer earlier :sweat_smile:. But perhaps I misunderstand your question.

Richard Feldman (Dec 30 2023 at 12:34):

Luke Boswell (Jan 04 2024 at 23:49):

I think we should add a "Developing Platforms" guide for the website, and include discussion or summary about the current state of things.

After our plugin meeting earlier this morning, I had the thought that it would be helpful to communicate some of the known issues (like glue being in-development) and what the plan/vision is. I can draft something if this would be helpful.

Richard Feldman (Jan 04 2024 at 23:55):

sounds good! I think there may be other informational things floating around - maybe ask in channel?

Richard Feldman (Jan 05 2024 at 00:12):

Luke Boswell (Jan 05 2024 at 00:14):

Richard Feldman (Jan 05 2024 at 00:29):

Anton (Jan 05 2024 at 10:16):

The examples repo has sort of become a place for both examples and more in-depth explainers as well, should we just put it there? It's useful for users to have just one place they need to go to to find the answers to their questions.

Richard Feldman (Jan 05 2024 at 10:29):

interesting, although I think the language reference should really go on roc-lang.org rather than in a repo

Anton (Jan 05 2024 at 10:34):

Anton (Jan 05 2024 at 10:43):

Perhaps we can find a better name instead of "examples", but I don't think we should further split up our "knowledge libraries", we already have the tutorial, docs, examples, faq, website pages and everything spread out on zulip and github.

Richard Feldman (Jan 05 2024 at 13:04):

maybe we could just add lang ref entries in the same sidebar as the buildin modules

Richard Feldman (Jan 05 2024 at 13:04):

Richard Feldman (Jan 05 2024 at 13:05):

I want to have separate articles on things like how conditionals and pattern matching and such work, which don't fit into any particular builtin module, so I think we need more than just those - but maybe they can be located in the same place

Anton (Jan 05 2024 at 13:14):

Yeah, that's a good spot for it, I do think we should then do something like add a ToC at the top, or collapse the headers by default. Scrolling to search for information is annoying.

Anton (Jan 05 2024 at 13:21):

One option is to to give these articles their own space and to combat fragmentation; create a powerful search page on the website that can search through faq, examples repo, articles, tutorial, zulip and github. The one-stop page for Roc knowledge :)

Richard Feldman (Jan 05 2024 at 13:26):

as a reader, I have different motivations for wanting to view each of those, and I would prefer to have them separate

Richard Feldman (Jan 05 2024 at 13:26):

I think if we have some documentation that only exists in examples, that's a bug :big_smile:

Richard Feldman (Jan 05 2024 at 13:27):

I can see arguments for and against having all the language docs (both builtin modules and also the rest of the language reference) in the same place, but I like the idea of trying out having both in one place

Anton (Jan 05 2024 at 14:13):

I agreed with this at first but then I thought I could definitely see a user searching through all three (tutorial, examples, docs) to learn about a specific concept in Roc.

Anton (Jan 05 2024 at 14:20):

It does seem like that would require substantial remodeling of roc-lang.org/builtins to make it well integrated.

Brendan Hansknecht (Jan 08 2024 at 23:10):

Huge change that Rob really wishes that they had done: default int type should be an arbitrary sized int type. Of course, you can still have sized integer types with that.

Richard Feldman (Jan 08 2024 at 23:27):

interesting - the first benefit he mentioned was "security" - which I assume is because Go has wrapping overflow by default

Richard Feldman (Jan 08 2024 at 23:29):

I guess something along the lines of "when doing any arithmetic operation, on overflow upcast to arbitrary int and try again" but Go is statically typed - so what happens to the integer after that?

Richard Feldman (Jan 08 2024 at 23:31):

like what happens if I have a struct with a u8 in it, I take it out and multiply it by a gazillion and then try to put it back?

Richard Feldman (Jan 08 2024 at 23:32):

seems like just kicking the overflow can down the road? :face_with_raised_eyebrow:

Brendan Hansknecht (Jan 08 2024 at 23:53):

He is just saying that the default int type would be an arbitrary precision int that never overflows

Brendan Hansknecht (Jan 08 2024 at 23:54):

Probably would use 1 but to signify if It is a pointer to the real int or just an int stored locally on the stack.

Brendan Hansknecht (Jan 08 2024 at 23:55):

His point is just about changing the default to something safer and without overflow

Luke Boswell (Jan 09 2024 at 01:10):

Just looking at some things for roc docs and I'm wondering what we should use for the platform name? Currently we could use the platform "name" ... but under the module params proposal that changes to the following which no longer includes a string in the module header.

platform package [Stdout, Stdin, echo, read]
    requires [main]
    provides [mainForHost] to "prebuilt-hosts/"
    packages [
        Foo, Bar, Baz from "https://…",
        Something as Smt from "https://…",
    ]
    hosts [
        echo : Str -> Task {} [],
        read : Task Str [],
    ]

Luke Boswell (Jan 09 2024 at 01:11):

The root path is usually something like platform/main.roc which isn't very descriptive

Richard Feldman (Jan 09 2024 at 03:15):

Richard Feldman (Jan 09 2024 at 03:16):

as opposed to just a list of modules, and possibly some customizable introductory text (e.g. the doc comment for the package module)

Luke Boswell (Jan 09 2024 at 03:24):

I don't mind if we dont, I guess the main thing I was wanting was to be able to include doc comments in the platform module which don't currently generate

Luke Boswell (Jan 09 2024 at 03:25):

Isaac Van Doren (Jan 09 2024 at 04:09):

Ayaz Hafiz (Jan 09 2024 at 04:51):

Agus Zubiaga (Jan 09 2024 at 23:03):

Isaac Van Doren (Jan 10 2024 at 00:12):

Richard Feldman (Jan 10 2024 at 00:41):

Richard Feldman (Jan 10 2024 at 00:42):

Isaac Van Doren (Jan 10 2024 at 01:24):

Isaac Van Doren (Jan 10 2024 at 01:25):

Luke Boswell (Jan 11 2024 at 01:06):

I'm looking for help to find a name for a concept... most roc platforms will generate an executable, so roc build on your application and you have a program you can execute.

Some platforms are different and they are for making a plugin, so roc build produces a library instead which will ultimately be loaded and used by another program.

Is this a fair distinction between two use cases? are there others? What would be good names for these? Executable Platforms and Embedding Platforms?

Luke Boswell (Jan 11 2024 at 01:10):

I'm trying to research what I can to collect infromation for a guide on platform development

Luke Boswell (Jan 11 2024 at 01:30):

Zeljko Nesic (Jan 12 2024 at 10:29):

Concept-wise I would not use the word "plugin" to describe non-application artifact.

Brendan Hansknecht (Jan 15 2024 at 05:19):

In this specific case, N Tasks is fine, N + 1 is broken, but N + 2 is fine again.

Anton (Jan 15 2024 at 09:47):

Anton (Jan 15 2024 at 18:56):

I'm taking some time off and will be back on the 26th of January :wave:
I'll check my mentions and direct messages on zulip once a day for urgent stuff.

Brian Carroll (Jan 15 2024 at 19:02):

Richard Feldman (Jan 15 2024 at 19:25):

Brendan Hansknecht (Jan 16 2024 at 00:22):

Jared Cone (Jan 22 2024 at 00:30):

Hello, I'm checking in on Roc progress and I see it's able to build the compiler on Windows now which is awesome. Is the roc compiler itself able to work on Windows yet?

Brendan Hansknecht (Jan 22 2024 at 00:32):

@Luke Boswell would know a lot better, but it looks like we are getting close to enabling all the codegen tests: #6408

Luke Boswell (Jan 22 2024 at 00:51):

I've been using roc on Windows. All the LLVM tests pass now as of last night. :grinning: There is at least one bug with the surgical linker on Windows so you have to use --linker=legacy. Most of the current platforms have been developed without support for Windows, so there is a bit of work to upgrade them and test them out. The roc-wasm4 platform works really well though, and I've not had any issues with my zig platforms. I think we might want to track down and fix the surgical linker before we make a release?? though it hasn't been discussed yet.

Jared Cone (Jan 22 2024 at 02:46):

d:\External\Roc>target\debug\deps\roc build --linker=legacy Roctris\Roctris.roc

thread 'main' has overflowed its stack

Luke Boswell (Jan 22 2024 at 03:23):

Yes that looks right. What platform is roctris using? Ive had similar issues using basic cli

Jared Cone (Jan 22 2024 at 04:45):

It's a custom CLI platform from before the CLI example was a thing. Yea I get the same error when typing any expression into the repl

Brendan Hansknecht (Jan 22 2024 at 05:22):

Jared Cone (Jan 22 2024 at 05:59):

Definitely something fishy going on. Here I'm telling it to build an empty file:

d:\External\Roc>target\debug\deps\roc Roctris\Roctris.roc
←[36m── MISSING HEADER in Roctris\Roctris.roc ───────────────────────────────────────←[0m

I am expecting a header, but got stuck here:

←[36m1←[0m←[36m│←[0m
    ←[31m^←[0m

I am expecting a module keyword next, one of ←[32minterface←[0m, ←[32mapp←[0m, ←[32mpackage←[0m
or ←[32mplatform←[0m.

Jared Cone (Jan 22 2024 at 06:16):

d:\External\Roc>target\release\deps\roc build Roctris\Roctris.roc
🔨 Rebuilding platform...
An internal compiler expectation was broken.
This is definitely a compiler bug.
Please file an issue here: https://github.com/roc-lang/roc/issues/new/choose
thread '<unnamed>' panicked at 'Error:
    Failed to rebuild src/main.rs:
        The executed command was:
            rustup run nightly-2023-05-28 cargo build --bin host
        stderr of that command:
            error: toolchain 'nightly-2023-05-28-x86_64-pc-windows-msvc' is not installed
', crates\compiler\build\src\link.rs:1414:21
stack backtrace:
   0: std::panicking::begin_panic_handler
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library\std\src\panicking.rs:593
   1: core::panicking::panic_fmt
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library\core\src\panicking.rs:67
   2: roc_build::link::preprocess_host_wasm32
   3: roc_build::link::rebuild_host
   4: roc_build::program::build_file
   5: <anstyle::style::StyleDisplay as core::fmt::Display>::fmt
   6: <anstyle::style::StyleDisplay as core::fmt::Display>::fmt
   7: std::sys::windows::thread::impl$0::new::thread_start

Brendan Hansknecht (Jan 22 2024 at 06:26):

Second looks to be a real error. I think windows requires nightly currently for some rust stuff

Luke Boswell (Jan 22 2024 at 07:22):

Folkert de Vries (Jan 22 2024 at 16:42):

Jared Cone (Jan 23 2024 at 02:56):

Luke Boswell (Jan 23 2024 at 03:00):

Luke Boswell (Jan 23 2024 at 05:35):

Have we ever discussed having fuzzing within roc, maybe using something like expect? How might something like this work in the future?

Brendan Hansknecht (Jan 23 2024 at 05:36):

I know it was discuss at some point, but no idea where currently. We also discussed property based testing some.

Brendan Hansknecht (Jan 23 2024 at 05:37):

Probably a nice property based testing framework will be more useful to most users. Given roc is a safe language fuzzing should be less needed except for low level internals testing. Though both are useful and can be used together to some extent.

Luke Boswell (Jan 23 2024 at 05:42):

Brendan Hansknecht (Jan 23 2024 at 05:53):

Hmm...I think I could modify the roc fuzz platform to make a nice property based testing setup with multiple tests.

Of course baking it into roc would be best. Cause then you don't need a platform at all.

John Murray (Jan 23 2024 at 11:15):

Not much, ran into some issues with having the platform call arbitrary roc closures.

Brendan Hansknecht (Jan 23 2024 at 17:03):

One thing that is a bit sad is that fuzzing tools are not easier to setup and use. In a perfect world, all property testing tools would be based off of a fuzzing engine. Cause fuzzing uses coverage in the actual source code to explore inputs better.

Like property tests will never generate a proper gzip header (without explicit user guidance), but a fuzzer will.

That said, property testing is faster and generally doesn't store a bunch of data to disk. Fuzzing has to build a corpus to run well.

Luke Boswell (Jan 24 2024 at 08:50):

Richard Feldman (Jan 24 2024 at 13:37):

Brendan Hansknecht (Jan 24 2024 at 16:04):

Feels off to me. Sure, they spend the donations on paying contributors, but the money isn't going to the specific person who fixes the bounty. That person may not be on zigs payroll at all.

Brendan Hansknecht (Jan 24 2024 at 16:05):

Fabian Schmalzried (Jan 25 2024 at 10:09):

It's not about bugs, it's about features. I see it like this: if I want to spend some of my free time to implement something for a cool project, then I might as well make the project some money by implementing a bounty. If I spend my time that means I like the project, therefore I would be happy if the project would have more funding (even if I don't get any of it).

Brendan Hansknecht (Jan 25 2024 at 15:41):

Maybe it is more of a problem with how they worded/presented it. By talking about how the money is going towards paying contributors, it makes it sound like people who fix the issue or add the new feature will get some part of the value. They are contributors after all. This is only true if that person happens to be on zigs payroll
So I would label it as misleading at best. Not any sort of mal intent, but as a company donating, I think it would be easy to miss that point.

Fabian Schmalzried (Jan 26 2024 at 13:54):

I'm interested in how this turns out. How the whole zig community is reacting to this and if there even will be a significant amount of bounties.

Brendan Hansknecht (Jan 26 2024 at 15:36):

Rupert Smith (Jan 26 2024 at 17:15):

Hi, I have been conversing with @GordonBGood on the Elm Discourse and got onto the subject of how Roc optimizes away imutables and copying values sometimes by directly modifying structures on the heap. I have also been reading the papers on how this is implemented in Lean4 which seems to have quite a sophisticated implementation of it.
I was hoping to compare with the code for Roc.
Grepping the source I found this: https://github.com/roc-lang/roc/blob/main/crates/vendor/morphic_lib/src/api.rs#L176
Which comments on it, but isn't the implementation.
Sorry, I am not a Rust programmer so not sure how to find my way around... Can anyone point me to where the implementation is? Hoping there are maybe some comments I can read too to understand how it works, or failing that any docs or other sources I can consult (like a GitHub Issue or Pull Request)?
Thanks - first post here. I thought maybe I should post this in #beginners since I cannot even really claim that level of Roc ability!

Brian Carroll (Jan 26 2024 at 19:45):

This is not really in just one place. There are some aspects in the compiler and some in the standard library built-in functions.

One of the compiler features is reuse of allocations. Search for the Reuse variant of the monomorphic IR in crates/compiler/mono

Luke Boswell (Jan 26 2024 at 22:19):

Looks like WASI Preview 2 just launched. This will be fun to play with. :laughing:

Folkert de Vries (Jan 26 2024 at 22:45):

@Rupert Smith our implementation is now based on the one in Koka, but before that we basically implemented the Lean4 strategy. The current approach is better for algebraic data types, but has some nasty edge cases for our built-in lists and strings. So we'll need to move to more of a hybrid of the two systems.

the core of our logic is in crates/compiler/mono/src/inc_dec.rs and some related modules in the same folder like reset_reuse.rs.

Folkert de Vries (Jan 26 2024 at 22:45):

this is all based on a bunch of papers that I think are good for getting the ideas (going from the paper to an implementation is really hard though, so that's why it is very helpful that the lean and koka sources are available).

Folkert de Vries (Jan 26 2024 at 22:47):

Folkert de Vries (Jan 26 2024 at 22:50):

(we like these papers, have read them already, and are always happy to talk about them)

Rupert Smith (Jan 27 2024 at 09:24):

Rupert Smith (Jan 27 2024 at 09:31):

Interesting that all these papers are in the last 3 years - I might have thought this to have been a significant research area in FP in the 90s, whem computers were slower and had way less memory. I guess we didn't have the compiler infrastructure (LLVM etc) that we do now, and also a lot of research focussed on the theory side. MLton was highly optimised and never did this? But then ML did have mutation so I guess not needed.

The authors of the "counting immutable beans" paper suggest that their work is only scratching the surface of what ought to be possible.

Although, looking at the bilbiographies I see work by Hoffman in 2000 referenced and lots of other stuff around that time. I guess its just a topic that has come back for another round of consideration.

Folkert de Vries (Jan 27 2024 at 22:00):

it's an old idea, but does not work in most languages because it is so easy to create RC cycles: in languages with mutation it is trivial (java, ocaml) and in haskell laziness relies on cyclic values. So in most PL tradictions RC just did not make sense

Folkert de Vries (Jan 27 2024 at 22:00):

Luke Boswell (Jan 28 2024 at 01:04):

I was thinking maybe we should add this to RustGlue.roc for the benefit of all. Brendan pointed out that Buf and Body are hyper specific, but maybe there is a generic rust trait that would be more suitable.

Brendan Hansknecht (Jan 28 2024 at 01:09):

Looks like we get Buf through Cursor. To get Cursor just requires AsRef<[u8]> which we should be able to implement in general.

Brendan Hansknecht (Jan 28 2024 at 01:10):

Luke Boswell (Jan 28 2024 at 01:11):

Brendan Hansknecht (Jan 28 2024 at 18:47):

Folkert de Vries (Jan 28 2024 at 19:24):

Folkert de Vries (Jan 30 2024 at 14:29):

Kiryl Dziamura (Jan 30 2024 at 14:41):

I can run it locally, would it be helpful if I provide the IR file? or I can dig into the problem on my own but it will be very ineffective I think

anyway, let me check, maybe I can do smth about it. at least I already reproduced the problem. it's clear that it's an inconsistency between the expected return type and the actual one

Kiryl Dziamura (Jan 30 2024 at 15:30):

pub fn powC(arg1: RocDec, arg2: RocDec) callconv(.C) i128 {
    return @call(.always_inline, RocDec.pow, .{ arg1, arg2 }).num;
}

Folkert de Vries (Jan 30 2024 at 15:31):

Folkert de Vries (Jan 30 2024 at 15:34):

Folkert de Vries (Jan 31 2024 at 22:29):

Richard Feldman (Jan 31 2024 at 23:18):

Brendan Hansknecht (Feb 01 2024 at 06:22):

In this specific case, I am implementing TryFrom to go from a target_lexicon::Triple to a roc_target::Target. target_lexicon has tons of triples we don't support. So the error type would literally just be a single element enum if I create a full error type. Just UnsupportedTriple.

Brendan Hansknecht (Feb 01 2024 at 06:25):

I guess since there is only one possible failure, I could also just implement it with Err(())

Brian Carroll (Feb 01 2024 at 10:07):

If you make a single variant enum then it's equivalent to () but at least there's an informative name in the source code. And it's easy to extend later if needed.
I think that's our recommendation at work for Elm and Haskell code.

Brendan Hansknecht (Feb 01 2024 at 15:44):

Just feels kinda strange to make a one off super specific error enum. But I guess it doesn't hurt anything that it exists.

pub enum TargetFromTripleError {
    TripleUnsupported,
}

Richard Feldman (Feb 06 2024 at 01:10):

SIMD is wild: decoding a bunch of UTF-8 bytes faster than memcpy, and that's only using 128-bit SIMD

Brendan Hansknecht (Feb 06 2024 at 01:18):

I remember chatting with someone who is working on future CPUs and also in memory mini CPU like devices. He made a big complaint about memcpys being horridly slow compared to how fast it could be.

Richard Feldman (Feb 06 2024 at 03:04):

I heard there was some plan to do dedicated CPU instructions for memcpy in some mainstream chips

Richard Feldman (Feb 06 2024 at 03:04):

Brendan Hansknecht (Feb 06 2024 at 03:05):

With the right CPU and extensions for microcode, I think it tends to be a reasonable speed. That said, a number of CPUs it is worse than the generic simd versions

Brendan Hansknecht (Feb 06 2024 at 03:06):

Brendan Hansknecht (Feb 06 2024 at 03:07):

I think risc v has a generic use the largest simd you have then let me deal with the little bit of leftover bytes type vector instructions.

Brendan Hansknecht (Feb 06 2024 at 03:15):

This is starting with ice lake where rep mov should be pretty competitive but still isn't always the best on Intel.

Brendan Hansknecht (Feb 06 2024 at 03:25):

From a stack overflow post they suggest that a fully optimized and prefecthed sse2 memcopy can still be ~25% faster for large memory copies. That said it is decently old and theoretically rep.mov should be even better now. That said sse will be consistent across CPUs, rep mov will not, so I think a lot of implementations defensively use sse.

Brendan Hansknecht (Feb 06 2024 at 03:34):

Digging around a bit more, depending on exact CPU, small memcpys (which is most memcpys) that use rep mov could have anywhere from 7 to 50ish cycle of startup latency. For something that is under let's say 64 bytes, it will generally still be way faster to just use 8 byte wide mov operations that target registers or similar.

This is also why the advice is to avoid memcpys if you know the size (and it is small). The cost of looping for something like this can be pretty darn heavy.

Richard Feldman (Feb 06 2024 at 03:35):

Brendan Hansknecht (Feb 06 2024 at 03:39):

Not just that, but like the actual number of extra instructions between the memory movement instructions. Each cycle might load 8 instructions. Those 8 instructions might be check, conditional jump, load, store, inc counter, jump to top, 2 unused after loop instructions.

Instead it could be 4 load and 4 store operations for the entire copy if the move is 32 bytes and copied with registers.

Brendan Hansknecht (Feb 06 2024 at 03:40):

Loop unrolling of course helps some with this, but then you hit the branch misprediction issues more due to the loop running so few times.

Anton (Feb 07 2024 at 17:59):

Panicked during a panic. Aborting

Brendan Hansknecht (Feb 07 2024 at 18:00):

Richard Feldman (Feb 16 2024 at 03:10):

Brendan Hansknecht (Feb 16 2024 at 05:31):

Brendan Hansknecht (Feb 17 2024 at 03:00):

So they updated one of the examples in the Mojo vs Rust blog and this is actually a really useful footgun in rust to know of:

fn recursive(x: usize){
    if x == 0 {
        return;
    }
    let mut stuff = Vec::with_capacity(x);
    for i in 0..x {
        stuff.push(i);
    }
    recursive(x - 1)
}

Brendan Hansknecht (Feb 17 2024 at 03:00):

Another reason why rust is not really a true functional language (despite having many functional features). If you do a lot of true functional programming patterns you may hit cases like this that blow up.

Brendan Hansknecht (Feb 17 2024 at 03:01):

Essentially, the drop function for the Vec is run at the end of the scope. This means that it runs after the recursive call.

Brendan Hansknecht (Feb 17 2024 at 03:01):

Anton (Feb 17 2024 at 09:55):

Richard Feldman (Feb 17 2024 at 11:58):

Richard Feldman (Feb 17 2024 at 16:42):

Anton (Feb 17 2024 at 19:12):

I'd like to try out the llvm interpreter lli to help me debug something. Does anybody here have experience with it?

Richard Feldman (Mar 09 2024 at 13:19):

Anton (Mar 12 2024 at 16:24):

Brendan Hansknecht (Mar 12 2024 at 16:24):

When implemented, it will allow you to mock effects so that you can run an expect on an effectful function

Anton (Mar 12 2024 at 16:25):

Richard Feldman (Mar 12 2024 at 16:42):

Brendan Hansknecht (Mar 12 2024 at 16:44):

Richard Feldman (Mar 12 2024 at 18:21):

Richard Feldman (Mar 12 2024 at 18:22):

Brendan Hansknecht (Mar 12 2024 at 18:26):

Luke Boswell (Mar 14 2024 at 01:22):

I'm seeing this error

thread 'test::lookup_clone_result' panicked at 'misaligned pointer dereference: address must be a multiple of 0x8 but is 0x16dc05d74', crates/repl_expect/src/app.rs:57:45

. on this PR https://github.com/roc-lang/roc/pull/6586 but I'm pretty sure it is unrelated. (when I run cargo test locally on my mac, I've pushed to CI to see if that is just my setup)

Also, I wasn't sure if I should have committed all the glue/tests/fixtures/. Would appreciate if someone could confirm if that is the correct thing to do when they are changed.

Luke Boswell (Mar 15 2024 at 03:03):

There were a bunch of files generated and I'm not sure if they should be committed. This is the header from one of them

# ⚠️ READ THIS BEFORE MODIFYING THIS FILE! ⚠️
#
# This file is a fixture template. If the file you're looking at is
# in the fixture-templates/ directory, then you're all set - go ahead
# and modify it, and it will modify all the fixture tests.
#
# If this file is in the fixtures/ directory, on the other hand, then
# it is gitignored and will be overwritten the next time tests run.

Luke Boswell (Mar 15 2024 at 03:04):

I'm guessing I should remove these from the PR and update our gitignore so these aren't included

Brendan Hansknecht (Mar 15 2024 at 03:13):

Jonathan Schear (Mar 15 2024 at 15:22):

Jonathan Schear (Mar 15 2024 at 15:29):

Folkert de Vries (Mar 16 2024 at 13:17):

slightly more detailed benchmarks of what part of compilation takes so long in rust

Richard Feldman (Mar 16 2024 at 14:31):

Richard Feldman (Mar 16 2024 at 14:32):

I remain very confused why nobody on the Rust team is even talking about doing a direct to machine code backend

Richard Feldman (Mar 16 2024 at 14:33):

Brendan Hansknecht (Mar 16 2024 at 15:20):

Most surprising part of that article to me was how much work the rust frontend must do in release builds.

Brendan Hansknecht (Mar 16 2024 at 15:21):

I expected the release and dev full compilation builds to be more starkly different due to llvm eating way way more time.

Brendan Hansknecht (Mar 16 2024 at 15:22):

Brendan Hansknecht (Mar 16 2024 at 15:27):

I guess to be fair, he said it was using the default 16 different units. So llvm could be running on up to 16 threads. So if it was single threaded it would be a lot slower.

Richard Feldman (Mar 16 2024 at 15:29):

Brendan Hansknecht (Mar 16 2024 at 15:30):

Anton (Mar 16 2024 at 15:35):

Richard Feldman (Mar 16 2024 at 15:35):

yeah that's what @matklad was telling me - more, smaller crates improves caching

Richard Feldman (Mar 17 2024 at 11:18):

I don't think LLVM automatically bit-packs booleans (let alone tag unions) does it?

Folkert de Vries (Mar 17 2024 at 12:03):

Folkert de Vries (Mar 17 2024 at 12:04):

Richard Feldman (Mar 17 2024 at 12:33):

Richard Feldman (Mar 17 2024 at 15:20):

Brendan Hansknecht (Mar 17 2024 at 16:31):

That's kinda surprising. If every crate is split into 256 pieces, you would think that small crate size wouldn't matter. At least not much.

Richard Feldman (Mar 17 2024 at 16:34):

well in the sense of like - as opposed to having big crates with lots of modules

Richard Feldman (Mar 17 2024 at 16:34):

because things are cached at the level of crates and not modules, so more crates means a higher % of the build that's cached

Brendan Hansknecht (Mar 17 2024 at 16:49):

Sure, but assuming they are actually able to split into fine grain enough pieces (with incremental compilation having a goal of 256 pieces per crate), you would think that most things are split well enough to have decent incremental compilation. Cause even if you have 1024 functions in your crate, that is only compiling 4 functions per code gen unit. So a small change only has to recompile 4 functions (and things that depend on those 4 functions). I guess any crate 256 functions or less means function level caching and dependency graph.

Brendan Hansknecht (Mar 17 2024 at 16:50):

Brendan Hansknecht (Mar 19 2024 at 14:48):

Anton (Mar 19 2024 at 15:18):

Richard Feldman (Mar 19 2024 at 15:59):

Eli Dowling (Mar 19 2024 at 21:59):

Wow, very interesting read.
I think about this very time I use a language server :sweat_smile:. Like this should be able to run on the worst device imaginable, it's actually one of the reasons I started working on developer tooling, seeing that many folks would find some of these tools impossible to use because of their "everyone has a fast PC" design.

On that, I can confidently say rocls runs comfortably on my laptop in its power saving mode with the CPU limited to 800mhz, but we can definitely improve caching to make that even better.

Folkert de Vries (Mar 20 2024 at 08:38):

Brendan Hansknecht (Mar 21 2024 at 02:46):

Finally working on roc again. Continuing my changes to how we handle targets. I am now super deep in letting the types guide me. Very much type error driven development. Kinda satisfying, but also such a gigantic footprint.

Brendan Hansknecht (Mar 21 2024 at 02:49):

Also, I really want to remove target from type checking only flows. It shouldn't be needed, but ends up being required cause uses farther down the compiler need it. Might reorchestrate that. Would require passing it in later instead of storing it in a bunch of types. May not be worth it, but want to look into it more.

Richard Feldman (Mar 21 2024 at 10:42):

matklad (Mar 21 2024 at 12:48):

Not exactly: this is not as much about the _size_ of the crate, as about the topology of crate graph. If you have one thousand tiny crates which are linearly dependent on each other, this is more or less the same as one giant crate.

On the other hand, if you have a sort-of start topology with 1 central crate, and 10 mutually-independent supporting crates, than that's great for incremental.

Richard Feldman (Mar 21 2024 at 13:58):

sure, I meant smaller just in the sense of breaking up a larger crate to be more granular necessarily means making it smaller

Luke Boswell (Mar 22 2024 at 02:39):

I think we should make the --linker=legacy the default option for Windows for now as we know we have a surgical linker bug that segfaults. @Folkert de Vries what do you think?

Folkert de Vries (Mar 22 2024 at 17:03):

well we never really investigated that bug right? on the other hand, using the legacy linker as the default might be pragmatic for now

Folkert de Vries (Mar 22 2024 at 17:03):

Richard Feldman (Mar 23 2024 at 15:10):

Brendan Hansknecht (Mar 24 2024 at 04:36):

Anton (Mar 25 2024 at 09:05):

Folkert de Vries (Mar 25 2024 at 09:06):

$ time $(sleep 1 | sleep 1)

real    0m1.020s
user    0m0.002s
sys 0m0.006s

Brendan Hansknecht (Mar 25 2024 at 14:26):

I'm surprised more people don't know this. Pipe can only store like 4k characters by default. The inputting app will get stuck on IO if the next isn't processing fast enough.

Also, you want them all to start at the same time so that it runs faster as a whole without eating all your memory buffering.

This is also why you can pipe through a programming like tee and see live output.

Richard Feldman (Mar 25 2024 at 15:37):

Brendan Hansknecht (Mar 25 2024 at 23:52):

Not sure if anyone here has read more details of the new apple hardware vulnerability, but I hate how nice things that make CPUs faster are almost always the source of security vulnerabilities: https://gofetch.fail/

Brendan Hansknecht (Mar 25 2024 at 23:53):

In this case, this vulnerability was fully explored on apple hardware, but probably also affects the newest intel hardware.

Brendan Hansknecht (Mar 25 2024 at 23:54):

It probably will also affect other new hardware if they don't make changes/fixes. Apple was just ahead of the game on adding data dependent memory prefetchers to their cpus (part of the reason why the apple M chips are so fast)

Brendan Hansknecht (Mar 25 2024 at 23:54):

Basically when they load a cache line, they check it for things that look like pointers. If they find one, they prefetch it. Huge gain when loading a linked list for example.

Brendan Hansknecht (Mar 25 2024 at 23:55):

That said, I don't know how much this vulnerability matters. It has to be exploited by a process running locally on your machine. If an attacker has a process running locally on your machine, you are probably already screwed.

Folkert de Vries (Mar 26 2024 at 22:15):

Richard Feldman (Mar 26 2024 at 22:45):

Fabian Schmalzried (Mar 27 2024 at 09:45):

Luke Boswell (Mar 28 2024 at 08:45):

I've been wanting a thing to help me print debug parser issues. I've been using snapshot tests which are great, but don't quite seem to be enough.

So I started threading a boolean through the parser functions and didn't really stop to think if that was a good idea. It started as a helper so I could pass in from the top of a module and then I could print debug for just that modules information. Just wondering if this would be useful to keep around, or if I shouldn't include in a PR -- there's a lot of parser functions and when you touch one it kind of explodes to all of them.

Luke Boswell (Mar 28 2024 at 08:47):

I figure something like this might be optimised out of any release build if it is a constant false value and there was nothing using it.

Luke Boswell (Mar 28 2024 at 08:57):

For example I can do something like the following and it will only print out for the modules where I have passed in true.

    if print_debug {
        dbg!(&result);
    }

Brendan Hansknecht (Mar 28 2024 at 15:22):

A constant would be better. Optimizing away a parameter that is passed deep down the stack can happen but is unlikely.

Ayaz Hafiz (Mar 28 2024 at 18:08):

Richard Feldman (Mar 30 2024 at 22:20):

Richard Feldman (Mar 31 2024 at 04:42):

Brendan Hansknecht (Mar 31 2024 at 18:04):

-## This means the #U64 this function returns can always be safely converted to #I64 or #I32, depending on the target.
+## This means the #U64 this function returns can always be safely converted to #I64 or #I32, depending on the target.

Folkert de Vries (Mar 31 2024 at 18:05):

Brendan Hansknecht (Mar 31 2024 at 18:05):

Luke Boswell (Apr 01 2024 at 23:45):

I'm trying to run the fuzzer for the parser and a bit stuck with rust; does anyone have any pointers?

10:43:02 ~/Documents/GitHub/roc/crates/compiler/test_syntax/fuzz optional-unit-assign $ cargo +nightly fuzz run -j2 fuzz_expr -- -dict=../parse/fuzz/dict.txt
error: failed to parse manifest at `/Users/luke/Documents/GitHub/roc/crates/compiler/test_syntax/fuzz/Cargo.toml`

Caused by:
  error inheriting `version` from workspace root manifest's `workspace.package.version`

Caused by:
  `workspace.package.version` was not defined

Luke Boswell (Apr 02 2024 at 01:12):

Brendan Hansknecht (Apr 02 2024 at 01:34):

Fuzzers are good at finding the same bug in many ways, so be a bit careful with that.

Anton (Apr 02 2024 at 15:59):

Does the surgical linker require position independent executables(pie) as input?

Brendan Hansknecht (Apr 02 2024 at 16:57):

I would need to think about this more. That just means the executable can be loaded to anywhere in memory. That shouldn't strictly matter, but it probably does. My default thought is definitely yes.

Brendan Hansknecht (Apr 05 2024 at 01:28):

I'm finally starting to look at the big refcounting change for lists again. Where we need to pass the element dec functions into list functions so that when a list is free, all of the elements are decremented. Instead of doing recursive increfs and decrefs that kill performance everywhere.

I am so glad that llvm ir is strongly typed. Makes it list out all of the function signature mismatches for me. That said, still have hundreds of tests to fix....will be a journey.

Brendan Hansknecht (Apr 05 2024 at 01:49):

PASS [ 0.665s] test_gen::test_gen gen_list::basic_int_list_len
First heart beat. Got a super basic test passing. Now just to wire everything else up....

Brendan Hansknecht (Apr 05 2024 at 01:50):

~70 passing gen_list tests and ~130 failing.... What is that 35% passing. Not bad for most functions not being wired yet.

Richard Feldman (Apr 05 2024 at 02:24):

Brendan Hansknecht (Apr 05 2024 at 02:47):

Will probably be slow progress with mem leaks and segfaults, but pushing it forward

Brendan Hansknecht (Apr 05 2024 at 14:58):

This is actually going surprisingly fast (though I have only touched the llvm backend so far).

Luke Boswell (Apr 13 2024 at 04:02):

Anton (Apr 13 2024 at 09:52):

It needed a new release of basic-cli, the bundle is made now, so I expect we'll be able to merge it today

Anton (Apr 13 2024 at 09:58):

basic-cli also typically uses nightly Roc, but in this case we needed to use the branch, so I had to make some changes for that.

Luke Boswell (Apr 13 2024 at 11:18):

Luke Boswell (Apr 14 2024 at 03:46):

error: cvt doesn't compile for this platform yet
  --> /Users/m1ci/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cvt-0.1.2/src/lib.rs:22:9
   |
22 |         compile_error!("cvt doesn't compile for this platform yet");
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: could not compile `cvt` (lib) due to previous error

Luke Boswell (Apr 14 2024 at 04:18):

It seems like something that shouldn't have passed CI the first time, and not a flake.

Jackson Wambolt (Apr 14 2024 at 22:35):

I'm investigating #6641, and I'm curious why macros like skip_second are macros when they could be functions - I've only verified that for skip_second but it looks like its true for others like and and between. Wouldn't it be faster to compile a regular function? And any runtime performance loss from could be replicated with #[inline(always)].

Folkert de Vries (Apr 14 2024 at 22:36):

this _might_ be a renmant of the earlier parser where we wanted to use functions but that caused the rust compiler to just grind to a halt

Folkert de Vries (Apr 14 2024 at 22:36):

Folkert de Vries (Apr 14 2024 at 22:37):

the original problem was that types got too large but both the rust compiler and our parser implementation have improved in the meantime

Jackson Wambolt (Apr 14 2024 at 22:38):

Awesome, I'll see which ones can be replaced and if it compiles faster now I'll submit a PR

Folkert de Vries (Apr 14 2024 at 22:39):

I think functions are preferable it they are not slower (so if compile time stays the same that's also fine)

Richard Feldman (Apr 14 2024 at 22:45):

a lot of those used to be functions and then the rust type checker was giving "recursion limit reached" errors

Richard Feldman (Apr 14 2024 at 22:46):

the solution was to replace them with macros so that they didn't participate in type checking as functions

Anton (Apr 15 2024 at 09:29):

@Luke Boswell on what OS are you running this? Based on the relevant source it should work on macos, linux and windows...

Luke Boswell (Apr 15 2024 at 09:39):

Luke Boswell (Apr 15 2024 at 09:40):

Anton (Apr 15 2024 at 09:52):

It fails when building for wasm so the relevant source code makes sense at least. There were some changes to dependencies in that PR so I think cvt is now accidentally included in the wasm build and the cvt functionality doesn't make sense for wasm.

Luke Boswell (Apr 15 2024 at 09:53):

Anton (Apr 15 2024 at 09:53):

Luke Boswell (Apr 15 2024 at 09:55):

[package]
name = "roc_error_macros"
description = "Provides macros for consistent reporting of errors in Roc's rust code."

authors.workspace = true
edition.workspace = true
license.workspace = true
version.workspace = true

[dependencies]

[package]
name = "roc_test_utils"
description = "Utility functions used all over the code base."

authors.workspace = true
edition.workspace = true
license.workspace = true
version.workspace = true

[dependencies]
pretty_assertions.workspace = true
remove_dir_all.workspace = true

[dev-dependencies]

Anton (Apr 15 2024 at 09:56):

It may be further down the dependency tree, remove_dir_all probably does not make sense for wasm

Luke Boswell (Apr 15 2024 at 09:58):

[target.'cfg(not(windows))'.dependencies]
cvt = "0.1.1"
libc = "0.2"

Luke Boswell (Apr 15 2024 at 09:59):

Just compare strings normally. It's just to show pretty colors for the snapshots

Luke Boswell (Apr 15 2024 at 10:00):

Anton (Apr 15 2024 at 10:02):

We could also split some test_utils off into test_utils_no_wasm, there we can put all the dir stuff in.

Luke Boswell (Apr 15 2024 at 10:38):

Luke Boswell (Apr 21 2024 at 01:53):

It looks like glue has regressed somehow, or at least there is a bug in module imports which prevents us from generating the glue types for roc-wasm4.

$ roc glue ../roc/crates/glue/src/ZigGlue.roc platform/glue/ platform/main-glue.roc

── MODULE NOT IMPORTED in ../roc/crates/glue/src/../platform/Types.roc ─────────

The `TypeId` module is not imported:

37│          id = TypeId.fromU64 index
                  ^^^^^^^^^^^^^^

Did you mean to import it?

Luke Boswell (Apr 21 2024 at 01:55):

Luke Boswell (Apr 24 2024 at 02:14):

Just discovered that using dbg statements can mess with ! suffixes... needs further investigation.

Anton (Apr 24 2024 at 10:10):

Luke Boswell (Apr 24 2024 at 10:18):

Brendan Hansknecht (Apr 29 2024 at 04:45):

Looking at timing of our compilation, almost all of the frontend time is spent in "Other". do we know what that is likely to be? Is it mono? Something else?

            0.036 ms   Read .roc file from disk
            0.045 ms   Parse header
            0.147 ms   Parse body
            1.059 ms   Canonicalize
            0.000 ms   Constrain
            1.395 ms   Solve
            0.058 ms   Find Specializations
            1.337 ms   Make Specializations (Pass 0)
            0.170 ms   Make Specializations (Pass 1)
            0.030 ms   Make Specializations (Pass 2)
           15.413 ms   Other

Ayaz Hafiz (Apr 29 2024 at 21:50):

Brendan Hansknecht (Apr 29 2024 at 21:56):

Even our basic rocLovesZig example where main is just a string spends most of the time in other across essentially all modules. Though I guess it is compiling all of the standard library.

timings

Here's how long each module took to compile:

    Bool
            0.000 ms   Read .roc file from disk
            0.000 ms   Parse header
            0.098 ms   Parse body
            0.144 ms   Canonicalize
            0.000 ms   Constrain
            0.020 ms   Solve
            0.013 ms   Find Specializations
            0.010 ms   Make Specializations
            1.003 ms   Other

            1.289 ms   Total

    #Derived_gen
            0.000 ms   Read .roc file from disk
            0.000 ms   Parse header
            0.000 ms   Parse body
            0.000 ms   Canonicalize
            0.000 ms   Constrain
            0.000 ms   Solve
            0.001 ms   Find Specializations
            0.005 ms   Make Specializations
            0.000 ms   Other

            0.000 ms   Total

    Str
            0.000 ms   Read .roc file from disk
            0.000 ms   Parse header
            0.580 ms   Parse body
            0.796 ms   Canonicalize
            0.000 ms   Constrain
            0.076 ms   Solve
            0.053 ms   Find Specializations
            0.023 ms   Make Specializations
            4.636 ms   Other

            6.164 ms   Total

    Dict
            0.000 ms   Read .roc file from disk
            0.000 ms   Parse header
            2.035 ms   Parse body
            3.452 ms   Canonicalize
            0.000 ms   Constrain
            0.247 ms   Solve
            0.226 ms   Find Specializations
            0.039 ms   Make Specializations
            4.616 ms   Other

           10.615 ms   Total

    List
            0.000 ms   Read .roc file from disk
            0.000 ms   Parse header
            1.550 ms   Parse body
            2.359 ms   Canonicalize
            0.000 ms   Constrain
            0.136 ms   Solve
            0.140 ms   Find Specializations
            0.030 ms   Make Specializations
            0.995 ms   Other

            5.209 ms   Total


            0.021 ms   Read .roc file from disk
            0.015 ms   Parse header
            0.503 ms   Parse body
            0.044 ms   Canonicalize
            0.000 ms   Constrain
            0.415 ms   Solve
            0.017 ms   Find Specializations
            0.068 ms   Make Specializations
           13.683 ms   Other

           14.766 ms   Total

    Num
            0.000 ms   Read .roc file from disk
            0.000 ms   Parse header
            0.605 ms   Parse body
            1.051 ms   Canonicalize
            0.000 ms   Constrain
            0.185 ms   Solve
            0.150 ms   Find Specializations
            0.042 ms   Make Specializations
            0.622 ms   Other

            2.656 ms   Total

    Result
            0.000 ms   Read .roc file from disk
            0.000 ms   Parse header
            0.087 ms   Parse body
            0.098 ms   Canonicalize
            0.000 ms   Constrain
            0.044 ms   Solve
            0.030 ms   Find Specializations
            0.006 ms   Make Specializations
            1.042 ms   Other

            1.306 ms   Total

    #UserApp
            0.036 ms   Read .roc file from disk
            0.028 ms   Parse header
            0.002 ms   Parse body
            0.044 ms   Canonicalize
            0.000 ms   Constrain
            0.516 ms   Solve
            0.005 ms   Find Specializations
            0.034 ms   Make Specializations
           13.651 ms   Other

           14.317 ms   Total

    Decode
            0.000 ms   Read .roc file from disk
            0.000 ms   Parse header
            0.130 ms   Parse body
            0.181 ms   Canonicalize
            0.000 ms   Constrain
            0.210 ms   Solve
            0.009 ms   Find Specializations
            0.006 ms   Make Specializations
            4.937 ms   Other

            5.475 ms   Total

    Inspect
            0.000 ms   Read .roc file from disk
            0.000 ms   Parse header
            0.454 ms   Parse body
            0.634 ms   Canonicalize
            0.000 ms   Constrain
            0.267 ms   Solve
            0.034 ms   Find Specializations
            0.017 ms   Make Specializations
            5.733 ms   Other

            7.139 ms   Total

    Box
            0.000 ms   Read .roc file from disk
            0.000 ms   Parse header
            0.039 ms   Parse body
            0.100 ms   Canonicalize
            0.000 ms   Constrain
            0.017 ms   Solve
            0.032 ms   Find Specializations
            0.011 ms   Make Specializations
            1.074 ms   Other

            1.273 ms   Total

    Encode
            0.000 ms   Read .roc file from disk
            0.000 ms   Parse header
            0.095 ms   Parse body
            0.261 ms   Canonicalize
            0.000 ms   Constrain
            0.178 ms   Solve
            0.010 ms   Find Specializations
            0.017 ms   Make Specializations
            2.596 ms   Other

            3.156 ms   Total

    Set
            0.000 ms   Read .roc file from disk
            0.000 ms   Parse header
            0.349 ms   Parse body
            0.539 ms   Canonicalize
            0.000 ms   Constrain
            0.185 ms   Solve
            0.037 ms   Find Specializations
            0.017 ms   Make Specializations
           10.311 ms   Other

           11.438 ms   Total

    Hash
            0.000 ms   Read .roc file from disk
            0.000 ms   Parse header
            0.133 ms   Parse body
            0.152 ms   Canonicalize
            0.000 ms   Constrain
            0.025 ms   Solve
            0.011 ms   Find Specializations
            0.012 ms   Make Specializations
            5.887 ms   Other

            6.219 ms   Total

 (note, total time ~20ms)

Brendan Hansknecht (Apr 29 2024 at 22:01):

We are measuring time spent waiting on dependencies as part of compiling a module
List: 5.2ms
Dict (which depends on list): 10.6ms with 4.6ms other
Set (which depends on dict): 11.4ms with 10.3ms other

UserApp depends on everything else but is just a string and super trivial. So it has 14.317 ms total and 13.651 ms for other. I think the 13.651 ms was waiting on all deps.

Ayaz Hafiz (Apr 29 2024 at 22:09):

yeah that makes sense. the earlier phases cannot really be parallelized across the entire dependency graph, and even mono cannot really be, so the wall time for a single module is likely to be much higher than the actual work done for it

Luke Boswell (Apr 29 2024 at 23:19):

It the process of updating roc-json to the latest basic-cli release I've discovered a strange compiler bug I think. Haven't tracked down what the issue is, but have been able to isolate it to the package/Option.roc module.

$ RUST_BACKTRACE=1 roc test package/Option.roc
thread 'main' panicked at /Users/luke/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bumpalo-3.14.0/src/lib.rs:1854:5:
out of memory
stack backtrace:
   0: rust_begin_unwind
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:595:5
   1: core::panicking::panic_fmt
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/panicking.rs:67:14
   2: bumpalo::oom
   3: roc_repl_eval::eval::addr_to_ast
   4: roc_repl_eval::eval::struct_to_ast
   5: roc_repl_expect::get_values
   6: roc_repl_expect::run::render_expect_failure
   7: roc_cli::test
   8: roc::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Luke Boswell (Apr 29 2024 at 23:21):

Luke Boswell (Apr 30 2024 at 11:49):

Is there any particular reason we use a host.c in basic-cli and build that using clang, and then link using ld to get a prebuilt binary when "rebuilding platform"? Im wondering if I could replace this with another crate to produce the static library using just cargo?

Luke Boswell (Apr 30 2024 at 11:50):

In the (near) future I'm hoping to remove this rebuilding platform from the cli, and so the platform needs to producs the prebuilt binary / static library itself.

Richard Feldman (Apr 30 2024 at 11:54):

Richard Feldman (Apr 30 2024 at 11:57):

basically host.c is a workaround way to get a .o executable out of Rust, because apparently there's no way to convince pure rust to compile an executable into a .o file

Richard Feldman (Apr 30 2024 at 11:57):

or at least that's the only way we were able to figure out how to do it in that forum :sweat_smile:

Luke Boswell (Apr 30 2024 at 17:28):

Ok nice, thats good to know. Though we've since added support for using archives .a and cargo happily produces them with staticlib crates, so we could do it all with cargo now I think.

Brendan Hansknecht (Apr 30 2024 at 17:31):

IIRC, the problem isn't producing a .a/o it's that the .a/o needs to contain everything to generate the executable. I don't think cargo is willing to put a main function in a .a.

Luke Boswell (Apr 30 2024 at 17:36):

Though the issue then is using that library in a bin crate that also has a main, in which case there is duplicate symbols called main.

I'm not suggesting the way Its currently written in basic-ssg is the right way, because I think that was a mistake. But I think it's possible. So I am thinking we might be able to have another crate that depends on the core library and just adds a main fn, similar to how its done in main.rs

Luke Boswell (May 01 2024 at 07:01):

One approach I can think of is to include a script alongside each platform called build.roc and when we run any of the tests, we first run this script which is responsible for rebuilding the host.

For most platforms, this is as simple as running zig build-lib -lc host.c to produce a c-archive and then running mv libhost.a macos-arm64.a to rename it for the current architecture.

Alternatively we could leave the current rebuilding host functionality, and rewire it so that it is called before running a test to ensure we have the prebuilt binaries available.

Luke Boswell (May 01 2024 at 07:18):

Richard Feldman (May 01 2024 at 12:23):

I think if the tests are in Rust it'll be best if we keep that logic in Rust, as like a test helper function

Richard Feldman (May 01 2024 at 12:24):

otherwise we get into situations like "the actual problem was that the .roc file to build the test's platform didn't run because that .roc file's platform was broken because of an in-progress change to the current compiler code base..."

Luke Boswell (May 01 2024 at 18:21):

Luke Boswell (May 01 2024 at 18:23):

Where we currently build the host anyway, except the logic to do that is in the compiler.

Brendan Hansknecht (May 01 2024 at 18:49):

Luke Boswell (May 01 2024 at 19:41):

Brendan Hansknecht (May 01 2024 at 19:48):

I have no major thought on if the build script should be in roc vs in whatever the native language would use. I guess I prefer native language if possible so the platforms feel more familiar to people who use the host language. That said, a lot of the platforms are super tiny and some languages like c/c++ don't have clear build tooling. Also can't just use a bash script due to eventually needing to support Windows. So that pushes for build.roc for simple consistency.

Richard Feldman (May 01 2024 at 20:08):

do you mean in tests in the roc-lang/roc repo specifically? Or for other projects?

Richard Feldman (May 01 2024 at 20:08):

personally I have no preference on what other projects want to use (if they want to use build.roc, great!)

Brendan Hansknecht (May 01 2024 at 20:08):

Richard Feldman (May 01 2024 at 20:08):

Richard Feldman (May 01 2024 at 20:09):

actually I guess it's probably fine to try out build.roc in the roc-lang/roc tests too and see if it's a problem in practice

Brendan Hansknecht (May 01 2024 at 20:09):

Fundamentally, roc no longer builds the host. It is up to the host to decide how it will get built. Roc will essentially run as if --prebuilt-playform is always true.

Richard Feldman (May 01 2024 at 20:09):

Brendan Hansknecht (May 01 2024 at 20:11):

Also, thinking about this more, I think I actually like forcing everything to be build.roc. Then I don't need to figure out the build script/command. always just roc build.roc

Brendan Hansknecht (May 01 2024 at 20:11):

Luke Boswell (May 01 2024 at 23:32):

For the changes I'm working on to remove the platform rebuilding... I've hit a problem now where I need glue gen to build the platforms, but to gen glue I need to be able to build platforms. It's a vicious cycle.

So I think the best approach is to make a release of the glue platform just like any other platform. We could a URL for the glue platform in the GH releases.

TBH I've been thinking about this for other reasons anyway. Currently whenever I use glue elsewhere I have to have roc repository cloned locally to reference the platform which isn't ideal.

Richard Feldman (May 01 2024 at 23:43):

Luke Boswell (May 02 2024 at 06:26):

I hadn't realised how big of a change this is. I apologise in advance for how large this PR is going to be. I'm trying to keep it reasonable, but I'm also trying to clean things up as I go and leave it in a better state.

Luke Boswell (May 02 2024 at 06:27):

For example, I've had to touch all of the roc_cli tests and update the platform implementations

Richard Feldman (May 02 2024 at 11:06):

Luke Boswell (May 03 2024 at 09:08):

Luke Boswell (May 03 2024 at 09:10):

Luke Boswell (May 03 2024 at 09:27):

Luke Boswell (May 03 2024 at 09:31):

In the space of about 15-30mins I was able to update roc-json, roc-parser, roc-ansi, and roc-random to use the new syntax. Super quick and painless upgrade. Thank you @Agus Zubiaga for putting the effort in to make that possible.

JRI98 (May 03 2024 at 09:37):

Luke Boswell (May 03 2024 at 09:38):

Agus Zubiaga (May 03 2024 at 10:50):

Ah, interesting. Wasn’t the annotation already mandatory in the old syntax? I guess we can make it optional and have it behave as if you wrote:

import "data.json" as data : _

Agus Zubiaga (May 03 2024 at 10:55):

Richard Feldman (May 03 2024 at 10:56):

I like the type annotation being optional here, because type annotations are optional for all other values :big_smile:

Agus Zubiaga (May 05 2024 at 19:33):

Hm, this is not as easy as I thought. The way solve works for ingested files means that:

import "data.json" as data : _

import "data.json" as data : List U8

Instead, I think we want data to be inferred as Str or List U8 from usage.

Richard Feldman (May 05 2024 at 19:41):

Richard Feldman (May 05 2024 at 19:42):

Agus Zubiaga (May 05 2024 at 19:42):

Yeah, I'll probably need some because I don't have any experience in this part of the compiler.

Richard Feldman (May 05 2024 at 19:43):

I'd like to use this feature in the live coding on Tuesday :grinning_face_with_smiling_eyes:

Richard Feldman (May 05 2024 at 19:44):

Agus Zubiaga (May 05 2024 at 19:44):

Agus Zubiaga (May 05 2024 at 19:45):

If it's ok, I'd like to continue working on params before shifting focus to the inference part

Agus Zubiaga (May 05 2024 at 19:45):

Richard Feldman (May 05 2024 at 19:46):

Agus Zubiaga (May 05 2024 at 19:49):

Agus Zubiaga (May 05 2024 at 20:06):

Agus Zubiaga (May 05 2024 at 20:07):

Luke Boswell (May 12 2024 at 18:03):

Ayaz Hafiz (May 13 2024 at 00:03):

I think we should make a repository called "rfcs" or something similar to keep design documents of the sort typically shared in ideas. I used to do it in Notion, and I know Richard sometimes does them in Google Docs, but for me at least writing in google docs is quite laborious. It would also be good to centralize this. Any thoughts?

Richard Feldman (May 13 2024 at 01:16):

yeah I totally agree! Someone talked about this at some point in the past but I don't know how much progress was actually made on it :sweat_smile:

Ayaz Hafiz (May 13 2024 at 01:20):

Anton (May 13 2024 at 08:12):

Luke Boswell (May 13 2024 at 09:13):

Ok, well I've been converting the platforms to build using roc as the scripting language. Maybe we should use bash instead then? I think Roc is nicer for the scripting, and will be even better soon with builtin Task and we can move a bunch of duplicate logic into a package.

Brendan Hansknecht (May 13 2024 at 09:19):

Anton (May 13 2024 at 09:20):

We could build the compiler first and add roc to the path at the start of the CI workflow

Brendan Hansknecht (May 13 2024 at 09:20):

So even if you don't do roc build.roc, you still have a call to roc in the equivalent build.sh script

Brendan Hansknecht (May 13 2024 at 09:20):

So either way, the user has to build roc and add it to the path (or make it otherwise accessible)

Brendan Hansknecht (May 13 2024 at 09:21):

Luke Boswell (May 13 2024 at 09:39):

Well the current approach is that each test will rebuild the platform when running the example. So it doesn't really use roc as a separate fork or child process or anything.

Luke Boswell (May 13 2024 at 09:39):

Richard Feldman (May 20 2024 at 17:49):

it's really cool how many people in #beginners are looking to make their own platforms :smiley:

Anton (May 20 2024 at 17:54):

Richard Feldman (May 26 2024 at 12:51):

Brendan Hansknecht (May 26 2024 at 13:50):

Floating point sum reductions (same thing as the summation) happen in machine learning quite often. In a neutral net that I was debugging, I noticed that one ml framework had significantly less accurate results for the neural net as a whole. After digging into the network, I realized that the results deviated the most when reaching a reduction. I originally assumed the framework was cutting corners in reduction in the name of performance. It wasn't. Turns out that the framework actually had the most accurate floating point reductions. So a more numerically accurate floating point reduction made the neural network as a whole noticeably less accurate. Why? The neural net was trained with a less accurate reduction and had optimized the weights for a less accurate reduction.

Richard Feldman (May 26 2024 at 17:25):

Brendan Hansknecht (May 26 2024 at 18:16):

Definitely could. Though if floats are for speed, I don't think we should really care about summation accuracy. If anything, I would argue for dumb simd and more speed instead of accuracy.

Richard Feldman (May 26 2024 at 19:20):

Luke Boswell (Jun 03 2024 at 03:11):

    Code Generation
         5737.190 ms   Generate final IR from Mono IR
        16199.271 ms   Generate object

        21936.461 ms   Total

Finished compilation and code gen in 28207 ms

Produced a app.o file of size 999952

Finished linking in 1115 ms

0 errors and 13 warnings found in 29325 ms

Luke Boswell (Jun 03 2024 at 03:39):

thread 'tokio-runtime-worker' panicked at 'The Roc app crashed with: Erroneous: Expr::Call', src/lib.rs:46:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

    Code Generation
        13051.603 ms   Generate final IR from Mono IR
        35184.772 ms   Generate object

        48236.376 ms   Total

Finished compilation and code gen in 66731 ms

Produced a app.o file of size 965568

Finished linking in 4643 ms

0 errors and 13 warnings found in 71376 ms

Luke Boswell (Jun 03 2024 at 03:41):

These aren't optimised builds... just running with $ DB_PATH=test.db roc --time src/main.roc

Luke Boswell (Jun 03 2024 at 04:32):

Trying to figure out what's causing this... I think I'm going backwards :sweat_smile:
0 errors and 6 warnings found in 179487 ms <-- 180 seconds

Luke Boswell (Jun 03 2024 at 05:17):

Looks to be mac specific. With my (tiny) linux server
0 errors and 6 warnings found in 3671 ms

Luke Boswell (Jun 03 2024 at 09:43):

Ok, so I've now got it working normally 0 errors and 6 warnings found in 2853 ms

I cleaned basically every cache I could find. Upgraded my MacOS version, updated all the packages I could find, and rebuilt roc and roc_ls from latest main using nix.

Richard Feldman (Jun 03 2024 at 16:30):

Luke Boswell (Jun 03 2024 at 21:56):

Yeah, I was a little worried for a bit. I suspect I had done something strange like built roc from a branch somewhere or there was some combination of things in caches. Glad it's back to normal.

Luke Boswell (Jun 04 2024 at 01:09):

Actually, I think it's still a problem. It's like every time I compile my app it gets a bit slower, and slower to build.

Richard Feldman (Jun 04 2024 at 02:01):

hm, what's really odd about that is that we don't currently store anything on disk during builds

Richard Feldman (Jun 04 2024 at 02:01):

Luke Boswell (Jun 04 2024 at 02:03):

Luke Boswell (Jun 04 2024 at 02:05):

Richard Feldman (Jun 04 2024 at 02:06):

Luke Boswell (Jun 04 2024 at 02:08):

I definitely "fixed" it when I rebuilt everything, and it was fine on my linux server. I'll keep playing with it and see if I can find any correlations between things.

Luke Boswell (Jun 04 2024 at 02:09):

I am doing a lot with Tasks in basic-webserver, so it could be something related to nesting those maybe.

Luke Boswell (Jun 04 2024 at 02:32):

I think I know what the issue might be. The language server looks to be hogging all the CPU

Luke Boswell (Jun 04 2024 at 02:33):

If I turn it off I'm back down to a more normal 0 errors and 0 warnings found in 18753 ms

Richard Feldman (Jun 04 2024 at 03:00):

Luke Boswell (Jun 04 2024 at 03:05):

Folkert de Vries (Jun 04 2024 at 08:27):

Brendan Hansknecht (Jun 06 2024 at 20:02):

Brendan Hansknecht (Jun 07 2024 at 04:10):

Two other interesting reads. Mojo is testing new areas related to simple ownership (or at least more automatic and less explicit)

Luke Boswell (Jun 07 2024 at 08:46):

Found a new class of bug I think Roc app crashed with: voided tag constructor is unreachable commit

Folkert de Vries (Jun 28 2024 at 16:15):

that needs some cleanups, but hopefully this'll fix a bunch of performance problems

Luke Boswell (Jun 30 2024 at 11:12):

We were experiencing some hard to track down bugs on basic-cli... which we suspect are from rust glue generated code in roc_app.

So we used a PR on roc-platform-template-zig to test the implementation instead. It looks like task as builtin is good to go. :tada:

We had issues not that long ago with the glue types in basic-cli when implementing the API changes just before SYCL. I had removed a lot of them around that time.

Sam and I have been working on removing the remaining few glue types in #221 and replacing these with roc_std or hand rolled rust types. This should eliminate this as a significant variable for debugging, and hopefully even eliminate the bug and unlock builtin task.

Richard Feldman (Jun 30 2024 at 12:01):

Sam Mohr (Jun 30 2024 at 12:04):

It turns out that Tasks aren't super hard to implement because there's nothing going on under the hood, really, it's just a lambda returns a result. They just happen to work very well as a contract for IO management, which we can enforce for free using opaque types!

Luke Boswell (Jul 01 2024 at 05:13):

I've been upgrading platforms for Task as a builtin... and just couldn't wait to test out platform independent packages (even though we don't quite have module params yet).

# /package/main.roc
package [
    ReversePrint
] {}

# /package/ReversePrint.roc
module [line]

line : Str, (Str -> Task a b) -> Task a b
line = \msg, echo ->
    reversed =
        msg
        |> Str.toUtf8
        |> List.reverse
        |> Str.fromUtf8
        |> Result.withDefault "BAD UTF8"

    echo reversed

# example application
app [main] {
    pf: platform "platform/main.roc",
    test: "package/main.roc",
}

import pf.Stdout
import test.ReversePrint

main = ReversePrint.line "Roc loves Zig" Stdout.line

$ roc test.roc
🔨 Rebuilding platform...
giZ sevol coR
Runtime: 0.027ms

Agus Zubiaga (Jul 01 2024 at 19:48):

That's awesome! I can't wait to rewrite roc-pg with Task as a builtin and params

Luke Boswell (Jul 02 2024 at 12:18):

It's a bit of a different direction to effectively remove platform rebuilding from the cli for end users.

I'm also trying to add some documentation in the build pipeline and refactor as much as I can.

Luke Boswell (Jul 02 2024 at 23:54):

Luke Boswell (Jul 02 2024 at 23:55):

Luke Boswell (Jul 02 2024 at 23:56):

Luke Boswell (Jul 03 2024 at 00:11):

Joshua Warner (Jul 04 2024 at 02:09):

Possibly quick question: anyone know why AnnotatedBodys exist in the AST, instead of being separate type def + Body? What's the intuition behind why those are paired up?

Luke Boswell (Jul 04 2024 at 02:36):

Richard Feldman (Jul 04 2024 at 12:11):

Folkert de Vries (Jul 04 2024 at 12:12):

I think that's for making can easier, and possibly error message generation too?

Folkert de Vries (Jul 04 2024 at 12:13):

Folkert de Vries (Jul 04 2024 at 12:14):

if there's good reasons to refactor that now, then if we can keep generating good error messages I don't think there is a reason not to do that refactor

Kiryl Dziamura (Jul 05 2024 at 19:31):

just a note, AnnotatedBody is the only way to define a type annotation for a variable right now

Kiryl Dziamura (Jul 07 2024 at 20:47):

Kiryl Dziamura (Jul 07 2024 at 20:53):

Folkert de Vries (Jul 07 2024 at 21:04):

yes that strangeloop talk is on my list of talks to watch before giving a talk: on a second (or more) watch you start to notice some of the timing and structure that makes it so good while it treats a bunch of hard technical subject matter

Ayaz Hafiz (Jul 08 2024 at 03:59):

Been working with a lot of retail distribution data lately, and there is one distribution center owned by UNFI called UNFI ROC (Rocklin, CA) - always makes me think of this community

Sam Mohr (Jul 08 2024 at 03:59):

Ayaz Hafiz (Jul 08 2024 at 03:59):

Luke Boswell (Jul 08 2024 at 04:00):

Ayaz Hafiz (Jul 08 2024 at 04:00):

I will say the retail supply chain could use a lot of love in terms of software products, so totally maybe

Luke Boswell (Jul 08 2024 at 04:01):

Ironically, I'm currently staring at Roc code that attempts to improve the situation in supply chain management

Luke Boswell (Jul 08 2024 at 04:01):

Sam Mohr (Jul 08 2024 at 04:15):

Ayaz Hafiz (Jul 08 2024 at 04:16):

Brendan Hansknecht (Jul 08 2024 at 21:17):

So not that it is particularly meaningful, but I finally have actually committed to llvm. I have a single commit that adds like 10 lines to expose and mlir c API that I wanted for debugging.

Sam Mohr (Jul 08 2024 at 21:18):

How quick is their release cycle? Will you be able to use it anytime soon without building LLVM from source?

Brendan Hansknecht (Jul 08 2024 at 21:19):

My job builds llvm from source, so should be pulled in for our weekly update. Otherwise, llvm releases like every 6 months

Brendan Hansknecht (Jul 08 2024 at 21:21):

Brendan Hansknecht (Jul 08 2024 at 21:22):

Zeljko Nesic (Jul 09 2024 at 00:54):

The Italian guide dude from the SYCL Conf from the Zig crew that we sat down had actually worked on supply chain models software. Maybe he can give you some directions.

Luke Boswell (Jul 09 2024 at 05:53):

Does anyone know how the benchmarks work? I'm trying to figure out how/where they get built for the run in CI

Luke Boswell (Jul 09 2024 at 06:06):

Luke Boswell (Jul 10 2024 at 03:20):

It looks like roc build --bundle doesn't pick up and include any windows binaries even if they exist. :sad:

Luke Boswell (Jul 10 2024 at 03:25):

Found the issue, easy fix -- need to add .libas a recognised legacy host file type

Luke Boswell (Jul 10 2024 at 04:55):

Does anyone know how to find a rust dependency that is using Tcp? I've got basic-ssg building on Windows -- but when I go to link it there are a bunch of missing symbols that all look related to Tcp.

Luke Boswell (Jul 10 2024 at 04:56):

I suspect I might need to turn a feature off on a dependency using a feature flag.

Sam Mohr (Jul 10 2024 at 04:56):

Luke Boswell (Jul 10 2024 at 05:06):

I've narrowed my issue down to definitely just a missing library when linking on Windows

Luke Boswell (Jul 12 2024 at 07:22):

Listening on <http://localhost:8000>
2024-07-12T07:21:16Z GET /dashboard
thread 'tokio-runtime-worker' panicked at 'The Roc app crashed with: Can't create record with improper layout', src/lib.rs:46:5

Luke Boswell (Jul 12 2024 at 07:44):

baseWithBody = \content, navBar -> Generated.Pages.baseWithBody {
        content,
        navBarRtl: navBar,
        headerRtl,
        isWhiteBackground: Bool.true,
    }

baseWithBody = \content, navBar -> Generated.Pages.baseWithBody {
        content,
        navBar,
        headerRtl,
        isWhiteBackground: Bool.true,
    }

Roc turns this into a runtime error and happily roc builds and outputs the built file -- which means I don't pick this up until it panics at runtime.

But because roc returns Warnings as a non-zero exit code I've been ignoring the exit code and just checking the app was built. I probably need to revisit this strategy.

I'd love a way to say, don't roc build if there are any errors. I think this should be the dafault actually.

Anton (Jul 12 2024 at 10:05):

I've had this thought as well and mentioned it on zulip before but I can't find the conversation anymore...
I'd say the problem is that we want both ways to be discoverable and want the user to be aware if their command ignores errors or not.

Luke Boswell (Jul 12 2024 at 10:08):

Luke Boswell (Jul 12 2024 at 10:09):

I think after we land the rebuild host changes that are almost done, this will be the next step. There's just been a lot of preparation to get to this point and enable ^^&

Anton (Jul 12 2024 at 10:13):

I wonder if we can improve upon the name run to make it obvious that it runs even with errors, some suggestions:

Luke Boswell (Jul 12 2024 at 10:17):

I don't think we need it anymore -- if we want to be able to pipe input into just roc, or run without any arguments, we otherwise would be back in the situation of having multiple alternative ways to run a script if we keep run subcommand.

Anton (Jul 12 2024 at 10:38):

Oh ok, I assumed it was still there given the "run workflow" title. I'll read through the whole thing in a bit

Anton (Jul 12 2024 at 12:21):

Richard Feldman (Jul 13 2024 at 18:44):

Richard Feldman (Jul 14 2024 at 21:53):

Richard Feldman (Jul 14 2024 at 22:07):

Richard Feldman (Jul 14 2024 at 22:09):

@Ayaz Hafiz @Folkert de Vries imagine if we could adopt this for --optimize and use it to defunctionalize instead of lambda sets, then use heap-allocated closures in dev builds

Richard Feldman (Jul 14 2024 at 22:11):

Folkert de Vries (Jul 14 2024 at 22:12):

real question is whether they've solved the problems we have. e.g. is it efficient/possible to perform the analysis on a per-module basis?

Folkert de Vries (Jul 14 2024 at 22:13):

we can spin a cool story about lambda sets just ignoring the practical problems we face as a production language

Richard Feldman (Jul 14 2024 at 22:14):

Folkert de Vries (Jul 14 2024 at 22:14):

I really think programs just get too big for that to be a practical solution long-term

Richard Feldman (Jul 14 2024 at 22:21):

Folkert de Vries (Jul 14 2024 at 22:22):

yeah here the details start to matter, and I wonder if they have run into that sort of problem with their implementation

Folkert de Vries (Jul 14 2024 at 22:23):

caching etc could go a long way though, but I think that only gets investigated once performance problems come up

Richard Feldman (Jul 14 2024 at 22:27):

Richard Feldman (Jul 14 2024 at 22:39):

but the idea of having one process that does all of these and more is pretty intriguing:

Luke Boswell (Jul 16 2024 at 07:28):

Interesting / amusing bug -- if you name an Effect tmpDir roc won't expose it for some reason.

Luke Boswell (Jul 16 2024 at 07:33):

Hmmm, having trouble getting an effect that just returns a string to work someTmpDir : Effect Str

Luke Boswell (Jul 16 2024 at 07:35):

# Effect.roc
someTmpDir : Effect Str

# Dir.roc
tmpDir : Task Str []_
tmpDir =
    Effect.someTmpDir
    |> Effect.map Ok
    |> InternalTask.fromEffect

$ roc build --no-link examples/hello-world.roc
0 errors and 0 warnings found in 116 ms
 while successfully building:

    examples/hello-world.o
$ objdump -t examples/hello-world.o | grep "roc_fx_someDir"
$

Luke Boswell (Jul 16 2024 at 07:56):

Luke Boswell (Jul 18 2024 at 08:42):

Luke Boswell (Jul 19 2024 at 11:42):

I just checked if I needed to update the glue platform package as it's been a while -- re-generated the bundle and it's got an identical hash. :smiley:

Brendan Hansknecht (Jul 20 2024 at 23:08):

With our joinpoints, we guarantee that everything that lives for multiple iterations is explicitly passed to the jump expression. As such, anything that is alloca'ed in the middle of the loop created by the joinpoint should be safe to have it's alloca be part of the entry block. Correct?

It' just that at the jump instruction, we have to copy out of any temporary allocas and into the allocas for the joinpoint. Otherwise, we may run into mutation bugs where the the the next run of loop mutate values that became joinpoint args.

If so, I think that means we can hoist all allocas period to the entry block. Then cleanup jointpoints/jumps and hopefully have fixed a number of bugs.

Brendan Hansknecht (Jul 20 2024 at 23:08):

Brendan Hansknecht (Jul 21 2024 at 04:27):

So turns out I should have tried to fix this a long time ago. Turned out to be pretty easy:

Brendan Hansknecht (Jul 21 2024 at 05:18):

Also, this cut a whole minute off of 1brc in roc. llvm really does optimize better with all allocas in the entry block.

Benchmark 1: ./1brc data/measurements_1_000_000_000.txt
  Time (mean ± σ):     118.589 s ±  0.643 s    [User: 116.121 s, System: 1.720 s]
  Range (min … max):   118.009 s … 119.280 s    3 runs

Benchmark 2: ./1brc-old data/measurements_1_000_000_000.txt
  Time (mean ± σ):     188.869 s ±  3.279 s    [User: 186.282 s, System: 1.635 s]
  Range (min … max):   185.188 s … 191.478 s    3 runs

Summary
  ./1brc data/measurements_1_000_000_000.txt ran
    1.59 ± 0.03 times faster than ./1brc-old data/measurements_1_000_000_000.txt

Brendan Hansknecht (Jul 21 2024 at 15:32):

Also, most of the flamegraph is now allocating for RocList::extend_from_slice So platform primitives instead of roc itself.

Luke Boswell (Jul 23 2024 at 23:10):

This may not help at all, but on the refactor-host branch of basic-webserver I was able to get a working executable by using zig within the nix shell, and compiling my app using --no-link.

zig build-exe ./target/release/libhost.a  app.o -lc -lunwind -fstrip

This is a workaround for the linux musl and linking issue, that I doubt is really bothering anyone, but sharing just in case.

I suspect our linking issue with basic-cli may be related to -lunwind which isn't included by default in our roc linker flags. I'll try and test that more later.

Luke Boswell (Jul 23 2024 at 23:12):

Brendan Hansknecht (Jul 25 2024 at 16:01):

So assuming the numbers in this repo are correct (which I expect them to be), we apparently shouldn't be using linear search anymore. We should be using monobound binary search: https://github.com/scandum/binary_search/

This is thinking about the many cases in the compiler where for small maps, we attempt to use vec map and linear searching. Hmm, though that doesn't take into account the cost of keeping the list sorted so you can binary search.

Richard Feldman (Jul 25 2024 at 16:19):

since we use VecMap in so many places, might not be a big effort to swap its implementation for something which does that, and just see what happens

Brendan Hansknecht (Jul 25 2024 at 16:28):

Brendan Hansknecht (Jul 29 2024 at 17:14):

We just landed task as builtin...didn't we...That is probably why my code using task suddenly broke on main?

Brendan Hansknecht (Jul 29 2024 at 17:15):

── UNRECOGNIZED NAME in ../sorting/builtin.roc ─────────────────────────────────

Nothing is named `Task` in this scope.

56│>      Stdout.line!
57│>          (
58│>              if testSort answrlst then
59│>                  "List sorted correctly!"
60│>              else
61│>                  "Failure in sorting list!!!"
62│>          )

Did you mean one of these?

    Hash
    main
    List
    U8

────────────────────────────────────────────────────────────────────────────────

Anton (Jul 29 2024 at 17:20):

Anton (Jul 29 2024 at 17:21):

Sam Mohr (Jul 29 2024 at 17:21):

Brendan Hansknecht (Jul 29 2024 at 17:22):

Interesting. Why is that needed today but not yesterday (or a few days ago, don't have exact bisect)

Anton (Jul 29 2024 at 17:23):

Anton (Jul 29 2024 at 17:24):

Brendan Hansknecht (Jul 29 2024 at 18:07):

I guess once we have task as built-in this will be fixed cause task will be automatically imported.

Sam Mohr (Jul 29 2024 at 18:07):

Musab Nazir (Jul 29 2024 at 21:04):

I ran into this exact issue yesterday and was confused (using testing releases). I don't think the docs / tutorial mentions the need to use exposing [Task] or maybe I didn't look hard enough

Luke Boswell (Jul 29 2024 at 21:15):

We only just merged that change, and I don't think anyone realised it would impact the imports in this way. The Task as builtin change is basically ready and will eliminate this entirely. We just are going through the process to make sure that is well tested before making anither breaking change and new release.

Luke Boswell (Jul 29 2024 at 21:16):

When Task is a builtin there wont be any need to import it manually, it will be automatically available in every app.

Brendan Hansknecht (Jul 30 2024 at 18:49):

Brendan Hansknecht (Jul 30 2024 at 18:51):

Theoretically would also be a more optimal way to do large List.map operations assuming the mapping function is simple enough (too expensive of a mapping function and doing more complex memory loading may actually hurt a lot).

Folkert de Vries (Jul 30 2024 at 18:51):

someone suggested using large pages somewhere for basically the same effect with less effort

Brendan Hansknecht (Jul 30 2024 at 18:59):

How do large pages fix the issue? I thought the core problem was needing multiple memory streams to increase memory load throughput and keep the load buffer full.

Luke Boswell (Aug 01 2024 at 23:38):

Maybe this is a completely crazy idea... but LLVM bitcode isn't specific to an os/arch and is basically an IR. Right?

Is there any reason why a platform host couldn't be a LLVM bitcode file? So platform authors provide that one generic-llvm.bc or something and then roc can build and link targeting anything supported by LLVM.

It woudn't be compiled into machine code already, so it would be much slower than just linking, and probably much larger. But maybe it provides a more flexibility or is a good option for some use-cases?

Brendan Hansknecht (Aug 01 2024 at 23:59):

Versioning is a problem (no guarantee that llvm 16 ir will load with llvm 17). Also wouldn't work with dev backends.

Luke Boswell (Aug 02 2024 at 00:29):

So it could work then? We could version the file, like generic_llvm_18_1_8.bc.

Even is this only worked for slower optimised builds, it might be useful, particularly for e.g. getting support for various long tail of targets.

Luke Boswell (Aug 02 2024 at 00:30):

Brendan Hansknecht (Aug 02 2024 at 00:48):

Notification Bot (Aug 03 2024 at 11:07):

Brendan Hansknecht (Aug 04 2024 at 18:50):

Luke Boswell (Aug 04 2024 at 22:46):

Is this the work of the alias analysis, lambda sets? I imagine this is what is happening in the "Specialise" stage of the compiler.

Luke Boswell (Aug 04 2024 at 22:47):

And thinking about Jasper's comment the other day, could this be represented in Roc style syntax, or is this completely something different?

Brendan Hansknecht (Aug 05 2024 at 03:59):

I think it is related. Like it happens for all types that unify, lambdasets being one of them. I think this specific error may come from morphic related code? Caues I think it use things like bag<(heap_cell,)>. Theoretically you could represent this as a roc style tag union, but I don't think that would erally be helpful.

As a note, this is generating from basic webserver with sqlite. I ported the todos example over. Using the query function twice in different contexts seems to be leading to this somehow. If I comment out either the createTodo path or the listTodos path, it seems to fix things. I am very much not sure what all is going on/the real issue. I think the first type has extra empty () values. Maybe something should be collapsing those together? Cause union { (), () } is probably the same thing as union { () } (assuming this are raw unions.

Brendan Hansknecht (Aug 05 2024 at 17:35):

@Folkert de Vries just curious if you have any thoughts or comments on the above. Specifically around alias analysis union simplification and if it is valid. Like are we missing a canonicalization step here that would lead to all the () in the union merging into a single (). I'm not sure the semantics expected here.

Brendan Hansknecht (Aug 07 2024 at 01:16):

Ayaz Hafiz (Aug 07 2024 at 01:31):

Ayaz Hafiz (Aug 07 2024 at 01:32):

there's probably a way to simplify it, but simplifying it is also likely covering up a bug

Ayaz Hafiz (Aug 07 2024 at 01:32):

i would run the IR checker - that will point to an issue if there is one earlier on, which i suspect there is

Brendan Hansknecht (Aug 07 2024 at 01:33):

I think it crashes before check mono ir runs. That or it passes and then gets to this failure anyway.

Ayaz Hafiz (Aug 07 2024 at 01:33):

there should be a mono IR check pass that runs before alias analysis (morphic). this is a panic in morphic

Ayaz Hafiz (Aug 07 2024 at 01:34):

I would check roc_debug_flags, i think its ROC_CHECK_IR_AFTER_SPECIALIZATION or something

Brendan Hansknecht (Aug 07 2024 at 01:35):

Ok. I'll double check when I get the chance. I'm pretty sure I ran it with ROC_CHECK_MONO_IR and it still got to the error above and crashed.

Anton (Aug 07 2024 at 09:13):

Should we enable these env vars in CI when doing cargo test without --release to catch any bugs that may otherwise go undetected?

ROC_VERIFY_RIGID_LET_GENERALIZED
ROC_VERIFY_OCCURS_ONE_RECURSION
ROC_CHECK_MONO_IR

Brendan Hansknecht (Aug 07 2024 at 19:13):

Anton (Aug 09 2024 at 15:21):

After reading the descriptions of the verify flags (in crates/compiler/debug_flags/src/lib.rs), I ended up enabling only ROC_CHECK_MONO_IR (PR#6976)

Brendan Hansknecht (Aug 10 2024 at 00:00):

check failure

Full source: procedure `pf.Sqlite.87` (`pf.Sqlite.464`):
    let `pf.Sqlite.865` : [C [C [], C Str, C Str [C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C ], C U8], C List Str] = TagId(1) `pf.Sqlite.464`;
    ret `pf.Sqlite.865`;
IR PROBLEMS FOUND:
── SYMBOL LAYOUT DOESN'T MATCH ITS USE ─────────────────────────────────────────

in 87 : (List Str) ->
[C [C [], C Str, C , C Str [C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C ], C , C U8, C ], C List Str] ((niche
{}))

0│  procedure `pf.Sqlite.87` (`pf.Sqlite.464`):
1│>     let `pf.Sqlite.865` : [C [C [], C Str, C Str [C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C ], C U8], C List Str] = TagId(1) `pf.Sqlite.464`;
2│      ret `pf.Sqlite.865`;

Sqlite.865 defined here with layout
[C [C [], C Str, C Str [C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C ], C U8], C List Str]

1│      let `pf.Sqlite.865` : [C [C [], C Str, C Str [C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C ], C U8], C List Str] = TagId(1) `pf.Sqlite.464`;
2│>     ret `pf.Sqlite.865`;

Sqlite.865 used as a return value here with layout
[C [C [], C Str, C , C Str [C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C ], C , C U8, C ], C List Str]

Brendan Hansknecht (Aug 10 2024 at 00:04):

[
    C [
        C [],
        C Str,
-       C ,
        C Str [C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C , C ],
-       C ,
        C U8,
-       C
    ],
    C List Str
]

Brian Carroll (Aug 15 2024 at 18:15):

Just saw someone share this collection of "Resources for Amateur Compiler Writers". I haven't dug into them yet, but thought I'd pass it on anyway!
https://c9x.me/compile/bib/

Anton (Aug 16 2024 at 18:42):

I just tried IDA free for the first time, it is definitely the best assembly debugger I have ever used
IDA-screenshot

Sam Mohr (Aug 16 2024 at 18:43):

Brendan Hansknecht (Aug 16 2024 at 19:00):

Sam Mohr (Aug 16 2024 at 19:03):

Zeljko Nesic (Aug 16 2024 at 22:24):

Anton (Aug 17 2024 at 08:02):

Anton (Aug 17 2024 at 10:14):

The debugger is in beta and crashes easily on the binary I'm testing it with, it definitely has potential though!

Brendan Hansknecht (Aug 21 2024 at 02:21):

I was looking at basic webserver and the cost of always copying the request and response bodies. Cause currently it always copies the input and output data from rust vectors into roc lists and back.

Raw rust, fully async but still manifesting the fully body into bytes (aka avoid streaming in the request body):
787482 requests in 20.10s, 72.66GB read

Roc with spawn_blocking and copying all inputs and outputs between rust vector and roc lists:
627501 requests in 20.00s, 57.90GB read

So 2 extra copies of all bytes only costs ~20% loss in overall perf. Still a hefty loss, but I was expecting a lot more.

Note: if we also constrain rust to use spawn_blocking it takes a ~4% perf hit:
755249 requests in 20.10s, 69.69GB read

Zeljko Nesic (Aug 21 2024 at 22:00):

See if you can try copying 1MB of data back and forth and see how much of penalty does it take! :)

Brendan Hansknecht (Aug 21 2024 at 23:59):

Luke Boswell (Aug 26 2024 at 07:10):

Sam Mohr (Aug 26 2024 at 07:10):

Sam Mohr (Aug 26 2024 at 07:11):

Luke Boswell (Aug 26 2024 at 07:12):

Brendan Hansknecht (Aug 26 2024 at 07:31):

Anton (Aug 26 2024 at 08:16):

Anton (Aug 30 2024 at 08:06):

Sam Mohr (Aug 30 2024 at 08:07):

Sam Mohr (Aug 30 2024 at 08:08):

Anton (Aug 30 2024 at 08:17):

Sam Mohr (Aug 30 2024 at 08:17):

Brendan Hansknecht (Sep 03 2024 at 16:46):

Brendan Hansknecht (Sep 03 2024 at 16:51):

Luke Boswell (Sep 04 2024 at 11:39):

How do I fix a snapshot failure like this? I made a completely unrelated change (as far as I can tell) and now this is failing...

Richard Feldman (Sep 04 2024 at 11:40):

looks like there's a newline at the end of the line in the source file, and maybe it got saved and your editor automatically trimmed it off?

Richard Feldman (Sep 04 2024 at 11:41):

that used to happen to me all the time, but I edited the tests in question to no longer need a trailing newline

Luke Boswell (Sep 04 2024 at 11:41):

Luke Boswell (Sep 04 2024 at 11:43):

alloc.reflow("Tip: Learn more about builtins in the tutorial:\n\n<https://www.roc-lang.org/tutorial#builtin-modules>"),

Richard Feldman (Sep 04 2024 at 11:47):

Richard Feldman (Sep 04 2024 at 11:48):

Luke Boswell (Sep 04 2024 at 11:51):

I'm just going to leave that for now and come back to it later... in the build-host PR / rebuild-platform branch I've managed to remove all references to basic-cli from roc (aside from the scripts for building the website) now.

Luke Boswell (Sep 04 2024 at 11:52):

Brendan Hansknecht (Sep 06 2024 at 18:40):

Scale of waste is mind boggling to me sometimes. I get the computers are fast, so a company can make a ton of money well also wasting a ton, but it just feels wrong to me.

jitted multithreaded ruby using sqlite as a local database... on a modern high end bare metal server with 48 cores.... only getting 15k requests per second.

Basic webserver still has wasted copies and is missing the effect interpreter to enable full async. Yet it can do 120k requests per second on 4 cores of my m1 Mac.

Obviously an apples to oranges comparison to some extent. But the rough scale is real.

Related aside: shouldn't ruby have async and await that turns this into just waiting on the network card? Shouldn't this still be able to saturate the network card for something so simple? Is Ruby with rails all blocking io?

Sam Mohr (Sep 06 2024 at 18:41):

My company primarily runs a Ruby on Rails monolith, tell me about it... :smiling_face_with_tear:

Sam Mohr (Sep 06 2024 at 18:46):

Sam Mohr (Sep 06 2024 at 18:47):

It seems like Ruby 3.0 onwards does async IO automatically, but we run Ruby 2.7 because any company using something so outdated as Rails will avoid the cost of upgrading things whenever possible

Eli Dowling (Sep 16 2024 at 23:48):

Hahaha, this is deeply relatable. I just started on a codebase with a few hundred thousand lines of nightmarish PHP.
We have pages that take over 10 seconds to load because they query all 250k users on the database to display a stat.

I can promise you, nobody has any idea how many requests per second we manage.... But I can also promise it ain't good :sweat_smile:

Luke Boswell (Sep 20 2024 at 07:17):

define void @roc__mainForHost_1_exposed_generic(ptr %0) !dbg !21

mainForHost : Str -> Str

Luke Boswell (Sep 20 2024 at 07:18):

Brendan Hansknecht (Sep 20 2024 at 12:07):

Ayaz Hafiz (Sep 27 2024 at 02:07):

when x is
  Error e -> Error e
  Ok y -> y + 1

Sam Mohr (Sep 27 2024 at 02:07):

Sam Mohr (Sep 27 2024 at 02:08):

when x is
    Err e -> return Err e
    Ok y ->
        y +1

Ayaz Hafiz (Sep 27 2024 at 02:08):

Sam Mohr (Sep 27 2024 at 02:09):

Sam Mohr (Sep 27 2024 at 02:10):

Returning early from the continuation will return to the parent function's return value immediately, since the parent function doesn't do anything after calling the continuation

Sam Mohr (Sep 27 2024 at 02:10):

Ayaz Hafiz (Sep 27 2024 at 02:28):

Ayaz Hafiz (Sep 27 2024 at 02:29):

wouldn't the early return immediately pass the result value to the next continuation in the chain? if you return to the parent function wouldn't that break the sequence of continuations?

Luke Boswell (Oct 16 2024 at 23:28):

Is zstd a dependency for using a roc nightly on intel macos? I'm trying to setup a GH runner and it's given me an error like

dyld[4651]: Library not loaded: /usr/local/opt/zstd/lib/libzstd.1.dylib
  Referenced from: <4E33E9A3-ECE7-3A94-90F3-D55423A81AF9> /Users/runner/work/roc-ray/roc-ray/roc_nightly/roc
  Reason: tried: '/usr/local/opt/zstd/lib/libzstd.1.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/opt/zstd/lib/libzstd.1.dylib' (no such file), '/usr/local/opt/zstd/lib/libzstd.1.dylib' (no such file), '/usr/local/lib/libzstd.1.dylib' (no such file), '/usr/lib/libzstd.1.dylib' (no such file, not in dyld cache)
/Users/runner/work/_temp/97a43c70-cd5c-446e-a653-669d45f49f87.sh: line 1:  4651 Abort trap: 6           ./roc_nightly/roc version

Sam Mohr (Oct 16 2024 at 23:37):

Luke Boswell (Oct 16 2024 at 23:44):

Run ./roc_nightly/roc version

dyld[2644]: Library not loaded: /usr/local/opt/zstd/lib/libzstd.1.dylib
  Referenced from: <4E33E9A3-ECE7-3A94-90F3-D55423A81AF9> /Users/runner/work/roc-ray/roc-ray/roc_nightly/roc
  Reason: tried: '/usr/local/opt/zstd/lib/libzstd.1.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/opt/zstd/lib/libzstd.1.dylib' (no such file), '/usr/local/opt/zstd/lib/libzstd.1.dylib' (no such file), '/usr/local/lib/libzstd.1.dylib' (no such file), '/usr/lib/libzstd.1.dylib' (no such file, not in dyld cache)
/Users/runner/work/_temp/e7c7fbc1-dcab-4aba-8b61-ee456c77d93c.sh: line 1:  2644 Abort trap: 6           ./roc_nightly/roc version

Luke Boswell (Oct 16 2024 at 23:55):

Luke Boswell (Nov 21 2024 at 03:54):

I think we've managed to break the --no-link build pipeline sometime recently... I used to be able to do this.

 $ roc build --no-link --emit-llvm-ir examples/simple.roc
Legacy linking failed: Failed to find any legacy linking files; I need one of these three paths to exist:
    examples/../platform/macos-arm64.a
    examples/../platform/macos-arm64.o
    examples/../platform/libhost.a

Luke Boswell (Nov 21 2024 at 03:55):

I didn't think this was --rebuild-host PR... but that's probably the most likely candidate

Brendan Hansknecht (Nov 21 2024 at 04:10):

Luke Boswell (Nov 21 2024 at 05:25):

Joshua Warner (Nov 22 2024 at 03:20):

QQ: Is there any particular reason why there are two phases to canonicalization? (desugaring, and then conversion to can::Expr/etc)

Joshua Warner (Nov 22 2024 at 03:24):

Sam Mohr (Nov 22 2024 at 03:26):

Sam Mohr (Nov 22 2024 at 03:27):

Joshua Warner (Nov 22 2024 at 03:27):

Joshua Warner (Nov 22 2024 at 03:28):

Sam Mohr (Nov 22 2024 at 04:38):

Richard Feldman (Nov 22 2024 at 05:09):

Luke Boswell (Nov 22 2024 at 05:40):

Joshua Warner (Nov 22 2024 at 05:44):

Joshua Warner (Nov 22 2024 at 05:45):

I've written basically that ~5 times now. Every time I need to either look up how exactly it's done, and/or spend hours debugging.

Joshua Warner (Nov 22 2024 at 05:45):

Anton (Nov 22 2024 at 09:49):

Joshua Warner (Nov 22 2024 at 15:12):

Richard Feldman (Nov 22 2024 at 16:16):

it's basically the gold standard for how to efficiently resolve precedence without needing a separate pass later, and lots of compilers use it

Brendan Hansknecht (Nov 22 2024 at 16:24):

Brendan Hansknecht (Nov 22 2024 at 16:25):

Anthony Bullard (Nov 22 2024 at 16:56):

In my own language I was working on, I didn't have bin/unary ops and let the user decide the precedence through function call. Was that principal or laziness? I'll let you make the call

Richard Feldman (Dec 01 2024 at 02:13):

Luke Boswell (Dec 01 2024 at 02:13):

Richard Feldman (Dec 01 2024 at 02:13):

I need a bunch of cleanup before it's ready for a PR, but it actually works end to end now

Richard Feldman (Dec 01 2024 at 02:15):

(for those who haven't been following along, this resolves the extremely longstanding issue where if you expose a type alias of a type from another module - such as Http.Request : InternalHttp.Request in this example) it would render as nothing - which is why the Http docs today are not very helpful :sweat_smile:

Isaac Van Doren (Dec 01 2024 at 03:40):

Eli Dowling (Dec 02 2024 at 05:17):

Hey, would anyone a little more familiar with the inner workings of the compiler be able to suggest how I could get types for the tag completions I just added to the language server?
Currently I get the tag names from the subs tag_names field

Initially as an experiment I just tried to brute force it by iterating every variable entry in the current module subs looking for Tags
But that approach didn't seem to find many of the tags that exist in tag_names

 subs.variables
                .iter()
                .flat_map(|var1| {
                    let res = subs2.get(var1.clone()).content.clone();
                    match res {
                        Content::Structure(structure) => match structure {
                            FlatType::FunctionOrTagUnion(names, _, ext) => {
                                let res = subs.get_subs_slice(names).iter().filter_map(|(label)| {
                                    if label.as_ident_str().starts_with(prefix) {
                                        // let type_str = SubsFmtFlatType()
                                        Some(CompletionItem {
                                            label: label.as_ident_str().to_string(),
                                            kind: Some(CompletionItemKind::ENUM),
                                            documentation: Some(lsp_types::Documentation::String(
                                                format_var_type(
                                                    ext.var(),
                                                    &mut subs2,
                                                    module_id,
                                                    interns,
                                                ),
                                            )),
                                            ..Default::default()
                                        })
                                    } else {
                                        None
                                    }
                                });
                                res.collect()
                            }
                            FlatType::TagUnion(name, ext)
                            | FlatType::RecursiveTagUnion(_, name, ext) => {
                                let res = name.iter_from_subs(&subs).filter_map(|(label, var)| {
                                    if label.as_ident_str().starts_with(prefix) {
                                        // let type_str = SubsFmtFlatType()
                                        Some(CompletionItem {
                                            label: label.as_ident_str().to_string(),
                                            kind: Some(CompletionItemKind::ENUM),
                                            documentation: Some(lsp_types::Documentation::String(
                                                format_var_type(
                                                    ext.var(),
                                                    &mut subs2,
                                                    module_id,
                                                    interns,
                                                ),
                                            )),
                                            ..Default::default()
                                        })
                                    } else {
                                        None
                                    }
                                });
                                res.collect()
                            }
                            _ => vec![],
                        },

                        _ => vec![],
                    }
                })
                .collect::<Vec<_>>() //we have to collect so that we can release the lock

How can I go from tag_names to types? or maybe where should I add in the compiler some way of saving the tag unions that exist so i can use them later in the completion?

Luke Boswell (Dec 13 2024 at 01:40):

Please file an issue here: <https://github.com/roc-lang/roc/issues/new/choose>
Invalid decimal for float literal = 1e10. This should be a type error!
Location: crates/compiler/mono/src/ir/literal.rs:115:25

Brendan Hansknecht (Dec 13 2024 at 01:49):

Brendan Hansknecht (Dec 13 2024 at 01:50):

Brendan Hansknecht (Dec 15 2024 at 01:03):

Apparently my new version of the false interpreter found a bug in drop specialization.

Brendan Hansknecht (Dec 15 2024 at 01:03):

Brendan Hansknecht (Dec 15 2024 at 01:04):

I was hoping to just update and land this change quickly, not to round about debug a bunch of things due to the update (I guess it at least proves the value of having false interpreter as a test case)

Brendan Hansknecht (Dec 15 2024 at 01:25):

It is a debug assert being hit in drop specialization. Can be reproed on the fix-false branch on any target with cargo test -p roc_cli -- false_

J.Teeuwissen (Dec 15 2024 at 12:01):

Opened pr that should fix this specific issue: https://github.com/roc-lang/roc/pull/7376
I haven't touched Roc in a while, i suggest checking/testing the changes thoroughly ;)

Luke Boswell (Dec 19 2024 at 00:31):

Does this error message mean anything to anyone? for context, it's a CI failure in basic-cli after upgrading hyper and removing "ring" and replacing with "aws-lc-sys" crate. Only failing in macOS-13 -- passes everywhere else.

cargo:warning=In file included from /Users/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.24.0/aws-lc/crypto/fipsmodule/bcm.c:150:
  cargo:warning=In file included from /Users/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.24.0/aws-lc/crypto/fipsmodule/rand/urandom.c:69:
  cargo:warning=/nix/store/hmchx96ysd0h19p7kx0w12pljkbp30j3-Libsystem-1238.60.2/include/CommonCrypto/CommonRandom.h:35:9: error: unknown type name 'CCCryptorStatus'
  cargo:warning=typedef CCCryptorStatus CCRNGStatus;
  cargo:warning=        ^
  cargo:warning=In file included from /Users/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.24.0/aws-lc/crypto/fipsmodule/bcm.c:150:
  cargo:warning=/Users/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.24.0/aws-lc/crypto/fipsmodule/rand/urandom.c:394:42: error: use of undeclared identifier 'kCCSuccess'
  cargo:warning=  if (CCRandomGenerateBytes(out, len) == kCCSuccess) {
  cargo:warning=                                         ^
  cargo:warning=2 errors generated.

Luke Boswell (Dec 19 2024 at 00:32):

darwinInputs = with pkgs;
    lib.optionals stdenv.isDarwin
    (with pkgs.darwin.apple_sdk.frameworks; [
        Security
        CoreFoundation
    ]);

Brendan Hansknecht (Dec 19 2024 at 00:34):

Brendan Hansknecht (Dec 19 2024 at 00:35):

Notification Bot (Dec 29 2024 at 03:33):

Brendan Hansknecht (Dec 29 2024 at 20:26):

Tag1: [A]
Tag2: [B]

Tag3: [C]Tag1,Tag2

Ayaz Hafiz (Dec 29 2024 at 20:28):

Brendan Hansknecht (Dec 29 2024 at 20:35):

── INVALID_EXTENSION_TYPE in examples/../platform/Sqlite.roc ───────────────────

This record extension type is invalid:

50│  Tag3 : ([C]Tag1)Tag2
                     ^^^^

Note: A record extension variable can only contain a type variable or
another record.


── INVALID_EXTENSION_TYPE in examples/../platform/Sqlite.roc ───────────────────

This tag union extension type is invalid:

50│  Tag3 : ([C]Tag1)Tag2
                ^^^^

Note: A tag union extension variable can only contain a type variable
or another tag union.

Brendan Hansknecht (Dec 29 2024 at 20:36):

Anthony Bullard (Dec 29 2024 at 21:08):

Brendan Hansknecht (Dec 29 2024 at 21:14):

Luke Boswell (Dec 29 2024 at 23:39):

Luke Boswell (Dec 29 2024 at 23:40):

I think his idea is good and any potential (unforeseen negative) impact limited to the formatter.

Anthony Bullard (Dec 30 2024 at 01:38):

Anthony Bullard (Dec 30 2024 at 01:42):

Anthony Bullard (Dec 30 2024 at 01:44):

But more seriously I wish this design could work well with some sort of line width constraints (if it does and I missed it somehow i apologize), me and Richard were talking about that for better formatting of docs code blocks than what we have today

Anthony Bullard (Dec 30 2024 at 01:47):

Joshua Warner (Dec 30 2024 at 02:42):

Joshua Warner (Dec 30 2024 at 02:43):

When I've floated that previously (and seen it discussed separately), I think Richard has been somewhat against it

Joshua Warner (Dec 30 2024 at 02:44):

There's probably a happy medium where there aren't dramatic changes from how things operate today, but line-length constraints / adjustments really only operate on the fringes / fix up some of the edge cases

Joshua Warner (Dec 30 2024 at 02:45):

Sam Mohr (Dec 30 2024 at 02:46):

I think we'd want line length constraints that can be applied to type signatures for docs generation, but otherwise are ignored

Richard Feldman (Dec 30 2024 at 02:46):

Richard Feldman (Dec 30 2024 at 02:47):

in other words, have the line width be configurable, and have roc format set it to infinity, but set it to different numbers when generating docs

Sam Mohr (Dec 30 2024 at 02:47):

Because line length constraints in code just tend to force awkward breaking up of lines that need to be longer than the arbitrary limit

Joshua Warner (Dec 30 2024 at 02:51):

FWIW for normal formatting, I've been thinking of experimenting with soft line _width_ constraints rather than line length constraints - i.e. not counting indentation against you

Joshua Warner (Dec 30 2024 at 02:52):

Joshua Warner (Dec 30 2024 at 02:55):

Joshua Warner (Dec 30 2024 at 02:56):

Joshua Warner (Dec 30 2024 at 02:57):

Joshua Warner (Dec 30 2024 at 02:58):

Joshua Warner (Dec 30 2024 at 03:02):

The "simple" solution of just having everything authored to fit a reasonably narrow viewport (i.e. manually insert newlines to force multi-line representations in the formatter) feels not _terrible_.

Joshua Warner (Dec 30 2024 at 03:02):

The big thing that tends to blow line length limits accidentally in roc code would be comments, and I don't know if there's an easy solution for that.

Sam Mohr (Dec 30 2024 at 03:02):

Currently, the type signatures in docs have a max width about 600px wide even on really wide screens. So type signatures can get narrower than that, but are pretty much the same size on desktop and most mobile views

Joshua Warner (Dec 30 2024 at 03:03):

Sam Mohr (Dec 30 2024 at 03:04):

Sam Mohr (Dec 30 2024 at 03:06):

On formatting comments: since doc comments are markdown, any group of lines that doesn't have an empty line between its lines can be treated as a giant, single line and wrapped based on the word lengths. But I think normal comments just need to be left as-is

Joshua Warner (Dec 30 2024 at 03:08):

Joshua Warner (Dec 30 2024 at 03:10):

It may look a little weird if there's a very long type decl, but it'll copy-paste just fine

Sam Mohr (Dec 30 2024 at 03:12):

For function types, we do let the browser do line wrapping after forcing multilining on long types

Sam Mohr (Dec 30 2024 at 03:13):

A good way to see this is to compare List.walk in the search results vs. its main section in the List module

Sam Mohr (Dec 30 2024 at 03:15):

walkFrom :
    List elem,
    U64,
    state,
    (state, elem -> state)
    -> state

walkFrom : List elem, U64, state, (state, elem -> state)
    -> state

Sam Mohr (Dec 30 2024 at 03:17):

List.walkFromUntil : List elem, U64, state,
    (state, elem -> [ Continue state, Break
    state ]) -> state

Sam Mohr (Dec 30 2024 at 03:17):

Joshua Warner (Dec 30 2024 at 03:18):

Joshua Warner (Dec 30 2024 at 03:19):

Why is the developer ok with that super long type in the source but not ok when it’s in docs?

Sam Mohr (Dec 30 2024 at 03:22):

Because the developer can decide to break up a long def with a newline. Just putting a newline in the middle will make Roc multiline it for you

Sam Mohr (Dec 30 2024 at 03:22):

This lets people choose for themselves in wide display environments how to display their signatures

Sam Mohr (Dec 30 2024 at 03:23):

So keeping them in a single line and autowrapping leads to potentially many List.walkFromUntils

Sam Mohr (Dec 30 2024 at 03:24):

I personally don't feel that strongly about this, I don't think it matters that much

Joshua Warner (Dec 30 2024 at 12:54):

I think I agree with your logic there, but to me it more strongly indicates the formatting should adapt to the viewport in all situations where that’s possible.

Joshua Warner (Dec 30 2024 at 12:57):

Eg in the editor with LSP, it would ideally be possible to do a “view-only” reformat or something, where the on-disk version has longer lines

Joshua Warner (Dec 30 2024 at 12:59):

Or maybe this is done at ‘git checkout’ time: code is formatted to your preference when checked out, and formatted back to some generic standard when committed. Pretty sure hooks for that sort of thing do already exist.

Richard Feldman (Dec 30 2024 at 13:57):

I understand in theory, but having spent years with formatters that do and don't enforce line width, I've found that the line width enforcement is frequently annoying and almost never helpful.

Richard Feldman (Dec 30 2024 at 13:58):

for example, sometimes I have two lines that are doing things that are very similar, and will be easier to understand if they are side by side and formatted the same way - so the similarities between them are visually obvious - but one happens to call functions with slightly longer names, causing the formatter to make it multiline.

Richard Feldman (Dec 30 2024 at 14:00):

I've wasted a bunch of time trying to coax line-width-enforcing formatters to not make my code harder to read, and I've wasted zero time doing that with formatters that don't enforce it

Richard Feldman (Dec 30 2024 at 14:02):

the difference is that when there is a hard width cap, like in types in docs, if there's going to be wrapping anyway, it's better if we can apply a wrapping algorithm that looks nicer than the browser default of just word wrapping without taking into account indentation, whether an entire type should now render as multiline, etc :big_smile:

Joshua Warner (Dec 30 2024 at 14:06):

Joshua Warner (Dec 30 2024 at 14:11):

The same view-width logic applies to my editor (I run it full screen and I'm not reducing the font size - so it's not getting any wider). I would love my editor to have nicer line wrapping behavior that's guided by a language server. Or alternatively, an easy git config to format my code to some line limit on checkout (supposing it's somehow perfectly invertable).

The practical difference there is in difficulty. Docs (in theory) have one width across all users. Editor config is often unique-ish for each user.

Joshua Warner (Dec 30 2024 at 14:12):

I would add tho that many times I've come across a docs site that looked silly because of how much it restricted the content width, and I've gone into the browser inspector and removed that width constraint while I read.

Joshua Warner (Dec 30 2024 at 14:12):

(which is to say, I would push back a bit on the notion that 600px is great for everyone)

Joshua Warner (Dec 30 2024 at 14:13):

Here's an idea: ship the formatter as a wasm binary in the browser that will dynamically re-format lines to match whatever the real display width is :)

Richard Feldman (Dec 30 2024 at 15:01):

Joshua Warner (Dec 30 2024 at 15:03):

That's definitely lighter-weight, in terms of what we ship to the browser, so probably better.

Joshua Warner (Dec 30 2024 at 15:03):

Joshua Warner (Dec 30 2024 at 15:04):

Joshua Warner (Dec 30 2024 at 15:28):

Ok, anyway, with that in mind: yes, I think this "Node" refactor for the formatter that I've been working on could be updated to (optionally) do line-length limits

Joshua Warner (Dec 30 2024 at 15:29):

That's not high on my priority list right now, but happy to provide some guidance if someone else wanted to tackle that

Richard Feldman (Dec 30 2024 at 17:00):

Jasper Woudenberg (Dec 31 2024 at 09:02):

I'm working on migrating my blog to the Roc-based static site generator I'm working on. I was doing some styling and trying to figure out the best approach to display code snippets on mobile.

Rewriting the code snippets to have very short line widths is one approach. The thing is, while that means you don't have to scroll horizontally anymore, I think it creates a different flavor of poor readability where even simple expressions need to be broken across multiple lines.

One approach that I'm wondering if others have tried before (haven't ran into this myself): Suppose code snippets on mobile are replaced with an icon, that when pressed open the snippet full screen, in landscape mode. The user presses the code example, flips their phone, then only needs to scroll vertically.

In a brief experiment, I get a line width of about 65 characters with reasonble fontsize that way. I believe 50-70 characters or about 10 words per line is considered about ideal for readability of regular text, so if main-body width is around 50-70 characters on a big screen, then 65-character wide examples will look great on the big screen too, at least typographically speaking.

Plus, if we can have a simple rule like "code examples should have a max linewidth of 65", and the result is the code is readable _and_ looks like same across different screen sizes, that creates a pretty nice experience writing code examples, because what-you-see-is-what-you-get.

Luke Boswell (Jan 04 2025 at 05:09):

I've been trying to isolate it and make a minimal repro. Been hacking code out and deleting things.

Luke Boswell (Jan 04 2025 at 05:13):

It's not related to the platform at all... this is just the package and compiler

Luke Boswell (Jan 04 2025 at 05:19):

Luke Boswell (Jan 04 2025 at 05:27):

Brendan Hansknecht (Jan 04 2025 at 15:24):

Brendan Hansknecht (Jan 04 2025 at 15:25):

WTF???
WTF???
app [main] { pf: platform "platform/main.roc" }

...

Anton (Jan 04 2025 at 15:25):

Anthony Bullard (Jan 04 2025 at 15:26):

Anthony Bullard (Jan 04 2025 at 15:27):

Anthony Bullard (Jan 04 2025 at 15:28):

Brendan Hansknecht (Jan 04 2025 at 15:28):

We probably should just force a merge and worry about the fuzzing issue separately

Brendan Hansknecht (Jan 04 2025 at 15:28):

Anthony Bullard (Jan 04 2025 at 15:29):

Anthony Bullard (Jan 04 2025 at 15:30):

Brendan Hansknecht (Jan 04 2025 at 15:31):

Not sure if it is a new regression that slipped in or something older that just took a while for the fuzzer to happen upon.

Anthony Bullard (Jan 04 2025 at 15:34):

Anthony Bullard (Jan 04 2025 at 15:35):

{
    some_field?
    """
    """,
}

Anthony Bullard (Jan 04 2025 at 15:36):

Brendan Hansknecht (Jan 04 2025 at 15:36):

Anthony Bullard (Jan 04 2025 at 15:37):

{
    some_field??
    """
    """,
}

Brendan Hansknecht (Jan 04 2025 at 15:38):

Brendan Hansknecht (Jan 04 2025 at 15:39):

Anthony Bullard (Jan 04 2025 at 15:41):

Anthony Bullard (Jan 04 2025 at 15:42):

Anthony Bullard (Jan 04 2025 at 15:44):

I would think this should be a PNC apply with the result of the whitespace application of @Y @Y as the func and then result of the whitespace application of @Y @Y as the first arg, so just (@Y @Y)(@Y @Y)

Anthony Bullard (Jan 04 2025 at 15:45):

Anthony Bullard (Jan 04 2025 at 15:47):

Anthony Bullard (Jan 04 2025 at 15:50):

Anthony Bullard (Jan 04 2025 at 15:57):

Just a formatting error, parses correctly. I’ll put the fix in with my current PR

Anthony Bullard (Jan 04 2025 at 19:27):

Anthony Bullard (Jan 04 2025 at 20:28):

Luke Boswell (Jan 05 2025 at 02:29):

$ cargo run -- test bug.roc

Luke Boswell (Jan 05 2025 at 02:29):

Anthony Bullard (Jan 05 2025 at 02:33):

Anthony Bullard (Jan 05 2025 at 02:39):

❯ RUST_BACKTRACE=1 ../roc/target/debug/roc test bug.roc
thread 'main' panicked at crates/compiler/mono/src/reset_reuse.rs:1244:42:
Expected symbol to have a layout. It should have been inserted in the environment already.
stack backtrace:
   0: rust_begin_unwind
             at /rustc/25ef9e3d85d934b27d9dada2f9dd52b1dc63bb04/library/std/src/panicking.rs:647:5
   1: core::panicking::panic_fmt
             at /rustc/25ef9e3d85d934b27d9dada2f9dd52b1dc63bb04/library/core/src/panicking.rs:72:14
   2: core::panicking::panic_display
             at /rustc/25ef9e3d85d934b27d9dada2f9dd52b1dc63bb04/library/core/src/panicking.rs:196:5
   3: core::panicking::panic_str
             at /rustc/25ef9e3d85d934b27d9dada2f9dd52b1dc63bb04/library/core/src/panicking.rs:171:5
   4: core::option::expect_failed
             at /rustc/25ef9e3d85d934b27d9dada2f9dd52b1dc63bb04/library/core/src/option.rs:1988:5
   5: expect<&roc_mono::reset_reuse::LayoutOption>
             at /rustc/25ef9e3d85d934b27d9dada2f9dd52b1dc63bb04/library/core/src/option.rs:894:21
   6: get_symbol_layout
             at /Users/anthonybullard/Development/roc/crates/compiler/mono/src/reset_reuse.rs:1244:9
   7: insert_reset_reuse_operations_stmt
             at /Users/anthonybullard/Development/roc/crates/compiler/mono/src/reset_reuse.rs:444:41
   8: insert_reset_reuse_operations_stmt
             at /Users/anthonybullard/Development/roc/crates/compiler/mono/src/reset_reuse.rs:516:36
   9: insert_reset_reuse_operations_stmt
             at /Users/anthonybullard/Development/roc/crates/compiler/mono/src/reset_reuse.rs:213:36
  10: insert_reset_reuse_operations_stmt
             at /Users/anthonybullard/Development/roc/crates/compiler/mono/src/reset_reuse.rs:516:36
  11: insert_reset_reuse_operations_stmt
             at /Users/anthonybullard/Development/roc/crates/compiler/mono/src/reset_reuse.rs:757:39
  12: insert_reset_reuse_operations_stmt
             at /Users/anthonybullard/Development/roc/crates/compiler/mono/src/reset_reuse.rs:516:36
  13: insert_reset_reuse_operations_stmt
             at /Users/anthonybullard/Development/roc/crates/compiler/mono/src/reset_reuse.rs:757:39
  14: insert_reset_reuse_operations_stmt
             at /Users/anthonybullard/Development/roc/crates/compiler/mono/src/reset_reuse.rs:516:36
  15: insert_reset_reuse_operations_stmt
             at /Users/anthonybullard/Development/roc/crates/compiler/mono/src/reset_reuse.rs:516:36
  16: insert_reset_reuse_operations_stmt
             at /Users/anthonybullard/Development/roc/crates/compiler/mono/src/reset_reuse.rs:213:36
  17: insert_reset_reuse_operations_stmt
             at /Users/anthonybullard/Development/roc/crates/compiler/mono/src/reset_reuse.rs:516:36
  18: insert_reset_reuse_operations_stmt
             at /Users/anthonybullard/Development/roc/crates/compiler/mono/src/reset_reuse.rs:213:36
  19: insert_reset_reuse_operations_proc
             at /Users/anthonybullard/Development/roc/crates/compiler/mono/src/reset_reuse.rs:82:20
  20: insert_reset_reuse_operations
             at /Users/anthonybullard/Development/roc/crates/compiler/mono/src/reset_reuse.rs:44:24
  21: update
             at /Users/anthonybullard/Development/roc/crates/compiler/load_internal/src/file.rs:2921:21
  22: state_thread_step
             at /Users/anthonybullard/Development/roc/crates/compiler/load_internal/src/file.rs:1765:25
  23: {closure#1}
             at /Users/anthonybullard/Development/roc/crates/compiler/load_internal/src/file.rs:2104:23
  24: {closure#0}<roc_load_internal::file::load_multi_threaded::{closure_env#1}, core::result::Result<roc_load_internal::file::LoadResult, roc_load_internal::file::LoadingProblem>>
             at /Users/anthonybullard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/crossbeam-utils-0.8.16/src/thread.rs:163:65
  25: call_once<core::result::Result<roc_load_internal::file::LoadResult, roc_load_internal::file::LoadingProblem>, crossbeam_utils::thread::scope::{closure_env#0}<roc_load_internal::file::load_multi_threaded::{closure_env#1}, core::result::Result<roc_load_internal::file::LoadResult, roc_load_internal::file::LoadingProblem>>>
             at /rustc/25ef9e3d85d934b27d9dada2f9dd52b1dc63bb04/library/core/src/panic/unwind_safe.rs:272:9
  26: do_call<core::panic::unwind_safe::AssertUnwindSafe<crossbeam_utils::thread::scope::{closure_env#0}<roc_load_internal::file::load_multi_threaded::{closure_env#1}, core::result::Result<roc_load_internal::file::LoadResult, roc_load_internal::file::LoadingProblem>>>, core::result::Result<roc_load_internal::file::LoadResult, roc_load_internal::file::LoadingProblem>>
             at /rustc/25ef9e3d85d934b27d9dada2f9dd52b1dc63bb04/library/std/src/panicking.rs:554:40
  27: ___rust_try
  28: try<core::result::Result<roc_load_internal::file::LoadResult, roc_load_internal::file::LoadingProblem>, core::panic::unwind_safe::AssertUnwindSafe<crossbeam_utils::thread::scope::{closure_env#0}<roc_load_internal::file::load_multi_threaded::{closure_env#1}, core::result::Result<roc_load_internal::file::LoadResult, roc_load_internal::file::LoadingProblem>>>>
             at /rustc/25ef9e3d85d934b27d9dada2f9dd52b1dc63bb04/library/std/src/panicking.rs:518:19
  29: catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<crossbeam_utils::thread::scope::{closure_env#0}<roc_load_internal::file::load_multi_threaded::{closure_env#1}, core::result::Result<roc_load_internal::file::LoadResult, roc_load_internal::file::LoadingProblem>>>, core::result::Result<roc_load_internal::file::LoadResult, roc_load_internal::file::LoadingProblem>>
             at /rustc/25ef9e3d85d934b27d9dada2f9dd52b1dc63bb04/library/std/src/panic.rs:142:14
  30: scope<roc_load_internal::file::load_multi_threaded::{closure_env#1}, core::result::Result<roc_load_internal::file::LoadResult, roc_load_internal::file::LoadingProblem>>
             at /Users/anthonybullard/.cargo/registry/src/index.crates.io-6f17d22bba15001f/crossbeam-utils-0.8.16/src/thread.rs:163:18
  31: load_multi_threaded
             at /Users/anthonybullard/Development/roc/crates/compiler/load_internal/src/file.rs:2040:29
  32: load
             at /Users/anthonybullard/Development/roc/crates/compiler/load_internal/src/file.rs:1550:35
  33: load
             at /Users/anthonybullard/Development/roc/crates/compiler/load/src/lib.rs:39:5
  34: load_and_monomorphize
             at /Users/anthonybullard/Development/roc/crates/compiler/load/src/lib.rs:143:11
  35: test
             at /Users/anthonybullard/Development/roc/crates/cli/src/lib.rs:586:27
  36: main
             at /Users/anthonybullard/Development/roc/crates/cli/src/main.rs:83:17
  37: call_once<fn() -> core::result::Result<(), std::io::error::Error>, ()>
             at /rustc/25ef9e3d85d934b27d9dada2f9dd52b1dc63bb04/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Luke Boswell (Jan 05 2025 at 02:40):

Ok, good to know... maybe it's only on Mac somehow. @Sam Mohr must have been using a linux machine

Anthony Bullard (Jan 05 2025 at 02:41):

Anthony Bullard (Jan 05 2025 at 02:42):

Luke Boswell (Jan 05 2025 at 02:42):

Yea've I've tried that. It doesn't give me much more info. I'm not sure how to get any more detail than just a symbol number

Anthony Bullard (Jan 05 2025 at 02:42):

Anthony Bullard (Jan 05 2025 at 02:43):

Luke Boswell (Jan 05 2025 at 02:43):

thread 'main' panicked at crates/compiler/mono/src/reset_reuse.rs:1244:42:
Expected symbol `bug.60` to have a layout. It should have been inserted in the environment already.
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Anthony Bullard (Jan 05 2025 at 02:43):

Anthony Bullard (Jan 05 2025 at 02:44):

Luke Boswell (Jan 05 2025 at 02:48):

This part of the compiler is harder for me to follow... it's like peering into the dark

Anthony Bullard (Jan 05 2025 at 02:50):

You could instrument all of the calls to Symbol::new and see what kind of thing goes in, and the symbol that comes out

Anthony Bullard (Jan 05 2025 at 02:51):

Sam Mohr (Jan 05 2025 at 02:57):

Brendan Hansknecht (Jan 05 2025 at 03:45):

Luke Boswell (Jan 05 2025 at 03:53):

Richard Feldman (Jan 05 2025 at 04:08):

Luke Boswell (Jan 05 2025 at 22:38):

I've got a couple of hours free... I'm going to poke at PR's and CI and try and land a bunch of things.

Luke Boswell (Jan 05 2025 at 22:39):

I'd like to finish the examples/ folder cleanup, and then start on removing Task from the builtins and tests

Anthony Bullard (Jan 05 2025 at 22:48):

Luke Boswell (Jan 05 2025 at 22:49):

It's a lesser of two evils, we're only going to give ourselves grief trying to keep Task around. The sooner we transition things across we can eliminate a lot of known bugs and rough edges.

Anthony Bullard (Jan 05 2025 at 22:49):

Anthony Bullard (Jan 05 2025 at 22:50):

Sam Mohr (Jan 06 2025 at 00:34):

I have about a 400 file collection of updating almost all idents to snake_case idents

Sam Mohr (Jan 06 2025 at 00:34):

I have a couple tests left to fix, but we can actually maybe update everything at once

Sam Mohr (Jan 06 2025 at 00:35):

Sam Mohr (Jan 06 2025 at 00:36):

Sam Mohr (Jan 06 2025 at 00:51):

Luke Boswell (Jan 06 2025 at 01:11):

I don't follow the logic of breaking it up. Is that to make it easier to roll back?

Sam Mohr (Jan 06 2025 at 01:14):

I'd rather not break it up, I just read the lack of response as people being unhappy with a big change.

Sam Mohr (Jan 06 2025 at 01:14):

Joshua Warner (Jan 06 2025 at 01:15):

Sam Mohr (Jan 06 2025 at 01:15):

Anthony Bullard (Jan 06 2025 at 01:19):

Anthony Bullard (Jan 06 2025 at 01:20):

Sam Mohr (Jan 06 2025 at 01:20):

Luke Boswell (Jan 06 2025 at 01:23):

@Sam Mohr to clarify, this is basically upgrading all the Builtins to snake_case right? not doing the sneaky Can case change thing in the background?

Sam Mohr (Jan 06 2025 at 01:23):

Sam Mohr (Jan 06 2025 at 01:24):

Luke Boswell (Jan 06 2025 at 01:24):

Sam Mohr (Jan 06 2025 at 01:24):

Sam Mohr (Jan 06 2025 at 01:25):

Ian McLerran (Jan 06 2025 at 01:27):

I'm all in favor of this as one big bandaid. I want to start migrating my packages to snake case, but hesitant to do so until I can migrate everything all at once.

Sam Mohr (Jan 06 2025 at 16:37):

module []

println! : Str => Result {} _

print_something! = \{} ->
    println!("Hello, world!")?

    Ok({})

❯ roc check test.roc
── UNKNOWN OPERATOR in test.roc ────────────────────────────────────────────────

This looks like an operator, but it's not one I recognize!

6│      println!("Hello, world!")?
                                 ^

I have no specific suggestion for this operator, see
https://www.roc-lang.org/tutorial#operator-desugaring-table for the
full list of operators in Roc.⏎

Anthony Bullard (Jan 06 2025 at 16:38):

Sam Mohr (Jan 06 2025 at 16:39):

Joshua Warner (Jan 06 2025 at 16:42):

This makes sense. The parsing logic for question mark suffixes needs to be moved to the same spot as we’re handling pnc args

Anthony Bullard (Jan 06 2025 at 16:43):

Luke Boswell (Jan 07 2025 at 22:13):

Just upgrading basic-cli... there is only one external dependency on another package. roc-json in one example.

Sam Mohr (Jan 07 2025 at 22:14):

I agree with this. We can just have an example using roc-json on the website or in examples

Sam Mohr (Jan 08 2025 at 00:13):

Seemingly, the last CI test failure in the snake_case change is that we are failing to generate the docs for basic-cli to host on the website.

Sam Mohr (Jan 08 2025 at 00:13):

Sam Mohr (Jan 08 2025 at 00:14):

But presumably for outdated reasons, we generate the docs for basic-cli in the compiler repo and host them with the main website

Sam Mohr (Jan 08 2025 at 00:14):

Luke made a good point that this may be something we want because it lets us host multiple versions of the docs

Sam Mohr (Jan 08 2025 at 00:14):

Sam Mohr (Jan 08 2025 at 00:15):

But I think we should just generate the docs for basic-cli in its own repo and host them there, as we do for basic-webserver. Any objections?

Anthony Bullard (Jan 08 2025 at 00:18):

I think we need to figure out a better solution long term, but that works for now

Sam Mohr (Jan 08 2025 at 00:18):

Luke Boswell (Jan 08 2025 at 00:21):

I wonder if we could just commit the html files into the repo under a folder structure...

www/
- 0.16.0/
- 0.17.0/

Sam Mohr (Jan 08 2025 at 00:21):

Luke Boswell (Jan 08 2025 at 00:21):

Then we don't need all the CI magic (and complications) in all the different repos

Luke Boswell (Jan 08 2025 at 00:22):

It's a bit handraulic, when making a release, but brain dead simple and easy to manage.

Sam Mohr (Jan 08 2025 at 00:22):

Sam Mohr (Jan 08 2025 at 00:23):

Do we want to block updating Roc by changing this process, or are we okay with just stealing the basic-webserver code for now and doing that later?

Luke Boswell (Jan 08 2025 at 00:25):

Luke Boswell (Jan 08 2025 at 00:26):

If it's reliable to host the static site versions this way, I'd lean towards removing the build stuff from roc's CI and just point at the basic-cli repo docs like webserver

Sam Mohr (Jan 08 2025 at 00:28):

Luke Boswell (Jan 08 2025 at 00:29):

Yeah, I just tried that... even making a test release in basic-cli but we cant without a new testing release of roc

Sam Mohr (Jan 08 2025 at 00:29):

Luke Boswell (Jan 08 2025 at 00:29):

So my idea above was just to do it manually (locally), and we can side-step all the CI madness when making/testing breaking changes like this

Luke Boswell (Jan 08 2025 at 00:30):

Sam Mohr (Jan 08 2025 at 00:31):

Luke Boswell (Jan 08 2025 at 00:43):

Luke Boswell (Jan 08 2025 at 00:44):

── EXPECT FAILED in package/Json.roc ───────────────────────────────────────────

This expectation failed:

657│>  # Test decode of F32
658│>  expect
659│>      actual : DecodeResult F32
660│>      actual = Str.toUtf8 "12.34e-5" |> Decode.fromBytesPartial utf8
661│>      numStr = actual.result |> Result.map Num.toStr
662│>
663│>      Result.withDefault numStr "" == "0.00012339999375399202"

When it failed, these variables had these values:

actual : DecodeResult F32
actual = { rest: [], result: Ok 0.0001234 }

numStr : Result Str DecodeError
numStr = Ok "0.0001234"


1 failed and 111 passed in 1252 ms.

Luke Boswell (Jan 08 2025 at 00:44):

Luke Boswell (Jan 08 2025 at 00:45):

Luke Boswell (Jan 08 2025 at 00:46):

Brendan Hansknecht (Jan 08 2025 at 00:48):

Luke Boswell (Jan 08 2025 at 00:48):

# Test decode of F64
expect
    actual : DecodeResult F64
    actual = Str.toUtf8 "12.34e-5" |> Decode.fromBytesPartial utf8
    numStr = actual.result |> Result.map Num.toStr

    Result.withDefault numStr "" == "0.0001234"

Brendan Hansknecht (Jan 08 2025 at 00:48):

Notification Bot (Jan 08 2025 at 01:17):

Anton (Jan 08 2025 at 10:00):

Can we keep the docs for 0.18.0, 0.17.0 etc. available? They don't need to be rebuild every time, they use a docs.tar.gz that's in the release assets, so it should not create issues with breaking changes in Roc

Luke Boswell (Jan 08 2025 at 10:01):

Luke Boswell (Jan 08 2025 at 10:02):

docs/
- 0.17.0/
- 0.18.0/

Luke Boswell (Jan 08 2025 at 10:05):

Ok, I'm going to merge that, configure and deploy the Pages site, and update the README

Luke Boswell (Jan 08 2025 at 10:07):

Anton (Jan 08 2025 at 10:12):

Luke Boswell (Jan 08 2025 at 23:04):

# default --migrate gives, but this looked a little confusing
try(Stdout.line!, "$(datetime) $(Inspect.to_str(req.method)) $(req.uri)")

# works, but has the dreaded interrobang
Stdout.line!?("$(datetime) $(Inspect.to_str(req.method)) $(req.uri)")

# what I expected, but doesn't compile right now
Stdout.line!("$(datetime) $(Inspect.to_str(req.method)) $(req.uri)")?

Sam Mohr (Jan 08 2025 at 23:05):

I might fix it accidentally in the Task removal, we'll see once I finish updating the code

Sam Mohr (Jan 08 2025 at 23:05):

Anthony Bullard (Jan 08 2025 at 23:06):

Luke Boswell (Jan 08 2025 at 23:07):

Anthony Bullard (Jan 08 2025 at 23:09):

Yeah, some guy is giving me a hard time in the review, but it'll get merged soon

Luke Boswell (Jan 08 2025 at 23:09):

Anthony Bullard (Jan 08 2025 at 23:09):

Anthony Bullard (Jan 08 2025 at 23:10):

Luke Boswell (Jan 08 2025 at 23:10):

Anthony Bullard (Jan 08 2025 at 23:10):

Luke Boswell (Jan 08 2025 at 23:11):

Run cargo nextest-gen-llvm --release --no-fail-fast --locked -E "package(test_gen) - test(gen_str::str_append_scalar)"
error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `/Users/username1/.cargo/bin/sccache /Users/username1/.rustup/toolchains/1.77.2-x86_64-apple-darwin/bin/rustc - --crate-name ___ --print=file-names --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=split-debuginfo --print=crate-name --print=cfg` (exit status: 2)
  --- stderr
  sccache: error: Timed out waiting for server startup
error: command `/Users/username1/.rustup/toolchains/1.77.2-x86_64-apple-darwin/bin/cargo test --no-run --message-format json-render-diagnostics --package test_gen --release --locked` exited with code 101
Error: Process completed with exit code 101.

Luke Boswell (Jan 08 2025 at 23:12):

Anthony Bullard (Jan 08 2025 at 23:12):

Luke Boswell (Jan 08 2025 at 23:13):

Anthony Bullard (Jan 08 2025 at 23:14):

Luke Boswell (Jan 08 2025 at 23:14):

Joshua Warner (Jan 08 2025 at 23:15):

FWIW I took a brief look at your replies on that PR @Anthony Bullard. Made sense to me. I can re-review after work today, but no concerns with going ahead and merging if you want to maintain momentum

Luke Boswell (Jan 08 2025 at 23:16):

I think we can either ignore it or restart the run... I lean towards merging now, and we can follow up if needed

Sam Mohr (Jan 08 2025 at 23:16):

Luke Boswell (Jan 08 2025 at 23:16):

Sam Mohr (Jan 08 2025 at 23:16):

Luke Boswell (Jan 08 2025 at 23:17):

And restarting the refcount PR in CI so we can see that's still ready to go (and also confirm the PNC change was a non-issue)

Luke Boswell (Jan 08 2025 at 23:31):

# THIS?
cwd =
    Env.cwd!({})
    |> Result.map_err(\CwdUnavailable -> Exit(1, "Unable to read current working directory"))
    |> try

# OR THIS?
cwd =
    Result.map_err(
        Env.cwd!({}),
        \CwdUnavailable -> Exit(1, "Unable to read current working directory")
    )?

# OR EVEN??
cwd =
    Env.cwd!({})
    |> Result.map_err?(\CwdUnavailable -> Exit(1, "Unable to read current working directory"))

Luke Boswell (Jan 08 2025 at 23:33):

I'm leaning towards the second because it feels closest to the PNC vision despite not having static dispatch to chain it

Sam Mohr (Jan 08 2025 at 23:33):

Luke Boswell (Jan 08 2025 at 23:34):

Sam Mohr (Jan 08 2025 at 23:39):

Though it's not quite the point of your example, this case is specifically handled by binop ?:

cwd =
    Env.cwd!() ? |CwdUnavailable|
        Exit(1, "Unable to read current working directory")

Sam Mohr (Jan 08 2025 at 23:41):

I think it's relevant here because the problematic cases are function calls with only a couple args in my eyes

Sam Mohr (Jan 08 2025 at 23:41):

Luke Boswell (Jan 08 2025 at 23:55):

Minor thing... but I don't think --migrate catches idents in import with as e.g.

import "todos.html" as todoHtml : List U8

Sam Mohr (Jan 08 2025 at 23:55):

Anthony Bullard (Jan 08 2025 at 23:57):

Sam Mohr (Jan 08 2025 at 23:58):

Luke Boswell (Jan 09 2025 at 00:34):

Sam Mohr (Jan 09 2025 at 00:37):

Luke Boswell (Jan 09 2025 at 00:38):

Sam Mohr (Jan 09 2025 at 00:38):

main! : _ => Result {} _
main! = \args ->

    parsed_args =
        Cli.parse_or_display_message(cli_parser, args, Arg.to_os_raw)
        |> try Result.on_err! \message -> Err (Exit 1 message)

Sam Mohr (Jan 09 2025 at 00:38):

Luke Boswell (Jan 09 2025 at 00:39):

I guess here we could... my heuristic has been -- if it's going to be a chain in SD then leave it for now

Sam Mohr (Jan 09 2025 at 00:41):

Fine by me, but we might need to make another pass when whitespace calling is removed, since I think that'll happen before static dispatch is ready

Sam Mohr (Jan 09 2025 at 00:41):

I expect that's the case because static dispatch is hard to implement, and \args -> going to |args| is easy to implement

Luke Boswell (Jan 09 2025 at 00:43):

I guess we can just cut a few testing releases and keep these breaking platform PRs in sync and do an actual release once we get |args| and ${interpolation} etc

Luke Boswell (Jan 09 2025 at 00:43):

It's easier to update them as we go, but does look a bit strange being in this intermediate state -- so we probably don't want to do a full release with what we currently have

Sam Mohr (Jan 09 2025 at 00:44):

Luke Boswell (Jan 09 2025 at 00:45):

We don't even need testing releases with the nix CI's, we just land the breaking change in roc main and we're good to go

Luke Boswell (Jan 09 2025 at 00:47):

Brendan Hansknecht (Jan 09 2025 at 18:12):

So much talking on Zulip lately. Have to skim tons of things cause I don't have the time to keep up.

Ayaz Hafiz (Jan 09 2025 at 18:21):

Sam Mohr (Jan 09 2025 at 18:22):

Richard Feldman (Jan 09 2025 at 18:30):

Notification Bot (Jan 09 2025 at 22:52):

Sam Mohr (Jan 10 2025 at 00:23):

I'm presuming that foo_! is how we'd represent a re-assignable effectful closure. Or would it be foo!_?

Luke Boswell (Jan 10 2025 at 00:24):

Sam Mohr (Jan 10 2025 at 00:24):

Brendan Hansknecht (Jan 10 2025 at 00:47):

Sam Mohr (Jan 10 2025 at 00:48):

Brendan Hansknecht (Jan 10 2025 at 01:42):

I don't actually feel that strongly against it. I could see building up a lazy computation with a for loop

Notification Bot (Jan 11 2025 at 04:03):

Joshua Warner (Jan 11 2025 at 05:41):

Sam Mohr (Jan 11 2025 at 05:56):

Ideally, an IgnoredValue field would be a runtime error outside of a record builder

Sam Mohr (Jan 11 2025 at 05:57):

Luke Boswell (Jan 12 2025 at 07:08):

Regarding PNC... my muscle memory definitely hasn't caught up yet. I'm really glad we have the --migrate flag, because I'm finding I write a lot of code before I realise, and then it's easy to just fix it up.

I assume at some point it will be natural to add the ( ) but for now it's taking mental effort to remember.

Luke Boswell (Jan 14 2025 at 22:07):

I'm WFH today, so should have time between things to poke at PR's etc. Let me know if you need anything.

Anthony Bullard (Jan 14 2025 at 22:07):

Sam Mohr (Jan 14 2025 at 22:08):

Luke Boswell (Jan 14 2025 at 22:08):

Luke Boswell (Jan 17 2025 at 02:54):

Did we accidentally or deliberately remove the todos.roc example from basic-cli? ... (I'm probably the guilty one)... I can't remember

Luke Boswell (Jan 17 2025 at 02:56):

Oh nvm, I think it's just @Brendan Hansknecht added todos.db for the sqlite example...

Luke Boswell (Jan 17 2025 at 02:56):

Anton (Jan 23 2025 at 18:50):

[crates/compiler/mono/src/reset_reuse.rs:1244:9] symbol = `temp.64`
[crates/compiler/mono/src/reset_reuse.rs:1244:9] symbol = `temp.data2`
[crates/compiler/mono/src/reset_reuse.rs:1244:9] symbol = `temp.orientation_str`
[crates/compiler/mono/src/reset_reuse.rs:1244:9] symbol = `temp.56`
[crates/compiler/mono/src/reset_reuse.rs:1244:9] symbol = `temp.54`
[crates/compiler/mono/src/reset_reuse.rs:1244:9] symbol = `temp.name_str`
[crates/compiler/mono/src/reset_reuse.rs:1244:9] symbol = `temp.64`
[crates/compiler/mono/src/reset_reuse.rs:1244:9] symbol = `temp.58`

Are the numbered symbols like temp.58 for variables that we make ourselves like during desugaring or something like that?

Luke Boswell (Jan 23 2025 at 20:42):

Brendan Hansknecht (Jan 24 2025 at 02:31):

I think they are also for all intermediates when we breakdown expressions, but would need to double check that.

Sam Mohr (Jan 24 2025 at 02:31):

Sam Mohr (Jan 24 2025 at 02:32):

Sam Mohr (Jan 26 2025 at 10:08):

If we always converted if statements to when bool is 1 -> ...; _ -> ... there wouldn't be a perf drop, would there?

Sam Mohr (Jan 26 2025 at 10:08):

Sam Mohr (Jan 26 2025 at 10:09):

Sam Mohr (Jan 26 2025 at 10:28):

Richard Feldman (Jan 26 2025 at 13:33):

should be the same perf, although we can give nicer error messages for type mismatches if we know it was an if compared to a when

Sam Mohr (Jan 26 2025 at 13:33):

Richard Feldman (Jan 26 2025 at 13:33):

and that relies on constraint gen knowing it was an if compared to a when, and constraint gen (currently) does a separate pass over the canonical IR

Sam Mohr (Jan 26 2025 at 13:34):

Richard Feldman (Jan 26 2025 at 13:34):

Sam Mohr (Jan 26 2025 at 13:34):

Richard Feldman (Jan 26 2025 at 13:35):

incidentally, I think we can potentially do it differently someday (where we generate constraints at the same time as making the canonical IR) but it's tricky because order of constraints matters, and we need them to be sorted by dependencies (e.g. first constrain functions whose types don't depend on other functions, etc.) - right now, doing the constraint pass after canonicalization (and sorting the canonical IR) takes care of that, but if we wanted to do (constrain + canonicalize) at the same time, then we'd need to have the ability to sort the constraints separately after the fact

Richard Feldman (Jan 26 2025 at 13:35):

since we don't know the dependency order until after canonicalization has completely finished

Richard Feldman (Jan 26 2025 at 13:36):

so I don't think now is the right time to do that since it might break things if we get it wrong :big_smile:

Sam Mohr (Jan 26 2025 at 13:36):

It seems like the lesson of the moment is to do everything in separate passes for greater likelihood of correctness

Richard Feldman (Jan 26 2025 at 13:36):

Richard Feldman (Jan 26 2025 at 13:37):

and then in the future we can experiment with incrementally combining things to see if they make that one step faster while maintaining correctness

Richard Feldman (Jan 26 2025 at 13:37):

Anton (Jan 27 2025 at 11:20):

emma (Jan 27 2025 at 12:31):

is there any docs/papers on how tags are inferred? it seems like they break the fundamental unification rule of not allowing type variables to be solved to a type containing themself.

Anton (Jan 27 2025 at 12:38):

Ayaz Hafiz (Jan 27 2025 at 14:51):

@emma the topic in literature is "polymorphic variants", roc takes a combination of approaches

Ayaz Hafiz (Jan 27 2025 at 14:51):

Can you elaborate on the unification rule you're describing? I'm not sure I follow the problem fully

Brendan Hansknecht (Jan 27 2025 at 16:41):

This makes some sense, but generally nano pass compilers are measurably very slow. I think it is better to make reasonably sized passes than nano passes. But perf depends on a lot of things. So leaning toward the simplest first makes the most sense.

Joshua Warner (Jan 29 2025 at 02:00):

For a long time I've wanted to try to "nanopass, but use some compiler magic to merge passes and make it fast"

Sam Mohr (Jan 29 2025 at 11:26):

Seems like the auto-annotate type signatures PR is ready to merge. Anyone against doing that now? I don't think it will interfere with the release process we're working on right now

Anton (Jan 29 2025 at 11:32):

Ayaz Hafiz (Jan 30 2025 at 06:44):

Sam Mohr (Jan 30 2025 at 06:58):

Ayaz Hafiz (Jan 30 2025 at 06:58):

Sam Mohr (Jan 30 2025 at 06:59):

Ayaz Hafiz (Feb 02 2025 at 23:21):

Anthony Bullard (Feb 02 2025 at 23:21):

Ayaz Hafiz (Feb 02 2025 at 23:21):

Anthony Bullard (Feb 02 2025 at 23:22):

Ayaz Hafiz (Feb 02 2025 at 23:22):

Luke Boswell (Feb 02 2025 at 23:23):

My plan is to hang in there as long as possible... and hopefully I'll learn the dark arts of roc's compiler internals by osmosis -- reading all the discussion and seeing the PR's roll in.

Sam Mohr (Feb 02 2025 at 23:45):

Man, I'm glad we have the rest of the team for the low-level stuff. I never had a good introduction to it and it still feels like black magic. I look forward to reading lots of PRs that might educate me!

Anthony Bullard (Feb 02 2025 at 23:57):

jan kili (Feb 03 2025 at 16:29):

3489a045-b487-43da-b4c6-48ab2b961b0b.png
lol I was caught up on all threads yesterday, this has gotta be some kind of record in here :clap::saluting_face::popcorn:

Brendan Hansknecht (Feb 03 2025 at 19:27):

Wait.... #compiler development is a different color for different people....that bugs me.

Brendan Hansknecht (Feb 03 2025 at 19:28):

Isaac Van Doren (Feb 03 2025 at 20:45):

Sam Mohr (Feb 03 2025 at 20:46):

Jasper Woudenberg (Feb 03 2025 at 20:48):

emma (Feb 05 2025 at 14:47):

@Ayaz Hafiz apologies for the very late reply - I'm aware of the term "polymorphic variants" but I haven't found any (simple) descriptions of how it gets implemented on top of typical HM systems - it seems to be hard to fit into the usual unification rules. Do you know if theres any good explanations of how roc or other languages implement it?

Ayaz Hafiz (Feb 05 2025 at 14:54):

there are a ton of papers from Leijen etc. but I don't know how interpretable they might be. Daan Leijen’s “Extensible records” paper is the simplest approach I know of, and is similar to the approach Elm/Roc use. I'll give a short description of the mechanism in Roc, feel free to ask for elaboration if it's confusing or incomplete. The approach for records is to bind a type variable to a record that unifies freely; for example the call (fun r -> r.x) {x: 1, y: 1} types the function as {x: a}b -> a (where b is free) and the record as {x: int, y: int}c (where c is free); {x: a}b ~ {x: int, y: int}c yields b = {y: int}d, c = {}d (where d is fresh).
The approach for variants is the dual. That is, an expression `A 1 = `B "foo" types the LHS and RHS as [A int]a and [B str]b respectively, the constraint [A int]a ~ [B str]b then solves a = [B str]c, b = [A int]c and both the LHS and RHS have type [A int, B str]c. However, a function [A, B]a -> {} is materially different than [A, B] -> {} - if you have a pattern match over the input on the former, you always need a catch-all branch, whereas a pattern match on the latter only needs a branch for A and B. To handle this, Roc selectively removes the remaining unbound variable (a in [A, B]a above) depending on the usage pattern of the variable, for example if it is used in an pattern match without a catch-all branch.

osa1 (Feb 05 2025 at 15:35):

osa1 (Feb 05 2025 at 15:38):

We also had a discussion here with @Ayaz Hafiz on variants and some of the counter intuitive type checking with rows, that may also be helpful. (gotta run now so can't find the thread, sorry)

Wizard ish (Feb 07 2025 at 03:28):

Ok yall this might just be sleep deprivation talking but what do yall think about a C++ concepts / rewriting rules feature in Roc

Wizard ish (Feb 07 2025 at 03:29):

That is to say, we have a way to say “if this expression is well formed, then for some other expression we can derive its meaning”

Sam Mohr (Feb 07 2025 at 03:31):

Wizard ish (Feb 07 2025 at 03:33):

Wizard ish (Feb 07 2025 at 03:34):

So, if we have a function f on x to y, we can apply f to Maybe x without specifying that it is a map

Wizard ish (Feb 07 2025 at 03:36):

Basically, allowing code to be fully modular (a more complex structure can always be used in place of a simpler one)

Sam Mohr (Feb 07 2025 at 03:43):

So this is a combo of Ocaml's module functors and Zig's "everything is a struct", right?

Wizard ish (Feb 07 2025 at 03:45):

Wizard ish (Feb 07 2025 at 03:46):

Sam Mohr (Feb 07 2025 at 03:46):

I think we've so far been successful in reducing the complexity of features in Roc to basically flavors of functions and values (values being sum or product types) with some light desugaring thrown on top. This comes with the recent push to remove proper typeclasses in the form of abilities and module params, since the simpler Roc without them can still be as productive as necessary

Sam Mohr (Feb 07 2025 at 03:46):

I think adding a feature like that would be cool and I can definitely think of places we could get use out of it

Sam Mohr (Feb 07 2025 at 03:46):

Sam Mohr (Feb 07 2025 at 03:47):

Wizard ish (Feb 07 2025 at 03:49):

Anton (Feb 07 2025 at 18:49):

I didn't expect zig to be so similar to Roc. If zig did not exist and we made a low-level language starting from our Roc experience, we could have ended up very close to zig :)

jan kili (Feb 07 2025 at 18:53):

Brendan Hansknecht (Feb 07 2025 at 19:29):

What makes you say this? Just curious. I wouldn't consider them very similar, but I guess it depends on which aspect you re comparing.

Anton (Feb 08 2025 at 11:19):

error union, try, switch, compiled, likes to go fast, type inference, structs are similar to records, no shadowing, generics, snake_case variables, PascalCase types, likes to keep it simple

Lucas Rosa (Feb 08 2025 at 17:21):

saw the switching to zig thing, very cool, very very cool. Also nice that it's happening right in the repo next to the existing code. I'll keep an eye out, been needing an excuse to write some zig. I come to you now at the turn of the tide.

Anton (Feb 08 2025 at 17:25):

Lucas Rosa (Feb 08 2025 at 17:28):

nice to see you too Anton. My Aiken side quest was fruitful and it got the niche adoption it needed to last haha. New gig is compiler stuff for this language called compact that compiles to zero knowledge circuits.

Brendan Hansknecht (Feb 08 2025 at 17:32):

Lucas Rosa (Feb 08 2025 at 17:37):

Anton (Feb 08 2025 at 18:31):

Interesting, I've forgotten the details of how zero knowledge proofs work but I do remember that I thought it was cool :big_smile:

Brendan Hansknecht (Feb 09 2025 at 02:38):

I just locally added a .ignore file that ignores the crates directory. So nice to have helix only seeing the new compiler and limited files.

Anthony Bullard (Feb 09 2025 at 15:34):

Brendan Hansknecht (Feb 09 2025 at 17:42):

Sam Mohr (Feb 10 2025 at 16:40):

Seems like a lot of people are excited to contribute because of the accessibility of a rewrite! You love to see it

Sam Mohr (Feb 10 2025 at 22:30):

Is there even a syntax planned to allow making custom tags for custom unions usable within the same module in the way that you can import exposing [Result.[Ok, Err]]?

Sam Mohr (Feb 10 2025 at 22:31):

Sam Mohr (Feb 12 2025 at 07:18):

@Joshua Warner do you have a plan for how the compiler post-parsing will know the "region" for some entity?

Sam Mohr (Feb 12 2025 at 07:19):

Will it be a pair of token IDs, a start and an end index into the source file, or a line and a column?

Sam Mohr (Feb 12 2025 at 07:19):

Or is that info not expected to last past parsing/maybe formatting in this compiler

Joshua Warner (Feb 12 2025 at 15:12):

Joshua Warner (Feb 12 2025 at 15:13):

Joshua Warner (Feb 12 2025 at 15:14):

Joshua Warner (Feb 12 2025 at 15:16):

The possibly compelling alternative would be to use a node id for a parser node - that’s a single u32 that can be mapped back to a precise source range when needed, and is smaller that the u32 pair needed for a byte range

Sam Mohr (Feb 23 2025 at 06:23):

When someone runs roc check Module.roc and Module.roc is not the root of its package, or at least not the only file in the package, which set of diagnostics do we want to show them?

It seems pretty easy to do the last one, and I'm not sure if someone would want the other cases, though maybe I'd understand the second one. It also shouldn't be that hard to implement.

Sam Mohr (Feb 23 2025 at 10:22):

Also, we currently allow setting an alternative package main file using a query fragment in the package download URL, and I don't think there's a need for that anymore. Anyone opposed to me removing that?

Richard Feldman (Feb 23 2025 at 13:03):

Richard Feldman (Feb 23 2025 at 13:04):

one way to think about this is "if I want a particular set of outputs, is there any possible way to convince the compiler to give them to me?"

Sam Mohr (Feb 23 2025 at 13:04):

Richard Feldman (Feb 23 2025 at 13:04):

there's certainly already a way to say "give me all the errors from the package"

Richard Feldman (Feb 23 2025 at 13:04):

Richard Feldman (Feb 23 2025 at 13:05):

so then the question is, if I say "I want roc check to check just this one file" - do I also want it to give me the errors from its dependencies, or not?

Sam Mohr (Feb 23 2025 at 13:05):

The decision between the first two in my mind is "if we give only the errors for a single module even if its deps are wrong, then is that correct?"

Richard Feldman (Feb 23 2025 at 13:05):

like "I know about those, I'm not working on them right now" vs. "I want to know if this thing completely works"

Sam Mohr (Feb 23 2025 at 13:06):

Lets say my module imports something that's malformed. We won't report two errors, so it'll look like my usage is correct even if the thing I imported is broken

Sam Mohr (Feb 23 2025 at 13:07):

Doing just the current file and nothing else should be easier to implement actually

Sam Mohr (Feb 23 2025 at 13:07):

We naively can't use the toposorted module list because its sorted over all modules

Sam Mohr (Feb 23 2025 at 13:08):

Sam Mohr (Feb 23 2025 at 13:09):

Sam Mohr (Feb 23 2025 at 13:10):

And we can do the option of "my module and its deps" later since it's more work but not obviously the right option

Brendan Hansknecht (Feb 23 2025 at 16:28):

The big thing for me is that I think it should fully check the single module if possible

Brendan Hansknecht (Feb 23 2025 at 16:28):

Brendan Hansknecht (Feb 23 2025 at 16:30):

I don't think it needs to recursively check all dependencies even things that aren't used.

Sam Mohr (Feb 23 2025 at 21:57):

Yes, I think we should do typechecking and all for at least the module and its deps. I'm just suggesting that we only report the errors for that module

Richard Feldman (Feb 23 2025 at 21:59):

Wizard ish (Feb 23 2025 at 22:15):

out of curiosity, what is the theoretical / computational model that Roc's type system uses?

Luke Boswell (Feb 23 2025 at 22:18):

Luke Boswell (Feb 23 2025 at 22:19):

This is a new architecture that @Ayaz Hafiz laid out to solve our issues with compiling Lambda Sets

Luke Boswell (Feb 23 2025 at 22:20):

Sam Mohr (Mar 03 2025 at 08:30):

My first pass at implementing the coordinate logic has just been merged in (PR), meaning that the main thrust of the work to typecheck or build a module (and its deps) is put together, outside of running the code with the interpreter/LLVM. I'm sure there are some bugs in the impl, but we'll want to get something basic working for canonicalization and import resolution to work said bugs out, so those stages are my next project.

As a side benefit, I was able to remove most of the refAllDeclsRecursive calls from our testing entrypoint.

Sam Mohr (Mar 04 2025 at 00:46):

So... argument order. I tend to write functions with "stuff to focus on" near the front of the arg list, and "needed context" near the end of the arg list. This is the Roc way to do it, and usually the Rust way as well. But in Zig, the allocator seems to usually get passed first (or second after the method target). What's the right pattern here?

Brendan Hansknecht (Mar 04 2025 at 00:50):

So I would say env/allocator first (and I assume env stores the allocator so you never need both)

Brendan Hansknecht (Mar 04 2025 at 00:51):

But ultimately it is pretty minor, it is more important we aim to be consistent than which choice we pick

Sam Mohr (Mar 04 2025 at 00:51):

But just the allocator? A lot of our functions in the Rust-based compiler use env and arena. Would it be better to do (gpa, important, other, env) or (gpa, env, important, other)?

Sam Mohr (Mar 04 2025 at 00:51):

Yeah, I was gonna vote we put the gpa in the env to make it less of a pain to use unmanaged lists, and then convert the Safe(Multi)List types to unmanaged

Brendan Hansknecht (Mar 04 2025 at 00:52):

I personally would lean to env, gpa, important, other. It feels most conventionally correct.

Brendan Hansknecht (Mar 04 2025 at 00:52):

Then you only need to pass around local arenas (which I think will be relatively rare)

Sam Mohr (Mar 04 2025 at 00:52):

Sam Mohr (Mar 04 2025 at 04:18):

Sam Mohr (Mar 04 2025 at 04:19):

It's extremely mechanical as PRs come, but it's 32 files, so no pressure if you don't have time

Sam Mohr (Mar 04 2025 at 04:34):

@Brendan Hansknecht what's the strategy for avoiding for (items) |item| item.deinit() deeper in our data types? Making an arena just for that collection, or making a top-level arena that gets passed around, or something else?

Brendan Hansknecht (Mar 04 2025 at 05:00):

Depends on the specific use case. One option is definitely to just use an arena for the items if they never grow and that is reasonable. Another option would be to flatten things out more. That way instead of a list of lists, you have a list of slices and then a second list of data.

I think flattening is what would make most sense for exposed_idents. Cause exposed_idents is a static slice of data essentially. So instead of nesting you would have a larger context ollections.SafeList(Ident.Idx). Then you would have each ModuleImport include exposed_idents as a Slice into the list.

Brendan Hansknecht (Mar 04 2025 at 05:00):

Sam Mohr (Mar 04 2025 at 05:01):

Brendan Hansknecht (Mar 04 2025 at 05:02):

Sam Mohr (Mar 04 2025 at 09:13):

Do we want to support two imports of the same external module in a single module? Feels like yes or no, it doesn't matter much. It feels like there's not much of a reason to want that except for only exposing some ident in a more narrow scope. At least at the top level, I don't see it being much besides a fragmentation in imports that makes reading harder. Could we maybe put warnings on subsequent imports at the top level?

Sam Mohr (Mar 04 2025 at 09:14):

Sam Mohr (Mar 04 2025 at 09:16):

I'm being reminded by the parsed AST that we can do import Foo as AliasedFoo, and that might be useful to allow two of.

Sam Mohr (Mar 04 2025 at 09:16):

I'll forgo the warning for now to trim complexity and therefore implementation time

Richard Feldman (Mar 04 2025 at 12:44):

I think it makes sense to treat imports like declarations: shadowing is a warning

Richard Feldman (Mar 04 2025 at 12:45):

Anthony Bullard (Mar 05 2025 at 02:55):

Brendan Hansknecht (Mar 05 2025 at 02:56):

Anthony Bullard (Mar 05 2025 at 02:56):

It's not gonna be too bad, just tedious. I'll probably just checkout my copy of all of these and fix the errors by hand

Anthony Bullard (Mar 05 2025 at 02:57):

Sam Mohr (Mar 05 2025 at 02:57):

Anthony Bullard (Mar 05 2025 at 02:58):

Anthony Bullard (Mar 05 2025 at 02:59):

I could get rid of the iterator completely honestly, and would do that over returning a slice over the extra data

Sam Mohr (Mar 05 2025 at 03:01):

The *Iter functions just create an Iter around an ArrayList of the extra_data

Sam Mohr (Mar 05 2025 at 03:02):

So you can make that happen by making the desired slice into the UnmanagedArrayList and calling @ptrCast to change the []const u32 into []const StatementIdx

Anthony Bullard (Mar 05 2025 at 03:03):

            var i: = @as(usize, @intCast(t.annos.span.start));
            while (i < t.annos.span.start + t.annos.span.len)  {
                const ann = TypeAnnoIdx{ .id = fmt.ast.store.extra_data.items[i] };
                fmt.formatTypeAnno(an);
                if (i < (t.annos.span.len - 1)) {
                    fmt.pushAll(", ");
                }
                i += 1;
            }

            var anno_iter = fmt.ast.store.typeAnnoIter(t.annos);
            while (anno_iter.next()) |an| {
                fmt.formatTypeAnno(an);
                if (i < (t.annos.span.len - 1)) {
                    fmt.pushAll(", ");
                }
                i += 1;
            }

Sam Mohr (Mar 05 2025 at 03:03):

I literally made the change in my editor for expressions and everything compiled

Sam Mohr (Mar 05 2025 at 03:04):

            for (fmt.ast.store.typeAnnoSlice(t.annos)) |an| {
                fmt.formatTypeAnno(an);
                if (i < (t.annos.span.len - 1)) {
                    fmt.pushAll(", ");
                }
            }

Anthony Bullard (Mar 05 2025 at 03:04):

Sorry, I misread the code you wrote. I'm literally was trying to avoid slices here at all

Sam Mohr (Mar 05 2025 at 03:04):

Anthony Bullard (Mar 05 2025 at 03:05):

Well the first option doesn't create a struct of any sort if that's what we are optimizing for

Sam Mohr (Mar 05 2025 at 03:05):

Sam Mohr (Mar 05 2025 at 03:06):

Anthony Bullard (Mar 05 2025 at 03:06):

Sam Mohr (Mar 05 2025 at 03:06):

I think it's more terse and at least as performant, if not sketchy maybe faster

Sam Mohr (Mar 05 2025 at 03:07):

Anthony Bullard (Mar 05 2025 at 03:09):

Ok, I just wish that kind of feedback could have came on the first PR - whose purpose was to get such feedback so I applied something everyone was on-board with before doing it everywhere.

But luckily this is largely a mechanical change - maybe I could even make it comptime and reduce the number of unique functions I have to maintain here

Sam Mohr (Mar 05 2025 at 03:09):

Anthony Bullard (Mar 05 2025 at 03:09):

Anthony Bullard (Mar 05 2025 at 03:10):

Sam Mohr (Mar 05 2025 at 03:10):

Anthony Bullard (Mar 05 2025 at 03:11):

I didn't think about that before and was worried about slices in general - but that's only an issue when we are appending to them (and they can resize and invalidate pointers)

Sam Mohr (Mar 05 2025 at 03:12):

Anthony Bullard (Mar 05 2025 at 03:13):

Anthony Bullard (Mar 05 2025 at 03:26):

Yeah, not going to try to comptime it. it really won't save much code since Idxs and DataSpans aren't Generic. I'd have to write a complicated comptime block that seems too clever by half

Anthony Bullard (Mar 05 2025 at 03:27):

I'll just replace the Iter functions with Slice functions and remove the Iterator and IdIterator structs and the type aliases

Anthony Bullard (Mar 05 2025 at 12:14):

Jared (Mar 07 2025 at 15:08):

Do y'all already have a plan for how to do type inference w/ multiple static dispatch? (the planned replacement for abilities). I looked into how koka does it, and it leaves a lot to be desired.

Richard Feldman (Mar 07 2025 at 17:37):

Jared (Mar 07 2025 at 17:45):

Jared (Mar 07 2025 at 17:46):

I figured it meant koka-style "there are multiple functions in scope with the same name, and we use type inference to determine which one to use". It's a way of doing ability-like or typeclass-like behavior

Brendan Hansknecht (Mar 07 2025 at 18:23):

If we see x.foo(a, b, c). We query the type of x. When then load the module that type is from and call ModuleOf(x).foo(x, a, b, c)

Brendan Hansknecht (Mar 07 2025 at 18:24):

Sam Mohr (Mar 07 2025 at 18:25):

@Brendan Hansknecht we actually plan to allow multiple custom types per module, though a single one per module is preferred

Brendan Hansknecht (Mar 07 2025 at 18:25):

Sam Mohr (Mar 07 2025 at 18:25):

Brendan Hansknecht (Mar 07 2025 at 18:26):

Type doesn't really matter (past finding the module) cause we don't have namespaces within modules. So the module + function name is a unique identifier. Of course can still get a type mismatch if you use the wrong custom type.

Sam Mohr (Mar 07 2025 at 18:27):

Jared (Mar 07 2025 at 18:33):

yeah so "if we see x, we query the type of x", if implemented in the koka way, really restricts things in an annoying way. For example, in koka, if you have

fun x(y) {
   y.z() + 2
}

if there are two z functions available, one returning an int and one returning a string, you would think that this should be inferrable because the return value is being used as an int. But koka chokes on this, requiring you to add an annotation or disambiguate the z with a prefix. imo this forfeits one of the key benefits of hindley-milner-style type inference, namely that the "order" of expressions doesn't matter, all information is essentially considered "at once".

Jared (Mar 07 2025 at 18:34):

I've been thinking about how to implement something like this for my own language, and I think the route I'll take is, if you see a z that is ambiguous, just give it a type variable, and let inference continue. At the end of inference, find all the type variables that were created in this manner, and then use the type information that resulted from the inference to disambiguate.

Sam Mohr (Mar 07 2025 at 18:35):

x : y -> Num a where y.z() -> Num a
x = |y| y.z() + 2

Sam Mohr (Mar 07 2025 at 18:36):

In that y.z() is inferred as y where y.z() -> b and then b is constrained to some number type Num a because of the addition

Jared (Mar 07 2025 at 18:37):

ok and if there's later in that function something that makes the type of y concrete, you then do the module lookup to satisfy the "where" bound?

Sam Mohr (Mar 07 2025 at 18:38):

Jared (Mar 07 2025 at 18:38):

Jared (Mar 07 2025 at 18:39):

Sam Mohr (Mar 07 2025 at 18:39):

Jared (Mar 07 2025 at 18:39):

Sam Mohr (Mar 07 2025 at 18:40):

It means "in the module where the custom type y is defined, there needs to be a function z that returns a type that unifies with Num a, and the function z needs to take a y as the first and only arg"

Sam Mohr (Mar 07 2025 at 18:40):

We discussed what you're calling multiple dispatch a little bit and it got ruled out for ambiguity reasons

Jared (Mar 07 2025 at 18:43):

Brendan Hansknecht (Mar 07 2025 at 18:43):

Jared (Mar 07 2025 at 18:44):

this does mean you would be able to have multi-type traits, but that seems like a reasonable sacrifice

Brendan Hansknecht (Mar 07 2025 at 18:45):

Also, you can explicitly do module(y).foo(...). This is needed for things like decode where the type is not an arg.

Brendan Hansknecht (Mar 07 2025 at 18:45):

Brendan Hansknecht (Mar 07 2025 at 18:46):

Trevor Settles (Mar 19 2025 at 02:48):

Is anyone doing some collaborative coding, like pairing or ensemble programming while working on the compiler? I had tried to do some work on the rust compiler, but never quite found a foothold in the codebase. It seems like now might be a good time to get back into it, since things are smaller and a little more well-thought-out

Brendan Hansknecht (Mar 19 2025 at 03:12):

There definitely has been some. That said, less so recently (just the ebb and flow as folks getting busy). That said, I'm sure we can make more time for it.

Sam Mohr (Mar 19 2025 at 04:44):

Trevor Settles (Mar 19 2025 at 04:51):

Ok, cool. Thanks for the info! I'm coming back from vacation on Saturday, so I'll check them

Luke Boswell (Mar 19 2025 at 05:09):

I'm in the middle of a move interstate, so my availability for roc has dropped off a little this week. I'm definitely keen for more collaborative coding. @Trevor Settles happy to spend time with you or anyone else interested, and sharing what I know about things.

Now is a great time to get involved with the new zig compiler work. There are lots of parts that are basically just stubbed out with a TODO or placeholder waiting for someone to give it some love.

If there isn't a draft PR then you can probably assume it's not being looked at or worked on by someone else.

I think for anyone who wants to help -- getting involved by reviewing PR's and leaving comments, contributing ideas to discussions, and reading the code/making PR's are all really helpful.

Sam Mohr (Mar 19 2025 at 05:10):

And if you want specific feature work, we can carve our a specific chunk for you, big or small

Reed Harston (Mar 20 2025 at 03:49):

Hello everyone, my name is Reed Harston.
I just came here from the Zig Showtime interview about the compiler rewrite.
Roc looks interesting, and I’d like to try it, but I don’t have any projects right now that seem like a good fit. But as soon as Richard said Zig people are welcome to come pitch in without even using Roc I paused the video and came right here. :big_smile:

I won’t have time right now to really dig into development, but I’d love to help out so I was thinking about jumping in and commenting on PRs and the like, and it turns out that just two message ago Luke said that would be helpful, so I plan on starting there.

Sam Mohr (Mar 20 2025 at 03:55):

Luke Boswell (Mar 20 2025 at 06:41):

Awesome @Reed Harston, any feedback on PRs are very welcome. I'm a zig noob and have been having a blast learning as I go. It's a different vibe from Rust, but so far it's been really great. I'm super excited for the builtin fuzzing features.

Brendan Hansknecht (Mar 21 2025 at 00:27):

Anyone interested in an ad-hoc contributor meeting this weekend? Chat current status. Chat what could use support. Chat what is needed to unblock various work. Also, I'll show off tracy for profiling.

Richard Feldman (Mar 21 2025 at 00:32):

Brendan Hansknecht (Mar 21 2025 at 00:34):

Isaac Van Doren (Mar 21 2025 at 03:03):

Brendan Hansknecht (Mar 21 2025 at 19:28):

Ping for a few folks to see if they have interest in joining/availability to add: @Luke Boswell @Sam Mohr @Trevor Settles @Anthony Bullard @Joshua Warner

Joshua Warner (Mar 21 2025 at 19:34):

Would like to join if possible, but very hard to know ahead of time if any particular time slot works

Luke Boswell (Mar 21 2025 at 20:19):

Sorry, I'd love to but not available at all this weekend. In the middle of moving house (like the actual moving part).

Brendan Hansknecht (Mar 22 2025 at 03:27):

Sam Mohr (Mar 22 2025 at 17:31):

Brendan Hansknecht (Mar 22 2025 at 17:34):

Sam Mohr (Mar 22 2025 at 17:37):

Loris Cro (Mar 22 2025 at 17:40):

Richard Feldman (Mar 22 2025 at 17:42):

Anthony Bullard (Mar 25 2025 at 10:58):

Sorry y'all! I've been super heads down the past couple of weeks with a major work project and family stuff

Anthony Bullard (Mar 25 2025 at 10:59):

Anton (Mar 25 2025 at 11:46):

Anton (Apr 18 2025 at 14:00):

Reed Harston (Apr 18 2025 at 14:12):

Reed Harston (Apr 18 2025 at 14:13):

Anton (Apr 18 2025 at 14:14):

Roc can do it too :) but I was just modifying an existing bash script that I thought was going to be a quick fix :p

Reed Harston (Apr 18 2025 at 14:15):

Joshua Warner (Apr 18 2025 at 15:20):

Lol; at my last company we had a script called deploy.sh (the name of the entrypoint, but there were probably 100+ files involved). It was IIRC 25k+ lines.

Anthony Bullard (Apr 18 2025 at 21:22):

The only place I’ve worked where bash was reasonable was Google. They have so much tooling around it it’s pretty nice

Anthony Bullard (Apr 18 2025 at 21:22):

Joshua Warner (Apr 18 2025 at 22:07):

I was very impressed at how functional and usable that script was, despite all the problems

Brendan Hansknecht (Apr 18 2025 at 23:23):

Now that I have used python a lot more, I feel less and less confident that anyone should use it. Like bash, convenient in the small. But to go to medium, you really need type checking and python type checking is slow and pretty buggy. Yes it has tons of libraries and scales farther than bash, but it kinda really sucks except in super interactive notebook style flows. And even in that case you often get stuck when you realize that production requires leaving the interactive flow and then you want fast type checking and what not.

Brendan Hansknecht (Apr 18 2025 at 23:23):

So I guess I put python minorly ahead of bash, but a lot less far ahead than one would hope.

Wizard ish (Apr 19 2025 at 02:08):

yeah i litterally just had a discussion about how python is essentially the C(++) of scripting languages, it just had things piled on top of it until it barely works anymore

Brendan Hansknecht (Apr 19 2025 at 02:24):

Wizard ish (Apr 19 2025 at 02:30):

Yeah, it may be the C++ of scripting languages but it is the C++ of scripting languages :grinning_face_with_smiling_eyes:

Reed Harston (Apr 19 2025 at 03:53):

Reed Harston (Apr 19 2025 at 03:54):

The funny thing is, at work the two languages we use are C++ and Python. Just those two.

Brendan Hansknecht (Apr 19 2025 at 04:06):

Almost everything at my work is c++ or python. Though we have a ton of mojo too.

Reed Harston (Apr 19 2025 at 13:19):

Ooh, first time I’ve met someone that has used Mojo. And professionally! What have you thought of it so far? Live up to its promises?

Reed Harston (Apr 19 2025 at 13:22):

I knows it’s designed for accelerating the AI work being done in Python, but I wonder if anybody has started using it for anything else they’re doing in Python but want to be fasted.

Anton (Apr 19 2025 at 13:33):

Reed Harston (Apr 19 2025 at 16:30):

Richard Feldman (Apr 19 2025 at 16:40):

this is the SIMD-powered lexing strategy (based on simdjson) that I want to try out someday for Roc: :grinning_face_with_smiling_eyes:

Richard Feldman (Apr 19 2025 at 16:40):

Brendan Hansknecht (Apr 19 2025 at 16:56):

Wizard ish (Apr 20 2025 at 01:45):

honestly python is just so... python the moment I hear that a language has "whitespace sensitive syntax", I can't deal with it, even something like NIM

Wizard ish (Apr 20 2025 at 01:46):

because it has (that might be overstating it given that it never works) a way to write syntax in a way that is whitespace agnostic

Wizard ish (Apr 20 2025 at 01:48):

Wizard ish (Apr 20 2025 at 01:50):

Brendan Hansknecht (Apr 20 2025 at 02:07):

I think tabs verse spaces leads to some issues, but it is generally more readable

Reed Harston (Apr 20 2025 at 03:12):

I’ve had multiple issues at work where we release a Python script to production and then a certain flow is broken because of a white space issue and our testing did catch it because we didn’t hit that flow or something.

Yes there that flagged an area to improve our process, and we did, but it still is frustrating that it was possible at all for that to get out into production because to the human eye everything looked right, but only because the tabs were the right width to match the spaces on the other lines.

Reed Harston (Apr 20 2025 at 03:14):

(There were two files that had a nearly identical section of code and when one was edited that section was copied to the other file. But one file used tabs and the other used spaces. Why? Ask someone from 5 years ago. I wish I knew. When I pasted it over everything looked right, and I happened to test the file that was right, and not the one that was wrong. Yes, user error. I haven’t done that again!)

Brendan Hansknecht (Apr 20 2025 at 03:18):

Brendan Hansknecht (Apr 20 2025 at 03:19):

Brendan Hansknecht (Apr 20 2025 at 03:20):

But this is where I definitely prefer mojo (though it is young and needs other support) but it can be compiled and fully type checked and etc.

Reed Harston (Apr 20 2025 at 04:22):

Part of our process improvement for Python includes linters run in Jenkins when we push changes.
We didn’t have anything before because all our Python is written by C++ firmware developers that write their code like it is C with classes. So you can imagine the state of our Python. :sweat_smile:

Romain Lepert (Apr 20 2025 at 08:59):

@Brendan Hansknecht curious what part of Mojo you work on. Mojo is my #1 upcoming language of interest. Hugely impressed with what it can do already. You guys have been doing magic (:wink:).

Brendan Hansknecht (Apr 21 2025 at 04:06):

I don't work on mojo at all, just work with it some. I initially worked mostly on the graph compiler (compiles ai models down to kernels). Now I work at the max framework level helping build out the tools we use to write models. Compilers are all c++. Frameworks are all python. Kernels are all mojo. I interact with everything to some extent, but mostly python lately.

Luke Boswell (May 13 2025 at 08:21):

Looking at @Jared Ramirez's PR I can see a few comments like this... which I assume is because we have that lint to ensure all top-level decls have a comment.

Would it be possible to not require that where it's just re-exporting something from another file? I think Zig's language server/editor integration? picks up the referenced comments -- and just appends the additional comment.

/// Type Desc
pub const Desc = types.Descriptor;

 /// Type Rank
 pub const Rank = types.Rank;

 /// Type Mark
 pub const Mark = types.Mark;

Luke Boswell (May 13 2025 at 08:24):

Anton (May 13 2025 at 08:25):

Anton (May 13 2025 at 08:26):

Joshua Warner (May 13 2025 at 12:14):

All times I’ve seen an attempt to enforce doc comments via tooling, it has devolved in exactly this way

Joshua Warner (May 13 2025 at 12:15):

Anton (May 13 2025 at 12:30):

In the old compiler we had nothing to encourage comments, and so we had very few comments. In the new compiler we force comments on pub things and we have decent comments in a lot of places and some redundant ones. I feel like it's already working, how do you see this devolving?

Anton (May 13 2025 at 12:35):

Joshua Warner (May 13 2025 at 15:21):

In my experience, things start off great, and then the percentage of low-value comments creeps up and up until they’re the cast majority of comments

Brendan Hansknecht (May 13 2025 at 15:24):

Brendan Hansknecht (May 13 2025 at 15:25):

I still think this test/trail is well worth it given the experience of the rust compiler.

Jared Ramirez (May 13 2025 at 16:39):

FWIW, there were very few comments in the Rust unify/subs code, and there were many places where I think they would have been helpful.

I can see how we'll likely end up with many self-describing comments like above with the pub enforcement, but I would guess the total number of good comments will be higher than without

Joshua Warner (May 13 2025 at 16:45):

Anton (May 13 2025 at 16:46):

Joshua Warner (May 13 2025 at 16:46):

Being really appreciative of someone going out of their way to document things well, for one

Joshua Warner (May 13 2025 at 16:48):

Gentle prodding on the code review is good, to try to encourage that. But imo it’s really important that it’s actually a person making that code review comment rather than a bot. It’s much more likely people will respect and listen to a person.

Anton (May 13 2025 at 16:50):

Anton (May 13 2025 at 16:51):

One alternative approach that comes to mind is a small checklist that shows up when you make a PR in github:

Anton (May 13 2025 at 16:52):

Anton (May 13 2025 at 16:54):

Below this one we could provide a collapsible list with all pub functions that you added but did not provide a comment for. So they have an easy overview without a hard reject mechanism.

Anton (May 17 2025 at 13:25):

Anton (May 17 2025 at 18:32):

Brendan Hansknecht (May 17 2025 at 18:42):

This actually looks relatively one direction and decent....though base is definitely a bit sketch as a super aggregator

Brendan Hansknecht (May 18 2025 at 14:55):

Anton (May 19 2025 at 17:55):

Anthony Bullard (May 19 2025 at 18:03):

Anton (May 20 2025 at 09:15):

Luke Boswell (May 20 2025 at 22:17):

I was trying to run typos (like in CI) and were running an older version of rustc with our rust-toolchain.toml

$ cargo install typos-cli --version 1.32.0
    Updating crates.io index
error: cannot install package `typos-cli 1.32.0`, it requires rustc 1.80 or newer, while the currently active rustc version is 1.77.2
`typos-cli 1.28.2` supports rustc 1.75

Brendan Hansknecht (May 20 2025 at 22:27):

Brendan Hansknecht (May 20 2025 at 22:28):

Luke Boswell (May 20 2025 at 22:29):

I'm happy to poke at it. I'll wait for @Anton who may have ideas. I think we're just following the upgrade guide the toml file.

Anton (May 21 2025 at 09:06):

In CI I remove rust-toolchain.toml before I install typos, that makes it work. Upgrading rust can come with some serious clippy work but with today's AI assistance that may go smooth, you're welcome to give it a try.

Anton (May 21 2025 at 09:14):

You can also install typos outside nix (and outside the roc repo) locally , that may be the easiest way.

Anton (May 28 2025 at 17:26):

It's nice that by encouraging code to be pure it is also easier and faster to test it

Anton (May 28 2025 at 17:41):

I remember that my professor deducted me a ton of points on the exam for my first programming (java) college course. Not because my solution was incorrect, but because I didn't put stuff in classes. Ironically, now I am working on a programming language without any classes, full circle, haha :p

Anton (May 28 2025 at 17:42):

He did also end up teaching me functional programming in a different course, so that's an acceptable redemption arc :)

Luke Boswell (May 30 2025 at 23:58):

@Jared Ramirez I noticed your explanation about the occurs check and not using Mark, I thought that looked good. You're switching back to use Mark, why is that? I'm just curious.

Jared Ramirez (May 31 2025 at 02:34):

Yeah, I kinda went back and forth on it. I think using Marks will have slight better performance (though it may be negligible), and I figured if it was done that way in the rust compiler, there was probably a reason.

Jared Ramirez (May 31 2025 at 02:36):

I was also gonna see what Richard thought when he reviews it! I kept the “covert to use Marks” change isolated in a commit, which can easily be dropped if the original way sans Marks is preferred

Richard Feldman (May 31 2025 at 04:30):

I have an idea, which I'm not sure has precedent - what if we counted the number of transitive unifications we'd done for a given type (e.g. we visit a canonical IR node and kick off a unification of it and some other type; all the unifications resulting from that original, before we move on to the next canonical IR node, would be "transitive" here) and if that count is under some threshold, then all occurs checks are no-ops

Richard Feldman (May 31 2025 at 04:31):

so the idea there would be that the occurs checks are there to catch infinite types, which are a rare but very serious error case

Richard Feldman (May 31 2025 at 04:31):

rare bc it's a very uncommon mistake to make, and serious bc it hangs the compiler by default (unless there's an occurs check to detect that we're stuck in an infinite loop)

Richard Feldman (May 31 2025 at 04:32):

so the idea would be that if we're stuck in an infinite loop, our iteration count will exceed the threshold, and if we're not stuck in an infinite loop, then it very rarely will, and so it's much cheaper to skip the checks

Richard Feldman (May 31 2025 at 04:33):

this would have a perf cost in the case where we do actually have an infinite type, but I think that's totally fine bc it comes up so rarely in practice

Richard Feldman (May 31 2025 at 04:34):

(plus the perf cost would probably only be paid for that one type in the whole build, so likely still wouldn't be noticeable)

Richard Feldman (May 31 2025 at 04:34):

Jared Ramirez (May 31 2025 at 04:40):

So like we have a counter and every time we recurse in unify, we increment it, then if we exceed the threshold we do a full occurs check?

Jared Ramirez (May 31 2025 at 04:42):

I think we’d have to reset/decrement the count for each recursive branch (eg if count = 1 at the start of a function, we recurse to unify args 1, we’d have to reset the count back to 1 before unifying args 2). This is how tracking seen variables in a regular occurs check works though, so I think that should be fine

Jared Ramirez (May 31 2025 at 05:19):

Thinking more, I wonder if we couldn’t combine unify and occurs check. We could track seen variables in unification and check on each recursive call if we’ve already seen this var

Richard Feldman (May 31 2025 at 11:06):

yeah, my thinking was just that we could avoid checking the variables at all in the very common case where there aren't any infinite types

Richard Feldman (May 31 2025 at 11:06):

since incrementing a counter and checking if it's over a threshold is so cheap it's basically free

Brendan Hansknecht (May 31 2025 at 14:44):

One thing, to double check. Can we make sure that this checker allows infinite recursion through indirection? Also through non tag types. And multiple aliases

Brendan Hansknecht (May 31 2025 at 14:45):

Node a: { data: a, children: List(Node(a)) }

Brendan Hansknecht (May 31 2025 at 14:48):

Wrapper a : { data: a, next: Inner(a) }
Inner a : [ Next: Wrapper(a), None ]

Richard Feldman (May 31 2025 at 14:48):

Brendan Hansknecht (May 31 2025 at 14:48):

Yeah, just want to make sure they get thought about early in case it matters to design

Brendan Hansknecht (May 31 2025 at 14:49):

Richard Feldman (May 31 2025 at 16:29):

Joshua Warner (Jun 02 2025 at 14:58):

Luke Boswell (Jun 05 2025 at 03:28):

Can I have idents or fields in records that are using a wider set of characters?

Luke Boswell (Jun 05 2025 at 03:30):

// Then sort by name (ascending)
const lhs_str = ctx.env.idents.getText(lhs.name);
const rhs_str = ctx.env.idents.getText(rhs.name);
return std.mem.order(u8, lhs_str, rhs_str) == .lt;

Luke Boswell (Jun 05 2025 at 03:31):

But I guess this wouldn't mind what encoding is used... it's just using the raw bytes.

Joshua Warner (Jun 05 2025 at 03:33):

Idents are currently ascii-only in the new compiler, and I think the intent is to keep it that way unless/until there’s a strong signal otherwise

Luke Boswell (Jun 05 2025 at 03:36):

pub fn chompIdentGeneral(self: *Cursor) bool {
    var valid = true;
    while (self.pos < self.buf.len) {
        const c = self.buf[self.pos];
        if ((c >= 'a' and c <= 'z') or (c >= 'A' and c <= 'Z') or (c >= '0' and c <= '9') or c == '_' or c == '!') {
            self.pos += 1;
        } else if (c >= 0x80) {
            valid = false;
            self.pos += 1;
        } else {
            break;
        }
    }
    return valid;
}

Richard Feldman (Jun 05 2025 at 04:02):

yeah I think it's reasonable to consider non-ASCII in the future, but even assuming we want to support it, it's enough of a project that I don't think we need it for 0.1.0

Hannes (Jun 05 2025 at 04:35):

My roc-math package has constants like π and function arguments like θ, but I think I'm the only one who's used them before :sweat_smile:

Luke Boswell (Jun 07 2025 at 04:37):

It's back to compiling and tests passing now... but I've definitely missed a heap of things. I can already see things that I've missed between my "fixes" and merge conflicts.

I'm going to take my time going back through all changes and clean it up so we can merge it hopefully this weekend sometime.

Anton (Jun 07 2025 at 10:32):

Random tip; merging multiple Stdout.line into one can drastically reduce your build times! So like this:

    Stdout.line!(
        """
        Testing Tcp module functions...
        Note: These tests require a TCP server running on localhost:8085
        You can start one with: ncat -e `which cat` -l 8085

        """
    )?

    Stdout.line!("Testing Tcp module functions...")?
    Stdout.line!("Note: These tests require a TCP server running on localhost:8085")?
    Stdout.line!("You can start one with: ncat -e `which cat` -l 8085")?
    Stdout.line!("")?

Anton (Jun 07 2025 at 10:33):

Doing this for all subsequent Stdout.line in tests/tcp.roc reduced build time from 9315 ms to 728 ms!

Anton (Jun 07 2025 at 10:58):

Brendan Hansknecht (Jun 07 2025 at 15:04):

Brendan Hansknecht (Jun 07 2025 at 15:05):

Or hmm....actually with the old tasks it makes sense.....surprising now that it makes much of a difference

Brendan Hansknecht (Jun 07 2025 at 15:06):

Brendan Hansknecht (Jun 07 2025 at 15:35):

Feels like a bug or file system caching or something else crazy....bug is my biggest guess.

Brendan Hansknecht (Jun 07 2025 at 15:36):

Anton (Jun 07 2025 at 15:36):

Anton (Jun 07 2025 at 18:35):

Hmm, addr2line is being used after recording the flamegraph and it's taking its time... it's also single threaded :graveyard:
I've never had that before, could be caused by our perf issue :thinking:

Brendan Hansknecht (Jun 07 2025 at 18:36):

Addr2line, generally isn't fast, but it taking forever could mean essentially infinite recursion

Anton (Jun 07 2025 at 19:42):

Luke Boswell (Jun 08 2025 at 07:56):

I've been putting together as much as I can for this Can PR but I have some architectural questions.

Luke Boswell (Jun 08 2025 at 07:56):

The type store currently holds a reference to the ModuleEnv, however I feel like it should live in the ModuleEnv so we can make fresh variables in Can, and later solve them.

Luke Boswell (Jun 08 2025 at 09:54):

I have gone with Can IR owning the type store. It seems to be working ok for now.

Sam Mohr (Jun 08 2025 at 09:55):

Richard Feldman (Jun 08 2025 at 11:19):

Richard Feldman (Jun 08 2025 at 11:20):

rather, we should reuse can idx to be var, and initialize the types store with a capacity of at least the highest can idx

Anthony Bullard (Jun 08 2025 at 11:58):

Luke Boswell (Jun 08 2025 at 12:20):

It's probably past the point where I should stop and get some feedback. Maybe even worth merging now (it's passed all the CI checks)??

I feel like most of my effort has been on the fringes and infrastructure like SExpr stuff that I understand, and I haven't really touched core Can implementation things, aside from copying Anthony's lead I think.

I've got another day tomorrow I can probably spend hacking on this, so if anyone is able to skim through and give me any pointers that would be appreciated.

Anthony Bullard (Jun 08 2025 at 12:30):

Luke Boswell (Jun 08 2025 at 12:33):

Luke Boswell (Jun 09 2025 at 08:52):

I started refactoring Can a little now I'm getting my head around what is going on there. It's such a massive thing I feel like some module structure for the API will help in the long run.

I've been trying different things out, and I'm basically just rolling with what I think is nicest. I figure if there are any concerns I'm happy to change it back etc.

One change is instead of calling it IR I started calling it CIR short for Can IR. It just helps a lot to qualify things, particularly when we are mixing both Parser IR and Can IR in some places.

Luke Boswell (Jun 09 2025 at 08:54):

I'm really tempted to refactor and update Parser in a similar way, move Node and NodeStore out into separate files and rename to PIR. But I can wait until I get some feedback on these Can changes.

Jared Ramirez (Jun 10 2025 at 00:52):

I think we might need to initialize some fresh vars as well – for Can nodes that need more than 1 var. Like a tag needs the tag union var & an extensibility var, and we can't use the Can Idx for both. Though very possible I'm missing something

Luke Boswell (Jun 10 2025 at 01:05):

I just realised I misread this comment when you wrote it Sam, and understood it to mean the opposite. :sweat_smile:

Thinking about this more, I agree it needs to live in the ModuleEnv as that is where things should be that outlive any particular compiler stage IR.

Hatching plans with @Jared Ramirez to move it, but needs some coordination to update the type_store a little as it currently holds a pointer to ModuleEnv.

Richard Feldman (Jun 10 2025 at 01:17):

Richard Feldman (Jun 10 2025 at 01:18):

in other words, let's say we have a can node that needs 3 vars - instead of that can node getting 1 slot and then we make 2 extra vars for it, we just give it 3 can slots

Richard Feldman (Jun 10 2025 at 01:19):

it might sound like it's wasting memory, but my thinking is that the alternative is to spend the same amount of memory writing down which type vars they need

Richard Feldman (Jun 10 2025 at 01:20):

and the advantage of this is that if you zero out the empty can slots, it becomes bidirectional; not only can you go from any given Can Idx to a type var, you can also take any given type var and get back to the Can Idx where it came from

Richard Feldman (Jun 10 2025 at 01:21):

which in turn means (I think) that we could avoid propagating region info through to type checking, and still get error messages that map back to source

Richard Feldman (Jun 10 2025 at 01:21):

because you can go from the type var that had the mismatch back to the original source via the Can IR node

Richard Feldman (Jun 10 2025 at 01:22):

Luke Boswell (Jun 10 2025 at 01:23):

aside -- should the Can IR nodes have region info? only some do at the moment, and it was something I was looking at yesterday. I've threaded things through for the sexpr to display that for those nodes that do.

Brendan Hansknecht (Jun 10 2025 at 01:31):

Brendan Hansknecht (Jun 10 2025 at 01:32):

Luke Boswell (Jun 10 2025 at 01:35):

Luke Boswell (Jun 10 2025 at 01:36):

Also Re desugaring... should we have a similar to rust, single pass that runs before Can, or should we be doing that desugaring as we go through the Parse IR

Brendan Hansknecht (Jun 10 2025 at 01:36):

Brendan Hansknecht (Jun 10 2025 at 01:37):

I think this was discussed a bit, and the decision was likely we should just start with a new pass just for desugaring, but at the same time, if it is trivial to do on a different IR, that is also fine.

Richard Feldman (Jun 10 2025 at 01:40):

as in, don't desugar anything - just keep it around as a first-class thing for later passes to deal with separately

Luke Boswell (Jun 10 2025 at 01:41):

Luke Boswell (Jun 10 2025 at 01:42):

description=Simple string interpolation
 type=expr
 ~~~SOURCE
 "Hello ${name}!"
 ~~~PROBLEMS
 NIL
 ~~~TOKENS
 StringStart(1:1-1:2),StringPart(1:2-1:8),OpenStringInterpolation(1:8-1:10),LowerIdent(1:10-1:14),CloseStringInterpolation(1:14-1:15),StringPart(1:15-1:16),StringEnd(1:16-1:17),EndOfFile(1:17-1:17),
 ~~~PARSE


 (string (1:1-1:17)
    (string_part (1:2-1:8) "Hello ")
    (ident (1:10-1:14) "" "name")
    (string_part (1:15-1:16) "!"))
 ~~~FORMATTED
 NO CHANGE
 ~~~CANONICALIZE
 (call
    (lookup (ident "Str.concat"))
    (call
        (lookup (ident "Str.concat"))
        (str "Hello ")
        (lookup (ident "name")))
    (str "!"))
 ~~~END

Luke Boswell (Jun 10 2025 at 01:43):

Basically, de-sugared the interpolation into multiple Str.concat calls (that I imagine exist).

Richard Feldman (Jun 10 2025 at 03:00):

Richard Feldman (Jun 10 2025 at 03:01):

one reason for doing that is so that if there's an error message later (e.g. a type mismatch) we can do better than referring to Str.concat calls that aren't in the code

Richard Feldman (Jun 10 2025 at 03:01):

bc we'll still know it was interpolation, so we can just put that directly in the error message

Luke Boswell (Jun 11 2025 at 09:56):

In the same vein as the above ^^ should we make a node for Binop instead of desugaring to a call to Num.add etc

Brendan Hansknecht (Jun 11 2025 at 15:26):

Brendan Hansknecht (Jun 11 2025 at 15:27):

Luke Boswell (Jun 11 2025 at 22:54):

@Joshua Warner @Anthony Bullard -- any objections to me refactoring the Node and NodeStore out of the parser IR and into their own files (in the src/check/parse directory).

I think it helps clarify the abstraction a little more, basically the Store is a lower level abstraction for good memory/cache efficiency etc, and the IR is the higher level representation of the AST with helpers for parsing.

Luke Boswell (Jun 11 2025 at 22:56):

Also, we wen't back and forth a little over what to call the IR, but settled on IR instead of AST (I think because it's just another IR in the overall compiler pipeline)...

Luke Boswell (Jun 11 2025 at 22:57):

Maybe it's a little OCD or something, but I like fully qualifying the types and having a short and clear distinction between the parser and can types is helpful.

Joshua Warner (Jun 11 2025 at 22:57):

Joshua Warner (Jun 11 2025 at 22:58):

Calling everything an IR does make things sound uniform, but that also heavily dilutes the meaning of that term.

Luke Boswell (Jun 11 2025 at 22:58):

Luke Boswell (Jun 11 2025 at 22:59):

Joshua Warner (Jun 11 2025 at 22:59):

Luke Boswell (Jun 11 2025 at 23:01):

In the process of refacatoring the NodeStore out into it's own file... I'd like to ziggify the types a little using nesting... so instead of PatternRecordFieldIdx we have Pattern.RecordField.Idx, and getPatternRecordField becomes Pattern.RecordField.get etc

Luke Boswell (Jun 11 2025 at 23:06):

I know what I'm proposing is a little like moving the deckchairs on the titanic, and kind of pointless ... but I'm really enjoying these features of Zig and I think it helps to establish the patterns we want to use elsewhere.

Joshua Warner (Jun 11 2025 at 23:24):

Luke Boswell (Jun 12 2025 at 01:14):

Is there a particular reason we are using a struct instead of an enum for index types in the parser?

e.g. pub const BodyIdx = struct { id: u32 }; instead of pub const BodyIdx = enum(u32) { _ };

We use both in different places. From what I can tell it seems the enum approach is preferable, but I'm not sure if there was a specific reason we did this. I assume it was just the first thing we thought of at the time and it would be ok to use the enum.

Anthony Bullard (Jun 12 2025 at 01:15):

Luke Boswell (Jun 12 2025 at 01:18):

return .{ .crash = .{
    // using the struct approach
    .expr = .{ .id = node.data.lhs },
    .region = node.region,
} };

return .{ .crash = .{
    // using the enum approach
    .expr = @enumFromInt(node.data.lhs),
    .region = node.region,
} };

Anthony Bullard (Jun 12 2025 at 01:39):

Luke Boswell (Jun 12 2025 at 03:33):

I've been making these changes, and have found many places where one would work better, and other places where the other works better. :shrug:

I don't feel particularly strong about it either, but I'm a far way through making all the Idx's use the enum approach. I'll keep going and see how it looks.

Luke Boswell (Jun 13 2025 at 02:39):

I think we should merge my gigantic Can PR that I've been working against so he has a more mature structure to work with.

I've been in a move fast and break things / refactor mode so theres a lot of improvements there, but all low technical risk as I'm mostly fleshing out the new Can stage and wiring everything up so we have snapshots with diagnostic errors and region info etc (along with some very primitive Can analysis).

Luke Boswell (Jun 13 2025 at 08:07):

Improved the snapshots reporting so they should now display all the problems, and print out useful information. Haven't got region info, but for now just seeing the source is useful to understand where the issue is.

~~~SOURCE
app [main!] { pf: platform "../basic-cli/platform.roc" }

import pf.Stdout

main! = |_| Stdout.line!("Hello, world!")
~~~PROBLEMS
CANONICALIZE: ident_not_in_scope "Stdout.line!"
CANONICALIZE: can_lambda_not_implemented "|_| Stdout.line!(..."

For hello world we are almost there.. just need to figure out the final details on lambdas. I'm not sure if we should have a lambda Expr variant in the Can IR or if there is some analysis we should be doing here to make it something else. We could have just missed that variant when translating from rust.

~~~CANONICALIZE
(can_ir
    (top_level_defs
        (def
            "let"
            (pattern (5:1-5:6)
                (assign (5:1-5:6) (ident "main!")))
            (expr (5:9-5:42) (runtime_error (5:9-5:42) "can_lambda_not_implemented"))
            "#0")))

All the Can nodes have regions information now, and they display correctly in our SExpr snapshots.

I've implemented most of the Expr, Pattern, and Statment handling in the NodeStore now, theres a few nodes here and there that need to store things in extra_data but I figure we will do that as we get to them. It should be mush easier now to implement the analysis logic without thinking too much about the low-level store.

Luke Boswell (Jun 13 2025 at 09:58):

We've discussed it before previously I think... but should we consider having a constants file somewhere where we keen magic hardcoded numbers that configure the compiler?

I was looking at @Jared Ramirez PR's for unify and the max_depth_before_occurs variable made me think of it.

Richard Feldman (Jun 13 2025 at 14:08):

Richard Feldman (Jun 13 2025 at 14:09):

and putting it at the top level dir so everything can access it (and it shouldn't import anything)

Anton (Jun 16 2025 at 19:08):

When debugging the rust compiler, note that you need to comment out this line to see local vars with lldb or gdb

Luke Boswell (Jun 17 2025 at 22:56):

I'm tracking the implementation of Scope is not quite correct. I discussed with Richard the intended design to accomodate var and shadowing and I intend on revisiting that next.

Jared Ramirez (Jun 19 2025 at 00:15):

Heads up that I’m starting work on basic type solving, now that we have basic type variable generation in Can. Just positing for viz!

Luke Boswell (Jun 20 2025 at 09:13):

I'm experimenting with some additions to my .rules file. I thought I'd made a PR to share. I might give it a week or so and see if I feel like tweaking it.

So far it's been good. I find the agents are much quicker at finding the relevant information for a task.

Anton (Jun 20 2025 at 09:16):

Cool stuff, can you try mentioning the Glossary.md file? It has a lot of links and Roc compiler specific terms too, I would expect it to be helpful.

Luke Boswell (Jun 20 2025 at 09:20):

The Glossary is a very useful resource, and it's getting better all the time :smiley:

Luke Boswell (Jun 20 2025 at 22:50):

Are type declarations only valid at the top-level? (I'm reasonably sure they are valid everywhere) but I figured I'd clarify. I'm going to leave the type_decl in statement positions as a TODO for now.

Richard Feldman (Jun 21 2025 at 00:44):

Anthony Bullard (Jun 21 2025 at 01:09):

Brendan Hansknecht (Jun 21 2025 at 01:14):

SomeType := [InnerType]

SomeAlias : [InnerType]

x : SomeType
x = InnerType

Anthony Bullard (Jun 21 2025 at 01:15):

Brendan Hansknecht (Jun 21 2025 at 01:16):

I could see 2 being valid anywhere....though a bit strange.... 1 sounds pretty crazy given we anchor static dispatch to the module and it would just generally be odd to define a named type within a scope in roc.

Brendan Hansknecht (Jun 21 2025 at 01:17):

Why do we want the everywhere? I feel like that only makes sense if we allow nested modules or other more intense scoping of some form.

Richard Feldman (Jun 21 2025 at 01:42):

Brendan Hansknecht (Jun 21 2025 at 01:42):

Brendan Hansknecht (Jun 21 2025 at 01:43):

I can't wait to see the use of an opaque type that is defined in a function and returned.

Anthony Bullard (Jun 21 2025 at 01:43):

Richard Feldman (Jun 21 2025 at 01:43):

Anthony Bullard (Jun 21 2025 at 01:44):

Brendan Hansknecht (Jun 21 2025 at 01:44):

I think it literally could only be used if it also was returned with functions that can use the type

Anthony Bullard (Jun 21 2025 at 02:25):

wouldn't that mean that the type would have to live outside the scope of the function ?

Brendan Hansknecht (Jun 21 2025 at 02:34):

Richard Feldman (Jun 21 2025 at 03:13):

Luke Boswell (Jun 21 2025 at 03:16):

Yeah, based on trying to implement it, my vote is to restrict to top level for now. It seems strange having type decls in Scope

Joshua Warner (Jun 21 2025 at 04:20):

Type decls inside a function could be useful for making sure you understand the type of some intermediate variable - e.g. like a type assertion

Joshua Warner (Jun 21 2025 at 04:21):

for that purpose, you'd need to use 1+3 or 2+3 (i.e. together with a value defined)

Anthony Bullard (Jun 21 2025 at 10:19):

Kiryl Dziamura (Jun 23 2025 at 18:55):

is there a thread with the current work plan or smth like that for coordination? I'm trying to get into zig rewrite but don't understand current goals and workflow. just don't know where to start at. I see src/README.md, it implies loads of work to be done, but nothing really specific. maybe there are some shortcuts to avoid redundant job?
e.g. I see records canonicalization isn't implemented yet. I assume there's a way to adapt rust implementation, but not sure if there an agreement on best practices. shortly, what's the rule of thumb for the rewrite?

Luke Boswell (Jun 23 2025 at 23:14):

I would say the rule of thumb is to coordinate via zulip, let people know what you are up to by making Draft PRs etc.

I've been focussing on implementing more of Can along with the s-expressions for debugging and making nice error reports.

Richard has been laying the foundation for runtime representations so we can evaluating things using the interpreter.

I was thinking about looking at either how do we fuzz Can or start using the cache for single modules, next.

There's a mountain of unimplemented or hacked together things though in Can. Records haven't been touched. When (now match) hasn't been touched. There's many error reports that we can bring across from the rust implementation.

I've been picking a snapshot that represents a real bit of roc code and seeing what I can do to get it working end to end, and letting that take me on a yak shaving adventure.

I wouldn't recommend doing anything in the build (or later) stages yet. I think the immediate goal is getting a basic interpreter going that we have high confidence is working correctly.

Joshua Warner (Jun 24 2025 at 04:52):

I've been noodling around with adding a secondary output file for snapshot tests - an html file with a more interactive version of the snapshot, with things like hovering over tokens or ranges in the snapshot output causing the corresponding source range in the input to be highlighted (and visa-versa). My thinking is that we'd always generate these .html files next to the existing snapshots but have them git-ignored to avoid bloating the repo. We can also start to integrate some of the debugging/visualization tools for various compiler passes into this html output. Thoughts?

Luke Boswell (Jun 24 2025 at 05:03):

Luke Boswell (Jun 24 2025 at 05:06):

One of the s-expression changes I've been thinking about, is should we be displaying the node_idx on each node? We have some nodes that reference other nodes (for example a pattern for a declaration). I could imagine hovering over a reference like that, and it highlights the s-expression node being referenced.

Joshua Warner (Jun 24 2025 at 05:07):

Luke Boswell (Jun 24 2025 at 05:08):

For this reason I've ben thinking of making all node_idx's use a common formatting like #12 (I think this is similar to what we had in the rust debug IR formats)

Joshua Warner (Jun 24 2025 at 05:24):

Luke Boswell (Jun 24 2025 at 05:30):

Luke Boswell (Jun 24 2025 at 05:36):

We have flags on the snapshot tool like --verbose ... we could include others like --html or --watch.

Luke Boswell (Jun 24 2025 at 05:36):

Luke Boswell (Jun 24 2025 at 05:37):

The s-expressions could have a renderToHtml helper too instead of the plain text version

Luke Boswell (Jun 24 2025 at 05:37):

Kiryl Dziamura (Jun 24 2025 at 10:45):

Richard Feldman (Jun 24 2025 at 16:41):

sounds good! We decided to have binops represented directly in CIR rather than desugaring them to function calls

Richard Feldman (Jun 24 2025 at 16:42):

the idea is that it will simplify having error messages trace back to what was originally in the source code

Kiryl Dziamura (Jun 24 2025 at 16:42):

Richard Feldman (Jun 24 2025 at 16:42):

(as opposed to converting them to a Call node which then has to record "was I called via a binop, and if so which one?" which has the same information but with more redundancy and memory usage)

Kiryl Dziamura (Jun 24 2025 at 16:44):

Richard Feldman (Jun 24 2025 at 16:46):

one critical difference is that lookups have all been resolved to other CIR indices instead of using strings

another important one is that we've organized everything so that type variables map 1-to-1 with CIR indices, whereas they certainly do not map 1-to-1 with AST nodes :smile:

Richard Feldman (Jun 24 2025 at 16:48):

oh yeah we also have translated = into either declare or reassign, depending on whether there was a previous declaration under the same name made with var

Luke Boswell (Jun 25 2025 at 10:12):

checkNumber = |num| {
    if num < 0 then
        "negative"
    else if num == 0 then
        "zero"
    else if num > 100 then
        "large"
    else
        "positive"
}

Luke Boswell (Jun 25 2025 at 10:21):

module [checkNumber]

checkNumber = |num| {
    if num < 0 {
        "negative"
    } else if num == 0 {
        "zero"
    } else if num > 100 {
        "large"
    } else {
        "positive"
    }
}

Anthony Bullard (Jun 25 2025 at 12:34):

Richard Feldman (Jun 25 2025 at 13:31):

    if num < 0
        "negative"
    else if num == 0
        "zero"
    else if num > 100
        "large"
    else
        "positive"

Richard Feldman (Jun 25 2025 at 13:32):

but I think it simplifies the rules of the language to not require brackets there

Richard Feldman (Jun 25 2025 at 13:33):

x = if things.is_empty() 0 else 1

Anton (Jun 25 2025 at 13:38):

Anthony Bullard (Jun 25 2025 at 13:38):

I think in a lot of cases the squirly-less syntax is unreadable, especially without newline / indent

Anton (Jun 25 2025 at 13:40):

Anthony Bullard (Jun 25 2025 at 13:40):

I find myself wanting to put parens around the condition if I don't use squirlies

Anthony Bullard (Jun 25 2025 at 13:40):

Richard Feldman (Jun 25 2025 at 13:43):

x = if things.is_empty() 0 else 1

x = if (condition) 0 else 1

x = if condition 0 else 1

Richard Feldman (Jun 25 2025 at 13:44):

x = if (things.is_empty()) 0 else 1

Richard Feldman (Jun 25 2025 at 13:44):

as an aside, something I've wondered about is whether we should use curly braces like we use trailing commas - as an indicator of whether the formatter should use newlines

Richard Feldman (Jun 25 2025 at 13:45):

fn = |arg| arg + 1

fn = |arg| {
    arg + 1
}

Richard Feldman (Jun 25 2025 at 13:45):

x = if things.is_empty() 0 else 1

x = if things.is_empty() {
    0
} else {
    1
}

Anton (Jun 25 2025 at 13:46):

Anthony Bullard (Jun 25 2025 at 13:46):

Richard Feldman (Jun 25 2025 at 13:47):

Anthony Bullard (Jun 25 2025 at 13:47):

Anthony Bullard (Jun 25 2025 at 13:49):

Oh, if the block is retained it's multiline, but it's usually elided in the formatter if not necessary

Anthony Bullard (Jun 25 2025 at 13:49):

I think that would be easy to change. If I see a useless block that is single-line, keep it (it will naturally be multiline)

Richard Feldman (Jun 25 2025 at 14:09):

it would be really cool if we could get to a point where the author has control over single-line vs multiline, without needing the formatter to be aware of newlines

Richard Feldman (Jun 25 2025 at 14:10):

using a combination of blocks vs no blocks and trailing comma vs no trailing comma, etc.

Anthony Bullard (Jun 25 2025 at 14:17):

Richard Feldman (Jun 25 2025 at 14:28):

Richard Feldman (Jun 25 2025 at 14:29):

Anthony Bullard (Jun 25 2025 at 14:51):

Luke Boswell (Jun 25 2025 at 22:35):

Luke Boswell (Jun 27 2025 at 00:22):

I'm working on de-structuring record patterns now. I've merged main in and just adding to my Records PR. Also addressing the review feedback. Thank you @Jared Ramirez and @Anthony Bullard

Kiryl Dziamura (Jun 27 2025 at 09:50):

(e-tuple @1.1-17.2
    (e-binop @2.5-2.11 (op "+")
        (e-int @2.5-2.6 (raw "4"))
        (e-int @2.9-2.10 (raw "2")))

(e-tuple @1-1-17-2
    (e-binop @2-5-2-11 (op "+")
        (e-int @2-5-2-6 (raw "4"))
        (e-int @2-9-2-10 (raw "2")))

Norbert Hajagos (Jun 27 2025 at 15:13):

Does anyone have something I can help with? Open for coding any area that is zig! Last time I made the txt -> markdown conversion for our snapshots. I'm down to do something one step harder, or similar. Will be back later to read zulip.

Joshua Warner (Jun 28 2025 at 04:43):

Random side-note: git does terrrrrrribly with rebasing (or merging, I think) changes when one of the sides does a case-only rename (e.g. sexpr.zig->SExpr.zig), and you're operating on a case insensitive filesystem (e.g. mac).

Joshua Warner (Jun 28 2025 at 04:44):

error: Your local changes to the following files would be overwritten by merge:
    src/base/SExpr.zig
Please commit your changes or stash them before you merge.
Aborting
hint: Could not execute the todo command
hint:
hint:     squash bb9df942054bfb94f2a3d154e6a549ea510edfec Switch to two-column layout in snapshot html and jazz up parse output
hint:
hint: It has been rescheduled; To edit the command before continuing, please
hint: edit the todo list first:
hint:
hint:     git rebase --edit-todo
hint:     git rebase --continue

(note: I made no 'local changes' here - this was git rebase stepping on its own toes)

Joshua Warner (Jun 28 2025 at 04:48):

Joshua Warner (Jun 28 2025 at 04:51):

Joshua Warner (Jun 28 2025 at 04:52):

Joshua Warner (Jun 28 2025 at 04:58):

Joshua Warner (Jun 28 2025 at 04:59):

Already caught a few cases that look kinda fishy in terms of the token range we choose for various ast nodes

Joshua Warner (Jun 28 2025 at 04:59):

Joshua Warner (Jun 28 2025 at 05:00):

Side note: @Kiryl Dziamura I think I'm starting to agree with your suggestion to not have the js and css embedded as zig strings; I'll fix that soon.

Luke Boswell (Jun 28 2025 at 05:18):

Yeah, I've picked up on that in a few places when going through a snapshot and manually verifying the regions... but haven't circled back to investigate yet.

Brendan Hansknecht (Jun 28 2025 at 06:43):

If no one else has an answer, can also fix fuzzer bugs, but I bet someone has solid work that could use a hand

Norbert Hajagos (Jun 28 2025 at 07:55):

Okay, I'll go with fixing fuzz errors. Will start with
zig build repro-tokenize -- -b XzD/ -v
which is this input:
_0�

Luke Boswell (Jun 29 2025 at 23:26):

I'm thinking we rename When to Match in Can to avoid confusion in future... if we agree I'll save it for a later PR that just does that one change.

Notification Bot (Jun 30 2025 at 22:07):

Joshua Warner (Jul 01 2025 at 03:48):

Looking into fancy-ing up the html snapshot generation for canonicalization now, and one of the slight snags I've run into is that can regions are defined in terms of byte offsets rather than token indices - was that an intentional choice? Thoughts on just using token indices?

Joshua Warner (Jul 01 2025 at 03:49):

Tokens are a bit easier to work out highlighting for in the markup (versus arbitrary byte ranges)

Luke Boswell (Jul 01 2025 at 03:49):

I don't think it matters much for the internal representation. We can still convert that back to line/col pairs right?

Joshua Warner (Jul 01 2025 at 03:50):

Luke Boswell (Jul 01 2025 at 03:50):

Joshua Warner (Jul 01 2025 at 03:51):

You need to keep around the list of token offsets (or be okay with recomputing it if needed)

Joshua Warner (Jul 01 2025 at 03:51):

Luke Boswell (Jul 01 2025 at 03:52):

Luke Boswell (Jul 01 2025 at 03:53):

Luke Boswell (Jul 01 2025 at 03:54):

I was talking with him the other day about a hypothesis that the AST and CIR nodes can map 1-1 and therefore we could have region information in a side array and just use the AST/CIR node index to get the region information back out.

Luke Boswell (Jul 01 2025 at 04:00):

Also we need the line/col information for s-expressions and reporting only -- so it would be ok to calculate those as needed. I'm assuming we wouldn't need to re-read the file, but using information like line_starts etc.

Removing region from IR's would have the advantage of not polluting our IR and keeping the nodes much smaller, hopefully reducing cache misses etc.

Joshua Warner (Jul 01 2025 at 04:09):

Joshua Warner (Jul 01 2025 at 04:10):

Maybe I should just bite the bullet (byte the bullet?) and implement arbitrary byte range selection in the source view

Joshua Warner (Jul 01 2025 at 04:10):

Luke Boswell (Jul 01 2025 at 04:11):

Why? I find it really easy to manually validate using the line/col information my editor shows.

Joshua Warner (Jul 01 2025 at 04:12):

Joshua Warner (Jul 01 2025 at 04:13):

Anyway, snark aside, I want to have the conversion to line:col to happen in once place (probably at the point we're rendering the sexpr), rather than at 100 different callsites in the ast/can/etc.

Luke Boswell (Jul 01 2025 at 04:14):

I think it is already like that -- only calculated when generating the s-expressions

Joshua Warner (Jul 01 2025 at 04:14):

It's calculated when _generating_ the sexpr. I want to move it to being calculated when _rendering_ the sexpr

Anton (Jul 01 2025 at 14:35):

The image file of the new compiler dependency graph is now to large to be hosted on github :(

Anton (Jul 01 2025 at 14:40):

Kiryl Dziamura (Jul 01 2025 at 14:41):

Kiryl Dziamura (Jul 01 2025 at 14:42):

Anton (Jul 01 2025 at 14:49):

Kiryl Dziamura (Jul 01 2025 at 14:57):

Anton (Jul 01 2025 at 15:26):

Luke Boswell (Jul 03 2025 at 00:02):

match shape {
    Rectangle({ width, height }) as rect => ..
}

Luke Boswell (Jul 03 2025 at 00:13):

We have f64_literal in CIR.Pattern... what is this used for, we aren't matching on an exact float literal are we? we don't have eq for floats.

Richard Feldman (Jul 03 2025 at 00:18):

Luke Boswell (Jul 03 2025 at 00:18):

Are we keeping the default/optional value thing? { name, age ? 0 } => ... -- I understand this is being removed as it was confusing

Richard Feldman (Jul 03 2025 at 00:24):

Kiryl Dziamura (Jul 03 2025 at 14:46):

I noticed build.zig.zon requires zig 0.14, but nix installs 0.13. 0.14 seems to be required only for the fuzzer

Luke Boswell (Jul 03 2025 at 20:12):

Luke Boswell (Jul 03 2025 at 22:43):

@Anton -- could we look at making the default nix for our zig compiler, and use a flag or something to get the devshell for the rust compiler?

Brendan Hansknecht (Jul 03 2025 at 23:15):

Jared Ramirez (Jul 04 2025 at 00:56):

fwiw i do not use the devshell and just do nix shell 'github:mitchellh/zig-overlay#"0.14.0"'

Luke Boswell (Jul 04 2025 at 00:58):

That might be nice to add to our BUILDING_FROM_SOURCE.md or CONTRIBUTING.md docs.

Brendan Hansknecht (Jul 04 2025 at 01:20):

Brendan Hansknecht (Jul 04 2025 at 01:22):

Aside, now that we have check working would be good to instrument it all with Tracy and look at some more detailed traces. See how perf looks and such.

Luke Boswell (Jul 04 2025 at 01:29):

Brendan Hansknecht (Jul 04 2025 at 06:53):

General question, what should the limit be for numbers of errors printed by something like roc check?

Luke Boswell (Jul 04 2025 at 07:04):

It's not something we've talked about... Maybe in the hundreds? Is it different in the TTY or LSP use case?

Brendan Hansknecht (Jul 04 2025 at 07:14):

My million line of code file, which is just repeated code (and thus has tons of repeated declaration errors), essentially just prints forever.

Luke Boswell (Jul 04 2025 at 07:14):

Brendan Hansknecht (Jul 04 2025 at 07:14):

Made me realize that maybe it isn't the right UX. Also, due to the extract lines perf issue, hangs for a while before it finally starts printing.

Kiryl Dziamura (Jul 04 2025 at 07:58):

Is aggregation an option? How many distinct diagnostic messages we would have? It might be a message and a list of file:line:column grouped by the same kind of error

Anton (Jul 04 2025 at 10:22):

For the future it's great to be able to jump between commits and immediately get the correct zig version, so a nix flake would be nice. To keep friction for adding dependencies, we can set it up with github code owners so that changes to flake.nix require approval from a specific person. What do you think @Brendan Hansknecht?

Brendan Hansknecht (Jul 04 2025 at 15:06):

Quite possibly. Though in this case, it generated 3.6 million lines of output. Even aggregated I think we want to cut that off.

Kiryl Dziamura (Jul 04 2025 at 15:07):

Brendan Hansknecht (Jul 04 2025 at 15:07):

That's fair for auto updating the zig version. And I think as long as we have the right culture it shouldn't matter too much, but I think it could be reasonable to set code owners for build.zig and flake.nix

Brendan Hansknecht (Jul 04 2025 at 16:27):

For my million line of code file. We spend:
0.4s -> tokenize + parse
9.8s -> canonicalize
56s -> generating diagnostic from can (this is after my fix that made this part way way faster)
0.6s -> checkDefs
1.5s -> generating diagnostics from check defs.

haha.... generating diagnostics is definitely really slow right now. Though I guess that isn't much of problem if we limit the number of diagnostics.

Brendan Hansknecht (Jul 04 2025 at 16:29):

Richard Feldman (Jul 04 2025 at 16:50):

Richard Feldman (Jul 04 2025 at 16:51):

obviously canonicalization should not be taking 20x as long as type checking, since it has astronomically less work to do :joy:

Richard Feldman (Jul 04 2025 at 16:52):

but this is really cool to know! any chance we could automate this so we can see the number change over time as we make changes? :smiley:

Anton (Jul 04 2025 at 17:06):

I can quickly set up something basic with the raspberry pi 4 CI server, to minimize power consumption long term. Can you share the file somewhere @Brendan Hansknecht?

Brendan Hansknecht (Jul 04 2025 at 17:16):

Brendan Hansknecht (Jul 04 2025 at 17:18):

Also, we may want to switch it out for something more representive I don't think many users will run files that generated nearly 400k diagnostics

Anton (Jul 04 2025 at 17:45):

Richard Feldman (Jul 04 2025 at 18:33):

still, it's a fun one to have in the mix for exactly that reason! if everything is broken, we don't want the compiler to lock up your machine :joy:

Brendan Hansknecht (Jul 04 2025 at 18:36):

For sure. For example, streaming diagnostics here would at least show the user that something is happening instead of simply having a crazy long delay.

Brendan Hansknecht (Jul 04 2025 at 18:50):

Oh, and we use way to much memory collecting all the diagnostics. ~~I think this uses about a gig making diagnostics asts (which are old school allocating per node asts)~~ More like 400MB for diagnostics.

Brendan Hansknecht (Jul 04 2025 at 18:57):

So I have two giant files that I have used. One based on a now quite old copy of our syntax grab bag and one based on List.roc.
New-List.roc
new.roc

Brendan Hansknecht (Jul 04 2025 at 18:57):

Also, I guess we should add some form of the --time flag that counts time in each stage of the compiler and reports on it.

Brendan Hansknecht (Jul 04 2025 at 19:04):

For the list file (which if I recall is 1 million lines of code, but 1.6 million lines including comments), the breakdown is:
0.9s -> tokenize + parse
75s -> canonicalize
60s -> generating diagnostic from can (this is after my fix that made this part way way faster)
2.7s -> checkDefs
0.01s -> generating diagnostics from check defs.

Brendan Hansknecht (Jul 04 2025 at 19:08):

Of note, input source is 53MB. So roughly a 25:1 ratio from memory usage after can to original source memory size.

Richard Feldman (Jul 04 2025 at 19:19):

Brendan Hansknecht (Jul 04 2025 at 19:23):

Brendan Hansknecht (Jul 04 2025 at 20:17):

Kiryl Dziamura (Jul 04 2025 at 20:19):

Brendan Hansknecht (Jul 04 2025 at 20:20):

Brendan Hansknecht (Jul 04 2025 at 21:37):

Hmm....zls doesn't seem to work for me on macos anymore. I don't know what went wrong, but it always hits:

/Users/bren077s/vendor/zig-0.14.0/lib/std/posix.zig:4533:22: 0x102497013 in kevent (build)
            .BADF => unreachable, // Always a race condition.

This theoretically comes from the --watch flag, but I am not setting the --watch flag in zls.

Brendan Hansknecht (Jul 04 2025 at 21:38):

Luke Boswell (Jul 04 2025 at 22:10):

It's working for me, but I built from source a long time ago and haven't updated it since. I'm not sure if the 0.14.0 branch in zls repo gets updates

Brendan Hansknecht (Jul 04 2025 at 22:11):

Yeah, I haven't built it from source in a long time and I thought it was working last time I was working on roc, but I guess not anymore.

Brendan Hansknecht (Jul 04 2025 at 22:11):

Brendan Hansknecht (Jul 04 2025 at 22:32):

Also, I guess to clarify. I think the main issue is with diagnostics. I'm not getting build errors in roc currently.

Brendan Hansknecht (Jul 04 2025 at 22:32):

Luke Boswell (Jul 05 2025 at 08:26):

**UNEXPECTED TOKEN IN EXPRESSION**
The token **crash "** is not expected in an expression.

Richard Feldman (Jul 05 2025 at 12:03):

Jared Ramirez (Jul 06 2025 at 21:49):

What do y'all think about the fact that CIR owns ModuleEnv? Having used it for a while it feels a little off, and I've been thinking like having ModuleEnv be a sibling of the IRs might make sense. Then, each stage could just have a pointer to it, plus the a pointer to last stage's IR and this stage's.

module_env: *ModuleEnv
can_ir: *CIR, // already there
parse_ir: *AST, // already there

module_env: *ModuleEnv
can_ir: *CIR, // already there

Luke Boswell (Jul 06 2025 at 21:59):

CIR doesn't own the memory anymore. I had to change that to a pointer to avoid some hairy lifetime issues.

Luke Boswell (Jul 06 2025 at 22:01):

I don't see any reason check_types can't hold a reference to the module env also

Jared Ramirez (Jul 06 2025 at 22:01):

Richard Feldman (Jul 06 2025 at 22:39):

Richard Feldman (Jul 06 2025 at 22:40):

couldn't we pass in exactly the parts they need, e.g. the IR from the previous stage and gpa?

Luke Boswell (Jul 06 2025 at 22:41):

Richard Feldman (Jul 06 2025 at 22:48):

interesting, I think we could pass in const* pointers to those two after tokenization

Richard Feldman (Jul 06 2025 at 22:49):

basically formalize which parts of our process should make changes to which parts of the module env

Luke Boswell (Jul 08 2025 at 03:06):

dbg x
crash "msg"
expect 1 == 1
return x

Luke Boswell (Jul 08 2025 at 03:07):

I'm quite sure this is correct, that's how Anthony has it parsing by the look of things... so I'm going to proceed.

Joshua Warner (Jul 08 2025 at 03:37):

I have a (more) data-oriented-design version of sexpr that I've been working on. It's not a complete slam dunk, but currently ~30% faster than the existing sexpr: https://github.com/roc-lang/roc/pull/7965.
Usage is a bit more verbose, but hopefully not too bad. There's still some significant headroom for optimization.
Would be curious on folks' thoughts.

Brendan Hansknecht (Jul 08 2025 at 04:27):

Brendan Hansknecht (Jul 08 2025 at 04:28):

And dod sexpr sound like a nice idea now that we are scaling up snapshots so much

Brendan Hansknecht (Jul 08 2025 at 04:28):

Originally it was avoided cause they aren't really play of the compiler and are reasonable to keep super simple

Luke Boswell (Jul 08 2025 at 04:52):

I think the SExpr's have been invaluable for validating behaviour, particularly between the stages TOKEN -> PARSE -> CAN -> TYPES etc

Joshua Warner (Jul 08 2025 at 05:11):

Luke Boswell (Jul 08 2025 at 05:14):

dbg(x)
crash("msg")
expect(1 == 1)
return(x)

Luke Boswell (Jul 08 2025 at 05:18):

foo : U64 -> Result(Str, [TooBig])
foo = |num| {
    str = if (num > 10) {
        return Err(TooBig)
    } else {
        "SMALL"
    }
    Ok(str)
}

Luke Boswell (Jul 08 2025 at 05:24):

foo = |num| {
    # statement - prints out the value of num converted to a string
    dbg num.to_str()

    # expression - prints out the value of num and then returns it
    dbg(num)
}

Brendan Hansknecht (Jul 08 2025 at 05:25):

Update my comment. Was just saying that DOD sexprs sound like a reasonable idea.

Brendan Hansknecht (Jul 08 2025 at 05:27):

Anton (Jul 08 2025 at 11:27):

Luke Boswell (Jul 08 2025 at 11:47):

The fuzzer can find some really interesting bugs { i, Complete] ... who would think of that!

Luke Boswell (Jul 08 2025 at 12:02):

Ah, man... we really need to give the Parser NodeStore the same love we just did for the Can NodeStore and add a bunch of roundtrip unit tests. I've found a few hard to track down issues now stemming from us putting something in or getting something out that is different than we expect.

@Anthony Bullard -- After your refactor I'm happy to make this change. It was a bit mechanical but I got a pretty quick process down to implement it.

Anton (Jul 08 2025 at 14:31):

In comments especially, I think we regularly use just "variable" when we're talking about a "type variable". Let's try to always use "type variable" to avoid confusion with a variable as in var x = 0.

Luke Boswell (Jul 08 2025 at 21:52):

After sleeping on it, I realised adding unit tests roundtripping the AST NodeStore might actually help @Anthony Bullard with the refactor. That scope of work is internal representation right?

Kiryl Dziamura (Jul 09 2025 at 10:00):

I just realized that if nominal type encapsulates a list and its size never changes (e.g. Matrix or Vector types), it can be allocated and managed as a static array behind the scenes.
Am I naive?

Richard Feldman (Jul 09 2025 at 11:26):

Richard Feldman (Jul 09 2025 at 11:27):

Richard Feldman (Jul 09 2025 at 11:29):

also we can't offer a way to get list elements that doesn't return a Result, even if it's fixed-length and we know it can't fail

Richard Feldman (Jul 09 2025 at 11:30):

whereas with tuples we do track length in the type system, they're already stack-allocated, we can already get their elements without Result, and with static dispatch we can automatically add .iter() and .len() to any tuple whose elements are all type-compatible

Kiryl Dziamura (Jul 09 2025 at 11:31):

Luke Boswell (Jul 09 2025 at 11:32):

Kiryl Dziamura (Jul 09 2025 at 11:32):

Luke Boswell (Jul 09 2025 at 11:33):

Shouldn't be a problem. I can imagine tools might code-gen larger tuples for MxN sizes someday

Kiryl Dziamura (Jul 09 2025 at 11:33):

Richard Feldman (Jul 09 2025 at 12:03):

Joshua Warner (Jul 09 2025 at 16:31):

I discovered last night that just switching the allocator in the snapshot generator can make the snapshot tests ~6x faster - from ~1400ms to 229ms.

joshw@Joshuas-MacBook-Air-3 ~/s/g/r/roc (main)> zig build snapshot
info: processed 412 snapshots in 229 ms.
joshw@Joshuas-MacBook-Air-3 ~/s/g/r/roc (main)> git diff
diff --git a/src/snapshot.zig b/src/snapshot.zig
index baa63fc905..6d0cf05212 100644
--- a/src/snapshot.zig
+++ b/src/snapshot.zig
@@ -96,7 +96,7 @@ pub fn main() !void {
     defer {
         _ = gpa_impl.deinit();
     }
-    const gpa = gpa_impl.allocator();
+    const gpa = std.heap.c_allocator;

     const args = try std.process.argsAlloc(gpa);
     defer std.process.argsFree(gpa, args);

Do we get anything from the GeneralPurposeAllocator (aka DebugAllocator, apparently), that's worth this 6x perf cost?

Anthony Bullard (Jul 09 2025 at 16:34):

Anton (Jul 09 2025 at 16:34):

We could use the DebugAllocator for debugging only after valgrind finds something. But I'm not sure valgrind catches everything the DebugAllocator would.

Joshua Warner (Jul 09 2025 at 16:38):

Anthony Bullard (Jul 09 2025 at 16:39):

Anthony Bullard (Jul 09 2025 at 16:40):

it probably wouldn't make much of a difference given how the tool is typically used

Joshua Warner (Jul 09 2025 at 16:58):

Richard Feldman (Jul 09 2025 at 18:15):

it would be nice to use the testing allocator for snapshot generation on ci - that one is slower but it also detects memory leaks

Joshua Warner (Jul 09 2025 at 18:15):

Joshua Warner (Jul 09 2025 at 18:47):

With the c allocator and some simple multithreading, getting good numbers with html snapshot generation again:

$ zig build snapshot
info: collected 412 items in 1 ms, processed 412 snapshots in 50 ms.
$ zig build snapshot -- --html
info: collected 412 items in 1 ms, processed 412 snapshots in 167 ms.

Joshua Warner (Jul 09 2025 at 18:47):

Joshua Warner (Jul 09 2025 at 18:50):

$ zig build snapshot -- --singlethread
info: collected 412 items in 1 ms, processed 412 snapshots in 196 ms.
$ zig build snapshot -- --singlethread --html
info: collected 412 items in 1 ms, processed 412 snapshots in 396 ms.

Joshua Warner (Jul 09 2025 at 18:50):

Brendan Hansknecht (Jul 09 2025 at 19:57):

Brendan Hansknecht (Jul 09 2025 at 19:58):

Brendan Hansknecht (Jul 09 2025 at 19:59):

Oh hmm...actually we use the c allocator in roc main, so I guess that is our default allocator currently

Joshua Warner (Jul 09 2025 at 23:10):

Brendan Hansknecht (Jul 09 2025 at 23:26):

Joshua Warner (Jul 11 2025 at 01:50):

Random side note - in the future I think it'll be really useful/helpful to minimize fuzz crashes at least a bit before checking them in

Brendan Hansknecht (Jul 11 2025 at 02:49):

Brendan Hansknecht (Jul 11 2025 at 02:50):

Not sure which part specifically you are calling out. snapshots of fuzz tests or what appears on the website.

Luke Boswell (Jul 11 2025 at 02:51):

I have added quite a few snapshots fuzz_crash that I hadn't minimised. I'm not sure how we minimise them...

Brendan Hansknecht (Jul 11 2025 at 02:51):

Luke Boswell (Jul 11 2025 at 02:52):

Brendan Hansknecht (Jul 11 2025 at 02:52):

I think they have afl-tmin.bash instead and it is missing features and so still doesn't work for our executables on mac

Luke Boswell (Jul 11 2025 at 02:53):

It's still sitting in a box. It doesn't have a wifi card installed and the router is up the other end of the house :sad:

Luke Boswell (Jul 11 2025 at 02:54):

Joshua Warner (Jul 11 2025 at 02:54):

Luke Boswell (Jul 11 2025 at 02:55):

Joshua Warner (Jul 11 2025 at 02:55):

Also FWIW minimization is not a super hard algorithm. I have a version of it written up for the old compiler that we can easily port

Luke Boswell (Jul 11 2025 at 02:55):

Joshua Warner (Jul 11 2025 at 02:57):

Brendan Hansknecht (Jul 11 2025 at 03:15):

Brendan Hansknecht (Jul 11 2025 at 03:16):

Also, yeah, minimization is pretty simple in most systems. I think the main the afl-tmin does is delete various block sizes and test if it still repros. So like test deleting every 256 byte chunk one at a time and if any can be deleted with repro, remove it. Then try 128 then 64 then 32 then ... and see how small the repro gets.

Brendan Hansknecht (Jul 11 2025 at 03:23):

Anton (Jul 11 2025 at 17:42):

I'm debugging a segfault and I'd like to add some logging to print all the intermediate outputs. Just like the snapshots but live (as soon as the phase is done) , do you have any pointers before I start on this @Luke Boswell?

Luke Boswell (Jul 11 2025 at 17:49):

What's been working well for me is asking an LLM to isolate the bug using print debug statements. Basically tell it to drill down starting from this point and iterate. It usually find the bug very quickly, and gives me a nice summary of why it's occuring.

Luke Boswell (Jul 11 2025 at 17:51):

Anton (Jul 11 2025 at 17:51):

Luke Boswell (Jul 11 2025 at 17:54):

I've got ideas, but I'd put them in the experimental category. It's not something I have a lot of experience with.

Anton (Jul 11 2025 at 17:57):

I mainly want to know if you think it's a good idea to re-use code from snapshot.zig or modify snapshot.zig to share code with the logging etc.

Luke Boswell (Jul 11 2025 at 17:57):

I just watched a LamdbaConf video on Jai recently and he had some really nice debugging stuff for his compiler. Has put a lot of thought into how that is done.

Luke Boswell (Jul 11 2025 at 17:58):

Jared Ramirez (Jul 11 2025 at 20:48):

does anyone else run into frequent crashes with passing --watch to zig build <cmd>?

Anthony Bullard (Jul 11 2025 at 20:48):

Jared Ramirez (Jul 11 2025 at 20:50):

Yeah, I've been using watchexec as a work around. Does anyone object to me adding that as a dep in src/flake.nix? (At least until we see if 14.1 or 15 fixes it)

Richard Feldman (Jul 11 2025 at 20:59):

Brendan Hansknecht (Jul 11 2025 at 21:39):

We already use 14.1 (at least in nix and ci, locally, you may still be running 14.0)

Brendan Hansknecht (Jul 11 2025 at 21:39):

Joshua Warner (Jul 12 2025 at 03:11):

What's the latest on the foo.(local_func)(args) calling style? Is that supposed to be parsed in the new compiler?

Richard Feldman (Jul 12 2025 at 03:27):

we ended up settling on foo->local_func as the syntax and referring to them as "arrow calls" but I forget what their implementation status is in the new compiler

Joshua Warner (Jul 12 2025 at 03:31):

Currently going thru my migration script to update my benchmarking corpus and ran into the .(local_func) syntax

Joshua Warner (Jul 12 2025 at 03:31):

Richard Feldman (Jul 12 2025 at 03:43):

Richard Feldman (Jul 12 2025 at 03:44):

Anton (Jul 12 2025 at 09:13):

Seems fine, maybe put it in a separate devDependencies like we do in the old flake here.

Anton (Jul 12 2025 at 14:26):

Seeing all the visualizations made me think; it would be nice to have anomaly detection set up for the snapshots, like in CI it would alert us if some metric for a Roc program is 100x higher than the average for all snapshots. I don't know if we should actually do that but it's a cool idea :p

Joshua Warner (Jul 12 2025 at 15:39):

Is import Foo.Bar supposed to work? The new compiler doesn't parse that currently.

Brendan Hansknecht (Jul 12 2025 at 16:07):

Joshua Warner (Jul 12 2025 at 18:39):

Richard Feldman (Jul 12 2025 at 18:42):

Brendan Hansknecht (Jul 12 2025 at 18:48):

Brendan Hansknecht (Jul 12 2025 at 18:49):

Joshua Warner (Jul 12 2025 at 18:50):

My thinking is that it's really easy to just concat a bunch of random files, and that's actually a reasonable input to the tokenizer - but that's going to fail early in parsing.

Joshua Warner (Jul 12 2025 at 18:51):

Brendan Hansknecht (Jul 12 2025 at 18:58):

Brendan Hansknecht (Jul 12 2025 at 18:59):

Joshua Warner (Jul 12 2025 at 19:04):

@Brendan Hansknecht do I need to do anything special to get this to work with Tracy?

Brendan Hansknecht (Jul 12 2025 at 19:07):

Brendan Hansknecht (Jul 12 2025 at 19:08):

Brendan Hansknecht (Jul 12 2025 at 19:09):

General question this got me thinking of. should roc check be able to run over a directory of files? I guess that might lead to a bunch of redundant work

Brendan Hansknecht (Jul 12 2025 at 19:09):

Richard Feldman (Jul 12 2025 at 19:48):

Brendan Hansknecht (Jul 12 2025 at 19:54):

You mean like roc check dir/*.roc? That doesn't deal with recursive roc files and directory structures. And not all shells support **. That is why I generally prefer direct support of passing in a dir

Richard Feldman (Jul 12 2025 at 20:03):

Joshua Warner (Jul 12 2025 at 21:33):

Also, there are limits on the length of a command line on some platforms. Eg 32kb on windows, IIRC. It’s easy to hit those by doing shell expansion of large directories.

Richard Feldman (Jul 12 2025 at 23:30):

wow, I just successfully resolved a segfault in our snapshot testing system that only reproduced in one file, and only when running with the --debug flag - just by telling Claude to try to fix it and then being increasingly insistent that the bug had to be in our code somewhere and not a bug in zig's general purpose allocator.

Richard Feldman (Jul 12 2025 at 23:30):

Richard Feldman (Jul 12 2025 at 23:31):

Anton (Jul 13 2025 at 10:13):

Brendan Hansknecht (Jul 13 2025 at 12:44):

Anton (Jul 13 2025 at 13:04):

I have not had it cheat on me before, but I have heard it from others. I expect that these kinds of issues are temporary.

Brendan Hansknecht (Jul 13 2025 at 13:25):

Yeah, I bet some of the rl tuning is over optimized on getting some tasks done and doesn't check well enough on how it gets done

Brendan Hansknecht (Jul 13 2025 at 13:30):

Also, I have been messing around with it in a python + mojo project and that may confuse it more often.

Anton (Jul 13 2025 at 13:31):

Richard Feldman (Jul 13 2025 at 13:51):

I've gotten used to these now. It used to be that I'd think "wow, how embarrassing for Claude" and now I'm just like "oh yeah, that happens, hey Claude do it right, moving on."

Richard Feldman (Jul 13 2025 at 13:52):

it's also why I don't understand people who are like "I don't even read the code anymore" - I have never seen that be the optimal way to use these tools on a serious project

Richard Feldman (Jul 13 2025 at 13:53):

if you don't read the code you have no idea when tests are being silently deleted etc

Brendan Hansknecht (Jul 13 2025 at 13:57):

Yeah, Ive only just started using Claude (cause my work is paying for it), and so far, I have been impressed for super short scripts and trivial edits to many files, but in general, I think it slows me down currently.

Brendan Hansknecht (Jul 13 2025 at 13:57):

But I'm finding the patterns it can deal with quickly that would be less fast for me to do

Richard Feldman (Jul 13 2025 at 14:01):

yeah it took me awhile before it was a net speedup for me on complicated projects

Richard Feldman (Jul 13 2025 at 14:02):

a lot of trial and error getting a feeling for failure modes, how to scope tasks for it, etc.

Richard Feldman (Jul 13 2025 at 14:02):

now I'm definitely significantly faster with it than without it, but it took awhile for that to be true

Brendan Hansknecht (Jul 13 2025 at 14:40):

Yeah. Makes sense. And I bet it is more capable work things that have more examples documentation and tooling.... Mojo is a bit lacking

Brendan Hansknecht (Jul 13 2025 at 15:40):

Also, I bet that strong typing wpuld help it. I probably should add a claude.md and add an entry about type checking and ignoring things

Brendan Hansknecht (Jul 13 2025 at 15:50):

I think roc (once there are enough examples and the compiler is robust wnough) will be very llm friendly due to how the language works

Kiryl Dziamura (Jul 13 2025 at 15:51):

Kiryl Dziamura (Jul 14 2025 at 09:25):

image.png
when I call type information in zsl for a single quote, I get this nice table

Kiryl Dziamura (Jul 14 2025 at 09:48):

does roc want \u(...) escape or \u{...}? asking because string interpolation was \(...) initially and now it's ${...}

Kiryl Dziamura (Jul 14 2025 at 10:00):

and another question: why not implementing utf8 escape with int num literals? like \u{0x123} or \u{42} (yeah, historically this format is for hex so it would be unexpected to have dec here) or \u{0b1001}? hex is absolutely enough. I'm just wondering if num literals can give any advantage here?

Kiryl Dziamura (Jul 14 2025 at 10:26):

should escape seqs be processed during tokenization or during canonicalization? my thought is that it shouldn't hinder tokenization. the only case of escaping tokenizer should process is escape of double quote for strings ("\"") and single quote for single quote ('\'') so it knows where's the start and the end of the literal. and then process the literal at can time so the formatter can still format the code. but I'm likely missing smth.
@Joshua Warner I think you should know the answer

Richard Feldman (Jul 14 2025 at 11:24):

during tokenization the bytes we need to traverse will for sure already be in cache, so I think that'll be the cheapest time to do it

Richard Feldman (Jul 14 2025 at 11:24):

I'd also like to move the logic for turning number literals into actual numbers into tokenization for the same reason

Richard Feldman (Jul 14 2025 at 11:25):

that said, it might make sense to have a flag that the formatter can pass in to say "don't bother doing this because it won't get used"

Richard Feldman (Jul 14 2025 at 11:26):

I remember talking about this on Zulip years ago, and I think the conclusion was that Unicode escapes are a feature that will be used 99.99% of the time by copy/pasting from somewhere, so the most important thing is consistency, not flexibility

Richard Feldman (Jul 14 2025 at 11:27):

that said, I think the main thing is that we don't want to change the syntax here; however it works in the old compiler is exactly how it should work in the new compiler :smile:

Brendan Hansknecht (Jul 14 2025 at 13:53):

Isn't this in the boat of so fast it is better to just recompute on they fly at the last minute instead of shooting somewhere and spending extra memory

Brendan Hansknecht (Jul 14 2025 at 13:54):

Not actually sure it's the case but I thought Andrew talked directly about this in a zig talk and it being a win to not do it until the last minute

Brendan Hansknecht (Jul 14 2025 at 13:55):

On specific thing to consider for the long term, if we switch to a simd tokenizer, I'm pretty sure this will put a non-simd loop in the middle of it...which likely will make it cost more.

Richard Feldman (Jul 14 2025 at 14:04):

we need it during canonicalization to report "number literal too big for type" errors

Richard Feldman (Jul 14 2025 at 14:05):

Richard Feldman (Jul 14 2025 at 14:06):

fair point about SIMD tokenization, but right now we're doing it in the middle of canonicalization and every time we hit a number literal we go load memory that hasn't been touched since tokenization and won't be touched again unless there's an error to report, so definitely seems like it would be better to do it earlier :sweat_smile:

Brendan Hansknecht (Jul 14 2025 at 14:12):

Richard Feldman (Jul 14 2025 at 14:16):

actually I think it would be fine in the SIMDjson style, because after processing each 64-byte chunk of source with a bunch of SIMD passes, there are a series of non-SIMD passes that have to happen afterwards before moving onto the next 64B chunk, and they're all loops

Richard Feldman (Jul 14 2025 at 14:16):

Brendan Hansknecht (Jul 14 2025 at 14:18):

Brendan Hansknecht (Jul 14 2025 at 14:19):

Richard Feldman (Jul 15 2025 at 01:13):

Daniel Lemire doing experiments to see if using unaligned memory hurts perf on modern CPUs and...apparently it doesn't! :sweat_smile:

Brendan Hansknecht (Jul 15 2025 at 01:54):

Yeah, the only cost on modern machines should be around cache lines maybe not containing full data

Brendan Hansknecht (Jul 15 2025 at 01:56):

Brendan Hansknecht (Jul 15 2025 at 01:57):

So worst cost would be if it leads to loading two caches lines in a one off. Then that happens on repeat, but not linearly such that prefecthing helps

Brendan Hansknecht (Jul 15 2025 at 02:00):

Obviously don't want to do it anyway if you can avoid it cause it isn't a guarantee and your code may also run on less optimized architecture used for mobile or edge processors.

Luke Boswell (Jul 15 2025 at 11:04):

I fixed some minor issues with backticks in our can diagnostics. I've noticed there are a bunch of things with our problem reports that aren't ideal, and we tend to be copy-pasting the last ones etc.

Whenever you're looking at a new Diagnostic (or the existing ones if you feel like helping clean this up a little) can you pay special attention to the helpers like addCodeBlock or addInlineCode or addKeyword or addType etc and try and use the right one. It will just help to keep our reports pretty in every context.

Luke Boswell (Jul 15 2025 at 11:58):

Kiryl Dziamura (Jul 15 2025 at 12:56):

Yesterday I was thinking whether zig transforms switch by tokens to jmpifs or to a jumptable (I think it should but I haven't checked) and then started wondering if it's true for tokenizer itself (jump per character). So in place of a sequence of ifs we have a jumptable where offset is the character byte

Brendan Hansknecht (Jul 15 2025 at 14:51):

You should by default assume that ifs manifest and hurt optimizations....of course they can be cmov

Brendan Hansknecht (Jul 15 2025 at 14:52):

Also, zig turns a while true loop with a switch in it to a jump table. I believe that is the pattern...I know there was a doc or changelog entry about the format here

Brendan Hansknecht (Jul 15 2025 at 14:53):

But I'm sure some of these techniques will apply. I know we have also already shared a few zig specific blogs

Kiryl Dziamura (Jul 15 2025 at 14:55):

there's also a comment about simd in the clickhouse lexer: https://news.ycombinator.com/item?id=44562278
I heard a lot about simdjson, I guess it's time to finally look into it :smile:

Brendan Hansknecht (Jul 15 2025 at 15:01):

I only roughly know it. Also, I'm sure many incremental changes could be made to the current tokenizer to make it faster

Brendan Hansknecht (Jul 15 2025 at 15:01):

That said, of we want to improve compiler perf, we should focus on can and diagnostics

Brendan Hansknecht (Jul 15 2025 at 15:24):

One extra note...if tokenizer perf sounds fun (either tuning the current tokenizer or working on a more Simd version)....go for it....enjoy

Anton (Jul 15 2025 at 15:27):

I've been wondering, when we introduce simd to the tokenizer, should we just keep two versions of the tokenizer:

Kiryl Dziamura (Jul 15 2025 at 15:28):

Maybe next time :smile:just verifying my thoughts. I already have what to do in the repo

Brendan Hansknecht (Jul 15 2025 at 17:14):

Luke Boswell (Jul 15 2025 at 18:49):

Richard Feldman (Jul 15 2025 at 19:15):

Joshua Warner (Jul 15 2025 at 21:57):

A casual attempt at applying labeled switch to the main switch in the tokenizer yields a ~15% perf regression :/

Brendan Hansknecht (Jul 15 2025 at 22:04):

Kiryl Dziamura (Jul 17 2025 at 08:57):

in tokenize.zig, there are malformed tokens and diagnostic messages. my intuition was that malformed token is a twin of a valid token or group of tokens (e.g. valid Float, Int and their malformed version MalformedNumber) and diagnostic messages clarify the malfunction (e.g. NumberNoDigits, NumberBadSuffix, NumberNoExponentDigits) but I see the things, that I would expect to be messages only, are actually tokens:

                .MalformedNumberBadSuffix,
                .MalformedNumberNoDigits,
                .MalformedNumberNoExponentDigits,
                .MalformedNumberUnicodeSuffix,

Kiryl Dziamura (Jul 17 2025 at 13:39):

I just realized that unicode escape in single quote literal is redundant since roc considers single quote as an int. to me, base16 via 0x... is clearer than \u{...}:

what's the reason of keeping both? to comply string literal and for historical reasons?

Brendan Hansknecht (Jul 17 2025 at 14:17):

Joshua Warner (Jul 17 2025 at 15:20):

The intent here was to communicate to the parser that there was a vaguely-number-like-thing here, and you should probably construct the syntax tree to be consistent with there being a number-like-thing here.

Contrast that with the situation where we just have the tokenizer message and no token, where it'll seem to the parser like there was nothing there at all - definitely leading to a redundant error message, and possibly leading to cascading parse errors because the tree is structurally wrong.

Kiryl Dziamura (Jul 17 2025 at 15:23):

but number-like-thing can be just MalformedNumber. and tokenizer message has region. but I get what you mean. it sounds like it's better to ditch diagnostics during tokenization and generate unique malformed tokens instead. then push messages at canonicalization based on the tokens

Kiryl Dziamura (Jul 17 2025 at 15:24):

Joshua Warner (Jul 17 2025 at 15:26):

There are som diagnostics that we don’t want to convert to malformed tokens, eg the one for having an uppercase base (0Xff)

Joshua Warner (Jul 19 2025 at 17:38):

Joshua Warner (Jul 19 2025 at 17:42):

/// The source is always duplicated for the ModuleEnv, which owns the copy.
fn processSourceInternal(

Joshua Warner (Jul 19 2025 at 17:48):

Ah of course and the higher-level function has a leak in the case of ~a cache hit~ the cache being disabled :sweat_smile:

Richard Feldman (Jul 19 2025 at 19:45):

yeah my thinking there was that when the compiler is running in like a watch mode, it'll be common for files to change on disk and then we need to reload the source bytes as well as getting a new ModuleEnv, so it seemed like coupling those would be the least error-prone if we're going to be unloading ModuleEnvs at runtime

Joshua Warner (Jul 19 2025 at 20:19):

I ought to be able to invoke the compiler on code I manage the allocation for, without duplicating it.

Joshua Warner (Jul 19 2025 at 20:21):

Ran into this vis-à-vis benchmarking, where somehow I missed this and we were double-freeing all the sources because I wasn’t duping them every iteration.

Richard Feldman (Jul 19 2025 at 21:09):

Richard Feldman (Jul 19 2025 at 21:10):

ok feel free to change it so that module env owns everything else but the src :+1:

Joshua Warner (Jul 19 2025 at 21:29):

Joshua Warner (Jul 19 2025 at 22:53):

Joshua Warner (Jul 19 2025 at 22:54):

Side note: issue #8086 is fast approaching. Should try to get some sort of intel throwback in there :stuck_out_tongue:

Luke Boswell (Jul 19 2025 at 23:08):

I know this is a first world problem... but has anyone else ever had trouble with sub-millisecond precision measurements across different browsers?

Luke Boswell (Jul 19 2025 at 23:08):

Joshua Warner (Jul 19 2025 at 23:09):

Joshua Warner (Jul 19 2025 at 23:11):

It was that speculative execution thing a few years ago. It turns out if you can measure times really precisely, it becomes possible to exfiltrate secret data. Very hard to fully harden against, so a defense in depth is making sure you can measure super precisely.

Luke Boswell (Jul 19 2025 at 23:13):

Luke Boswell (Jul 19 2025 at 23:15):

I'm enjoying seeing the timing information for our compiler to type check some source :smiley:

Joshua Warner (Jul 20 2025 at 00:08):

Joshua Warner (Jul 20 2025 at 00:09):

If we could do wasm code emission and execution in anywhere close to that, that’d be awesome!

Luke Boswell (Jul 20 2025 at 01:10):

Thats the time taken to load the source into the wasm compiler, tokenize->parse->canonicalize->type check->serialise diagnostics to HTML

Luke Boswell (Jul 20 2025 at 01:11):

Joshua Warner (Jul 20 2025 at 01:38):

Is there something about diagnostics that makes them complicated/slow to generate?

Joshua Warner (Jul 20 2025 at 01:38):

Luke Boswell (Jul 20 2025 at 01:42):

Luke Boswell (Jul 20 2025 at 01:43):

But yeah, I haven't really looked at it. Just messing around with things. It's really cool how capable the compiler is already!

Brendan Hansknecht (Jul 20 2025 at 02:17):

Joshua Warner (Jul 20 2025 at 02:47):

Essentially, they have a system that can type-check a partially-written typescript file (i.e. a prefix of a file), such that it can determine whether or not any continuation of that prefix could possibly be well-typed.

Joshua Warner (Jul 20 2025 at 03:04):

Richard Feldman (Jul 20 2025 at 11:05):

so is the goal to be able to tell when someone is typing in an editor and the line they're currently writing is incomplete, and if we detect that it's definitely an error no matter what comes after it, then we can assume it's malformed and then continue trying to check the rest of the file as normal so you still get errors elsewhere?

Joshua Warner (Jul 20 2025 at 15:32):

The goal is to distinguish errors that are purely due to the input being incomplete, and errors that won’t be fixed no matter what more is typed at the cursor (or cursors, I suppose).

Joshua Warner (Jul 20 2025 at 15:35):

I think this is mostly useful in the context of providing some constraints during decoding for an llm (ie guaranteeing that the llm generates a well-typed program), but maybe there are some implications for what errors we immediately display in an editor vs which ones we wait for your cursor to move away before displaying.

Kiryl Dziamura (Jul 22 2025 at 07:38):

Luke Boswell (Jul 25 2025 at 05:37):

What do you think about adding to our zig CI lint, that you cannot leave any std.debug.print's in the codebase. These don't play nicely with WASM freestanding or anything that doesn't have stdio.

Luke Boswell (Jul 26 2025 at 11:05):

@Loris Cro or any Zig folk... are there any mature TUI libraries we should consider for roc cli?

Not sure it's the path we want to go down... but interested to explore the idea, particularly now we basically have things ready to build our REPL.

Loris Cro (Jul 26 2025 at 11:11):

yes libvaxis is the one I would recommend as well. just for completeness know that it has a high-level and a low-level API, so if you like the idea of having widgets and the whole runtime that manages them, then great, but if you decide you want direct control over everything, you can still do that, libvaxis will not fight you

Loris Cro (Jul 26 2025 at 11:12):

I use the low-level api in bork because I wanted to have my own interaction system with UI elements

Stream: compiler development

Topic: casual conversation

Ayaz Hafiz (Jul 07 2023 at 16:12):

Folkert de Vries (Jul 07 2023 at 16:13):

Folkert de Vries (Jul 07 2023 at 16:14):

Ayaz Hafiz (Jul 07 2023 at 16:16):

Ayaz Hafiz (Jul 07 2023 at 16:16):

Folkert de Vries (Jul 09 2023 at 16:28):

Folkert de Vries (Jul 10 2023 at 19:18):

Folkert de Vries (Jul 10 2023 at 19:18):

Ayaz Hafiz (Jul 12 2023 at 21:22):

Luke Boswell (Jul 12 2023 at 22:33):

Ayaz Hafiz (Jul 12 2023 at 23:22):

Richard Feldman (Jul 12 2023 at 23:23):

Richard Feldman (Jul 12 2023 at 23:23):

Ayaz Hafiz (Jul 15 2023 at 20:28):

Richard Feldman (Jul 18 2023 at 18:16):

Ayaz Hafiz (Jul 18 2023 at 18:18):

Andrew Kelley (Jul 18 2023 at 20:59):

Richard Feldman (Jul 18 2023 at 21:00):

Richard Feldman (Jul 18 2023 at 21:00):

Andrew Kelley (Jul 18 2023 at 21:03):

Andrew Kelley (Jul 18 2023 at 21:04):

Pearce Keesling (Jul 18 2023 at 21:26):

Ayaz Hafiz (Jul 18 2023 at 21:51):

Qqwy / Marten (Jul 18 2023 at 22:06):

Folkert de Vries (Jul 21 2023 at 12:20):

Folkert de Vries (Jul 21 2023 at 12:21):

Richard Feldman (Jul 24 2023 at 19:54):

Brendan Hansknecht (Jul 24 2023 at 20:32):

Brendan Hansknecht (Jul 24 2023 at 20:34):

Ayaz Hafiz (Jul 24 2023 at 21:17):

Folkert de Vries (Jul 27 2023 at 22:32):

Folkert de Vries (Aug 05 2023 at 16:44):

Folkert de Vries (Aug 05 2023 at 16:45):

Folkert de Vries (Aug 07 2023 at 15:09):

Folkert de Vries (Aug 09 2023 at 14:21):

Ayaz Hafiz (Aug 09 2023 at 14:36):

Richard Feldman (Aug 09 2023 at 14:38):

Folkert de Vries (Aug 09 2023 at 14:38):

Richard Feldman (Aug 09 2023 at 14:39):

Richard Feldman (Aug 09 2023 at 14:39):

Folkert de Vries (Aug 09 2023 at 14:39):

Folkert de Vries (Aug 09 2023 at 14:40):

Brendan Hansknecht (Aug 09 2023 at 15:05):

Folkert de Vries (Aug 09 2023 at 15:07):

Folkert de Vries (Aug 09 2023 at 15:07):

Richard Feldman (Aug 11 2023 at 05:23):

Folkert de Vries (Aug 18 2023 at 14:23):

Ayaz Hafiz (Aug 19 2023 at 06:05):

Ayaz Hafiz (Aug 19 2023 at 06:05):

ShalokShalom (Aug 20 2023 at 09:42):

Ayaz Hafiz (Aug 20 2023 at 14:29):

Folkert de Vries (Aug 20 2023 at 14:30):

Richard Feldman (Aug 23 2023 at 13:26):

Richard Feldman (Aug 23 2023 at 13:27):

Brendan Hansknecht (Aug 23 2023 at 14:20):

Brendan Hansknecht (Aug 23 2023 at 14:20):

Richard Feldman (Aug 23 2023 at 14:21):

Richard Feldman (Aug 23 2023 at 14:21):

Pit Capitain (Aug 24 2023 at 12:49):

Zeljko Nesic (Aug 24 2023 at 18:02):

Richard Feldman (Aug 24 2023 at 18:22):

Richard Feldman (Aug 24 2023 at 23:15):

Richard Feldman (Aug 26 2023 at 18:49):

Luke Boswell (Aug 31 2023 at 07:29):

Luke Boswell (Aug 31 2023 at 07:35):

Luke Boswell (Aug 31 2023 at 07:43):

Luke Boswell (Aug 31 2023 at 07:45):

Luke Boswell (Aug 31 2023 at 08:01):

Fábio Beirão (Aug 31 2023 at 13:13):

Fábio Beirão (Aug 31 2023 at 13:15):

Fábio Beirão (Aug 31 2023 at 13:17):

Fábio Beirão (Aug 31 2023 at 13:17):

Luke Boswell (Aug 31 2023 at 21:06):

Richard Feldman (Sep 04 2023 at 03:36):

Richard Feldman (Sep 04 2023 at 14:05):

Folkert de Vries (Sep 04 2023 at 14:15):

Richard Feldman (Sep 04 2023 at 17:07):

Luke Boswell (Sep 05 2023 at 10:13):