Stream: beginners

Topic: REPL Test Crashes

Sam Mohr (Jul 31 2024 at 08:00):

When I run cargo test --all in a nix dev shell on current main on my x86_64 Intel linuxbook, I get four failures in the REPL tests, like the following:

---- tests::identity_lambda stdout ----
thread 'tests::identity_lambda' panicked at crates/repl_test/src/cli.rs:96:13:
repl exited unexpectedly before finishing evaluation. Exit status was ExitStatus(unix_wait_status(25856)) and stderr was ""
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

If I run the example test in the repl myself, I get:

[smores@smoresbook:~/dev/roc]$ RUST_BACKTRACE=1 cargo run --bin roc -- repl
    Finished dev [unoptimized + debuginfo] target(s) in 0.40s
     Running `target/debug/roc repl`

  The rockin' roc repl
────────────────────────

Enter an expression, or :help, or :q to quit.

» \x -> x
thread 'main' panicked at crates/compiler/gen_dev/src/generic64/mod.rs:1085:9:
assertion `left == right` failed
  left: 0
 right: 1
stack backtrace:
...
   4: build_fn_call<roc_gen_dev::generic64::x86_64::X86_64GeneralReg, roc_gen_dev::generic64::x86_64::X86_64FloatReg, roc_gen_dev::generic64::x86_64::X86_64Assembler, roc_gen_dev::generic64::x86_64::X86_64SystemV>
             at ./crates/compiler/gen_dev/src/generic64/mod.rs:1085:9
   5: build_expr<roc_gen_dev::generic64::Backend64Bit<roc_gen_dev::generic64::x86_64::X86_64GeneralReg, roc_gen_dev::generic64::x86_64::X86_64FloatReg, roc_gen_dev::generic64::x86_64::X86_64Assembler, roc_gen_dev::generic64::x86_64::X86_64SystemV>>
             at ./crates/compiler/gen_dev/src/lib.rs:829:25
   6: build_stmt<roc_gen_dev::generic64::Backend64Bit<roc_gen_dev::generic64::x86_64::X86_64GeneralReg, roc_gen_dev::generic64::x86_64::X86_64FloatReg, roc_gen_dev::generic64::x86_64::X86_64Assembler, roc_gen_dev::generic64::x86_64::X86_64SystemV>>
             at ./crates/compiler/gen_dev/src/lib.rs:595:17
...

I'm not seeing this in the CI tests, which are all passing for recent changes, but this is very consistent for me. Has anyone seen this issue before?

Luke Boswell (Jul 31 2024 at 08:13):

Do you get any failures with cargo test --release?

Luke Boswell (Jul 31 2024 at 08:13):

I don't think all the tests are included in CI normally

Sam Mohr (Jul 31 2024 at 08:13):

Let's try it

Sam Mohr (Jul 31 2024 at 08:13):

I confirmed that this happens on a clean clone of the repo

Luke Boswell (Jul 31 2024 at 08:14):

It may be that that test has regressed and it's not in CI. Maybe some of the more expensive tests aren't automatic, and have to be checked manually.

Luke Boswell (Jul 31 2024 at 08:15):

@Anton will be able to tell us I'm sure

Sam Mohr (Jul 31 2024 at 08:16):

The issue seems to be coming from returning a value in the REPL that is a lambda/function. When I debug print running \x -> x in the REPL, I get this:

[smores@smoresbook:~/dev/roc]$ cargo run --bin roc -- repl
    Finished dev [unoptimized + debuginfo] target(s) in 0.23s
     Running `target/debug/roc repl`

  The rockin' roc repl
────────────────────────

Enter an expression, or :help, or :q to quit.

» \x -> x
[crates/compiler/gen_dev/src/generic64/mod.rs:1085:9] (dst, &fn_name, &args, &arg_layouts) = (
    `#UserApp.setjmp`,
    "roc_setjmp",
    [
        `#UserApp.buffer`,
    ],
    [
        InLayout(U64),
    ],
)
[crates/compiler/gen_dev/src/generic64/mod.rs:1085:9] (dst, &fn_name, &args, &arg_layouts) = (
    `#UserApp.result`,
    "#UserApp_replOutput_13093313482219921989",
    [],
    [
        InLayout(VOID),
    ],
)
thread 'main' panicked at crates/compiler/gen_dev/src/generic64/mod.rs:1086:9:
assertion `left == right` failed
  left: 0
 right: 1
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Which shows that the result we try to store our REPL output into has zero args, but a single arg layout InLayout(VOID)

Sam Mohr (Jul 31 2024 at 08:17):

I'm building the release tests now, but it may take a few minutes on this older laptop

Sam Mohr (Jul 31 2024 at 08:17):

The above issue is not the case for functions that are called, like (\x -> x) 123, or values, like 123

Sam Mohr (Jul 31 2024 at 08:18):

I'll look into it further while this builds, and also check if these tests run in the CI

Sam Mohr (Jul 31 2024 at 08:20):

Actually, we are successfully running this specific test identity_lambda in the CI tests successfully: https://github.com/roc-lang/roc/actions/runs/10164028461/job/28108433906?pr=6941#step:8:357

Sam Mohr (Jul 31 2024 at 08:23):

However, they run successfully in --release mode in the CI (specifically --profile release-with-lto), which is also true on my local machine. So this error occurs in dev mode, but not release mode, probably because the failing assertion is debug only.

Sam Mohr (Jul 31 2024 at 08:24):

Also, the assertion fails with 0 args and 1 layout, which means even when we don't check the assertion in release mode, we won't fail because we iterate over the args, not the layouts

Luke Boswell (Jul 31 2024 at 08:26):

Now that you say it, I remember why I just always use --release and it was something like this. I always assumed it was just some tests failed if they didn't have an optimised build. I'm really not sure though.

Sam Mohr (Jul 31 2024 at 08:26):

Sounds like we have a few places where we have failed assertions that point to issues, but don't cause failures in behavior

Anton (Jul 31 2024 at 13:29):

Yeah, I'd really like to give these assertions and the issues they raise some love because they're early checks that may highlight a problem that is much harder to debug without the assertion. I'll do tests with debug on CI and skip any that currently fail so we at least prevent future regressions.

Anton (Jul 31 2024 at 14:48):

Related issues: #6946 #6947
PR to run debug tests on CI: PR#6948

Last updated: Jul 06 2025 at 12:14 UTC