Ian McLerran (Jan 17 2025 at 17:59): Ian McLerran (Jan 17 2025 at 17:59):I have encountered an error where I get a panic when running an expect statement. Specifically, if I uncomment either of the 2 commented out comparisons, the program will panic with:
thread 'main' panicked at /Users/imclerran/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bumpalo-3.14.0/src/collections/raw_vec.rs:691:5:
capacity overflow
However, as you can see, the function being tested (the main logic) does not appear to be causing the crash, since the function is called, sent to Result.map_ok, and a comparison can be run asserting that the value != Ok ShouldNotPass, and this does not panic and the expect passes.
This tells me that the panic is actually occurring while accessing the value, or in the Boolean comparison itself, but the panic at least did not occur while assigning the value in the utf8_to_utf16 function.
Here is the expect in question:
expect
# Missing continuation bytes
utf8_2 = [0b1101_1111, 0b0011_1111]
utf8_3 = [0b1110_1111, 0b1011_1111, 0b0011_1111]
utf8_4 = [0b1111_0111, 0b1011_1111, 0b1011_1111, 0b0011_1111]
err_2 = Err(BadUtf8({ index: 1, problem: ExpectedContinuation }))
err_3 = Err(BadUtf8({ index: 2, problem: ExpectedContinuation }))
err_4 = Err(BadUtf8({ index: 3, problem: ExpectedContinuation }))
utf16_2 = utf8_to_utf16(utf8_2)
# Result.map_ok added to prove utf8_to_utf16 is being evaluated without panic.
utf16_3 = utf8_to_utf16(utf8_3) |> Result.map_ok(|_| ShouldNotPass)
utf16_4 = utf8_to_utf16(utf8_4) |> Result.map_ok(|_| ShouldNotPass)
(utf16_2 == err_2)
# && (utf16_3 == err_3) # panics if uncommented
# && (utf16_4 == err_4) # panics if uncommented
&& (utf16_3 != Ok ShouldNotPass)
&& (utf16_4 != Ok ShouldNotPass)
Interesting development:
Adding this to the expect produces the exact same panic:
(while leaving the two lines commented out)
&& Str.from_utf8(utf8_2) == Err(BadUtf8({ index: 1, problem: ExpectedContinuation }))
However, this does not panic:
module []
expect
utf8_2 = [0b1101_1111, 0b0011_1111]
Str.from_utf8(utf8_2) == Err(BadUtf8({ index: 1, problem: ExpectedContinuation }))
In the code sample I think the problematic value gets compiled away without the expect, hence no panic
Anthony Bullard (Jan 17 2025 at 18:29):The panic is definitely from the utf8_to_utf16 function
Anthony Bullard (Jan 17 2025 at 18:29):Did you run with RUST_BACKTRACE=1 ?
Ian McLerran (Jan 17 2025 at 18:38):Okay, further testing -
I added:
dbg utf16_2
dbg utf16_3
dbg utf16_4
All values print correctly.
The problem is that the index value does not match the index value in the corresponding err values. If I change the index values in the expected errors, it does not panic. This is definitely in the == operator.
Ian McLerran (Jan 17 2025 at 18:39):[Utf16.roc:257] utf16_2 = (Err (BadUtf8 {index: 1, problem: ExpectedContinuation}))
[Utf16.roc:258] utf16_3 = (Err (BadUtf8 {index: 1, problem: ExpectedContinuation}))
[Utf16.roc:259] utf16_4 = (Err (BadUtf8 {index: 2, problem: ExpectedContinuation}))
0 failed and 10 passed in 613 ms.
If I change the indexes in the expected values to match what is actually calculated, I can uncomment all lines and the expect passes. The panic only occurs if utf16_n...index != index err_n...index
Ian McLerran (Jan 17 2025 at 18:53):Okay, I changed the algorithm which computes the index slightly to align with Str.from_utf8 (now returns index of first byte in character, rather than index byte which is an invalid continuation).
Ian McLerran (Jan 17 2025 at 18:53):However, the panic is still easily reproduced, just with different values of index.
expect
utf8_2 = [0b1101_1111, 0b0011_1111]
# if index is changed to anything besides 0, it panics
err_2 = Err(BadUtf8({ index: 0, problem: ExpectedContinuation }))
utf16_2 = utf8_to_utf16(utf8_2)
(utf16_2 == err_2)
Here is a fully independent min repro, which does not depend on any of my code:
module []
expect
v1 = Tag { i: 1 }
v2 = Tag { i: 2 }
v1 == v2
Ian McLerran said:
Here is a fully independent min repro, which does not depend on any of my code:
module [] expect v1 = Tag { i: 1 } v2 = Tag { i: 2 } v1 == v2
What roc command (roc test?) and version do you use?
Ian McLerran (Jan 17 2025 at 19:05):Yeah, roc test, built from latest main about an hour ago
Ian McLerran (Jan 17 2025 at 19:05):roc built from commit 9d37c906fe, committed at 2025-01-16 04:38:55 UTC
I could not reproduce on ubuntu 22.04:
โฏ ./target/release/roc version
roc built from commit 9d37c906fe, committed at 2025-01-16 04:38:55 UTC
roc on ๎ HEAD (9d37c90) [?] is ๐ฆ v0.0.1 via ๐ฆ v1.77.2 via โ๏ธ impure (nix-shell-env)
โฏ ./target/release/roc test temp.roc
โโ EXPECT FAILED in temp.roc โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
This expectation failed:
3โ> expect
4โ> v1 = Tag { i: 1 }
5โ> v2 = Tag { i: 2 }
6โ> v1 == v2
When it failed, these variables had these values:
v1 : [Tag { i : Num * }]
v1 = Tag { i: 1 }
v2 : [Tag { i : Num * }]
v2 = Tag { i: 2 }
1 failed and 0 passed in 126 ms.
Are you on macos?
Ian McLerran (Jan 17 2025 at 19:10):Okay, I'm now on roc version: roc built from commit 841e823235, committed at 2025-01-17 18:35:15 UTC
Ian McLerran (Jan 17 2025 at 19:11):Same issue with 841e823235. And yeah, MacOS.
Ian McLerran (Jan 17 2025 at 19:11):Aarch64
Ian McLerran (Jan 17 2025 at 19:36):Okay, appologies. I had a different panic in that min repro from not using a proper module. Apparently this is not a min repro, but a different panic.
expect
v1 = Tag { i: 1 }
v2 = Tag { i: 2 }
v1 == v2
HOWEVER...
Ian McLerran (Jan 17 2025 at 19:37):The original min repro I posted before I edited (thought i found smaller) still produces it:
module []
expect
utf8 = [0b1101_1111, 0b0011_1111]
err = Err(BadUtf8({ index: 1, problem: ExpectedContinuation }))
str_res = utf8 |> Str.from_utf8
str_res == err
Appologies, didn't see this:
Anthony Bullard said:
Did you run with RUST_BACKTRACE=1 ?
Here is the backtrace...
thread 'main' panicked at /Users/imclerran/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bumpalo-3.14.0/src/lib.rs:1854:5:
out of memory
stack backtrace:
0: rust_begin_unwind
at /rustc/25ef9e3d85d934b27d9dada2f9dd52b1dc63bb04/library/std/src/panicking.rs:647:5
1: core::panicking::panic_fmt
at /rustc/25ef9e3d85d934b27d9dada2f9dd52b1dc63bb04/library/core/src/panicking.rs:72:14
2: bumpalo::oom
3: roc_repl_eval::eval::addr_to_ast
4: roc_repl_eval::eval::expr_of_tag
5: roc_repl_eval::eval::addr_to_ast
6: roc_repl_expect::get_values
7: roc_repl_expect::run::render_expect_failure
8: roc_cli::test
9: roc::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
That will help for sure
Brendan Hansknecht (Jan 17 2025 at 20:20):More stuff in the shared mem infra that is broke
Anton (Jan 18 2025 at 10:25):I was able to reproduce with:
module []
expect
utf8 = [0b1101_1111, 0b0011_1111]
err = Err(BadUtf8({ index: 1, problem: ExpectedContinuation }))
str_res = utf8 |> Str.from_utf8
str_res == err
Can you make an issue @Ian McLerran?
Ian McLerran (Jan 21 2025 at 14:57):Sure can!
Ian McLerran (Jan 21 2025 at 15:21):Issue @ #7539
Last updated: Jul 06 2025 at 12:14 UTC