Stream: beginners

Topic: Decoding an Aliased Opaque type causes Compiler crashes

view this post on Zulip Eli Dowling (Mar 18 2024 at 00:10):

When I try to decode an aliased opaque type it crashes the compiler
I made this repro example:

UserName := Str implements [
        Eq { isEq: userNameEq },
        Decoding {
            decoder: userNameDecode,
        },
    ]
userNameEq = \@UserName a, @UserName b -> a == b
userNameDecode = Decode.custom \bytes, fmt ->
    bytes
    |> Decode.fromBytesPartial fmt
    |> Decode.mapResult @UserName

#This is fine
Alias : { user: UserName }
expect
    prog =
        """
        {"user":"name"}
        """
        |> Str.toUtf8
    rec : Result Alias _
    rec = prog |> Decode.fromBytes TotallyNotJson.json
    rec == Ok { user: @UserName "name" }
# This will crash the compiler
AliasBad : UserName
expect
    prog =
        "name"
        |> Str.toUtf8
    rec : Result AliasBad _
    rec = prog |> Decode.fromBytes TotallyNotJson.json
    rec == @UserName "name" |>Ok

I get this error:

An internal compiler expectation was broken.
This is definitely a compiler bug.
Please file an issue here: https://github.com/roc-lang/roc/issues/new/choose
thread '<unnamed>' panicked at 'ambient functions don't unify', /home/eli/Code/roc/roc/crates/compiler/unify/src/unify.rs:201:18
stack backtrace:
   0: rust_begin_unwind
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/panicking.rs:593:5
   1: core::panicking::panic_fmt
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/panicking.rs:67:14
   2: panic_display<&str>
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/panicking.rs:150:5
   3: roc_unify::unify::Unified<M>::expect_success
             at ./crates/compiler/unify/src/unify.rs:201:18
   4: compact_lambda_set<roc_late_solve::LatePhase>
             at ./crates/compiler/solve/src/specialize.rs:577:73
   5: compact_lambda_sets_of_vars<roc_late_solve::LatePhase>
             at ./crates/compiler/solve/src/specialize.rs:434:37
   6: unify
             at ./crates/compiler/late_solve/src/lib.rs:403:17
   7: roc_mono::ir::Env::unify
             at ./crates/compiler/mono/src/ir.rs:1437:33
   8: specialize_proc_help
             at ./crates/compiler/mono/src/ir.rs:3506:20
   9: specialize_variable
             at ./crates/compiler/mono/src/ir.rs:4049:9
  10: call_by_name_help
             at ./crates/compiler/mono/src/ir.rs:8958:31
  11: call_by_name
             at ./crates/compiler/mono/src/ir.rs:8671:17
  12: with_hole
             at ./crates/compiler/mono/src/ir.rs:5420:21
  13: assign_to_symbol
             at ./crates/compiler/mono/src/ir.rs:8486:23
  14: from_can
             at ./crates/compiler/mono/src/ir.rs:6949:13
  15: specialize_proc_help
             at ./crates/compiler/mono/src/ir.rs:3541:32
  16: specialize_variable
             at ./crates/compiler/mono/src/ir.rs:4049:9
  17: specialize_external_help
             at ./crates/compiler/mono/src/ir.rs:3198:9
  18: specialize_external_specializations
             at ./crates/compiler/mono/src/ir.rs:3178:13
  19: specialize_all
             at ./crates/compiler/mono/src/ir.rs:3040:9
  20: roc_load_internal::file::make_specializations
             at ./crates/compiler/load_internal/src/file.rs:5745:13
  21: roc_load_internal::file::run_task
             at ./crates/compiler/load_internal/src/file.rs:6498:17
  22: worker_task
             at ./crates/compiler/load_internal/src/file.rs:2048:34
  23: roc_load_internal::file::load_multi_threaded::{{closure}}::{{closure}}
             at ./crates/compiler/load_internal/src/file.rs:1788:25
  24: crossbeam_utils::thread::ScopedThreadBuilder::spawn::{{closure}}
             at /home/eli/.cargo/registry/src/index.crates.io-6f17d22bba15001f/crossbeam-utils-0.8.16/src/thread.rs:440:31
  25: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/ops/function.rs:250:5
  26: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/alloc/src/boxed.rs:1993:9

Interestingly in my original application where this cropped up the error is different, perhaps that's because the opaque type takes type args?:
Code:

TT: Option Str
expect
    prog="""{"prog":"hi"}"""|>Str.toUtf8

    rec:Result {prog:TT} _
    rec=prog|>Decode.fromBytes Core.json
    rec==Ok {prog:Union.some "hi" }

Makes this error:

thread 'main' panicked at 'Error in alias analysis: error in module ModName("UserApp"), function definition FuncName("\x02\x00\x00\x00%\x00\x00\x00y0za5-\xc3Y"), definition of value binding ValueId(22): could not find func in module ModName("UserApp") with name FuncName("\x15\x00\x00\x00\x14\x00\x00\x00\xa8,\xfbK\xdc\xea\xc1\xe8")', crates/compiler/gen_llvm/src/llvm/build.rs:5761:19
stack backtrace:
   0: rust_begin_unwind
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/panicking.rs:593:5
   1: core::panicking::panic_fmt
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/panicking.rs:67:14
   2: build_procedures_help
             at ./crates/compiler/gen_llvm/src/llvm/build.rs:5761:19
   3: build_procedures_expose_expects
             at ./crates/compiler/gen_llvm/src/llvm/build.rs:5675:25
   4: expect_mono_module_to_dylib
             at ./crates/repl_expect/src/run.rs:675:24
   5: test
             at ./crates/cli/src/lib.rs:518:43
   6: main
             at ./crates/cli/src/main.rs:80:17
   7: core::ops::function::FnOnce::call_once
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/ops/function.rs:250:5

view this post on Zulip Ayaz Hafiz (Mar 18 2024 at 01:54):

ouch, that’s a nasty bug

view this post on Zulip Eli Dowling (Mar 18 2024 at 07:32):

I've got a few more for you too ;)

optionToEncode : Option val -> Encoder fmt  where val implements Encoding,fmt implements EncoderFormatting
optionToEncode = \@Option val ->
    when val is
        Some contents ->
            Encode.custom \bytes, fmt ->
                bytes |> Encode.append contents fmt
        None -> Encode.custom \bytes, fmt -> bytes

expect
    encoded =
        dat:{maybe:Option u8,other:Str}
        dat={maybe: none {},other:"hi" }
        Encode.toBytes dat Core.json
        |> Str.fromUtf8

    expected = Ok "{\"other\":\"hi\"}"
    expected == encoded

This causes:

thread '<unnamed>' panicked at 'unspecialized lambda sets left over during resolution:
LambdaSet([] + (<939>FlexAble(fmt, [`Encode.Encoding`]):`Encode.toEncoder`:2), ^<942>),
 UlsOfVar(VecMap { keys: [939, 971], values: [VecSet { elements: [938, 941] }, VecSet { elements: [970, 973] }] })',
 crates/compiler/mono/src/layout.rs:2065:17

Interestingly encoding a Some value works fine.

I've modified the Core.json record encoder slightly so that it just doesn't include the field if it gets a 0 byte response but even just encoding the Option on it's own fails with the exact same error:

expect
    encoded =
        dat:Option u8
        dat=@Option None
        Encode.toBytes dat Core.json
        |> Str.fromUtf8

    expected = Ok ""
    expected == encoded

view this post on Zulip Eli Dowling (Mar 18 2024 at 07:44):

Interestingly with some more testing i found that if i remove the encode from the Some encoder I don't get that error.

optionToEncode : Option val -> Encoder fmt  where val implements Encoding,fmt implements EncoderFormatting
optionToEncode = \@Option val ->
    when val is
        Some contents ->
            Encode.custom \bytes, fmt ->
                bytes |>List.concat ("hi"|>Str.toUtf8)
        None -> Encode.custom \bytes, fmt ->
            bytes |>List.concat ("hello"|>Str.toUtf8)

I was able to run the tests, but it mostly throws this error depending on how long the list i return in the encoder is.

thread 'main' panicked at 'misaligned pointer dereference: address must be a multiple of 0x8 but is 0x7ffff7e2c03c', crates/repl_expect/src/app.rs:57:45

view this post on Zulip Anton (Mar 18 2024 at 10:08):

thread 'main' panicked at 'misaligned pointer dereference: address must be a multiple of 0x8 but is 0x7ffff7e2c03c', crates/repl_expect/src/app.rs:57:45

Does this also happen with a release compiler?

view this post on Zulip Eli Dowling (Mar 18 2024 at 10:10):

Oh, haven't tested that, I can do that shortly though.

view this post on Zulip Eli Dowling (Mar 18 2024 at 10:26):

Thanks @Anton I got some improvement there.
The version that just returns lists no longer has weird pointer errors.
However if i actually try to encode in the Some branch it still gives me this error.

thread '<unnamed>' panicked at 'called `Option::unwrap()` on a `None` value', crates/compiler/mono/src/ir.rs:6143:56

Which looks like it's the same as the lambda set error from before, it just isn't being caught by the debug asserts

view this post on Zulip Ayaz Hafiz (Mar 18 2024 at 14:06):

The debug asserts probably fail earlier, at the first panic you got

view this post on Zulip Ayaz Hafiz (Mar 18 2024 at 14:07):

Also, pretty worrying that we get a bad pointer deref in dev but not in release... we should fix the one in dev

view this post on Zulip Eli Dowling (Mar 20 2024 at 08:45):

Well I figured out what the second set of errors were.. in my type annotations I had u8 instead of U8 :face_palm::face_palm::face_palm::face_palm:

Last updated: Jul 06 2025 at 12:14 UTC