Karakatiza (Mar 29 2024 at 00:29): Karakatiza (Mar 29 2024 at 00:29):I am trying to write a custom type to parse nulls in JSON, and the following code crashes when roc test:
interface Nullable
exposes [Nullable, null, full, nullableDecode, nullableEncode]
imports [
json.Core.{ json },
]
Nullable val := [Null, Full val] implements [
Decoding { decoder: nullableDecode },
Encoding { toEncoder: nullableEncode },
Inspect, # auto derive
Eq, # auto derive
]
null = @Nullable Null
full = \a -> @Nullable (Full a)
nullBytes = [110, 117, 108, 108]
nullableDecode : Decoder (Nullable val) fmt where val implements Decoding, fmt implements DecoderFormatting
nullableDecode = Decode.custom \bytes, fmt ->
if bytes |> List.startsWith nullBytes then
{ result: Ok (@Nullable (Null)), rest: List.dropFirst bytes 4 }
else
when bytes |> Decode.decodeWith (Decode.decoder) fmt is
{ result: Ok res, rest } -> { result: Ok (@Nullable (Full res)), rest }
{ result: Err a, rest } -> { result: Err a, rest }
nullableEncode : Nullable val -> Encoder fmt where val implements Encoding, fmt implements EncoderFormatting
nullableEncode = \val ->
Encode.custom
(\bytes, fmt -> List.concat
bytes
(
when val is
@Nullable Null -> nullBytes
@Nullable (Full a) -> Encode.toBytes a fmt
)
)
TestType : {
a : Nullable I32,
}
expect
actual : DecodeResult TestType
actual = "{\"a\":null}" |> Str.toUtf8 |> Decode.fromBytesPartial json
expected = Ok ({ a: @Nullable Null })
actual.result == expected
expect
actual : DecodeResult TestType
actual = "{\"a\":3}" |> Str.toUtf8 |> Decode.fromBytesPartial json
expected = Ok ({ a: full 3 })
actual.result == expected
expect
actual = { a: @Nullable Null } |> Encode.toBytes json |> Str.fromUtf8
expected = Ok ("{\"a\":null}")
actual == expected
expect
actual = { a: full 3 } |> Encode.toBytes json |> Str.fromUtf8
expected = Ok ("{\"a\":3}")
actual == expected
The culprit is test 3: when encoding a null value, the branch @Nullable (Some a) -> is chosen. And the crash only happens when val or matched a are accessed in any way anywhere in the body of nullableEncode during test 3, even with dbg.
Karakatiza (Mar 29 2024 at 00:34):The crash:
thread '<unnamed>' panicked at 'called `Option::unwrap()` on a `None` value', crates/compiler/mono/src/ir.rs:6143:56
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
There was an unrecoverable error in the Roc compiler. The `roc check` command can sometimes give a more helpful error report than other commands.
roc check doesn't find errors
Richard Feldman (Mar 29 2024 at 01:22):does roc check pass?
Karakatiza (Mar 29 2024 at 01:24):It does, 0 errors and the warnings only relate to no usage of some declarations
Richard Feldman (Mar 29 2024 at 13:11):huh, interesting! I'm seeing roc check errors when I copy/paste that source locally :thinking:
── UNRECOGNIZED NAME in Lib/Nullable.roc ───────────────────────────────────────
Nothing is named `json` in this scope.
50│ actual = "{\"a\": null}" |> Str.toUtf8 |> Decode.fromBytesPartial json
^^^^
Did you mean one of these?
Box
Bool
U8
F64
── UNRECOGNIZED NAME in Lib/Nullable.roc ───────────────────────────────────────
Nothing is named `json` in this scope.
56│ actual = "{\"a\": 3}" |> Str.toUtf8 |> Decode.fromBytesPartial json
^^^^
Did you mean one of these?
Box
Bool
U8
F64
── UNRECOGNIZED NAME in Lib/Nullable.roc ───────────────────────────────────────
Nothing is named `json` in this scope.
61│ actual = { a: @Nullable Null } |> Encode.toBytes json |> Str.fromUtf8
^^^^
Did you mean one of these?
Box
Bool
U8
F64
── UNRECOGNIZED NAME in Lib/Nullable.roc ───────────────────────────────────────
Nothing is named `json` in this scope.
66│ actual = { a: nonNull 3 } |> Encode.toBytes json |> Str.fromUtf8
^^^^
Did you mean one of these?
Box
Bool
U8
F64
────────────────────────────────────────────────────────────────────────────────
4 errors and 1 warning found in 16 ms
if you do roc --version what does it print?
Karakatiza (Mar 29 2024 at 13:19):Ah, I am just using roc-json library, did not mention that
Karakatiza (Mar 29 2024 at 13:21):The latest 0.6.3, and yesterday's roc nightly (will print version a bit later)
Karakatiza (Mar 29 2024 at 13:22):Json.Core.{ json }
Richard Feldman (Mar 29 2024 at 13:34):ah, can you maybe share a link to the full repo? Looks like this needs more than one file to reproduce :big_smile:
Karakatiza (Mar 29 2024 at 13:47):Not a link but the complete file to try as-is:
main.roc
app "simple-json"
packages {
pf: "https://github.com/roc-lang/basic-cli/releases/download/0.8.1/x8URkvfyi9I0QhmVG98roKBUs_AZRkLFwFJVJ3942YA.tar.br",
json: "https://github.com/lukewilliamboswell/roc-json/releases/download/0.6.3/_2Dh4Eju2v_tFtZeMq8aZ9qw2outG04NbkmKpFhXS_4.tar.br",
}
imports [
pf.Stdout,
json.Core.{ json },
]
provides [main] to pf
main =
Stdout.line "Hello Richard!"
Nullable val := [Null, Full val] implements [
Decoding { decoder: nullableDecode },
Encoding { toEncoder: nullableEncode },
Inspect, # auto derive
Eq, # auto derive
]
null = @Nullable Null
full = \a -> @Nullable (Full a)
nullBytes = [110, 117, 108, 108]
nullableDecode : Decoder (Nullable val) fmt where val implements Decoding, fmt implements DecoderFormatting
nullableDecode = Decode.custom \bytes, fmt ->
if bytes |> List.startsWith nullBytes then
{ result: Ok (@Nullable (Null)), rest: List.dropFirst bytes 4 }
else
when bytes |> Decode.decodeWith (Decode.decoder) fmt is
{ result: Ok res, rest } -> { result: Ok (@Nullable (Full res)), rest }
{ result: Err a, rest } -> { result: Err a, rest }
nullableEncode : Nullable val -> Encoder fmt where val implements Encoding, fmt implements EncoderFormatting
nullableEncode = \val ->
Encode.custom
(\bytes, fmt -> List.concat
bytes
(
when val is
@Nullable Null -> nullBytes
@Nullable (Full a) -> Encode.toBytes a fmt
)
)
TestType : {
a : Nullable I32,
}
expect
actual : DecodeResult TestType
actual = "{\"a\":null}" |> Str.toUtf8 |> Decode.fromBytesPartial json
expected = Ok ({ a: null })
actual.result == expected
expect
actual : DecodeResult TestType
actual = "{\"a\":3}" |> Str.toUtf8 |> Decode.fromBytesPartial json
expected = Ok ({ a: full 3 })
actual.result == expected
expect
actual = { a: @Nullable Null } |> Encode.toBytes json |> Str.fromUtf8
expected = Ok ("{\"a\":null}")
actual == expected
expect
actual = { a: full 3 } |> Encode.toBytes json |> Str.fromUtf8
expected = Ok ("{\"a\":3}")
actual == expected
Ok wow, this is definitely a bug, but I just found a workaround
Crashes:
expect
actual = { a: @Nullable Null } |> Encode.toBytes json |> Str.fromUtf8
expected = Ok ("{\"a\":null}")
actual == expected
Passes:
expect
actual = { a: null } |> Encode.toBytes json |> Str.fromUtf8
expected = Ok ("{\"a\":null}")
actual == expected
To help debugging, copying again the bug behavior:
The culprit is test 3: when encoding a null value, the branch @Nullable (Some a) -> is chosen. And the crash only happens when val or matched a are accessed in any way anywhere in the body of nullableEncode during test 3, even with dbg.
Richard Feldman (Mar 29 2024 at 14:27):huh! Might be a scoping issue involving expect that we haven't encountered before
Richard Feldman (Mar 29 2024 at 14:28):do you think you could minimize that and open an issue for it? I bet you don't need to get JSON involved at all, seems like the issue is with the opaque wrapper
Karakatiza (Mar 29 2024 at 14:29):Weird thing is the first test worked with
expect
actual : DecodeResult TestType
actual = "{\"a\":null}" |> Str.toUtf8 |> Decode.fromBytesPartial json
expected = Ok ({ a: @Nullable Null })
actual.result == expected
I'll try to minimize the repro today
Richard Feldman (Mar 29 2024 at 15:48):appreciate it, thank you!
Last updated: Jul 05 2025 at 12:14 UTC