Joshua Warner (Dec 14 2021 at 01:59): Joshua Warner (Dec 14 2021 at 01:59):Thinking about tackling a json parser library for roc; does that seem overly ambitious at this point? Any tips on the best way to lay out the interface? For instance, I see packages/parser; would it be appropriate to base things on that?
Lucas Rosa (Dec 14 2021 at 03:06):You should be fine. I think you just need stuff from the Str module for that
Lucas Rosa (Dec 14 2021 at 03:06):it could also be a good stress test and uncover some more bugs
Lucas Rosa (Dec 14 2021 at 03:07):I remember some discussion around encoders and decoders but I forget where that went.
Joshua Warner (Dec 14 2021 at 04:22):Fun times! Pretty sure I hit a compiler bug already.
Here's the code:
#!/usr/bin/env roc
app "json"
packages { base: "platform", }
imports [
base.Task.{ Task, await },
base.Stdout,
]
provides [ main ] to base
main : Task {} *
main =
{} <- await (Stdout.line "ready")
parsed = Number (42)
serialized = formatJson parsed
# If instead of the above two lines, I do the following, things work fine:
# serialized = formatJson (Number (42))
Stdout.line serialized
JsonValue : [
Number I64,
String Str,]
formatJson: JsonValue -> Str
formatJson = \value ->
when value is
Number n -> Num.toStr n
String s -> Str.concat "\"" (Str.concat (escape s) "\"")
escape: Str -> Str
escape = \text -> text # TODO
Here's the error:
joshw@Joshuas-MacBook-Air ~/s/g/r/roc (joshuawarner32/extract-spaced-enum-2) [101]> cargo run -- examples/cli/Json.roc
Finished dev [unoptimized + debuginfo] target(s) in 0.38s
Running `target/debug/roc examples/cli/Json.roc`
[compiler/load/src/file.rs:2606] loc_package_entry = |L 3-3, C 15-31| PackageEntry {
shorthand: "base",
spaces_after_shorthand: [],
package_or_path: |L 3-3, C 21-31| Path(
PlainLine(
"platform",
),
),
}
[compiler/load/src/file.rs:2608] existing_package = "base"
thread 'main' panicked at 'Error in alias analysis: error in module ModName("UserApp"), function definition FuncName("\x08\x00\x00\x00\x04\x00\x00\x00}\x0b\x1fU\xbe,v\x90"), definition of value binding ValueId(4): expected type '(union { ((),), ((heap_cell,),) },)', found type '((),)'', compiler/gen_llvm/src/llvm/build.rs:4051:19
stack backtrace:
🔨 Rebuilding host... Done!
0: rust_begin_unwind
at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:517:5
1: std::panicking::begin_panic_fmt
at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:460:5
2: roc_gen_llvm::llvm::build::build_procedures_help
at ./compiler/gen_llvm/src/llvm/build.rs:4051:19
3: roc_gen_llvm::llvm::build::build_procedures
at ./compiler/gen_llvm/src/llvm/build.rs:4024:5
4: roc_build::program::gen_from_mono_module_llvm
at ./compiler/build/src/program.rs:301:5
5: roc_build::program::gen_from_mono_module
at ./compiler/build/src/program.rs:207:50
6: roc_cli::build::build_file
at ./cli/src/build.rs:181:27
7: roc_cli::build
at ./cli/src/lib.rs:312:27
8: roc::main
at ./cli/src/main.rs:30:21
9: core::ops::function::FnOnce::call_once
at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Also I've learned that the compiler panics if you try to call List.map with the arguments reversed, e.g. List.map formatJson items. (rather than showing a type error)
Joshua Warner (Dec 14 2021 at 04:50)::grimacing: Been running into a non-stop stream of internal compiler errors. Some of them _might_ be my fault, not super clear based on the error messages.
Joshua Warner (Dec 14 2021 at 04:52):The most recent is:
thread '<unnamed>' panicked at 'TODO layout_from_flat_type for Apply(`#UserApp.Pair`, [])', compiler/mono/src/layout.rs:1483:21
It’s never the users fault!
Joshua Warner (Dec 14 2021 at 05:09):^^^ 100% agreed. What I meant by that was, "Maybe I wrote code that wasn't technically valid Roc, and the compiler is still learning how to express its emotions" :stuck_out_tongue_wink:
Joshua Warner (Dec 14 2021 at 05:22):For bugs in places like compiler/unify/src/unify.rs and compiler/solve/src/solve.rs, what's the best way to figure the things in the error message map back to source locations in the input program?
Lucas Rosa (Dec 14 2021 at 07:29):For sure, I understand :p. I was just being silly :)
Lucas Rosa (Dec 14 2021 at 07:30):I think the exprs are usually wrapped in Loc which should carry source info
Lucas Rosa (Dec 14 2021 at 07:31):I have to double check though or maybe I misunderstood your question
Folkert de Vries (Dec 14 2021 at 12:19):you get panics in type checking? Also are you using roc check or cargo run -- check ? that will usually show the (type) errors
Folkert de Vries (Dec 14 2021 at 12:19):the real problem is that we panic before those errors can be shown
Folkert de Vries (Dec 14 2021 at 12:20):or I mean the real real problem is that we're not correctly handling incorrect programs, but for now a quick fix might be just running the type checker until it is satisfied, and then hopefully codegen just works at that point
Joshua Warner (Dec 15 2021 at 00:51):cargo run -- check is definitely better. I came back to the problem with fresh eyes and realized I was missing a , between variants in my tag union / enum.
Joshua Warner (Dec 15 2021 at 00:52):It definitely seems like the parser _should_ have yelled at me in that condition.
Joshua Warner (Dec 15 2021 at 00:57):Ok, I think I'm past the main problems, but encountering some new interesting challenges.
Is it possible to use a self-referential type? e.g.
JsonValue :
[
Number I64,
String Str,
Array (List JsonValue),
Object (List { key: Str, value: JsonValue}),
]
When I try to construct a literal of that type (Object [ { key: "testing", value: Number (42) }]) and serialize it, I'm getting this compiler error:
── CIRCULAR TYPE ───────────────────────────────────────────────────────────────
I'm inferring a weird self-referential type for printKeyValue:
46│ printKeyValue = \{key, value} ->
^^^^^^^^^^^^^
Here is my best effort at writing down the type. You will see ∞ for
parts of the type that repeat something already printed out
infinitely.
{ key : Str, value : [ Array (List a), Number I64,
Object (List ∞), String Str ] as a } -> Str
it _should_ work, but recursive types with just aliases are tricky
Folkert de Vries (Dec 15 2021 at 01:00):we have plans for proper newtypes
Folkert de Vries (Dec 15 2021 at 01:00):which should make all this much more reliable
Joshua Warner (Dec 15 2021 at 01:00):Is there a hacky way I can make this work for now?
Travis (Oct 04 2022 at 14:05):Hello. I'm having fun learning while (attempting to) make a json parser with roc. Is this a good place to ask questions?
Anton (Oct 04 2022 at 14:09):Hi Travis,
I've moved your question to a new topic, ask away :)
Travis (Oct 04 2022 at 14:18):Hi Anton. Thanks for making a topic. I wonder is there a way to debug what type the compiler is expecting akin to 'holes' in haskell? Forgive me if thats the wrong term. The compiler does a pretty good job of explaining type errors. But it might be nice sometime to force it to stop and just output what the type of something is.
Anton (Oct 04 2022 at 14:27):@Ayaz Hafiz @Folkert de Vries
Ayaz Hafiz (Oct 04 2022 at 14:37):We don't have typed holes exposed in the surface syntax. I think it would be useful to add, but there are some implications regarding syntax that would need to be figured out - for example, we may not be able to use _, since that is used commonly for other purposes already
Richard Feldman (Oct 04 2022 at 14:40):a quick trick: replace whatever type you're curious about with [] so it will fail type checking, and then look at the error message to see what the compiler inferred
Travis (Oct 04 2022 at 14:46):thats pretty handy. i hadn't thought of using []. :+1:
Travis (Oct 06 2022 at 00:08):how can i format code? ```roc doesn't seem to work
Travis (Oct 06 2022 at 00:09):and ```haskell doesn't look great either
Brendan Hansknecht (Oct 06 2022 at 00:10):I tend to use ```coffee
or ```elixir
Travis (Oct 06 2022 at 00:14):Thanks.
Travis (Oct 06 2022 at 00:15):elixir looks passable
Travis (Oct 06 2022 at 00:15):any ideas on how i could allow for a leading minus in manyDigitsP? i'm relying on examples/parser/Parser/{Core,Str}.roc but haven't shown their imports. thinking i need a slightly different approach but not sure.
Error:
The argument is an anonymous function of type:
[Err [Nothing], Ok Str] -> [Ok Str]a
But map needs its 2nd argument to be:
Result U8 [Nothing] -> [Ok Str]a
minus = codeunit 45 # '-'
maybeMinus = maybe minus
|> map \res -> # <-- Here is the problem line
when res is
Ok m -> Ok m
Err _ -> Ok ""
manyDigitsP : Parser RawStr RawStr
manyDigitsP =
sepBy1 maybeMinus (oneOrMore (codeunitSatisfies (\c -> 48 <= c && c <= 59 )))
u64P : Parser RawStr Value
u64P =
manyDigitsP
|> map \rawStr ->
str = strFromRaw rawStr
when Str.toI64 str is
Ok num -> Ok (JsonNumber num)
Err _ -> Err "`\(str)` is not a I64"
|> flatten
seems like i just need to map a non-minus to an empty list
Ayaz Hafiz (Oct 06 2022 at 00:18):try changing
- Ok m -> Ok m
+ Ok _ -> Ok "-"
ah. thank you. so that gets rid of one error. now just this:
43│ manyDigitsP : Parser RawStr RawStr
44│ manyDigitsP =
45│ sepBy1 maybeMinus (oneOrMore (codeunitSatisfies (\c -> 48 <= c && c <= 59 )))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This Parser.Core.sepBy1 call produces:
Parser RawStr (List [Ok Str]a) ?
But the type annotation on manyDigitsP says it should be:
Parser RawStr (List U8) ?
maybe List.map? :thinking:
Travis (Oct 06 2022 at 00:29):think i just need to get change that type annotation and handle the result of manyDigitsP differently - as a List instead of as a raw string.
Travis (Oct 06 2022 at 00:45):any advice on how to transform a List [Ok Str] to a Str?
Travis (Oct 06 2022 at 00:45):is there maybe a List.mapConcat or something?
Travis (Oct 06 2022 at 00:48):nevermind. i think i'm onto something.
Travis (Oct 06 2022 at 00:57):found a solution. now returning List U8 rather than Ok Str from maybeMinus and just doing List.join on its result to get a raw str.
Travis (Oct 06 2022 at 00:59):now i've discovered that Str.toI64 doesn't allow for a leading minus which is what i was hoping for... :face_palm:
Travis (Oct 06 2022 at 01:06):or wait. no it looks like perhaps sepBy1 isn't doing what i thought it was. i expected to get '-123' but actually getting just '-'. hmm not sure why.
Travis (Oct 06 2022 at 01:06):While reading: -123 or - is not a I64
Ayaz Hafiz (Oct 06 2022 at 01:08):Probably - does not parse to an i64
Travis (Oct 06 2022 at 01:16):yeah i was confused what the error message meant. but i think its actually passing just '-' to Str.toI64
Travis (Oct 06 2022 at 01:17):yep. this prints -123:
main : Str
main = Num.toStr (Result.withDefault (Str.toI64 "-123") 0 )
so it looks like i'm not getting the full result from sepBy1
Travis (Oct 06 2022 at 01:28):well i'm taking a break. heres my code if anyone is curious. https://pastebin.com/VQjVetHt
also let me know if there is a better place to paste code.
Travis (Oct 06 2022 at 01:30):also any critiques are welcome :+1: i'm very new and likely making lots of mistakes.
Travis (Oct 06 2022 at 01:37):incase anyone wants to try running it. i've been running it like this from the roc repo's root directory :
$ roc examples/parser/parse-json.roc
Ayaz Hafiz (Oct 06 2022 at 01:40):this is super cool!!
Qqwy / Marten (Oct 06 2022 at 16:31):Nice work! :blush:
Travis (Oct 07 2022 at 03:32):so i figured out how to allow a leading minus. i was infact using sepBy1 incorrectly. the solution i found uses const and apply. it also disallows leading zeroes. here it is:
minus = codeunit 45 # '-'
maybeMinusP : Parser RawStr RawStr
maybeMinusP =
maybe minus
|> map \res ->
when res is
Ok _ -> [45u8]
Err _ -> []
oneToNineP = codeunitSatisfies (\c -> 49 <= c && c <= 59 ) # '1'..'9'
zeroToNineP = codeunitSatisfies (\c -> 48 <= c && c <= 59 ) # '0'..'9'
rawIntP : Parser RawStr RawStr
rawIntP =
const (\vals0 -> \val1 -> \vals -> List.join [vals0, [val1], vals])
|> apply maybeMinusP
|> apply oneToNineP
|> apply (many zeroToNineP)
intP : Parser RawStr Value
intP =
rawIntP
|> map \rawStr ->
str = strFromRaw rawStr
when Str.toI64 str is
Ok num -> Ok (Number num)
Err _ -> Err "`\(str)` is not a I64"
|> flatten
and now i'm trying to move to parsing arrays. but i can't understand what this error message needs me to do
── TYPE MISMATCH ────────────────────────────── examples/parser/parse-json.roc ─
Something is off with the body of the arrayP definition:
82│ arrayP : Parser RawStr Value
83│ arrayP =
84│> string "[]"
85│> |> map (\_ -> Ok (Array emptyArray))
This Parser.Core.map call produces:
Parser RawStr [Ok [Array (List Value)]b]c ?
But the type annotation on arrayP says it should be:
Parser RawStr [Array (List a), Boolean Bool, Null, Number I64,
String Str] as a ?
────────────────────────────────────────────────────────────────────────────────
here is the relevant code:
Value : [
Null,
Boolean Bool,
Number I64,
String Str,
Array (List Value),
]
#...
emptyArray: (List Value)
emptyArray = []
arrayP : Parser RawStr Value
arrayP =
# dummy impl just trying to get typing right
string "[]"
|> map (\_ -> Ok (Array emptyArray))
am i perhaps missing a type specifier somewhere?
Richard Feldman (Oct 07 2022 at 03:46):it looks like there's an Ok somewhere that shouldn't be there
Richard Feldman (Oct 07 2022 at 03:46):so maybe |> map (\_ -> Ok (Array emptyArray)) shouldn't have the Ok there?
Richard Feldman (Oct 07 2022 at 03:47):I think that because I'm seeing in the error message that it produces Parser RawStr [Ok [Array ... but expects Parser RawStr [Array ... (without the Ok)
Richard Feldman (Oct 07 2022 at 03:48):I'm guessing this is because map already takes care of unwrapping and re-wrapping the Result for you maybe?
Travis (Oct 07 2022 at 03:54):hmm yeah i think you're right. I've been using Ok elsewhere and copied it.
but now when i remove the Ok i get a segfault. here is what triggered it:
arrayP : Parser RawStr Value
arrayP =
string "[]"
|> map (\_ -> Array emptyArray)
does roc check give you any errors?
Travis (Oct 07 2022 at 03:56):nope.
$ roc check examples/parser/parse-json.roc
0 errors and 0 warnings found in 17 ms.
$ roc version
roc nightly pre-release, built from commit 2b91154 on Sa 01 Okt 2022 09:15:07 UTC
here is all the code if you need a repro: https://gist.github.com/travisstaloch/7add1274a085f1e6cba59ca93d991d9c
Richard Feldman (Oct 07 2022 at 04:05):gotcha - is there anyplace this code is pushed, so we can try to reproduce?
Richard Feldman (Oct 07 2022 at 04:05):perfect!
Travis (Oct 07 2022 at 04:06):that segfaults. the offending line is line 90
Richard Feldman (Oct 07 2022 at 04:06):does roc build succeed?
Travis (Oct 07 2022 at 04:06):yes it does
Richard Feldman (Oct 07 2022 at 04:07):ok, created an issue to track this: https://github.com/roc-lang/roc/issues/4233
Richard Feldman (Oct 07 2022 at 04:07):thanks for the concise reproductoin!
Richard Feldman (Oct 07 2022 at 04:07):*reproduction
Travis (Oct 07 2022 at 04:07):interesting. i thought it was a compiler segfault. no its a runtime fault.
Travis (Oct 07 2022 at 04:08):no problem. thanks for the help
Travis (Oct 07 2022 at 04:12):not sure if this is relevant, but does roc perhaps require a specific clang version? i've symlinked clang -> system installed clang-14 on debian
Richard Feldman (Oct 07 2022 at 04:19):oh yes - we need 13
Richard Feldman (Oct 07 2022 at 04:20):upgrading to 14 will be a significant project unfortunately :big_smile:
Richard Feldman (Oct 07 2022 at 04:21):although - do other examples work for you?
Richard Feldman (Oct 07 2022 at 04:22):that's the type of thing where I'd expect either everything works or nothing works; I wouldn't expect it to cause project-specific segfaults
Travis (Oct 07 2022 at 04:27):yeah i everything else has worked so far.
i'll check back later on. gotta run. thanks! o/
Travis (Oct 07 2022 at 05:23):can anyone else perhaps try running this file to see if they get a segfault too? i just want to make sure its not a local setup problem like using clang-14.
Travis (Oct 07 2022 at 05:23):https://gist.github.com/travisstaloch/7add1274a085f1e6cba59ca93d991d9c
Travis (Oct 07 2022 at 05:23):there are instructions at the top for running it
jan kili (Oct 07 2022 at 05:25):[jan@framey parser]$ roc run parse-json.roc
roc: /lib64/libtinfo.so.6: no version information available (required by roc)
🔨 Rebuilding platform...
── UNRECOGNIZED NAME ────────────────────────────────────────── parse-json.roc ─
The Parser.Str module does not expose anything by the name strToRaw.
────────────────────────────────────────────────────────────────────────────────
1 error and 13 warnings found in 744 ms.
Running program anyway…
────────────────────────────────────────────────────────────────────────────────
Segmentation fault (core dumped)
[jan@framey parser]$
ah thanks. i just noticed that. forgot i edited a file. updated now.
Travis (Oct 07 2022 at 05:33):here's a one liner if you like.
$ cd /tmp && git clone git@github.com:roc-lang/roc.git && cd roc && wget https://gist.github.com/travisstaloch/7add1274a085f1e6cba59ca93d991d9c/raw/a1fcbbd67e4fadc1d938303d192b022cfdd0da64/parse-json.roc -O examples/parser/parse-json.roc && roc examples/parser/parse-json.roc
i'm expecting a segfault
Travis (Oct 07 2022 at 05:34):here's what i see
$ roc examples/parser/parse-json.roc
🔨 Rebuilding platform...
Segmentation fault
[jan@framey parser]$ cd /tmp && git clone git@github.com:roc-lang/roc.git && cd roc && wget https://gist.github.com/travisstaloch/7add1274a085f1e6cba59ca93d991d9c/raw/a1fcbbd67e4fadc1d938303d192b022cfdd0da64/parse-json.roc -O examples/parser/parse-json.roc && roc examples/parser/parse-json.roc
Cloning into 'roc'...
remote: Enumerating objects: 165051, done.
remote: Counting objects: 100% (1076/1076), done.
remote: Compressing objects: 100% (494/494), done.
remote: Total 165051 (delta 617), reused 1002 (delta 573), pack-reused 163975
Receiving objects: 100% (165051/165051), 64.41 MiB | 6.44 MiB/s, done.
Resolving deltas: 100% (111974/111974), done.
--2022-10-06 23:34:13-- https://gist.github.com/travisstaloch/7add1274a085f1e6cba59ca93d991d9c/raw/a1fcbbd67e4fadc1d938303d192b022cfdd0da64/parse-json.roc
Resolving gist.github.com (gist.github.com)... 140.82.112.3
Connecting to gist.github.com (gist.github.com)|140.82.112.3|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://gist.githubusercontent.com/travisstaloch/7add1274a085f1e6cba59ca93d991d9c/raw/a1fcbbd67e4fadc1d938303d192b022cfdd0da64/parse-json.roc [following]
--2022-10-06 23:34:14-- https://gist.githubusercontent.com/travisstaloch/7add1274a085f1e6cba59ca93d991d9c/raw/a1fcbbd67e4fadc1d938303d192b022cfdd0da64/parse-json.roc
Resolving gist.githubusercontent.com (gist.githubusercontent.com)... 185.199.110.133, 185.199.109.133, 185.199.108.133, ...
Connecting to gist.githubusercontent.com (gist.githubusercontent.com)|185.199.110.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3208 (3.1K) [text/plain]
Saving to: ‘examples/parser/parse-json.roc’
examples/parser/parse-j 100%[============================>] 3.13K --.-KB/s in 0s
2022-10-06 23:34:14 (23.7 MB/s) - ‘examples/parser/parse-json.roc’ saved [3208/3208]
roc: /lib64/libtinfo.so.6: no version information available (required by roc)
── UNUSED IMPORT ────────────────────────────── examples/parser/parse-json.roc ─
Parser.Core.sepBy1 is not used in this module.
9│ parsePartial, ignore, between, many, flatten, oneOrMore, maybe, sepBy1, oneOrMore, alt },
^^^^^^
Since Parser.Core.sepBy1 isn't used, you don't need to import it.
── UNUSED IMPORT ────────────────────────────── examples/parser/parse-json.roc ─
Parser.Core.alt is not used in this module.
9│ parsePartial, ignore, between, many, flatten, oneOrMore, maybe, sepBy1, oneOrMore, alt },
^^^
Since Parser.Core.alt isn't used, you don't need to import it.
── UNUSED IMPORT ────────────────────────────── examples/parser/parse-json.roc ─
Parser.Str.parseStr is not used in this module.
11│ codeunitSatisfies, strFromRaw, digits, digit, parseRawStr, parseStr }
^^^^^^^^
Since Parser.Str.parseStr isn't used, you don't need to import it.
── UNUSED IMPORT ────────────────────────────── examples/parser/parse-json.roc ─
Parser.Str.parseRawStr is not used in this module.
11│ codeunitSatisfies, strFromRaw, digits, digit, parseRawStr, parseStr }
^^^^^^^^^^^
Since Parser.Str.parseRawStr isn't used, you don't need to import it.
── UNUSED IMPORT ────────────────────────────── examples/parser/parse-json.roc ─
Parser.Core.parse is not used in this module.
8│ Parser.Core.{ Parser, parse, map, apply, const, buildPrimitiveParser, Result, oneOf,
^^^^^
Since Parser.Core.parse isn't used, you don't need to import it.
── UNUSED IMPORT ────────────────────────────── examples/parser/parse-json.roc ─
Parser.Str.stringRaw is not used in this module.
10│ Parser.Str.{ string, stringRaw, RawStr, parseStrPartial, anyString, codeunit,
^^^^^^^^^
Since Parser.Str.stringRaw isn't used, you don't need to import it.
── UNUSED IMPORT ────────────────────────────── examples/parser/parse-json.roc ─
Parser.Str.digits is not used in this module.
11│ codeunitSatisfies, strFromRaw, digits, digit, parseRawStr, parseStr }
^^^^^^
Since Parser.Str.digits isn't used, you don't need to import it.
── UNUSED IMPORT ────────────────────────────── examples/parser/parse-json.roc ─
Parser.Core.ignore is not used in this module.
9│ parsePartial, ignore, between, many, flatten, oneOrMore, maybe, sepBy1, oneOrMore, alt },
^^^^^^
Since Parser.Core.ignore isn't used, you don't need to import it.
── UNUSED IMPORT ────────────────────────────── examples/parser/parse-json.roc ─
Parser.Str.anyString is not used in this module.
10│ Parser.Str.{ string, stringRaw, RawStr, parseStrPartial, anyString, codeunit,
^^^^^^^^^
Since Parser.Str.anyString isn't used, you don't need to import it.
── UNUSED IMPORT ────────────────────────────── examples/parser/parse-json.roc ─
Parser.Core.parsePartial is not used in this module.
9│ parsePartial, ignore, between, many, flatten, oneOrMore, maybe, sepBy1, oneOrMore, alt },
^^^^^^^^^^^^
Since Parser.Core.parsePartial isn't used, you don't need to import
it.
── UNUSED IMPORT ────────────────────────────── examples/parser/parse-json.roc ─
Parser.Core.buildPrimitiveParser is not used in this module.
8│ Parser.Core.{ Parser, parse, map, apply, const, buildPrimitiveParser, Result, oneOf,
^^^^^^^^^^^^^^^^^^^^
Since Parser.Core.buildPrimitiveParser isn't used, you don't need to
import it.
── UNUSED IMPORT ────────────────────────────── examples/parser/parse-json.roc ─
Parser.Str.digit is not used in this module.
11│ codeunitSatisfies, strFromRaw, digits, digit, parseRawStr, parseStr }
^^^^^
Since Parser.Str.digit isn't used, you don't need to import it.
── UNUSED IMPORT ────────────────────────────── examples/parser/parse-json.roc ─
Parser.Core.oneOrMore is not used in this module.
9│ parsePartial, ignore, between, many, flatten, oneOrMore, maybe, sepBy1, oneOrMore, alt },
^^^^^^^^^
Since Parser.Core.oneOrMore isn't used, you don't need to import it.
────────────────────────────────────────────────────────────────────────────────
0 errors and 13 warnings found in 50 ms.
You can run the program anyway with roc run examples/parser/parse-json.roc
[jan@framey roc]$
oh boy. i don't see those ununsed imports locally. could you try roc run ...?
Travis (Oct 07 2022 at 05:37):$ roc run examples/parser/parse-json.roc
Travis (Oct 07 2022 at 05:37):i'm just curious to see if the segfault is reproducible
jan kili (Oct 07 2022 at 05:39):Same output with the segfault after:
Segmentation fault (core dumped)
ok thanks. :+1: good to know its not just on my system.
jan kili (Oct 07 2022 at 05:39)::)
Travis (Oct 07 2022 at 05:41):i'll clean up those unused imports. wonder why i'm not seeing them :thinking:
Travis (Oct 07 2022 at 06:19):i wonder if this segfault is related to the recursive type?
Value : [
# ...
Array (List Value),
]
encountered a different compiler error while attempting float parsing. created issue https://github.com/roc-lang/roc/issues/4236
Richard Feldman (Oct 07 2022 at 09:21):what OS are you running on? (I'm guessing Linux?)
Travis (Oct 07 2022 at 09:54):yes this is a debian based distro
Brendan Hansknecht (Oct 07 2022 at 13:39):Yeah, that issue is known. It is an edge case in the surgical linker that is hit by some builtins. Mostly the float to/from string ones (but i am sure there are others). To be unblocked, user --linker=legacy for now
Travis (Oct 07 2022 at 14:13):yep can confirm i'm able to successfully run the reproduction using the --linker=legacy flag.
Travis (Oct 07 2022 at 14:31):now i'm seeing a different build error. goes away if i remove intP from this line.
valueP : Parser RawStr Value
valueP = oneOf [nullP, trueP, falseP, strP, intP, floatP]
intP and floatP are very similar
rawIntP : Parser RawStr RawStr
rawIntP =
const (\mminus -> \digit0 -> \digits -> List.join [mminus, [digit0], digits])
|> apply maybeMinusP
|> apply oneToNineP
|> apply (many zeroToNineP)
intP : Parser RawStr Value
intP =
rawIntP
|> map \rawStr ->
str = strFromRaw rawStr
when Str.toI64 str is
Ok num -> Ok (Number num)
Err _ -> Err "`\(str)` is not a valid I64"
|> flatten
rawFloatP : Parser RawStr RawStr
rawFloatP =
const (
\mminus -> \dig0 -> \digs ->
\ddot -> \digsRhs ->
List.join [
mminus, [dig0], digs,
ddot, digsRhs
]
)
|> apply maybeMinusP
|> apply oneToNineP
|> apply (many zeroToNineP)
|> apply maybeDotP
|> apply (many zeroToNineP)
floatP : Parser RawStr Value
floatP =
rawFloatP
|> map \rawStr ->
str = strFromRaw rawStr
when Str.toF64 str is
Ok num -> Ok (Float num)
Err _ -> Err "`\(str)` is not a valid F64"
|> flatten
here's the error trace:
stack backtrace:
0: rust_begin_unwind
at /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/std/src/panicking.rs:584:5
1: core::panicking::panic_fmt
at /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/core/src/panicking.rs:143:14
2: roc_mono::borrow::ParamMap::get_param_offset
3: roc_mono::borrow::BorrowInfState::collect_expr
4: roc_mono::borrow::BorrowInfState::collect_stmt
5: roc_mono::borrow::BorrowInfState::collect_stmt
6: roc_mono::borrow::BorrowInfState::collect_stmt
7: roc_mono::borrow::infer_borrow
8: roc_mono::ir::Proc::insert_refcount_operations
9: roc_load_internal::file::update
10: roc_load_internal::file::state_thread_step
11: roc_load_internal::file::load_multi_threaded
12: roc_load_internal::file::load
13: roc_load::load_and_monomorphize
14: roc_cli::build::build_file
15: roc_cli::build
16: roc::main
maybe i'll try do de-dupe some of this and see what happens.
Brendan Hansknecht (Oct 07 2022 at 14:35):Does roc check give a better error at all?
Travis (Oct 07 2022 at 14:37):roc check shows 0 errors 0 warnings
Travis (Oct 07 2022 at 14:39):same error when
Travis (Oct 07 2022 at 15:19):ended up working around the above issue by moving intP and floatP from the top-level oneOf to an alt parser:
numberP : Parser RawStr Value
numberP = alt floatP intP
$ roc examples/parser/parse-json.roc --linker=legacy
🔨 Rebuilding platform...
null
true
false
"null"
123
-123
-1.23
good news here. somehow the errors related to arrays i was getting earlier (https://roc.zulipchat.com/#narrow/stream/231634-beginners/topic/json.20parser/near/302777078) seem to have ceased. i did a bunch of refactoring and i think roc is telling me i'm on the right track :smiley:
Travis (Oct 07 2022 at 19:45):so now i'm able to use my recursive array type and i think i need to use examples/parsing/Parser/Core.roc#lazy
Travis (Oct 07 2022 at 19:47):does anyone have advice on how i can use it? i've searched and found some strange syntax i don't understand like this: lazy (\{} -> fn)
Travis (Oct 07 2022 at 19:48):here is the error i'm running into
── CIRCULAR DEFINITION ──────────────────────── examples/parser/parse-json.roc ─
The arrayP definition is causing a very tricky infinite loop:
163│ arrayP =
^^^^^^
The arrayP value depends on itself through the following chain of
definitions:
┌─────┐
│ arrayP
│ ↓
│ valueP
└─────┘
────────────────────────────────────────────────────────────────────────────────
here is my attempt to use lazy which yields the error above
arrayP =
elements = sepBy (lazy (\{} -> valueP)) (codeunit commaC)
# ...
valueP = oneOf [nullP, trueP, falseP, strlitP, numberP, lazy (\{} -> arrayP)]
You're using it correctly except you only need it once, not twice.
Brian Carroll (Oct 07 2022 at 20:46):Try getting rid of one of the lazy calls
Brian Carroll (Oct 07 2022 at 20:46):{} is the empty record. The equivalent in C or JavaScript would be calling a function with no arguments.
Brian Carroll (Oct 07 2022 at 20:48):The point of using lazy is to break the recursion cycle so it doesn't blow up. But you only need to break the cycle in one place.
Travis (Oct 07 2022 at 21:02):Thanks. That syntax does seem strange. Having someone explain its meaning helps a lot.
To your suggestion, thats what I thought at first and tried using lazy once at each location and got the same error before trying both locations as shown above.
Travis (Oct 07 2022 at 21:06):So perhaps this means its something else? Here is the full arrayP - after moving the lazy call to valueP below - incase it's relevant
arrayP =
elements = sepBy valueP (codeunit commaC)
const (\_ -> \_ -> \vals -> \_ -> \_ -> vals)
|> apply (codeunit lbraceC)
|> apply wsP
|> apply elements
|> apply wsP
|> apply (codeunit rbraceC)
|> map (\vals -> Array vals)
otherwise it seems like i'm either using lazy incorrectly or missing some detail of how its used.
Travis (Oct 07 2022 at 21:13):maybe it doesn't work or works differently than expected? i couldn't find any uses of it in the roc repo.
Travis (Oct 07 2022 at 21:15):here are two of the functions being used from examples/parser/Parser/Core.roc
# Internal utility function. Not exposed to users, since usage is discouraged!
#
# Runs `firstParser` and (only) if it succeeds,
# runs the function `buildNextParser` on its result value.
# This function returns a new parser, which is finally run.
#
# `andThen` is usually more flexible than necessary, and less efficient
# than using `const` with `map` and/or `apply`.
# Consider using those functions first.
andThen : Parser input a, (a -> Parser input b) -> Parser input b
andThen = \firstParser, buildNextParser ->
fun = \input ->
{ val: firstVal, input: rest } <- Result.try (parsePartial firstParser input)
nextParser = buildNextParser firstVal
parsePartial nextParser rest
buildPrimitiveParser fun
## Runs a parser lazily
##
## This is (only) useful when dealing with a recursive structure.
## For instance, consider a type `Comment : { message: String, responses: List Comment }`.
## Without `lazy`, you would ask the compiler to build an infinitely deep parser.
## (Resulting in a compiler error.)
##
lazy : ({} -> Parser input a) -> Parser input a
lazy = \thunk ->
const {}
|> andThen thunk
could you add type annotations to arrayP and valueP? What are their types exactly?
Ayaz Hafiz (Oct 07 2022 at 21:43):It's possible this is a type checker bug, I made it do some trickery so that if it sees an infinitely-sized type, you get that error. But it's possible it's overly conservative
Ayaz Hafiz (Oct 07 2022 at 21:44):(As an aside - we recently landed some changes so that where you have codeunit commaC, you should be able to just use codeunit ',' now)
Travis (Oct 07 2022 at 21:51):oh cool. glad to hear char literals are happening. i'll have to get a new compiler soon.
Travis (Oct 07 2022 at 21:52):here they are with type annotations:
arrayP : Parser RawStr Value
arrayP =
elements = sepBy valueP (codeunit commaC)
const (\_ -> \_ -> \vals -> \_ -> \_ -> vals)
|> apply (codeunit lbraceC)
|> apply wsP
|> apply elements
|> apply wsP
|> apply (codeunit rbraceC)
|> map (\vals -> Array vals)
valueP : Parser RawStr Value
valueP = oneOf [nullP, trueP, falseP, strlitP, numberP, lazy (\{} -> arrayP)]
i can post the whole file somewhere if you want too
Ayaz Hafiz (Oct 07 2022 at 21:57):Hm. Yeah this might be a compiler bug, unless I'm misunderstanding the program. Could you please file a bug report with a link to the whole file? If possible, we'd appreciate minimizing the file to a minimal reproduction of the error. I can take a look at this early next week.
Travis (Oct 07 2022 at 21:58):yes certainly will do. i'll work on whittling down to a smaller repro a bit later. will post back here then.
Travis (Oct 08 2022 at 07:30):@Ayaz Hafiz here is the issue. https://github.com/roc-lang/roc/issues/4246 let me know if the repro looks ok.
Ayaz Hafiz (Oct 08 2022 at 14:50):Thanks!
Ayaz Hafiz (Oct 08 2022 at 14:51):In that repro, though, the error makes sense - because you can't build a without having to inline it an infinite number of times
Ayaz Hafiz (Oct 08 2022 at 14:51):which makes me think, maybe the way to solve your problem is like this?
Ayaz Hafiz (Oct 08 2022 at 14:52):arrayP : Parser RawStr Value
arrayP =
lazy (\{} ->
elements = sepBy valueP (codeunit commaC)
const (\_ -> \_ -> \vals -> \_ -> \_ -> vals)
|> apply (codeunit lbraceC)
|> apply wsP
|> apply elements
|> apply wsP
|> apply (codeunit rbraceC)
|> map (\vals -> Array vals)
)
valueP : Parser RawStr Value
valueP = oneOf [nullP, trueP, falseP, strlitP, numberP, arrayP]
(sorry for the terrible formatting, can't tab-indent in Zulip
Ayaz Hafiz (Oct 08 2022 at 14:53):Or, conversely,
arrayP : Parser RawStr Value
arrayP =
elements = sepBy valueP (codeunit commaC)
const (\_ -> \_ -> \vals -> \_ -> \_ -> vals)
|> apply (codeunit lbraceC)
|> apply wsP
|> apply elements
|> apply wsP
|> apply (codeunit rbraceC)
|> map (\vals -> Array vals)
valueP : Parser RawStr Value
valueP = lazy (\{} -> oneOf [nullP, trueP, falseP, strlitP, numberP, arrayP])
So that the whole parser is forced to be lazy, rather a subpart
Ayaz Hafiz (Oct 08 2022 at 14:59):oh nevermind
Ayaz Hafiz (Oct 08 2022 at 14:59):The parser is a function so it's fine
Ayaz Hafiz (Oct 08 2022 at 15:04):okay, it's a compiler bug. I'll make a PR
Ayaz Hafiz (Oct 08 2022 at 15:11):https://github.com/roc-lang/roc/pull/4257 should fix it
Travis (Oct 08 2022 at 23:05):Wow very cool. Thanks for the quick response. Hope this gets merged so i can try it out.
Travis (Oct 08 2022 at 23:06):i don't have a proper dev setup yet and relying on download assets. no rust / cargo install here.
Richard Feldman (Oct 08 2022 at 23:21):just merged it!
Richard Feldman (Oct 08 2022 at 23:21):so it'll be included in tonight's nightly release
Travis (Oct 09 2022 at 16:09):looks like the new character literals are U32s not U8s. is this correct?
Travis (Oct 09 2022 at 16:10):i tried yesterday's nightly and seeing this new error
── TYPE MISMATCH ────────────────────────────── examples/parser/Parser/Str.roc ─
This 2nd argument to map has an unexpected type:
181│> |> List.map \digitNum ->
182│> digitNum
183│> |> codeunit
184│> |> map (\_ -> digitNum)
The argument is an anonymous function of type:
U8 -> Parser RawStr U8 ?
But map needs its 2nd argument to be:
Int Unsigned32 -> Parser RawStr U8 ?
────────────────────────────────────────────────────────────────────────────────
oops nevermind. the above uses Oct 7 release, not Oct 8
Travis (Oct 09 2022 at 16:17):and the error is gone in Oct 8 release
Travis (Oct 09 2022 at 16:19):character literals working now :tada: :rock_on:
Travis (Oct 10 2022 at 18:11):seems that the 2022-10-10-07ceabc build has problems. it no longer catches the cyclic dependency when i don't do the lazy calls.
Travis (Oct 10 2022 at 18:12):and it crashes trying to build if i do the lazy calls
$ roc build examples/parser/parse-json.roc
thread '<unknown>' has overflowed its stack
fatal runtime error: stack overflow
Aborted
I think your implementation was right all along because parsers are functions, so you may not strictly need lazy in your use case
Ayaz Hafiz (Oct 10 2022 at 18:12):well.. that's no good
Ayaz Hafiz (Oct 10 2022 at 18:14):does it build if you do it without lazy
Travis (Oct 10 2022 at 18:14):yep. it also crashes with the same message on build without the lazy calls
Ayaz Hafiz (Oct 10 2022 at 18:17):Do you have lldb or gdb on your machine?
Travis (Oct 10 2022 at 18:18):just gdb atm. shall i try for a backtrace? last time i tried it was garbled.
Ayaz Hafiz (Oct 10 2022 at 18:18):Yes please
Ayaz Hafiz (Oct 10 2022 at 18:18):this is an error in the compiler so it should be better I think
Travis (Oct 10 2022 at 18:19):yes its better now. here is the first page
$ gdb -args roc build examples/parser/parse-json.roc
GNU gdb (Debian 10.1-1.7) 10.1.90.20210103-git
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from roc...
(gdb) run
Starting program: /home/travis/.local/bin/roc build examples/parser/parse-json.roc
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff75ff700 (LWP 163044)]
[New Thread 0x7fffeffff700 (LWP 163045)]
[New Thread 0x7ffff6dfe700 (LWP 163046)]
[New Thread 0x7ffff65fd700 (LWP 163047)]
[New Thread 0x7ffff5dfc700 (LWP 163048)]
[New Thread 0x7ffff55fb700 (LWP 163049)]
[New Thread 0x7ffff4dfa700 (LWP 163050)]
[New Thread 0x7fffef7fe700 (LWP 163051)]
[New Thread 0x7fffeeffd700 (LWP 163052)]
[New Thread 0x7fffee7fc700 (LWP 163053)]
[New Thread 0x7fffedffb700 (LWP 163054)]
Thread 5 "roc" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff65fd700 (LWP 163047)]
0x00005555562eee34 in roc_mono::layout::layout_from_flat_type ()
(gdb) bt
#0 0x00005555562eee34 in roc_mono::layout::layout_from_flat_type ()
#1 0x00005555562ed57f in roc_mono::layout::Layout::new_help ()
#2 0x00005555562edeca in roc_mono::layout::Layout::from_var ()
#3 0x00005555562edeca in roc_mono::layout::Layout::from_var ()
#4 0x000055555630381f in roc_mono::layout::list_layout_from_elem ()
#5 0x00005555562f3ea4 in roc_mono::layout::layout_from_flat_type ()
#6 0x00005555562ed57f in roc_mono::layout::Layout::new_help ()
#7 0x00005555562edeca in roc_mono::layout::Layout::from_var ()
#8 0x00005555562f2e41 in roc_mono::layout::layout_from_flat_type ()
#9 0x00005555562ed57f in roc_mono::layout::Layout::new_help ()
#10 0x00005555562edeca in roc_mono::layout::Layout::from_var ()
#11 0x00005555562edeca in roc_mono::layout::Layout::from_var ()
#12 0x000055555630381f in roc_mono::layout::list_layout_from_elem ()
#13 0x00005555562f3ea4 in roc_mono::layout::layout_from_flat_type ()
#14 0x00005555562ed57f in roc_mono::layout::Layout::new_help ()
#15 0x00005555562edeca in roc_mono::layout::Layout::from_var ()
#16 0x00005555562f2e41 in roc_mono::layout::layout_from_flat_type ()
#17 0x00005555562ed57f in roc_mono::layout::Layout::new_help ()
#18 0x00005555562edeca in roc_mono::layout::Layout::from_var ()
#19 0x00005555562edeca in roc_mono::layout::Layout::from_var ()
#20 0x000055555630381f in roc_mono::layout::list_layout_from_elem ()
#21 0x00005555562f3ea4 in roc_mono::layout::layout_from_flat_type ()
#22 0x00005555562ed57f in roc_mono::layout::Layout::new_help ()
#23 0x00005555562edeca in roc_mono::layout::Layout::from_var ()
#24 0x00005555562f2e41 in roc_mono::layout::layout_from_flat_type ()
#25 0x00005555562ed57f in roc_mono::layout::Layout::new_help ()
#26 0x00005555562edeca in roc_mono::layout::Layout::from_var ()
#27 0x00005555562edeca in roc_mono::layout::Layout::from_var ()
#28 0x000055555630381f in roc_mono::layout::list_layout_from_elem ()
#29 0x00005555562f3ea4 in roc_mono::layout::layout_from_flat_type ()
#30 0x00005555562ed57f in roc_mono::layout::Layout::new_help ()
#31 0x00005555562edeca in roc_mono::layout::Layout::from_var ()
looks like similar repeated many times
Ayaz Hafiz (Oct 10 2022 at 18:21):yeah we're very familiar with this one :sweat_smile: happens a lot these days
Ayaz Hafiz (Oct 10 2022 at 18:22):and roc check passes?
Travis (Oct 10 2022 at 18:23):yeah roc check passed on both (lazy and no lazy). there is one warning but thats just unused warning
Ayaz Hafiz (Oct 10 2022 at 18:25):I'm really sorry you're blocked again right after the other fix :disappointed:
Ayaz Hafiz (Oct 10 2022 at 18:25):Would you mind trying to whittle down a smaller reproduction and filing a bug report again?
Ayaz Hafiz (Oct 10 2022 at 18:26):In the meantime, you could try removing all type annotations from your program - that might actually fix the compiler bug
Travis (Oct 10 2022 at 18:27):sure. no problem. understood that lots of development going on, young lang etc.
Travis (Oct 10 2022 at 18:28):sure i'll reopen the last issue. should work on repro pretty soon
Travis (Oct 10 2022 at 18:29):will let you know if removing type annotations works
Ayaz Hafiz (Oct 10 2022 at 18:29):thanks!
Travis (Oct 10 2022 at 18:29):looks like the end of that trace is different if you want to see it. i forgot to let it run out before.
Ayaz Hafiz (Oct 10 2022 at 18:33):It's okay, the trace you showed tells us enough about what the problem is. I'll dig deeper into it with a debug build of the compiler which provides more stack trace info
Travis (Oct 10 2022 at 18:37):ok. i'm seeing a different error after removing the annotations
# with lazy call
$ roc build examples/parser/parse-json.roc
thread '<unnamed>' panicked at 'not yet implemented: recursive closures', crates/compiler/mono/src/layout.rs:1524:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
okay, that is a lot more promising
Travis (Oct 10 2022 at 18:38):without lazy calls i see the same error message
Travis (Oct 10 2022 at 18:38):*as before
Travis (Oct 10 2022 at 18:39):this one
# no lazy calls and no annotations
$ roc build examples/parser/parse-json.roc
thread '<unknown>' has overflowed its stack
fatal runtime error: stack overflow
Aborted
sorry that sentence was ambiguous
Ayaz Hafiz (Oct 10 2022 at 18:41):all good
Travis (Oct 10 2022 at 19:59):shoot i can't reopen it. will make a new one w/ reference
Travis (Oct 10 2022 at 20:15):https://github.com/roc-lang/roc/issues/4288
Travis (Oct 10 2022 at 20:16):let me know if the repro needs any changes
Travis (Oct 10 2022 at 20:18):wasn't sure wether to include the last two which have annotations
Travis (Oct 10 2022 at 20:32):is it known or expected that '\b' and '\f' aren't legal char literals?
Richard Feldman (Oct 10 2022 at 21:11):I forget - what do those do?
Travis (Oct 10 2022 at 21:17):backspace and form feed
Travis (Oct 10 2022 at 21:17):i forget if they're legal in json but happened to have them being escaped for some reason
Richard Feldman (Oct 10 2022 at 21:24):oh yeah if I remember right, I think I evaluated those and concluded they probably weren't a good idea to have first class support for
Travis (Oct 10 2022 at 21:26):sounds good :+1: . just checking to make sure it was intended.
Travis (Oct 11 2022 at 03:06):Travis said:
@Ayaz Hafiz did you see this?
Ayaz Hafiz (Oct 11 2022 at 03:17):Yes - thanks for the report. I don't know when I'll get to it but it's definitely on my radar!
Richard Feldman (Oct 13 2022 at 15:38):@Travis do you have the current code for your json parser posted anywhere?
Travis (Oct 13 2022 at 15:49):not yet. i'll create a new gist incase the old one is of use for #4233.
Richard Feldman (Oct 13 2022 at 15:49):thanks! :smiley:
Travis (Oct 13 2022 at 15:49):here you go. https://gist.github.com/travisstaloch/acae64ec4d8346597f1aaecfe8c401e4
Travis (Oct 13 2022 at 15:50):no problem. as is, its using lazy in one place, the main valueP parser
Travis (Oct 13 2022 at 16:45):oh yeah i forgot i also removed all the type annotations trying to repro. i think there is a different error if they are added back.
Last updated: Jul 05 2025 at 12:14 UTC