If there's a weird way for features to interact, fuzzing seems to reliably find it. So, brain teaser:
When parsed and formatted, dbg dbg g g
yields: dbg (dbg g g)
Is that the same thing? Why or why not?
Why does it add the parens?
Wouldn't that parse as a function call with three arguments?
I would expect that to fail typechecking or something later, but for the sake of parsing/formatting it should be no different to say foo 1 2 3
right?
It _is_ different, because there are two different ways dbg
is parsed :/
The same thing doesn't happen with foo foo g g
* * * AST before formatting:
Expr(
SpaceAfter(
Apply(
@0-11 Dbg,
[
@4-11 Apply(
@4-7 Dbg,
[
@8-9 Var {
module_name: "",
ident: "g",
},
@10-11 Var {
module_name: "",
ident: "g",
},
],
Space,
),
],
Space,
),
[
Newline,
],
),
)
* * * AST after formatting:
Expr(
Apply(
Dbg,
[
Apply(
Dbg,
[
Apply(
Var {
module_name: "",
ident: "g",
},
[
Var {
module_name: "",
ident: "g",
},
],
Space,
),
],
Space,
),
],
Space,
),
)
The key is that dbg
in the outer context is parsing as a dbg statement, so it gobbles up the rest of the string - and then prior to formatting (haven't quite identified where...) this gets converted to a Dbg expression, which of course must follow the usual rules for how calls work - and so we introduce parens in the formatter to (try to) preserve that nesting order.
... but unbeknownst to the formatter, adding the parens triggers that inner thing to be a new "defs" context, which allows the inner dbg to parse as a dbg statement instead of a dbg expression.
I guess it needs to be parsed as a special keyword because we don't have statements in roc, like user space function call on a line by itself.
I'm looking at
fn dbg_kw<'a>() -> impl Parser<'a, Expr<'a>, EExpect<'a>> {
(move |arena: &'a Bump, state: State<'a>, min_indent: u32| {
let (_, _, next_state) =
parser::keyword(keyword::DBG, EExpect::Dbg).parse(arena, state, min_indent)?;
Ok((MadeProgress, Expr::Dbg, next_state))
})
.trace("dbg_kw")
}
In the parser::keyword
should we prevent two keywords in sequence, like if if
or dbg dbg
?
Joshua Warner said:
If there's a weird way for features to interact, fuzzing seems to reliably find it. So, brain teaser:
When parsed and formatted,
dbg dbg g g
yields:dbg (dbg g g)
Is that the same thing? Why or why not?
that seems correct, but I don't like that it seems correct :laughing:
in the sense that dbg
doesn't accept multiple arguments, so whatever comes after it must be its own self-contained expression (hence the parens)
but although it's logical, I don't like it :laughing:
The core of the weirdness here is that we have two fundamental the different precedence levels for dbg statements: at the statement level, it has a very low precedence, but if the expression level it has the same precedence as other function applications.
One way to fix that would be to standardize on dbg always having function level precedents - effectively, eliminating the statement level dbg.
On the other hand, we could move even the expression level dbg to have a very low precedence, and operate the same way as the statement level dbg.
Or, if we just want to fix this one particular thing, we can teach the formatter not to introduce parentheses in the case where it would matter and change how this inner statement is parsed. But that feels pretty hacky?
Actually not sure that last option will be very feasible...
personally I'm okay with the status quo
on the grounds that I don't think anyone is likely to ever write it on purpose
also in the parens-and-commas world you'd have to write it like dbg(dbg(g, g))
or dbg(dbg, g, g)
anyway so there wouldn't be any strangeness
I would like to fix this invariant violation somehow
I do eventually want to get "fuzz-clean", which then means we have strong properties that you can trust of the parser and formatter
And after that, move on to fuzzing other areas of the compiler ;)
IMO the status quo is a bit hard to understand because there are two different rules for how you have to parenthesize things in dbg
, depending on where it is
Concretely, dbg f g
is fine at the top level, and operates like dbg (f g)
- but if it's inside an expression somewhere you have to do dbg (f g)
.
This fuzzer-found bug is a very round about way of describing that weirdness
a straightforward fix would be to have dbg
"accept" multiple arguments, like a function call, at the parsing level
and then give a canonicalization error for too many args passed to dbg
Yep, and that's how dbg
works at the "expression" level
yeah so we could do the same at the statement level
Aha yeah; that's what I was thinking as well :)
A slightly further change would be to make us parse dbg
identically regardless of the context
Given the thing we just agreed upon above, the remaining difference is that dbg
at the statement level can currently introduce a block - e.g.:
dbg
x = 1
x
That seems much less useful than it is in the expect
context, so my initial instinct is to go ahead and remove that ability
Note that you can always do:
dbg (
x = 1
x
)
... if you really need to
why couldn't you do it in the expression? :thinking:
(moved discussion to #compiler development > Defs in arbitrary expressions )
Joshua Warner said:
I do eventually want to get "fuzz-clean", which then means we have strong properties that you can trust of the parser and formatter
It would be cool to have a "hey did you run the fuzzer" as a requirement whenever someone updates anything syntax related. I'm guessing at the moment they would immediately run into these kind of issues, and so it wouldn't be as helpful for them?
You could have CI run the fuzzer for 30 seconds or a minute on each pr. Or do it a bit longer each nightly.
Nightly seems best, we don't want to confuse people with failures unrelated to their changes
Do we have a script or single command for the fuzzing?
$ cd crates/compiler/test_syntax/fuzz
$ cargo +nightly fuzz run -j4 fuzz_expr --sanitizer=none -- -dict=dict.txt
It's described in crates/compiler/test_syntax/fuzz/README.md
Thanks :)
Assuming we iron out the current bugs, I think it is much nicer to do a bit per PR. Cause if it caches something it will almost certainly be a real bug from the PR.
Can always bisect instead, but it is more work
Also, if you can, make sure to save the corpus (and probably want to minimize it at the end of runs)
One caveat that I didn't realize immediately with moving _everything_ to use the expr-level dbg
is that things like dbg 1 + 1
will cease to work, and you'll have to write dbg (1 + 1)
.
Thoughts?
seems reasonable, we'll see if people complain about it :big_smile:
I was also surprised when I first used debug that you didn’t have add parens around expressions like that so I think this just decreases the amount of surprise :+1:
Given debug works as an expression, I would expect this to print 1:
x = dbg 1 + 1
This prints 2
x = dbg (1 + 1)
That said, as a statement, I would expect different precedence. Though same precedence is ok. I would expect 2 from this
dbg 1 + 1
Could we make those use a different syntax?
I think parens-and-commas fixes this, right?
Yeah, I think so
in that case I'd say we can go with whatever design most easily fixes the fuzzing case :big_smile:
can always revisit later
Here's the diff to do that, with some refactoring to make the required transformations in can
easier: https://github.com/roc-lang/roc/pull/7239
This relegates Expr::Defs to only be used between desugaring and the main canonicalization phase, and instead uses Expr::Stmts as the representation that comes out of the parser.
Note that the list of stmts already existed as an intermediate representation in the parser - so this isn't introducing a new step, it's just moving it around.
One of the ancillary benefits here is that some things that used to be a parse errors, and thus completely blocked running the code (e.g. missing a final expression in a defs), are now canonicalization errors where they can be non-fatal
Ok, with that out of the way, I was able to fix a bunch of other bugs found in fuzzing here: https://github.com/roc-lang/roc/pull/7267
Not yet at the end of it, but I at least feel like it's still finding interesting things that'd be possible for users to hit, rather than trivial problems.
Was hoping to get fully to "fuzz clean" before submitting the next PR - but these fixes have been piling up for too long now.
...
#1863400: cov: 47740 ft: 42904 corp: 11904 exec/s 1197 oom/timeout/crash: 0/0/0 time: 397s job: 52 dft_time: 0
#1936387: cov: 47842 ft: 42918 corp: 11911 exec/s 1351 oom/timeout/crash: 0/0/0 time: 412s job: 53 dft_time: 0
#1985095: cov: 47873 ft: 42933 corp: 11917 exec/s 885 oom/timeout/crash: 0/0/0 time: 427s job: 54 dft_time: 0
#2068034: cov: 47874 ft: 42945 corp: 11925 exec/s 1481 oom/timeout/crash: 0/0/0 time: 441s job: 55 dft_time: 0
#2105228: cov: 47882 ft: 42948 corp: 11928 exec/s 652 oom/timeout/crash: 0/0/0 time: 455s job: 56 dft_time: 0
#2184903: cov: 47914 ft: 42959 corp: 11937 exec/s 1373 oom/timeout/crash: 0/0/0 time: 471s job: 57 dft_time: 0
#2262528: cov: 47917 ft: 42960 corp: 11938 exec/s 1315 oom/timeout/crash: 0/0/0 time: 486s job: 58 dft_time: 0
#2335873: cov: 47986 ft: 42969 corp: 11944 exec/s 1222 oom/timeout/crash: 0/0/0 time: 501s job: 59 dft_time: 0
#2388073: cov: 48069 ft: 42995 corp: 11954 exec/s 855 oom/timeout/crash: 0/0/0 time: 517s job: 60 dft_time: 0
#2468634: cov: 48092 ft: 43009 corp: 11962 exec/s 1299 oom/timeout/crash: 0/0/0 time: 534s job: 61 dft_time: 0
#2548301: cov: 48139 ft: 43018 corp: 11970 exec/s 1264 oom/timeout/crash: 0/0/0 time: 550s job: 62 dft_time: 0
#2623369: cov: 48172 ft: 43021 corp: 11972 exec/s 1172 oom/timeout/crash: 0/0/0 time: 566s job: 63 dft_time: 0
#2701742: cov: 48202 ft: 43039 corp: 11983 exec/s 1205 oom/timeout/crash: 0/0/0 time: 583s job: 64 dft_time: 0
#2748245: cov: 48243 ft: 43046 corp: 11987 exec/s 704 oom/timeout/crash: 0/0/0 time: 601s job: 65 dft_time: 0
INFO: fuzzed for 601 seconds, wrapping up soon
INFO: exiting: 0 time: 602s
And there is the sweet sweet sound of the fuzzer not having found any problems in 10 minutes of fuzzing.
Yay for arbitrary goals
PR here: https://github.com/roc-lang/roc/pull/7301
Note that one of the changes I'm including here is removing the ability for joining neighboring annotations+bodies if the patterns are not "equivalent", discussed here: https://roc.zulipchat.com/#narrow/channel/395097-compiler-development/topic/Merging.20non-equivalent.20annotations.20.2B.20bodies.3F/near/485732867
That much is probably worth discussing prior to actually merging. I can fairly easily back that commit out and fix that fuzzing issue separately if need be.
Well done! This is great to see. Feels good
Indeed
That was a long time coming
If I could quickly TL;DR the #1 learning here, it's: spaces are "slippery", and if they have any room to slide around, chaos ensues.
For example, and eliding some of the fields of these things, you can have these two trees that are logically equivalent:
SpaceBefore(Apply(Dbg, [...]))
Apply(SpaceBefore(Dbg), [...])
i.e. both represent having placed the spaces at the same place in the file.
Or similarly, where each element in a series of Defs has both "spaces_before" and "spaces_after".
I'm not sure if the parser would ever produce that exact example, but there definitely are a bunch of other possible cases where spaces can "slide" around while still being in logically the same place.
That's what all the expr_lift_spaces
, pattern_lift_spaces
, etc stuff in these recent diffs have been: trying to normalize the placing of those spaces, "lifting" it up to the highest part of the tree it can be.
This "sliding" is particularly prone to happening if the formatter decides to add or remove parentheses
Another footgun in the formatter is that is_multiline()
must correctly predict what .format_with_options(...)
will do, and if we have some subtle logic that sometimes adds a newline to the output in the .format_with_options(...)
impl, it's easy to forget to make .is_multiline()
account for that. The rules to format a string as a block string if it contains a newline or quote char have been particularly annoying here.
The solution to both of those that I'm very slowly working towards is to add an intermediate stage of processing that gives a heavily-normalized version of the expression from which is_multiline
can be calculated trivially and 100% correctly.
Today that's taking the form of expr_lift_spaces
/etc - but there are hints of my eventual plans with ann_lift_to_node
: that's the barest sketch of that normalized form. I added that in this case in order to make sure the decision of whether a type annotation needs parens consistent in a couple key spots.
I put up an initial sketch of doing fuzzing in CI here: https://github.com/roc-lang/roc/pull/7316
Would love some help getting this properly set up since I really don't know what I'm doing, and this is me just flailing around.
In particular, I'm guessing that we'll have problems with making sure the nightly version of cargo is installed. There may also be problems with nix/
Does the fuzzer work inside nix? I'd just add it as another step for one of our existing nix runs
It doesn't need to run on all the different os/archs
Yeah, no need for different architectures.
Like in .github/workflows/nix_linux_x86_64.yml
The only thing I'm not sure about is how to get the nightly compiler working in CI with nix. Fuzzing requires running nightly compiler, and doesn't work on stable.
The rust compiler I mean.
Just copy the commands already there I think
nix develop -c
... maybe we add a bash.sh
script that runs the fuzzer
I'd give this a try
- name: run fuzz tests
run: |
cd crates/compiler/test_syntax/fuzz
nix develop -c cargo +nightly fuzz run -j4 fuzz_expr --sanitizer=none -- -dict=dict.txt -max_total_time=60
@Anton may not have the nightly toolchain on the self-hosted machine.. but that should be an easy fix
One reason we might want this to be part of a separate job is so that we can configure it to be not blocking merging of PRs until we're confident that it's very stable.
I'm just throwing ideas around for how to get it working... I think @Anton will definitely have some things to say on this. But the more we can do to set it up and have something working for him, the easier to get to the desired end state is my thinking.
Here's the current failure mode:
path '/home/small-ci-user/actions-runner/_work/roc/roc/crates/compiler/test_syntax/fuzz' does not contain a 'flake.nix', searching up
error: no such command: `+nightly`
Cargo does not handle `+toolchain` directives.
Did you mean to invoke `cargo` through `rustup` instead?
It appears the cargo
installed there is actual cargo instead of rustup, so it doesn't understand the +nightly
thing.
Can you do rustup +nightly cargo
(that might not be the right command, but something like that)
Cause I know we use rustup and the tool chain file
It's Anton's machine, we should just wait for him to jump online.
It's a self hosted runner
Ok, my most recent attempt seems to be getting closest... except the nightly toolchain isn't installed it looks like
Config:
- name: run fuzz tests
run: |
cd crates/compiler/test_syntax/fuzz
nix develop -c rustup run nightly cargo fuzz run -j4 fuzz_expr --sanitizer=none -- -dict=dict.txt -max_total_time=60
Output:
error: toolchain 'nightly-x86_64-unknown-linux-gnu' is not installed
Our nix rust version comes from here, getting the nightly in there too would require some fiddling. I recommend starting out with a new workflow using the runner runs-on: [self-hosted, i7-6700K]
. I'll install nightly(rust toolchain) nightly-2024-02-03 on there, that one matches our current rust version and that way we don't need to install the latest every day.
It's installed, I expect this will work for your command cargo +nightly-2024-02-03 fuzz ...
I think we'll also need a cargo +nightly-2024-02-03 install cargo-fuzz
command as a preparatory step. Or should I put that in the job?
Yeah, you can put that in the job, it'll basically be zero cost if it is already installed
Ok, seems to be working! I currently have this in ubuntu_x86_64.yml, which may not be the ideal spot.
that file should be fine :+1:
We got our first fuzzer bug in the wild! :bug: :ladybug:
In this CI run https://github.com/roc-lang/roc/actions/runs/12287783837/job/34290444406?pr=7335
INFO: exiting: 77 time: 33s
────────────────────────────────────────────────────────────────────────────────
Failing input:
artifacts/fuzz_expr/crash-7f53e1d94350d5255f7f9bfdbedaff7665f10a0b
Output of `std::fmt::Debug`:
[50, 45, 52, 10, 46, 116, 10, 10, 33, 10, 10, 38, 112, 122, 50, 112, 122, 46, 116, 10, 10, 33, 10, 38, 112, 114, 118, 111, 105, 100, 101, 115, 33, 61, 61, 101, 74]
Reproduce with:
cargo fuzz run --sanitizer=none fuzz_expr artifacts/fuzz_expr/crash-7f53e1d94350d5255f7f9bfdbedaff7665f10a0b
Minimize test case with:
cargo fuzz tmin --sanitizer=none fuzz_expr artifacts/fuzz_expr/crash-7f53e1d94350d5255f7f9bfdbedaff7665f10a0b
────────────────────────────────────────────────────────────────────────────────
Error: Fuzz target exited with exit status: 77
Error: Process completed with exit code 1.
2-4
.t
!
&pz2pz.t
!
&prvoides!==eJ
What's our plan if CI fails on a fuzzer bug... and it passes everything else? Merge the PR and log an issue with a repro?
Nvm, it looks like it cancels the rest of the run immediately
We should move fuzzing to the last step in that job, so we can be sure everything else succeeded
And obviously if this gets too noisy we should remove it or move it to a non-blocking job
Will take a look later tonight
The minimal repro here is:
4
!
&z.t
(found by putting that in a file called todo
and then running cargo run --bin minimize expr todo
)
Fixed here: https://github.com/roc-lang/roc/pull/7340
It'll still cancel all the other jobs early in CI Manager though right if this fails? I had a little investigation how to make it not do that, and I wasn't confident I could do it.
I didn't realize it canceled other jobs
Although, I think this is one of the longer running jobs, so maybe it will work out fine.
here's another! https://github.com/roc-lang/roc/actions/runs/12289072630/job/34293972798?pr=7337#step:8:5297
Ooof
This is a little noisier than I was hoping for :/
I think I'm going to go ahead and disable it for now
That was before merging the other fix into main
could they both be hitting the same failure?
oh, nvm
missed the last commit being merging main
I'll run it for a bit while I'm in a meeting and see if I can find anything
actually my first statement is correct. That failure is before the fix on main. So main may be clean for fuzzing.
I just found a couple more things locally
Definitely not clean
ok
Actually, interesting fuzzing question here
Or perhaps language design question
Here's another
x
!&
I've been running into a bunch of problems with this logic that sees what would usually be an Alias followed by a body and turns that into an annotation.
It's very sensitive to where exactly the whitespace is attached, as well as some details like whether we're adding/removing parens around the pattern in the alias turned annotation.
I'm not sure I've seen this logic kick in on actual example code. Is that needed/valuable?
Found another
1)
:thinking: Maybe my 10 minute no-bugs-found run was lucky. And/or because I'm running it on relatively modest hardware (M1 macbook air)
I can make issues instead if you'd prefer @Joshua Warner
I had to go into the office, for some meetings. I can sit here all afternoon poking at the fuzzer :smiley:
This is a task I can excel at
@Luke Boswell what were the errors those were failing with?
I'll make issues and copy it all in, with the minimisation too
crash-d8c7b906b169f03d93286cac954573468ff5aae9
I lost the history (can't scroll back far enough) for this one.
Joshua Warner said:
I've been running into a bunch of problems with this logic that sees what would usually be an Alias followed by a body and turns that into an annotation.
It's very sensitive to where exactly the whitespace is attached, as well as some details like whether we're adding/removing parens around the pattern in the alias turned annotation.
I'm not sure I've seen this logic kick in on actual example code. Is that needed/valuable?
I don't think this is something people mess up in any significant amount in practice, so I'd say in this situation it sounds reasonable to change the parser to be more resilient to failure - even if that makes the grammar stricter
Ok, I'm going to try and not post ones that look like duplicates. Just noticing some of these might be for the same thing
it sounds reasonable to change the parser to be more resilient to failure
@Richard Feldman What would you say to requiring parens around the pattern in this case?
e.g.
(UserId x) : [ UserId I64 ]
UserId x = UserId 42
... and have the non-parens equivalent parse strictly as an Alias?
that seems fine for now...I don't think I've ever seen anyone write an annotation like that in practice :big_smile:
that is a valid annotation?
Can we just ban the parens there?
Not sure what you mean by ban?
Like, give a parse error?
yeah
Though I guess roc is permissive...so remove with the formatter I guess...
I don't think that actually helps in this situation, but I could be misunderstanding
well there ought to be some way to annotate that if you really want to
I guess, in my mind:
This is normal code to see:
(UserId x) = UserId 42
This is abnormal code to see:
(UserId x) : [ UserId I64 ]
Why do you have to be able to annotate a pattern match?
Right; I think that's what I was getting at earlier...
Like you don't annotate branches of a when ... is
I don't think you should be able to annotate this
hm, that's interesting :thinking:
Or you have to annotate it indirectly:
y : [ UserId I64 ]
y = UserId 42
UserId x = y
I hadn't thought about that perspective, but it would certainly simplify things!
so you can only annotate plain identifiers, not destructures
so you also couldn't annotate like (a, b) =
yeah, that would be my take
do people annotate destructures? I don't think I have ever seen that happen.
given that people seem not to do that anyway, and that it definitely creates parsing problems, I'm on board with that plan
I think there's been plenty enough time of having it be supported to know that it hasn't seen significant use in practice :big_smile:
Also, when I initially saw this code, I thought it was a weird way to write a type alias.
(UserId x) : [ UserId I64 ]
Thought it was the same as:
UserId x : [ UserId I64 ]
FWIW when I first encountered Roc, I was very confused that aliases look so much like annotations.
I didn't know you could annotate a pattern match. My mental model was; Uppercase is an Alias, Lowercase is an Annotation
~30mins on commit 9f395e033dfb9ade5c0642567f0dda8bbd1f2e5a
@Joshua Warner
~12 hours on commit 9f395e033dfb9ade5c0642567f0dda8bbd1f2e5a
~6 hours on commit 0471993428739cc754516c0e6c8a432895dbdaff
https://github.com/roc-lang/roc/pull/7500 now has fixes for all these :)
~2 hours on commit 875e355b68d15334f13fad616d7fe5d0dbc055ce
These are all slow-unit-
or oom-
results; i.e. the test framework thought those inputs took excessively long or consumed too much memory.
... but when I run those, I don't see either thing happen :thinking:
I think in light of Josh's change....I might just throw away my current PR and focus on ||
lambda syntax
Do you mean this one? https://github.com/roc-lang/roc/pull/7470
No, https://github.com/roc-lang/roc/pull/7490
And maybe 7470 too
But 7490 on my local is _much larger_ and ambitious
And I think after your change, and Sam's from a day or two ago, it might be hard to rebase and a lot of the assumptions I'm making might not work out
Basically I'm trying to move ALL PncApply
likes to Collections, and also Pattern:RecordDestructure to use assignedfield and deprecate the RequiredField and OptionalFiled variants of the Pattern enum.
And just moving to align everything between Pattern and Expr to be more consistent
I definitely like the direction
Anything I can do to help?
Happy to work on fixing conflicts or something
@Joshua Warner maybe you could help merge this https://github.com/roc-lang/roc/pull/7470
That sounds like the work Anthony is referring to
And I guess the other one too?? https://github.com/roc-lang/roc/pull/7490
@Luke Boswell @Joshua Warner I have a lot more going on on my local
The weekend has been hectic, but I’ll try to make it pushable tomorrow morning
Started finding & fixing some problems in can
now!
https://github.com/roc-lang/roc/pull/7504
~6 hours on commit 9ffc671659c2c2fec55e9b0475535e6cd34eb278
That's a big tuple :smirk:
@0-21 AccessorFunction(
TupleIndex(
"18888888888888888888",
),
),
Honestly I think we should give the same error for anything more than say 32 or something. In that range it’ll still work with the warning. We can always raise the limit if someone autogenerates code that needs that or something.
~12 hours on commit 5ebd6e0884db65f2d099dfb0ebf255d0bdf80c2b
@Joshua Warner I push up where I'm at right now.
Still some snapshot failures I'm not happy with, but don't have the brain power to fix right now
But the changes that are in there I'm happy with - for now. I still need to implement the "short single-arg" collapsing
And obviously rebase and figure out some of the new newlines I have
Probably just an artifact of some of the pattern-only variance I removed (Some may need to be added back, some may need to be done in a different way).
stat::number_of_executed_units: 567703
stat::average_exec_per_sec: 15343
stat::new_units_added: 9912
stat::slowest_unit_time_sec: 0
stat::peak_rss_mb: 602
INFO: exiting: 19712 time: 240s
AST changed after formatting...
Before formatting:
MM//(#
z
(#
w)#\"
w)#\",/
!\"\"aC-\"\"\"!\"a\"\"!CCa\"a\"\"\"(
#w)##,(
interface
##w,(
w)?
After formatting:
MM
// ( #
z
#
w # "
w) # ",/
!""
aC
-
"""
!"a""!CCa"a
"""(
# w)##,(
interface
## w,(
w,
)?
AST diff
Expr(
Defs(
Defs {
tags: [
EitherIndex(2147483648),
],
regions: [
…,
],
space_before: [
Slice<roc_parse::ast::CommentOrNewline> { start: 0, length: 0 },
],
space_after: [
Slice<roc_parse::ast::CommentOrNewline> { start: 0, length: 0 },
],
spaces: [],
type_defs: [],
value_defs: [
Stmt(
BinOps(
[
(
Tag(
"MM",
),
DoubleSlash,
),
],
Defs(
Defs {
tags: [
EitherIndex(2147483648),
EitherIndex(2147483649),
],
regions: [
…,
…,
],
space_before: [
Slice<roc_parse::ast::CommentOrNewline> { start: 0, length: 0 },
Slice<roc_parse::ast::CommentOrNewline> { start: 0, length: 0 },
],
space_after: [
Slice<roc_parse::ast::CommentOrNewline> { start: 0, length: 0 },
Slice<roc_parse::ast::CommentOrNewline> { start: 0, length: 0 },
],
spaces: [],
type_defs: [],
value_defs: [
Stmt(
Var {
module_name: "",
ident: "z",
},
),
Stmt(
Var {
module_name: "",
ident: "w",
},
),
],
},
Var {
module_name: "",
ident: "w",
},
),
),
),
],
},
- Apply(
- UnaryOp(
- Str(
- PlainLine(
- "",
- ),
- ),
- Not,
- ),
+ BinOps(
[
- Var {
- module_name: "",
- ident: "aC",
- },
- UnaryOp(
- TrySuffix(
- PncApply(
+ (
+ Apply(
+ UnaryOp(
Str(
PlainLine(
- "!\"a\"\"!CCa\"a",
+ "",
),
),
- [
- Apply(
- Var {
- module_name: "",
- ident: "interface",
- },
- [
- Var {
- module_name: "",
- ident: "w",
- },
- ],
- Space,
- ),
- ],
+ Not,
),
+ [
+ Var {
+ module_name: "",
+ ident: "aC",
+ },
+ ],
+ Space,
),
- Negate,
+ Minus,
),
],
- Space,
+ TrySuffix(
+ PncApply(
+ Str(
+ PlainLine(
+ "!\"a\"\"!CCa\"a",
+ ),
+ ),
+ [
+ Apply(
+ Var {
+ module_name: "",
+ ident: "interface",
+ },
+ [
+ Var {
+ module_name: "",
+ ident: "w",
+ },
+ ],
+ Space,
+ ),
+ ],
+ ),
+ ),
),
),
)
crash-e5c5f2279bb018c4128ff6a25ca5e1130aae4952
That one's fixed in https://github.com/roc-lang/roc/pull/7510
~12 hours on 9c340302e2b83246ad3857337b7f5d22fcd58a7f
~ 1 min on commit d42af0b763440176e829d67f0646b5c59c3f0f6d
That was all one bug; pushed a fix to https://github.com/roc-lang/roc/pull/7510
(pretty easy-to-hit problem with pipe-based closures)
@Anthony Bullard (or perhaps @Luke Boswell) - that could use a re-review if you have some time :)
~3 hours on commit 300412a4da46d81c6f40d196d968045fb0ad973f
That, plus a pretty key oversight in the implementation of and/or, are fixed in https://github.com/roc-lang/roc/pull/7535
I think >= 50% of the recent bugs I've fixed have been regressions introduced with recent parser changes
Some of them have been caught either directly on the fuzzing job on the PR itself, or in a fuzzing job on some subsequent PR
Maybe it's time to start paying more attention to the fuzzing happening in CI?
Ahk that's good to know. I thought we might be mostly ignoring that until we get a solid run out of the fuzzer without issues.
I haven't seen it run for more than 10mins yet without a crash - edit after the recent PNC changes etc
But I guess I'm only running each time you've been putting a new syntax related PR in.
There was a stretch of time where it was going a few hours without a crash
Yeah, we were really close
It definitely feels like we're almost back there again.
Yeah we need to stabilize again after some pretty rapid
Maybe if there is a crash we should make it a policy to:
A fuzzing crash found in CI, I mean
@Joshua Warner would you prefer pushing a fix into a PR for new syntax, or landing that PR in main and then following up with any syntax/fuzzer related fixes?
If it does look related to the PR, we should probably be bugging the PR author to look at it :)
We've been moving a lot faster than usual and trying to land the breaking changes staged to unblock things.
Keeping master bug-free-ish is a nice-to-have. Agree rapid collaboration is important (particularly recently)
I'm not too picky if the fix goes in the same PR or a following one
(at least, supposing the breakage isn't to something fairly obvious!)
I've been a little more relaxed because I know the nightlies are paused, and you've been very effective at cleaning things up. So it's been a way to collaborate faster, by merging things into main and not backing up merge conflicts
Yeah I think for better or worse we’ve been in move fast and break things mode - but I think that’s going to slow down now. We can work as a team and smash the fuzzer issues
I think there is a lot of dissonance (much of it surely caused by me) between the parse grammar and the formatter
I’m happy to put down my current projects and jump on to help with that
If we see a legitimate crash - it shouldn’t be on you alone Josh, despite being a wizard - write up issues for these and mark them as P:Medium
And maybe tag them as nightly blocker
Haha fair point :)
And feel free to assign the guilty party
I await the flood of issues…
Adding auto-minimization to the fuzzer in ci: https://github.com/roc-lang/roc/pull/7537
crash-fd9fabc49ffdbd93271a9f8d8e75ad416b7f9d83
~12 hours on commit 5b4c8e70d873e7f5e30f995aedea0199bf1f99b1
Last updated: Jul 06 2025 at 12:14 UTC