Niclas Ahden (May 20 2024 at 09:51): Niclas Ahden (May 20 2024 at 09:51):Hi! I'm learning Roc and have gotten stuck on an issue where roc dev freezes while running my code. The issue occurs when running a database query using roc-pg while mapping over a list. The code builds and roc check passes.
It's probably due to a misunderstanding on my part as I'm new, so I'd appreciate help in understanding this better.
Here are three examples building up to the issue:
# Works as expected. Output:
# pg: 9000
record <- Pg.Client.withConnect
{
host: "localhost",
port: 5432,
user: "postgres",
database: "postgres",
auth: Password "hunter2",
}
(\client ->
Pg.Cmd.new "SELECT 9000 AS foo"
|> Pg.Cmd.expect1
(
Pg.Result.succeed {
foo: <- Pg.Result.str "foo" |> Pg.Result.apply,
}
)
|> Pg.Client.command client)
|> Task.await
Stdout.line! "pg: $(record.foo)"
This makes me believe I'm using roc-pg correctly.
# Works as expected. Output:
# foo: 0
# foo: 1
List.range { start: At 0, end: At 1 }
|> Task.forEach! \n ->
Stdout.line! "foo: $(Num.toStr n)"
This makes me believe I'm using Task.forEach correctly.
# Freezes when running `roc dev`. `.roc-wrapped` uses 199 % CPU (the equivalent of maxing out two cores).
# Expected output:
# foo: 0
# pg: 9000
# foo: 1
# pg: 9000
List.range { start: At 0, end: At 1 }
|> Task.forEach \n ->
Stdout.line! "foo: $(Num.toStr n)"
r <- Pg.Client.withConnect
{
host: "localhost",
port: 5432,
user: "postgres",
database: "postgres",
auth: Password "hunter2",
}
(\client ->
Pg.Cmd.new "SELECT 9000 AS foo"
|> Pg.Cmd.expect1
(
Pg.Result.succeed {
foo: <- Pg.Result.str "foo" |> Pg.Result.apply,
}
)
|> Pg.Client.command client)
|> Task.await
Stdout.line! "pg: $(r.foo)"
I've tried expressing the last example in different ways to understand why roc dev freezes, but haven't been able to figure it out. Can anyone help me understand this better?
Here's the full program: https://pastecode.io/s/n8mfzskt
macOS aarch64
Roc built from latest current source, commit: 7e1a82f048f8ac77b07c7f03ffd30bb13f92edba
Richard Feldman (May 20 2024 at 17:39):@Agus Zubiaga created roc-pg and might have an idea? :big_smile:
Anton (May 20 2024 at 17:45):Profiling the executable to check where it is spending it's time can also be helpful.
In the first presentation of our Dec 2023 meetup Brendan explains how you can profile Roc code.
Agus Zubiaga (May 20 2024 at 17:48):I’ll try this later when I’m home, but running roc check might help find the issue. There are some times where dev hangs instead of reporting an error.
Anton (May 20 2024 at 17:50):The code builds and
roc checkpasses.
roc check was good
Agus Zubiaga (May 20 2024 at 17:51):Oh, I missed that
Agus Zubiaga (May 20 2024 at 17:53):I wonder if this is somehow related to creating connections in a loop. That should work, but you can also create the connection once and use Task.forEach inside.
Agus Zubiaga (May 20 2024 at 18:32):Ok, it's the compiler that's hanging, not the binary. It seems that the interaction with Task.forEach causes the hang. It works as expected if I extract its body into a function and call it from main:
main =
run! 0
run! 1
run = \n ->
Stdout.line! "foo: $(Num.toStr n)"
r <- Pg.Client.withConnect
# ... the rest
If we use Task.loop instead of Task.forEach, it works fine also:
main =
Task.loop (List.range { start: At 0, end: At 1 }) \rem ->
when rem is
[] ->
Task.ok (Done {})
[n, .. as rest] ->
run n |> Task.map \{} -> Step rest
run = \n ->
Stdout.line! "foo: $(Num.toStr n)"
r <- Pg.Client.withConnect
{
host: "localhost",
port: 5432,
user: "postgres",
database: "postgres",
auth: Password "hunter2",
}
(\client ->
Pg.Cmd.new "SELECT 9000 AS foo"
|> Pg.Cmd.expect1
(
Pg.Result.succeed {
foo: <- Pg.Result.str "foo" |> Pg.Result.apply,
}
)
|> Pg.Client.command client)
|> Task.await
Stdout.line! "pg: $(r.foo)"
So there seems to be something about Task.forEach specifically that makes the compiler hang
Niclas Ahden (May 20 2024 at 18:41):Thank you! Besides Task.forEach the issue may also occur with Task.seq as that's one of the other alternatives I tried before asking. I don't have a reproduction of that right now though.
Richard Feldman (May 20 2024 at 18:41):interesting! can we get it down to a minimal repro, possibly not involving roc-pg?
Niclas Ahden (May 20 2024 at 18:42):I have tried to reproduce it without roc-pg but was unable (not to say it's not possible!) I can tinker with it bit more to see if I can nail it down.
Richard Feldman (May 20 2024 at 18:44):appreciate it!
Richard Feldman (May 20 2024 at 18:45):compiler bugs can be super hard to fix without a minimal repro because even a small application results in a ton of in-memory information to scan through looking for problems :sweat_smile:
Niclas Ahden (May 20 2024 at 18:45):Indeed, I can relate!
Agus Zubiaga (May 20 2024 at 18:47):I'm trying to repro with just Tcp, but it seems to work fine unfortunately
Agus Zubiaga (May 20 2024 at 18:48):or fortunately :smiley:
Richard Feldman (May 20 2024 at 18:49):try moving something into a separate module?
Richard Feldman (May 20 2024 at 18:49):there's a known bug related to things working until you introduce a separate module
Niclas Ahden (May 20 2024 at 18:49):I wonder if it may be because the "effect" comes from a separate package
Niclas Ahden (May 20 2024 at 18:50):Yes, I guess that's the same line of thinking
Agus Zubiaga (May 20 2024 at 18:54):It seems to work fine on a different module. I'll try package.
Agus Zubiaga (May 20 2024 at 18:56):That works too :confused:
Niclas Ahden (May 20 2024 at 18:59):Works: File.writeUtf8,Sleep.millis, Http.send
Agus Zubiaga (May 20 2024 at 19:11):roc-pg does a lot of Task chaining and looping internally. It might not be that the compiler is hanging, but that it's actually making super slow progress.
Niclas Ahden (May 20 2024 at 19:13):OK, I'm letting it run now with roc dev --max-threads 16
Agus Zubiaga (May 20 2024 at 19:14):The fact it doesn't happen with Task.loop gives me hope it's fixable
Niclas Ahden (May 20 2024 at 19:15):FYI: it does not seem to utilize more cores despite the flag (199 % CPU util). RAM usage is increasing steadily: 25 GB now
Niclas Ahden (May 20 2024 at 19:16):RAM utilization peaked at 26 GB and it's not increasing (staying 24-26 GB) despite plenty more being available
Luke Boswell (May 20 2024 at 19:16):Is the compiler really using that much RAM?
Niclas Ahden (May 20 2024 at 19:17):btop claims it is. the process name is .roc-wrapped
Niclas Ahden (May 20 2024 at 19:17):Decreasing slowly now, 15 GB, 10 GB ...
Richard Feldman (May 20 2024 at 19:18):is this a new record for most pathological compiler explosion discovered? :laughing:
Agus Zubiaga (May 20 2024 at 19:19):roc-pg keeps delivering those :smiley:
Richard Feldman (May 20 2024 at 19:21):better to find them now than when they block things in production!
Niclas Ahden (May 20 2024 at 19:21):It's been hovering around 3.4-4.1 GB for a while now. The process has been doing 199 % CPU util since start. The machine is an Apple Macbook Pro M1 Max with 64 GB of RAM.
Agus Zubiaga (May 20 2024 at 19:22):I bet it'd be easier on the compiler if I didn't use combinators for decoding, but yeah, it's good to find these now
Niclas Ahden (May 20 2024 at 19:23):Huh, could the process be reading/writing from disk a lot? I'm not sure if it's this process, but a lot of disk IO is happening
Agus Zubiaga (May 20 2024 at 19:24):I wonder if it's outputting a ton of LLVM IR
Agus Zubiaga (May 20 2024 at 19:24):I know Ayaz fixed a case like that when I first presented roc-pg
Niclas Ahden (May 20 2024 at 19:24):The process kernel_task just wrote over 10 GB to disk in 30s or so
Niclas Ahden (May 20 2024 at 19:25):aaand another 15 GB
Niclas Ahden (May 20 2024 at 19:26):Total since the process started kernel_task has written 103 GB to disk
Niclas Ahden (May 20 2024 at 19:27):Poor SSD :joy:
Agus Zubiaga (May 20 2024 at 19:27):but... you get type-safe queries
Niclas Ahden (May 20 2024 at 19:27):#worthit
Niclas Ahden (May 20 2024 at 19:28):OK, it finished!
fish: Job 1, 'roc dev --max-threads 16' terminated by signal SIGKILL (Forced quit)
I did not force kill it, so I'm guessing the OS did
Niclas Ahden (May 20 2024 at 19:29):What an adventure :sweat_smile:
Niclas Ahden (May 20 2024 at 19:32):FYI: I'm low on disk space (<100 GB left) so it's possible that it ran out of space if the writes were temporary (or maybe it's just a coincidence that the numbers kind of add up)
Niclas Ahden (May 20 2024 at 19:52):This is my smallest reproduction:
List.range { start: At 0, end: At 1 }
|> Task.forEach \n ->
Pg.Client.withConnect
{
host: "localhost",
port: 5432,
user: "postgres",
database: "postgres",
auth: Password "hunter2",
}
\_ -> Task.ok {}
@Agus Zubiaga I've tried several other effects but cannot reproduce without roc-pg. It seems the query itself does not matter, so that's good! I'll hop off for tonight, but do reach out if I can help in any way. I'll probably be testing this more tomorrow! Thanks for creating roc-pg, I'm really grateful that it exists! It's what got me over the hump to explore Roc seriously :+1:
Agus Zubiaga (May 20 2024 at 20:06):Thanks! That reduction in scope is significant
Niclas Ahden (May 22 2024 at 14:46):This program passes roc check but panics during compilation:
app [main] {
pf: platform "https://github.com/roc-lang/basic-cli/releases/download/0.10.0/vNe6s9hWzoTZtFmNkvEICPErI9ptji_ySjicO6CkucY.tar.br",
pg: "https://github.com/agu-z/roc-pg/releases/download/0.1.0/nb1q6kN1pu1xvv45w2tE7JjbQ60hOyR1NMxxhRMCVFc.tar.br",
}
import pf.Task exposing [Task, await]
import pf.Stdout
import pg.Pg.Cmd
import pg.Pg.Client
import pg.Pg.Result
main =
_ = queryA!
_ = queryB!
Stdout.line! "Done"
queryA =
Pg.Client.withConnect
{
host: "localhost",
port: 5432,
user: "postgres",
database: "postgres",
auth: Password "hunter2",
}
\client ->
Pg.Cmd.new "SELECT 9000 As foo"
|> Pg.Cmd.expect1
(
Pg.Result.succeed {
foo: <- Pg.Result.i64 "foo" |> Pg.Result.apply,
}
)
|> Pg.Client.command client
queryB =
Pg.Client.withConnect
{
host: "localhost",
port: 5432,
user: "postgres",
database: "postgres",
auth: Password "hunter2",
}
\_ -> Task.ok {}
The compilation panic comes from queryB and it happens when the query is \_ -> Task.ok {} instead of an actual query. Uncomment _ = queryB! and the code compiles and runs.
Here's the panic with backtrace:
$ RUST_BACKTRACE=1 roc dev pg3.roc
thread 'main' panicked at crates/compiler/gen_llvm/src/llvm/build.rs:5764:19:
Error in alias analysis: error in module ModName("UserApp"), function definition FuncName(".\x00\x00\x00\x11\x00\x00\x00=}\xef\t\x02\xf9\x17\xef"), definition of value binding ValueId(5): could not find func in module ModName("UserApp") with name FuncName("=\x00\x00\x00\x06\x00\x00\x00\x1b\x82%\x07f\x82R\xe4")
stack backtrace:
0: rust_begin_unwind
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:645:5
1: core::panicking::panic_fmt
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/panicking.rs:72:14
2: roc_gen_llvm::llvm::build::build_procedures_help
3: roc_gen_llvm::llvm::build::build_procedures
4: roc_build::program::gen_from_mono_module
5: roc_build::program::build_loaded_file
6: roc_build::program::build_file
7: roc_cli::build
8: roc::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
macOS aarch64
roc build from source, commit 33075285b7298dcd76772c8763368b1ca167101c
@Agus Zubiaga I think it might be a separate issue, but I'll post it here for completeness as my previous minimal reproduction used the same kind of "empty query". I hope this is helpful; please let me know if you prefer that I don't ping about this issue :ok:
Agus Zubiaga (May 22 2024 at 15:03):Yeah, unfortunately, “error in alias analysis” is caused by a longstanding compiler bug that’s not specific to roc-pg usage.
Last updated: Jul 06 2025 at 12:14 UTC