Qqwy / Marten (Jul 12 2022 at 14:03): Qqwy / Marten (Jul 12 2022 at 14:03):I'm trying to add some debug printing to the List.appendUnsafe builtin. If my understanding is correct, this should be done in the Zig code in compiler/builtins/bitcode/src/list.zig.
Adding a simple std.debug.print("appendUnsafe called!", .{}) does not work. Compilation fails with the error
'zig failed: /home/qqwy/.asdf/installs/zig/0.9.1/lib/std/os.zig:134:24: error: container 'std.os.system' has no member called 'fd_t'
pub const fd_t = system.fd_t;
This looks like this code is being compiled in some kind of 'no std'-like kind of mode.
Qqwy / Marten (Jul 12 2022 at 14:04):What is the proper way to do this?
Ayaz Hafiz (Jul 12 2022 at 14:05):you need to turn on DEBUG mode in builtins/build.rs
Ayaz Hafiz (Jul 12 2022 at 14:05):it's because WASM doesn't have that type
Qqwy / Marten (Jul 12 2022 at 14:06):Ah, so switching that flag to true essentially disables WASM?
Ayaz Hafiz (Jul 12 2022 at 14:06):it disables the wasm backend builtins but do you need that for your debugging?
Qqwy / Marten (Jul 12 2022 at 14:07):Nope! Just trying to understand how it works. Thanks :+1:
Qqwy / Marten (Jul 12 2022 at 14:27):Hmm. Using std.debug.print results in an immediate segfault on app startup.
Using std.os.write works, but there is no way to format numbers.
Ayaz Hafiz (Jul 12 2022 at 14:28):what are you running over? do you have a branch?
Qqwy / Marten (Jul 12 2022 at 14:56):Publishing the branch now
Qqwy / Marten (Jul 12 2022 at 15:04):https://github.com/rtfeldman/roc/issues/3494
Ayaz Hafiz (Jul 12 2022 at 15:09):how do you repro the segfault? is it only with the debug.print?
Folkert de Vries (Jul 12 2022 at 15:10):@Ayaz Hafiz Marten noted earlier that there is almost no debug info in the final binary. That used to work with debugir, I believe it worked with llvm13 too. Any idea what might have changed recently? (or am I misremenbering hand and has this not worked for a long time)
Brendan Hansknecht (Jul 12 2022 at 15:14):Surgical linking? It is default now and doesn't handle debug info
Folkert de Vries (Jul 12 2022 at 15:14):ah, that might be it
Folkert de Vries (Jul 12 2022 at 15:14):good to know
Qqwy / Marten (Jul 12 2022 at 15:14):Yep! That is the reason! I have line numbers now :heart_eyes:
Brendan Hansknecht (Jul 12 2022 at 15:16):Yeah, eventually need to look into linking dwarf debug info.
Ayaz Hafiz (Jul 12 2022 at 15:19):debugir shouldn't be touching debug symbols from the zig builtins anyway I think, it only aids in debugging the generated LLVM
Qqwy / Marten (Jul 12 2022 at 15:21):Seems like there is a double free of a refcounted object (which manifests as invalid read of the refcount -> invalid write of the refcount-> invalid free).
Ayaz Hafiz (Jul 12 2022 at 15:22):is there a free of the address before the first read?
Qqwy / Marten (Jul 12 2022 at 15:24):There is
Qqwy / Marten (Jul 12 2022 at 15:24):Interestingly memory is also leaked, depending on how long the input is to this example
Qqwy / Marten (Jul 12 2022 at 15:26):(Short inputs 'only' leak, inputs > 23 characters result in the double-free)
Qqwy / Marten (Jul 12 2022 at 15:26):Which makes me think that it might be related to the short string optimization
Ayaz Hafiz (Jul 12 2022 at 15:26):yes almost certainly
Qqwy / Marten (Jul 12 2022 at 15:26):Even though 99% of the code manipulates List U8
Qqwy / Marten (Jul 12 2022 at 15:27):It is turned from a Str into a List U8 at the very beginning and turned back at the very end.
Folkert de Vries (Jul 12 2022 at 15:57):hold up
Folkert de Vries (Jul 12 2022 at 15:57):are you using the hello world platform?
Folkert de Vries (Jul 12 2022 at 15:58):in that platform's source code, we have
// Write to stdout
if (write(1, str_bytes, str_len) >= 0) {
// Writing succeeded!
// NOTE: the string is a static string, read from in the binary
// if you make it a heap-allocated string, it'll be leaked here
return 0;
} else {
printf("Error writing to stdout: %s\n", strerror(errno));
// NOTE: the string is a static string, read from in the binary
// if you make it a heap-allocated string, it'll be leaked here
return 1;
}
Ah!
Qqwy / Marten (Jul 12 2022 at 15:58):That would explain the leaking
Qqwy / Marten (Jul 12 2022 at 15:59):But not the memory corruption
Folkert de Vries (Jul 12 2022 at 15:59):yeah, so what I usually do is use the platform-switching platform and pick the zig one there
Qqwy / Marten (Jul 12 2022 at 16:00):The sole reason I'm using hello-world while developing the parser rather than cli-platform is that that one currently does not compile :sweat:
Qqwy / Marten (Jul 12 2022 at 16:00):I'll try to use the zig one :+1:
Qqwy / Marten (Jul 12 2022 at 16:00):But I have to leave for a couple of hours first
Folkert de Vries (Jul 12 2022 at 16:01):that platform also has a convenient const DEBUG: bool = false;
Anton (Jul 12 2022 at 16:09):Can you make an issue for the cli-platform @Qqwy / Marten?
Folkert de Vries (Jul 12 2022 at 16:30):I got it down to
main =
when manyImpl [48u8] [] is
_ -> "done"
manyImpl = \input, vals ->
{ before: start, others: inputRest } = List.split input 1
when List.get start 0 is
Err _ ->
Ok { val: vals, input: input }
Ok _startCodepoint ->
manyImpl inputRest []
the allocation of [48u8] does not get freed, really not sure why
Folkert de Vries (Jul 12 2022 at 16:31):RC just loses track of it
Folkert de Vries (Jul 12 2022 at 16:31):no decrement for it is emitted
Qqwy / Marten (Jul 12 2022 at 16:45):Anton said:
Can you make an issue for the cli-platform Qqwy / Marten?
It was this one: https://github.com/rtfeldman/roc/issues/3438
Qqwy / Marten (Jul 12 2022 at 16:46):Folkert de Vries said:
no decrement for it is emitted
Really odd that the opposite (double free) happens for large inputs.
Folkert de Vries (Jul 12 2022 at 16:51):often the one snowballs into the other
Folkert de Vries (Jul 12 2022 at 16:51):it just means RC is wrong and then all bets are off
Folkert de Vries (Jul 12 2022 at 17:05):got it
Folkert de Vries (Jul 12 2022 at 17:06):when the slice we end up making is empty, the logic is wrong
Folkert de Vries (Jul 12 2022 at 17:07):just need to turn sublist into
sublist : List elem, { start : Nat, len : Nat } -> List elem
sublist = \list, config ->
if config.len == 0 then
[]
else
sublistLowlevel list config.start config.len
in List.roc
Qqwy / Marten (Jul 12 2022 at 19:06):Even with this fix in place, the larger example still fails with a use-after-free :sad:
Folkert de Vries (Jul 12 2022 at 19:07):can you minimize the example again?
Qqwy / Marten (Jul 12 2022 at 20:07):I'm not sure; when I tried minimizing it further before, Roc no longer used refcounting and then the problem disappears
Folkert de Vries (Jul 12 2022 at 20:08):so long as you use lists it should just be using RC
Qqwy / Marten (Jul 12 2022 at 20:35):I managed to make the example shorter
Qqwy / Marten (Jul 12 2022 at 20:35):# Use after free. List has 24 elems.
# Making list 1 elem shorter resolves problem (short string optimization?)
main =
[65u8,65u8,65u8,65u8,65u8,65u8,65u8,65u8,65u8,65u8,65u8,65u8,65u8,65u8,65u8,65u8,65u8,65u8,65u8,65u8,65u8,65u8,65u8,65u8,]
|> Str.fromUtf8
|> Result.withDefault "(not used)"
This breaks also in the REPL
Qqwy / Marten (Jul 12 2022 at 20:35):And obviously also with Valgrind
Qqwy / Marten (Jul 12 2022 at 20:35):I think that Str.fromUtf8 encounters a problem iff the resulting string is 'big'
Qqwy / Marten (Jul 12 2022 at 20:36):In the REPL, the output becomes "%v����AAAAAAAAAAAAAAAA" : Str. (Deterministic because of WASM?)
Qqwy / Marten (Jul 12 2022 at 20:37):And running the binary outside of valgrind gives arbitrary output for the first 8 characters of the 'hello world' example.
Folkert de Vries (Jul 12 2022 at 20:49):got that one too, now there is just a memory leak that remains
Qqwy / Marten (Jul 12 2022 at 20:49):Wait you already fixed this one somehow?
Folkert de Vries (Jul 12 2022 at 20:49):for the small string cases
Qqwy / Marten (Jul 12 2022 at 20:50):Do I need to update the current branch with trunk or something?
Folkert de Vries (Jul 12 2022 at 20:50):yeah it's an RC thing where you flip a bool and it works
Folkert de Vries (Jul 12 2022 at 20:50):oh, it's not on trunk
Folkert de Vries (Jul 12 2022 at 20:50):just locally
Folkert de Vries (Jul 12 2022 at 20:50):I can make a branch, then you can cherry-pick to get unblocked
Qqwy / Marten (Jul 12 2022 at 20:50)::+1:
Folkert de Vries (Jul 12 2022 at 20:55):see the string-memory-problems branch
Folkert de Vries (Jul 12 2022 at 20:55):eww that has a build error,
Folkert de Vries (Jul 12 2022 at 20:58):allright, all good. The example from the issue now compiles without valgrind errors
Qqwy / Marten (Jul 12 2022 at 21:03):OK, :cherries: :ok: time
Qqwy / Marten (Jul 12 2022 at 21:04):Your fix resolves the problem. I only see the one leak from the hello-world platform now
Brian Carroll (Jul 12 2022 at 21:04):Qqwy / Marten said:
In the REPL, the output becomes
"%v����AAAAAAAAAAAAAAAA" : Str. (Deterministic because of WASM?)
If you mean the web REPL, which runs in Wasm, then be aware that on a 32-bit platform, short strings are <=11 bytes rather than <=23
Brian Carroll (Jul 12 2022 at 21:04):Not sure if that affects your conclusions or not
Qqwy / Marten (Jul 12 2022 at 21:06):No, I meant the command-line roc repl
Qqwy / Marten (Jul 12 2022 at 21:07):Does that one use WASM or LLVM?
Brian Carroll (Jul 12 2022 at 21:07):LLVM
Folkert de Vries (Jul 12 2022 at 21:17):though given the speed of our wasm backend now, using wasm there might not be a bad idea
Folkert de Vries (Jul 12 2022 at 21:17):eventually we'd go for just assembly I think, but that backend is fairly immature
Last updated: Jul 26 2025 at 12:14 UTC