Stream: beginners

Topic: String garbled in Stdout when used with list destructuring.

Ian McLerran (Jan 27 2025 at 15:13):

So I have discovered what appears to be a strange bug somewhere in either Stdout in basic-cli, or with list destructuring (it occurs when the two are used in combination).

Below is a screenshot of the garbled output, along with a debug statement, which shows the string itself is intact:
Screenshot 2025-01-27 at 07.27.21.png
And here is the snippet of code which produced that garbled output.

dbg (messages |> List.map .content)
when messages is
    [.., message] ->
        Stdout.line!(message.content)
    _ -> Ok({})

Ian McLerran (Jan 27 2025 at 15:15):

This garbling is consistent and repeatable. Here are some interesting notes:

• The bug is found with list destructuring, but using List.last does not garble the output
• inserting a dbg statement inside the destructured when block prevents the bug

Ian McLerran (Jan 27 2025 at 15:18):

So the following does not produce the garbled output, but if the debug statement is removed, the output will consistently be garbled.

when messages is
    [.., message] ->
        dbg message
        Stdout.line!(message.content)
    _ -> Ok({})

Ian McLerran (Jan 27 2025 at 15:30):

I haven't been able to produce a true min-repro yet, but the following short program should produce it reliably: gist

Ian McLerran (Jan 27 2025 at 16:29):

Btw, on the newest builds of roc, the Path error in basic-cli is encountered during build, but if you run with roc run garble.roc, the program runs just fine, and the garbling error can be observed.

Brendan Hansknecht (Jan 27 2025 at 16:31):

I vote refcounting bug related to the data return from http or some why that data is processed....though a bit surprising it works with last but not the when....

Brendan Hansknecht (Jan 27 2025 at 16:32):

Hopefully you can find a repro that doesn't require hitting an open ai API endpoint.

Ian McLerran (Jan 27 2025 at 16:34):

Lemme try and play around with this and see what I can do. Should be able to reproduce without a paywall endpoint by running LM studio (or Ollama, etc), and pointing the client there instead.

Ian McLerran (Jan 27 2025 at 16:35):

But i'll see if I can reproduce without any extra dependencies and a free endpoint

Ian McLerran (Jan 27 2025 at 16:39):

If you've got LM studio (or Ollama, etc) installed, set the api parameter like below:

api = OpenAICompliant { url: "http://127.0.0.1:1234/v1/chat/completions" }
model = "phi-3.1-mini-128k-instruct" # nice small model, but any will do
client = Chat.new_client({ api, model,  api_key: "" })

Ian McLerran (Jan 27 2025 at 17:26):

Okay, new reproduction. Just relies on on roc-json, but not roc-ai, and pings a free, unauthenticated endpoint. Gist here

Ian McLerran (Jan 27 2025 at 17:27):

Screenshot 2025-01-27 at 11.35.53.png

Anton (Jan 27 2025 at 17:38):

Can you make an issue @Ian McLerran?

Ian McLerran (Jan 27 2025 at 17:38):

Yep, sure can

Ian McLerran (Jan 27 2025 at 18:01):

Okay, issue filed here: #7554

Anton (Jan 27 2025 at 18:16):

Thanks, I've labeled it high priority

Brendan Hansknecht (Jan 29 2025 at 16:44):

@Ian McLerran what os are you on?

Ian McLerran (Jan 29 2025 at 16:44):

MacOS / Aarch64

Brendan Hansknecht (Jan 29 2025 at 16:44):

I just tried to take a look at this on mac (latest roc) and it seems to be working.

Brendan Hansknecht (Jan 29 2025 at 16:44):

I wonder if it was a basic cli bug somehow and the new basic cli testing release fixed it

Brendan Hansknecht (Jan 29 2025 at 16:45):

Can you test and see if you are still hitting it? update to latest main on roc and run latest basic cli test release

Ian McLerran (Jan 29 2025 at 16:45):

Let me try rebuilding roc and cli, and see if I can still reproduce...

Brendan Hansknecht (Jan 29 2025 at 16:45):

new basic cli url: https://github.com/roc-lang/basic-cli/releases/download/0.19.0/Hj-J_zxz7V9YurCSTFcFdu6cQJie4guzsPMUi5kBYUk.tar.br

Ian McLerran (Jan 29 2025 at 16:56):

Okay, fresh build of roc, and updated the min repro to the new url for basic-cli 0.19.0 ("Hj-J_z...").

Still getting the same garbling:
Screenshot 2025-01-29 at 10.54.04.png

Brendan Hansknecht (Jan 29 2025 at 17:03):

It looks like it might be specific to release builds of roc (that or rare and only happen at random coincidence)

Ian McLerran (Jan 29 2025 at 17:06):

Hmmm... It happens to me every time I run the example, and has for several builds of roc (from source).

Ian McLerran (Jan 29 2025 at 17:06):

Sometimes it just truncates the beginning of the str when printing, presumably because the memory location contains non-printing whitespace.

Ian McLerran (Jan 29 2025 at 17:07):

But the str acquired from the list destructuring in this example will never contain "est rerum" as it should.

Ian McLerran (Jan 29 2025 at 17:10):

@Anton were you able to reproduce?

Anton (Jan 29 2025 at 17:11):

I'll give it a try

Anton (Jan 29 2025 at 17:15):

No, all good on Ubuntu 22.04 with latest TESTING release

Ian McLerran (Jan 29 2025 at 17:16):

Hmmm... let me try with the TESTING release. I'm on the latest from main

Ian McLerran (Jan 29 2025 at 17:55):

Still getting the garbled result with the TESTING release. Wonder what is unique on my machine...

Ian McLerran (Jan 29 2025 at 17:55):

Its very consistent for me

Ian McLerran (Jan 29 2025 at 18:03):

Hmm.. it is consistently corrupting exactly the first 10 bytes each time. I changed the json endpoint to use an endpoint with 3 strings instead of 2, where "body" is the 3rd field, thinking maybe the number of bytes corrupted might change if I changed the size of the record. No change. Still exactly 10 bytes corrupted.

Ian McLerran (Jan 29 2025 at 18:14):

I went back to my roc-ai reproduction, and that is also corrupting exactly 10 bytes.

Brendan Hansknecht (Jan 29 2025 at 18:14):

Does it repo with a debug build of roc? cargo run repro.roc

Ian McLerran (Jan 29 2025 at 18:17):

Thats the same as roc run repro.roc, right?

Ian McLerran (Jan 29 2025 at 18:17):

If so, then yes.

Ian McLerran (Jan 29 2025 at 18:25):

Screenshot 2025-01-29 at 12.18.00.png
Screenshot 2025-01-29 at 12.24.50.png

Anton (Jan 29 2025 at 18:29):

A debug build is different, for roc releases we pass --release to cargo.

Anton (Jan 29 2025 at 18:30):

So cargo run repro.roc in the roc repo will make a debug build of the compiler

Ian McLerran (Jan 29 2025 at 18:31):

I'm doing a debug build of roc right now. Should be done any second.

Ian McLerran (Jan 29 2025 at 18:34):

Screenshot 2025-01-29 at 12.31.50.png
Still occurring

Brendan Hansknecht (Jan 29 2025 at 18:35):

Intriguing

Brendan Hansknecht (Jan 29 2025 at 18:35):

Thanks

Ian McLerran (Jan 29 2025 at 18:37):

One other interesting thing I am seeing here -- the dbg print dropped the opening quotation around the string (The quote at the end is inserted by dbg, and is not part of the actual string).

Last updated: Jul 06 2025 at 12:14 UTC