forcing a new string allocation/copy · beginners

In my little test program, I am tokenizing s-expressions. Coming from manual memory management languages, I want to allocate a working buffer at the start of the process, and reset/reuse it as needed.

This is all fine, but I realised that my tests were failing when I run them under --optimize, so I immediately suspected my use of the working buffer:

    res =
        st.buf
        |> List.takeFirst st.bufIndex
        |> Str.fromUtf8
        |> Result.mapErr \_ -> Err (BadUtf8 st.start)
    Result.try res \r -> Ok (r, st |> stateResetBuf)

It seems I need to force a copy of the string r on the last line. I tried Str.concat r "" and Str.concat "" r but this didn't change the behaviour of the tests.

What worked is this: "$(r)". This seems accidental rather than purposeful, so I wanted to ask if there was a more supported or intended way to do what I'm doing?

Brendan Hansknecht (Dec 13 2024 at 03:36):

tests failing on optimized but passing in a normal build are definitely bugs

Brendan Hansknecht (Dec 13 2024 at 03:37):

Optimized should not change behaviour

Brendan Hansknecht (Dec 13 2024 at 03:37):

You could also try Str.releaseExcessCapacity and that is a more intentional potential copy.

Mo (Dec 13 2024 at 03:38):

Brendan Hansknecht said:

tests failing on optimized but passing in a normal build are definitely bugs

Ah ok, I didn't realise that. Seems pretty easy to make a small test case if that would be useful.

Brendan Hansknecht (Dec 13 2024 at 03:39):

:thank_you:

Brendan Hansknecht (Dec 13 2024 at 03:39):

Yes, that would be very useful

Mo (Dec 13 2024 at 03:39):

Brendan Hansknecht said:

You could also try Str.releaseExcessCapacity and that is a more intentional potential copy.

That also does the trick and seems more official :)

Mo (Dec 13 2024 at 03:42):

At the moment the tests that failed under --optimize without this fix are in: https://github.com/mocompute/roc-sexp/blob/main/src/Token.roc but the few lines I posted above should be a good basis for a repro case -- I'll work on that a bit later today...

Mo (Dec 13 2024 at 07:14):

I think this is an intriguing bug, because it might intersect seamless slicing as well as the order of execution of parts of an expression under optimisation. I managed to narrow down a test case which identifies the clear cause of the behaviour: https://github.com/roc-lang/roc/issues/7354