Integer from bytes · beginners · Zulip Chat Archive

Hey, I'm working on parsing a binary format in Roc and I need to parse a series of bytes into an integer. To be specific, I need to parse a little endian u64. Is there a builtin for this?

Anton (Feb 27 2024 at 18:27):

Hi @Louis Pearson,
There used to be builtins for this but they got removed. I believe because they detected endianness automatically and we did not want the same pure roc program to return different outputs on different devices.

Louis Pearson (Feb 27 2024 at 18:30):

leBytesToU64 : List U8 -> U64
leBytesToU64 = \bytes ->
    expect List.len bytes == 8
    .value (List.walk bytes {value: 0, index: 0} \{value, index}, byte ->
        { value: value + (Num.shiftRightBy (Num.toU64 byte) (index * 4))
        , index: index + 1
        })

Anton (Feb 27 2024 at 18:37):

It's important to be aware that inline expect statements are discarded when building a roc program for release (with roc build). So you probably want to use Result here, or if you're building something quick and dirty you can use the crash keyword.

Louis Pearson (Feb 27 2024 at 19:22):

Thank you! Took me a little bit to work out how to integrate it into the rest of my code, but I've changed the function to look like this:

leBytesToU64 : List U8 -> Result U64 [WrongListLength]
leBytesToU64 = \bytes ->
    if List.len bytes == 8 then
        Ok (.value (List.walk bytes {value: 0, index: 0} \{value, index}, byte ->
            { value: value + (Num.shiftRightBy (Num.toU64 byte) (index * 4))
            , index: index + 1
            }))
    else
        Err WrongListLength

Louis Pearson (Feb 27 2024 at 19:25):

My main problem was trying to do everything in one function chaining a bunch of Task.await calls. I fixed it by moving the functional code to it's own function that returns a result and then convert that to Task.Ok/Task.Err based on the result

Louis Pearson (Feb 27 2024 at 19:27):

Luke Boswell (Feb 27 2024 at 19:55):

Brendan Hansknecht (Feb 27 2024 at 21:01):

Brendan Hansknecht (Feb 27 2024 at 21:02):

I think it is just fractional types that will require methods to break them into pieces

Richard Feldman (Feb 27 2024 at 21:03):

Brendan Hansknecht (Feb 27 2024 at 21:04):

Brendan Hansknecht (Feb 27 2024 at 21:08):

leBytesToU64 : List U8 -> Result U64 [WrongListLength]
leBytesToU64 = \bytes ->
    when bytes is
        [b0, b1, b2, b3, b4, b5, b6, b7] ->
            Num.toU64 b0
            |> Num.bitwiseOr (Num.toU64 b1 |> Num.shiftLeftBy 8)
            |> Num.bitwiseOr (Num.toU64 b2 |> Num.shiftLeftBy 16)
            |> Num.bitwiseOr (Num.toU64 b3 |> Num.shiftLeftBy 24)
            |> Num.bitwiseOr (Num.toU64 b4 |> Num.shiftLeftBy 32)
            |> Num.bitwiseOr (Num.toU64 b5 |> Num.shiftLeftBy 40)
            |> Num.bitwiseOr (Num.toU64 b6 |> Num.shiftLeftBy 48)
            |> Num.bitwiseOr (Num.toU64 b7 |> Num.shiftLeftBy 56)
    _ ->
        Err NotEnoughData

Brendan Hansknecht (Feb 27 2024 at 21:12):

Oh, also, in your example, you probably want to use List.walkWithIndex. Something like this is a bit more cleaned up:

leBytesToU64 : List U8 -> Result U64 [WrongListLength]
leBytesToU64 = \bytes ->
    if List.len bytes == 8 then
        List.walkWithIndex bytes 0 \value, byte, index ->
           value + (Num.shiftRightBy (Num.toU64 byte) (index * 4))
        |> Ok
    else
        Err WrongListLength

Louis Pearson (Feb 27 2024 at 22:38):

Hmm, just realized my implementation is multiplying index by 4 when it should be multiplying by 8

Louis Pearson (Feb 27 2024 at 22:39):

Louis Pearson (Feb 27 2024 at 22:52):

Brendan Hansknecht (Feb 27 2024 at 22:59):

Yeah, definitely is a learning curve. Roc was my first langauge where I really dove more into FP.

Brendan Hansknecht (Feb 27 2024 at 23:00):

Like I have done some elm, ocaml, and haskel, but not enough to actually feel like I know FP. Just basic tutorial level stuff.

Louis Pearson (Feb 27 2024 at 23:01):

Louis Pearson (Feb 27 2024 at 23:04):

I found elm to be a really compelling language when I first encountered it, but I don't do that much frontend web development so I didn't have a use case for it. I know Roc is still early days but I'm excited by the idea of it

Brendan Hansknecht (Feb 27 2024 at 23:06):

yeah, same here. Elm has been in my back pocket but I essentially never do web dev.

Luke Boswell (Feb 28 2024 at 07:57):

@Louis Pearson would you be interested in making a library/package that implements the Decoding and Encoding abilities like JSON does? If so I could help you with that. It's been on my list for a couple of months now but I haven't quite got around to it. We don't quite have all the pieces like for Dec/Frac etc, but I guess we could leave them unimplemented for now.

Luke Boswell (Feb 28 2024 at 07:59):

Also, I'm interested to know what binary format you are looking at. It sounds interesting

Louis Pearson (Feb 28 2024 at 07:59):

Luke Boswell (Feb 28 2024 at 08:00):

Louis Pearson (Feb 28 2024 at 08:01):

Brendan Hansknecht (Feb 28 2024 at 15:36):

I don't think this is a candidate for decode and encode. It is a specific binary header rather than a generic binary format

Louis Pearson (Feb 28 2024 at 16:22):

I thought the question of packaging was for encode/decode of little-endian or big-endian integers, not for APK signing

Brendan Hansknecht (Feb 28 2024 at 17:46):

Ah fair, though either way, it can't use the actual Encoding and Decoding abilities unless it is a full generic binary serialization format.