import + decode? · ideas · Zulip Chat Archive

Just writing this down cause I think it could be super cool. Not exactly sure how it would work though (in terms of when it would be initialized). Probably would fit a lot better once roc has compile time execution of top levels.

Anyway, the idea is to enable importing a file as any type that supports decode.
So you could write something like:

import [
    # Currently supported but not as awesome
    "some_image.png" as someImage : List U8,

    # With new proposal
    "some_image.png" as someSprite : Sprite with PngDecoder,
]

I am not exactly sure of the syntax and all of the wiring, but the basic concept is that you could be able to specify which decoder to use and which type to decode into. In this case instead of just getting a png as a List U8 that is unprocessed, it would use the PngDecoder to generate the Sprite type.

Assuming we have comptime, it would be able to ensure the decode is successful as well. Without comptime, Probably would need to make it return a Result Sprite DecoderError.

Agus Zubiaga (Jan 07 2024 at 10:45):

It’s not as concise, but you’ll will be able to do this with inline imports:

someSprite =
    import "some_image.png" as img : List U8
    Decode.fromBytes img Png.decode

Agus Zubiaga (Jan 07 2024 at 10:51):

Assuming we have comptime, we could crash in the Err case, so that decode failure is caught at compile time too

Asier Elorz (he/him) (Jan 07 2024 at 11:05):

If I understand correctly, the decoded object would be a global constant in the data section of the binary, right? Similar to what you get when you #embed in C or you include_bytes! in Rust.

Kiryl Dziamura (Jan 07 2024 at 14:05):

I kinda like the idea that Decode is a meta-programming tool. It's very similar to serde macro if I understand correctly. So it would even provide a meaningful message to the editor/lsp since it's comptime. Wouldn't it be a problem for DX though?

Kiryl Dziamura (Jan 07 2024 at 14:16):

I anticipate such an import mechanism makes sense only with comptime validation, right? And because of the nature of Decode it's possible to derive only data structures? Let's assume I want to build a VM using Roc and include bytecode in the bundle. Such a powerful mechanism would allow me to validate the bytecode during import. Or, even crazier, with this tool it would be possible to implement AST roc parser in roc with Decode ability and then implement an interpreter of the roc in roc and include both a roc script and the roc interpreter to the output bundle... that's sounds very cool, but also crazy.
Maybe I just don't get the Decode ability correctly.

Kiryl Dziamura (Jan 07 2024 at 14:23):

It feels like the feature demands comptime. Without it, it would mean inserting possibly invalid binary data (which is known at comptime) to the app (tho good for obfuscation purpose haha)

Kiryl Dziamura (Jan 07 2024 at 14:37):

Another question: if decode happens during comptime - the input data can be converted into an effective data structure (e.g. json datasets or configs), right? But if decode happens during runtime - roc adds the file to the binary as is?
upd. just noticed the runtime part of the question is already asked above

Brendan Hansknecht (Jan 07 2024 at 15:27):

I do think it heavily relies on comptime but I don't think it requires comptime. Without conptime, it just would have to work like what we have today. You have to include the file as a List U8. Then at runtime decode to a result something. That might just give you an error (but tests can check that)

Brendan Hansknecht (Jan 07 2024 at 15:29):

I guess when we get comptime we may automatically get something like this feature. Probably still need to do some extra cleanup and processing to ensure the original bytes don't get included in the binary and the result is guaranteed not to be an error (though that should probably be opt in).

Brendan Hansknecht (Jan 07 2024 at 15:31):

Assuming we have comptime, yep. Would decode to the object at comptime and store it as bytes in the binsry

Brendan Hansknecht (Jan 07 2024 at 15:35):

Decode is essentially the same as derserializing with serde. It goes from a list of bytes to any type that supports Decoding.

Hmm....I guess that means it wouldn't actually work for PNG cause PNG isn't a serialization format for arbitrary types.

So I guess this would really need arbitrary function execution at comptime and not decode at comptime.

Agus Zubiaga (Jan 07 2024 at 17:29):

Yeah, good point. We could allow anonymous imports (only for ingested files), so you can do:

someSprite = decodePng (import "some_image.png")

Stream: ideas

Topic: import + decode?

Brendan Hansknecht (Jan 07 2024 at 06:46):