Brian Teague (Jan 08 2024 at 22:01): Brian Teague (Jan 08 2024 at 22:01):Is it worth creating a placeholder issue on Github for a metaprogramming approach? Similar to Zig comptime to parse and run ROC functions that are only applicable at compile time instead of runtime to generate ROC code before it is compiled?
To avoid DRY (don't repeat yourself). Anyone have a good example of how metaprogramming would be beneficial to ROC code base like in Num.roc for example?
comptime bytesToX = map [[bytesToU8, U8, 0],[bytesToU16, U16, 1],[bytesToU32, U32, 3]] \[name, type, offset] ->
{name}:List {type}, Nat -> Result {type} [OutOfBounds]
{name}: List {type}, Nat -> Result {type} [OutOfBounds]
{name} = \bytes, index ->
# we need at least {offset} more bytes
offset = {offset}
if Num.addSaturated index offset < List.len bytes then
Ok ({name}Lowlevel bytes index)
else
Err OutOfBounds
I would guess that it wouldn't end up fitting the simplicity focus of roc. The general recommendation would probably be to use a higher level function if possible, just repeat the code, or to use some form of code gen if truly needed.
We do want compile time execution in general, but I would guess we are quite likely to veer away from the metaprogramming part.
Brendan Hansknecht (Jan 08 2024 at 22:07):For example, the code above could be:
bytesToU8 = \bytes, index -> bytesToHelper bytes index 8 bytesToU8LowLevel
bytesToU16 = \bytes, index -> bytesToHelper bytes index 16 bytesToU16LowLevel
...
bytesToHelper = \bytes, index, size, lowLevel
if Num.addSaturated index size < List.len bytes then
Ok (lowlevel bytes index)
else
Err OutOfBounds
The code example above is not 100% accurate, but the point is it would generate the below function signatures for U8, U16, U32, etc. with the correct types, offset, and call to the lowlevel function for that matching type.
I was looking at Num.roc as an example on repetitive code that maybe metaprogramming can reduce.
bytesToU32 : List U8, Nat -> Result U32 [OutOfBounds]
bytesToU32 = \bytes, index ->
# we need at least 3 more bytes
offset = 3
if Num.addSaturated index offset < List.len bytes then
Ok (bytesToU32Lowlevel bytes index)
else
Err OutOfBounds
For sure, but there should be a correct helper like what I shared above that would fix this duplication.
Brendan Hansknecht (Jan 08 2024 at 22:12):I guess my offset is just wrong. I was thinking it was size in bit (obviously that doesn't make sense, probably in bytes)
Brendan Hansknecht (Jan 08 2024 at 22:15):My overall point is that there are some solutions that help (though definitely are less powerful and don't fix everything). To justify adding something as complex as metaprogramming to roc would need some compelling examples of major pain points.
Brian Teague (Jan 09 2024 at 03:23):I added a little more detail to both examples. Both approaches do have pros and cons to consider.
One additional function call is added at runtime. (very cheap but not sure how lots of helper functions could compound)
byteHelper : List U8, Nat, U8, (List U8, Nat -> a) -> Result a [OutOfBounds]
byteHelper = \bytes, index, offset, lowLevel ->
if Num.addSaturated index offset < List.len bytes then
Ok (loweLevel bytes index)
else
Err OutOfBounds
bytesToU16 : List U8, Nat -> Result U16 [OutOfBounds]
bytesToU16 = \bytes, index -> byteHelper bytes, index, 1, bytesToU16Lowlevel
bytesToU32 : List U8, Nat -> Result U16 [OutOfBounds]
bytesToU32 = \bytes, index -> byteHelper bytes, index, 3, bytesToU32Lowlevel
bytesToU64 : List U8, Nat -> Result U16 [OutOfBounds]
bytesToU64 = \bytes, index -> byteHelper bytes, index, 7, bytesToU64Lowlevel
bytesToU128 : List U8, Nat -> Result U16 [OutOfBounds]
bytesToU128 = \bytes, index -> byteHelper bytes, index, 15, bytesToU128Lowlevel
No additional function call at runtime
Leverage string-interpolation and ROC evaluation logic to parse and run the function below at compile time (maybe a new eval keyword?)
Would require multiple line string support.
# new eval keyword
eval bytesToUNumType = map [
{ type: "U8", offset: 0 },
{ type: "U16", offset: 1 },
{ type: "U32", offset: 3 },
{ type: "U64", offset: 7 }
] \byteFunc -> " #Would need multiple line string support
bytesTo\(byteFunc.type) : List U8, Nat -> Result \(byteFunc.type) [OutOfBounds]
bytesTo\(byteFunc.type) = \bytes, index ->
# we need at least \(byteFunc.offset) more bytes
offset = \(byteFunc.offset)
if Num.addSaturated index offset < List.len bytes then
Ok (bytesTo\(type)Lowlevel bytes index)
else
Err OutOfBounds
"
|> List.walk "" Str.concat
Both reduce code. Helper functions add additional runtime function calls. Meta-programming can leverage the existing ROC syntax and can prevent unneeded function calls while also reducing the amount of boiler plate to write.
Agus Zubiaga (Jan 09 2024 at 03:46):If we don’t count blank lines, the helper example is actually the same number of lines as the meta-programming one
Agus Zubiaga (Jan 09 2024 at 03:48):I guess it’s one more line if we skip the first comment
Brian Teague (Jan 09 2024 at 03:48):Hmm, let me make the List of records one line :wink: Just kidding.
Agus Zubiaga (Jan 09 2024 at 03:50):As per the extra function call, I’m pretty sure that will get inlined, so there should be no difference
Brian Teague (Jan 09 2024 at 03:52):Yeah, there are probably some instances where meta-programming can genuinely reduce runtime performance, but I would need someone else to provide a better example.
Brendan Hansknecht (Jan 09 2024 at 03:55):They definitely exist. They just require a more complex example where the function won't just get directly inlined.
Brendan Hansknecht (Jan 09 2024 at 03:55):Also, there should be cases where helpers don't work due to type complications or other reasons.
Brendan Hansknecht (Jan 09 2024 at 03:56):Metaprogramming is definitely more powerful.
Agus Zubiaga (Jan 09 2024 at 03:56):Yeah, I know those exist. I just haven’t really come across a case where I actually wanted something like this in Roc.
Artur Swiderski (Jan 12 2024 at 01:20):I want to scream when I see this, "Don't do things like metaprogramming in Roc, please". It will kill the language for me . What is benefit here ? Just make language garbage for sake of making it garbage . Why some people tend to turn good things into ashes is beyond me
Hannes (Jan 12 2024 at 03:24):I've felt the need for generated Roc code a few times for performance reasons, e.g. recently I was using a large lookup table and wanted to generate the code for that instead of parsing the data at runtime, so I used a python script to generate Roc code. With comptime constant evaluation that wouldn't be a problem though.
Artur Swiderski (Jan 12 2024 at 03:44):if one needs speed so badly go for gpu and hardware acceleration, metaprograming is just sick idea (imho) entire concept
Brendan Hansknecht (Jan 12 2024 at 04:25):go for gpu and hardware acceleration
Many things that need performance also are algorithms tailored to the CPU. I don't want metaprogramming in Roc, but suggesting that it isn't useful and should requite a totally different platform is incorrect. Metaprogramming can have strong measurable performances gains for code that only makes sense to run on the CPU.
Brendan Hansknecht (Jan 12 2024 at 04:25):In many languages it is just done in a way that leads to a lot of complications in the syntax and general complexity in the code.
Brendan Hansknecht (Jan 12 2024 at 04:26):I think zig is probably the language with the cleanest way to do may forms of metaprogramming.
Brendan Hansknecht (Jan 12 2024 at 04:28):I also kinda like odins take (pretty sure it was odin, really hope I am not misremembering the language). In odin, they just include a parser and formatter for the language in the standard library. That way, it is easy to write an odin program to parse odin code, modify the ast, and then format that code back into files to output.
So instead of in language level meta programming, they enable super simple development of code generators.
Brendan Hansknecht (Jan 12 2024 at 04:29):personally I think something of that nature might fit much nicer with the simplicity of roc.
Brendan Hansknecht (Jan 12 2024 at 04:32):Roc definitely falls into a strange middle grey area as a whole. Trying to mix essentially elm with systems programming. Elm is a language of simplicity. Systems programming tends to be a world of harse complexities and real systems problems that need to be solved holistically. That sometimes requires a lot of power (code gen, meta programming, something).
Artur Swiderski (Jan 12 2024 at 11:01):I think differently, CPU is for slow to average performance and that's good enough. Accelerators are for fast performance, if anything there is not enough proliferation of those techniques and proper hardware in industry. Better API-s, better hardware, increase awareness etc. In reality I used only once hardware acceleration in my entire career, it because those techniques are not adequately promoted. Btw. there is a reason why in some languages I can't do stuff like
asm { MOV .. , ADDS .. ADC .. } although one could argue that there are performance gains potentially. Compilation time computation I think that world would be so much better place without it
Brendan Hansknecht (Jan 12 2024 at 15:55):Accelerators are limited pieces of hardware that are often very specialized. It is common for many things to be either impossible to run on accelerators or exceptionally slow on accelerators. Plus, accelerators are expensive to make and hard to program.
Yes, we have GPUs which are relatively easy to use and quite flexible, but past GPUs the cliff is quite steep.
Yes, if there is an accelerator for your specific application, it will be faster than CPU, but even deep learning specific accelerators often fail to be faster than GPU (and sometimes even CPU) because of a few ops that don't map well to the accelerators. It can take a lot of man power to discover hacks to deal with these performance walls. (And this is ignoring the number of companies that don't make it cause accelerators are so expensive to make)
In practice, CPUs are often the fastest reasonable device to use for most people and programs. It is not fair to others to write them off into the should be using accelerators crowd. This is not simply about education, better apis, and promotion of hardware. It is about accessibility, ease of use, suitability to the task at hand, cost, and level of complexity.
Brendan Hansknecht (Jan 12 2024 at 15:57):People should be able to use Roc to create code that approaches the performance of languages like rust and c++. Will that require metaprogramming, i hope not. Would adding metaprogramming make it easier, probably.
Anton (Jan 12 2024 at 17:50):Good nuanced take :)
Artur Swiderski (Jan 12 2024 at 18:33):I will have chance to test your statement soon because my next project will be all about hardware acceleration then I will see first hand (still Roc as high level language to manage it all) : )
Brian Teague (Jan 12 2024 at 19:02):Brendan Hansknecht said:
I also kinda like odins take (pretty sure it was odin, really hope I am not misremembering the language). In odin, they just include a parser and formatter for the language in the standard library. That way, it is easy to write an odin program to parse odin code, modify the ast, and then format that code back into files to output.
So instead of in language level meta programming, they enable super simple development of code generators.
I like this approach a lot:
1) The ROC syntax doesn't need any modifications
2) You get to see the generated code before it gets compiled
3) Debugging ROC won't have to worry about compile time generated code, only the ROC source files.
4) You still get the advantage of generating large portions of repetitive code and optimizing runtime performance.
Odin parser and formatter for reference:
https://pkg.odin-lang.org/core/
Would it be worth adding a parser and formatter to the ROC standard library?
Anton (Jan 12 2024 at 19:29):Let's make a new thread/topic for that :)
Eli Dowling (Jan 13 2024 at 11:36):Brendan Hansknecht said:
I also kinda like odins take (pretty sure it was odin, really hope I am not misremembering the language). In odin, they just include a parser and formatter for the language in the standard library. That way, it is easy to write an odin program to parse odin code, modify the ast, and then format that code back into files to output.
So instead of in language level meta programming, they enable super simple development of code generators.
I actually strongly disagree with this, because of tooling. One of the amazing things about nim, in my experience, the best meta-programming heavy language I've used (though I've not used zig). Is that you can make functions that do really wild stuff and are very safe, because you can embed the checks into them. Things like being able to make a function that sets a gpio pin by number but only accepts ones that are available within the embedded board you're targeting.
You can always see the results of a macro, and it can bubble up errors and provide useful info to the user. This code-gen system sounds much more like ocaml's ppx, which is a bit of a disaster in my opinion because it's just raw AST access done outside of the standard build system.
Brendan Hansknecht (Jan 13 2024 at 17:06):Never used nim (to any significant amount). i have used zig, and my gut feeling is that it's system wouldn't suite roc. If nim's system is at all the same, I would guess that I would feel the same.
Too much complexity for what roc is trying to achieve as a language. The advantage of something like Odin's solution is that it adds no complexity to the language itself.
Brendan Hansknecht (Jan 13 2024 at 17:07):But it still gives a clean way to do complex code gen without needing arbitrary lang scripts with bad support for understanding the language.
Last updated: Jun 16 2026 at 16:19 UTC