Alexander Pyattaev (Jan 05 2024 at 18:25): Alexander Pyattaev (Jan 05 2024 at 18:25):Currently roc allows custom data structures to be implemented in roc only, unless they are bundled with compiler. This is nice for safety, but limits platform-specific data interchange options. For example, a KD tree may be desirable for 3D graphics case, but it can only be exposed to roc code via Tasks, which is cumbersome and kinda inefficient.
The proposal is to allow platforms to define custom data types in native languages such that code using the platform can access them as if they were "builtin types".
Richard Feldman (Jan 05 2024 at 18:41):can you say more about the use case? Why wouldn't it work to implement the KD tree in pure Roc?
Brendan Hansknecht (Jan 05 2024 at 18:58):I want to add some context here. This came up in the plugins discussion yesterday.
A very important detail of this proposal is that it only allows platforms to create special libraries that do arbitrary ffi (the goal use case being enabling custom data structures). So this does not bring any of the concerns of arbitrary libraries doing custom ffi (and the broken ecosystem guarentees that causes). As such, I think it is a much more reasonable baseline.
Note: a platform is already all controlling and can do many funky things if they wanted. For example, you could use roc_dealloc to do arbitrary ffi or constantly poll roc data in another thread to do ffi. So a platform already is required to be trusted. Adding custom data structures does not change this. Though it does make it easier if a specific platform wants to break purity without the use of tasks.
The main gain here is enable easy access to data structures from the host language instead of forcing everything to either:
For 1, I think the use case is very clear. I don't want to write X in roc, but I know that it will be needed for my platform/users. X already exists in a well tested rust package. Just wrap X get the benefits with little to no hassle. (Also, in some cases, certain data structures may not be possible to write performantly in roc due to roc being pure)
For 2, we essentially force the platform to use roc types. So if the platform is a wrapper around bevy and using bevy types, we have a problem. The bevy types will constantly be paying a cost to be converted into generic roc types. This can totally destroy perf and be non-viable. Maybe we can force bevy to also use roc types for certain things to avoid the conversion, but that just makes all the rust game code much more annoying to write. Generally it is preferable to use the rust ecosystem for your rust game and then force the plugin language to use those rust types instead of forcing your entire rust game to use the plugin languages types and ecosystem.
For 3, this is just really inconvenient on the Roc side. If you need a platform data type or data structure, your entire code may just be a pile of tasks instead of something nice a pure. It will make roc a lot less friendly and usable. For a game engine, you may have the world state, many different game objects, and other complex types. All of those types exist on the host side but are accessed by the plugin. So the entire plugin will just be tasks and nothing else. It really just wants to read from a dictionary of data or similar. The task are pure unnecessary bloat.
Brendan Hansknecht (Jan 05 2024 at 19:03):Of course, the biggest concern with accepting something like this into the language is that a platform can abuse it to totally destroy the purity of roc. At least I am not sure of any sort of design to avoid this. Though, as noted above, a platform can already do that if they wanted to. It is just more effort today. Maybe there is some way to limit or scope this, but I'm not sure.
peeps (Jan 06 2024 at 16:59):Just wanna throw myself in here to say that I personally find this to be a very important feature for the projects I plan to use with Roc. A lot of the issues Brendan mentioned are issues that came up a lot when researching other plugin solutions. And every time I ran into resistance because they had some or all of those problems, I would just throw my hands in the air in frustration and think "there must be a better way!" and start googling again. I felt like I was juggling between different tradeoffs and never arriving at something acceptable. But these tradeoffs didn't feel like they HAD to be there, so I just gave up and decided to table it for another day.
Fast forward 6 months: I run into a random youtube video talking about Roc. I found the app-platform design immediately interesting. I feel like Roc is very close to being the perfect solution for my use case. But I don't know if I will use Roc if the issues Brendan mentioned are not addressed. And if it does end up happening, I think this will give Roc an edge over many other solutions that are out there (sort of an untapped market). Which would be good for the project in the long term imo.
Brendan Hansknecht said:
Of course, the biggest concern with accepting something like this into the language is that a platform can abuse it to totally destroy the purity of roc. At least I am not sure of any sort of design to avoid this. Though, as noted above, a platform can already do that if they wanted to. It is just more effort today. Maybe there is some way to limit or scope this, but I'm not sure.
I'm still very new to Roc so I'm not capable of proposing solutions, but I'll follow this thread and give feedback. But this almost feels like a non-issue (I'm not trying to be dismissive here). It makes sense to not help people destroy the purity of Roc, but I think in most cases platform developers are going to be responsible and not actively abuse it because if you're going to do that, why use Roc at all? If you're already drawn to Roc for its specific features as a language, going out of your way to undo some of that feels unnatural and counterproductive. I'd like to believe most people would not do that and instead pursue something else. This is exactly what I did when I ran into solutions that didn't fit well for my specific use case.
But if there is a way to catch/prevent accidental cases, such as with proper documentation and maybe even some compiler magic, that would be great regardless of whether this proposal is implemented. Now would be a good time to experiment with all that, since Roc is still in its early stages of development. And I would be very willing to help out with that.
Brendan Hansknecht (Jan 06 2024 at 17:13):Yeah. I totally agree with you. I think that we should add these features assuming they can fit into Roc.
Anton (Jan 06 2024 at 17:29):Yeah, it seems worth it
Asier Elorz (he/him) (Jan 06 2024 at 17:35):I also agree with Anton. I like the idea of letting platforms write functions in the native language under the promise that the function will be pure. I think we can make an analogy with unsafe Rust. Libraries use unsafe Rust to implement more efficient solutions and keep the promise of making the outer interface safe. This is not enforced by the language. You can write a function that is unsafe but pretends to be safe, but people don't usually do that. It is enforced through culture and by having the big libraries of the ecosystem set an example that others learn and imitate. Roc could seek something similar for purity.
Asier Elorz (he/him) (Jan 06 2024 at 17:37):This design could also give platforms a way to provide pure interfaces over operations that are impure but where the impurity can be made impossible to observe with a good abstraction, such as GPU computation.
Richard Feldman (Jan 06 2024 at 18:49):sorry, I'm very confused - I don't understand what is concretely being proposed here :sweat_smile:
Richard Feldman (Jan 06 2024 at 18:51):I have two guesses about what the concrete proposal actually is; one guess is no different from how things work today, so I'm assuming that's not the proposal, and the other guess is equivalent to allowing arbitrary C FFI, which I'm also assuming is not the proposal based on the comment saying it's not that :big_smile:
Richard Feldman (Jan 06 2024 at 18:52):so I'd love to see some code with made-up syntax
Brendan Hansknecht (Jan 06 2024 at 18:56):It is arbitrary CFFI, but only for platforms.
Brendan Hansknecht (Jan 06 2024 at 18:56):So it doesn't affect the library ecosystem as a whole. Just platforms, which have to be trusted anyway.
Brendan Hansknecht (Jan 06 2024 at 19:03):As for syntax, I don't think it really matters here. It would be like the Effect module, but for arbitrary C FFI instead such that the platform author can make a custom data type equivalent to List for example. With full C FFI for the underlying functionality, but not Effect or Task type.
Richard Feldman (Jan 06 2024 at 19:06):Brendan Hansknecht said:
It is arbitrary CFFI, but only for platforms.
interesting - let's start a new thread which just says it this way in the first post; I'd like to discuss the implications/concerns/philosophy around the idea of allowing this
Richard Feldman (Jan 06 2024 at 19:08):and I don't think that discussion would be limited to platform specific data structures/bevy/etc :big_smile:
Richard Feldman (Jan 06 2024 at 19:11):ok I made another thread for that: https://roc.zulipchat.com/#narrow/stream/304641-ideas/topic/allow.20platforms.20to.20do.20arbitrary.20C.20FFI
Last updated: Jun 16 2026 at 16:19 UTC