Plugins in roc · beginners · Zulip Chat Archive

I'm trying to figure out how to use roc as a plugin language in a game engine or similar app. Consider, I have a main app body in rust, and it is currently scripted with Python via PyO3. I'm not very happy with performance of the python scripts, so I am considering enabling scripting via roc.

I am currently at a loss, however, as to how exactly this is supposed to be done. All examples assume that main is in roc, not in any other language. Or maybe I am unable to find them. Either way, some help on getting this going would be nice. Rewriting the app to have roc doing main does not sound compelling as it would involve a massive amount of changes, and I still want to keep legacy python scripting support as well.

For reference, an example of what I'm looking to achieve is here (in rust/python) https://github.com/PyO3/pyo3/tree/main/examples/plugin One very important consideration is that the plugin needs to be able to create opaque types that are provided by the host application (so as to have the ability to use data structures defined by the host app).

Hannes (Dec 15 2023 at 10:35):

I think someone else is better qualified to answer this, but I can add one piece of the puzzle which is that none of the examples actually have the "true" main function in Roc.

A Roc application compiles down to something like a C library which has no entrypoint, that compiled library then gets linked with the platform which is the one in charge of calling the Roc functions. It just happens that example platforms like basic-cli call a function called "main" immediately. For you, the platform would be your existing Rust codebase and you could choose to call any Roc function at any time.

Hannes (Dec 15 2023 at 10:37):

Like I said, someone else will expand/correct my answer :sweat_smile: hopefully that's a good start though

Luke Boswell (Dec 15 2023 at 10:40):

Alexander Pyattaev (Dec 15 2023 at 11:50):

Thanks, I have been suspecting that it should not be a huge leap. However, the key stumbling block in this case is that the way the whole thing is linked together. Particularly, I am not sure how to make the "engine" part link against the roc library at runtime, rather than at build time.
PS the python example is helpful, yes, but it does not expose any python objects to roc code...

Richard Feldman (Dec 15 2023 at 11:55):

just to double-check, this means you want the .roc file to compile to a shared (dynamic) library, right? Rather than one big executable?

Richard Feldman (Dec 15 2023 at 11:56):

I'm curious about the opaque part here - what would be an example of how this would be used? :thinking:

Richard Feldman (Dec 15 2023 at 12:01):

as an aside, I'm excited to talk about this because this is always one of the use cases I've had in mind for Roc, and as far as I know you're the first person to actually take steps to try it out! :smiley:

Alexander Pyattaev (Dec 15 2023 at 12:03):

Yes, the idea is to compile to .so or equivalent format. Let me give a concrete example. A game engine consumes a certain "scenario" for every level/map to be played. The scenario contains assets such as 3d models, but also scripts needed to drive logic such as AI for characters in the game. For a commercial game with no modding community all this logic can be built into a .so. However, in order to support modding, it would be nicer to have the scenario scripts that are written in roc, so they can be "safe" in a sense that they will not destroy your files when you load a map for your game.

Hot reloading is also a highly relevant feature, as it enables scenario developer to iterate quickly without reloading the entire engine.

Currently people use lua or python for this sort of tasks, but lua is very obnoxious to use, python is not sandboxed, and both are fairly slow compared to native code.

Richard Feldman (Dec 15 2023 at 12:25):

ok cool! So what would be some examples of data that would be sent from the host to the roc app?

Alexander Pyattaev (Dec 15 2023 at 12:33):

The example I have seen is that the host app would want to pass into roc a readonly view of the world state (or slices of world state), which would include positions and types of entities in the world. On top of that, the host app would need to provide an api to construct new entities in the game world. Naturally, the entities can be expected to include reaources such as raw pointers into various parts of the world state, so they should be partially opaque to the plugin. Same is true if the host defines some sort of custom datastructure that the plugin may be able to use, one would not want to serialize it to make it visible to roc. Instead one would nearly always prefer to expose some api to mutate it via effects.

Alexander Pyattaev (Dec 15 2023 at 12:34):

I think the best way to get good intuition about it would be to check what bevy game engine is doing. In an ideal world, one would have a bevy system that calls a roc plugin that is loaded together with game scenario. Naturally, when you load another scenario, a new implementation for the same system might get loaded.

Richard Feldman (Dec 15 2023 at 13:26):

Vladimir Zotov (Dec 15 2023 at 15:08):

@Richard Feldman Interesting. Actually I originally approached Roc with an intent of embedding it in Bevy. Ideally I'd write Systems (as in Entity-Component-System) in Roc and a lot of things I coded were basically game state reducers that didn't really need to be mutating and mutation could be hidden behind the embedding glue code. Also I get easily distracted waiting for Rust to compile, with is unbearable after working with react/vite ecosystem and getting used to snappy hot reloading.
But now I'm leaning towards writing a pure Roc-driven backend with a Bevy frontend. Bevy heavily relies on compile-time magic for dependency injection of world queries and embedding Roc would mean I have to abandon all of that goodness and best case write a Roc-based DI.
If you know someone exploring this already (as in a game engine using Roc in any capacity), I'd be interested to learn how they do it.

Brendan Hansknecht (Dec 15 2023 at 15:59):

Probably a lot of the data from the host would be exposed as a Box {} (essentially an opaque pointer). Then Roc would be required to send that back to the host to make calls on it. So roc would be glue code calling a pronanly rather large API exposed by the host.

Alexander Pyattaev (Dec 16 2023 at 18:55):

Bevy is just an example, in my current usecase I am not relying on bevy (as it is not really a game, just very similar). However, if we can get roc adopted by bevy community, it would drive adoption pretty hard =)

Alexander Pyattaev (Dec 16 2023 at 18:57):

Did you get to a point where some of that works and can be a starting point to make decent examples? I'd be happy to make some that would be properly polished, but I just do not want to bash my head against the wall of figuring out the overall architecture of how this stuff is even supposed to function without running against the grain of roc.

Luke Boswell (Dec 16 2023 at 21:17):

I haven't updated it for a couple of months, but it shouldn't be too hard to revive if you would like me to. Its not suitable as a platform to share with most people right now, becuase both Zig and the hexops/mach-core library are changing frequently enough, so you need to be able to track the zig nightly and if you aren't comfortable manually upgrading zig package dependencies then it's tricky. (I'm still terrible at this, just silly enough to try and stumble through it blind anyway)

At the time I was using mainForHost : List U8 -> List U8 so passing list of bytes between Roc and Zig to do Model View Update, which worked well enough for a simple proof of concept.

Since then I've made progress on how to write an Effect manually for Zig (without Roc glue) and we have a working example over at ostcar/roc-wasi-platform. This is how we give Roc the ability to interact with the outside world with more than just passing something into main.

Here is a demo of what that experiment looked like roc-graphics-2.gif moving a coloured bird around using the keyboard.

I used TinyVG to interface because it looked simple with a zig implementation, and I was focussed on how to wire everything up.

I think there is a lot of potential here to do more useful things graphics wise.

One of the ideas I was thinking about researching further was if it is possible or ergonomic to use Roc Tasks to set up and drive the WebGPU shader pipelines. The idea is to basically wrap Zig's webgpu, and give Roc the ability to pass through shader logic and set up buffers for processing. This would be a more generic interface, provide higher performance, and standardised low-level primitives for roc package authors to build upon.

Luke Boswell (Dec 16 2023 at 22:28):

@Alexander Pyattaev if you have anything you would like assistance with, I'm happy to help out where I can. I could maybe help wire something minimal up for you?

Alexander Pyattaev (Dec 17 2023 at 06:34):

That would be absolutely amazing! I am totally stuck trying to sort out how the linking between rust binary and roc library module should work when main is not in roc, and critically which tools would i even want to use to do said linking. I think once I comprehend how the linking should work out, I should be able to make a compelling bevy example for the community.

I've set up a separate repo with more specific description and some code i used to investigate this on github, can you share your ideas if you find the time https://github.com/alexpyattaev/roc-plugin-example. The intent is to merge it into roc examples once it takes shape, obviously.

Vladimir Zotov (Dec 19 2023 at 14:57):

So funny Alexander that you mentioned TA Spring, for which this weekend I was looking into using ChatGPT to generate a new widget in Lua.

Vladimir Zotov (Dec 19 2023 at 15:07):

@Luke Boswell @Alexander Pyattaev I could explain/demo (voice chat) the basics of Bevy architecture and how I used it in my pet project, if you want. Maybe it could give you a better understanding of the depth of integration needed on Roc's behalf.
There is also an experimental integration of Typescript in Bevy (I believe the person actually made it work, but there hasn't been any recent updates) https://github.com/jakobhellermann/bevy_mod_js_scripting

Alexander Pyattaev (Dec 20 2023 at 21:59):

No need yet, once we figure out the basics the fun stuff like integration with asset manager would become relevant. For now we are dealing with basics, so we'll be in touch once we're done. Having clean integration with bevy would be next level, naturally.

peeps (Dec 21 2023 at 01:42):

@Vladimir Zotov I'd like to hear/read this explanation you speak of. I just discovered Roc and have no idea how it works on any level, but I'd like to know if Roc will be worth my time before I go deep into it.

Luke Boswell (Dec 21 2023 at 04:04):

I would love this. I'm reasonably comfortable with the basics of building Roc platforms but don't have any experience with Bevy.

I just skimmed through the bevy book as a reference to find the most basic plugin type thing I could think of to glue/integrate Roc with Bevy. The breakout example looked simple enough, so the idea was build a plugin so that end users could "mod" the game by changing the colors using Roc.

I've added a minimal Roc part to @Alexander Pyattaev's roc-plugin-example above. It currently builds Roc as either a standalone executable (for testing) or as a dynamic library for loading into rust as bevy plugin.

Hopefully we can flesh that out and get the basics working end to end, and then I imagine as Alexander mentioned the next steps will be to add more capabilities like commands to spawn entities or adding systems etc.

Luke Boswell (Dec 21 2023 at 04:07):

Also, that typescript integration looks like a really great starting point for our experiment. We can lean on some of this work to do similar things for Roc.

peeps (Dec 21 2023 at 04:25):

In regards to the roc bevy plugin example: I noticed that under "goals" it says "the Roc 'plugin' or 'script' would get loaded into the game on level startup". Does this mean that I could change the script, recompile the script, and then see the changes take effect if I restart the level (not the whole game)? It would be nice if mod developers didn't have to restart their game every time they make a change. I don't want to expand the scope to include hot reloading at the moment, but if we can create an example that can at least support what I'm talking about with some manual steps, it will allow for possible hot reloading in the future, which would be awesome.

peeps (Dec 21 2023 at 04:28):

So something like:
1) change the script
2) recompile script
3) press a button in the game that triggers a reload of the new script

Luke Boswell (Dec 21 2023 at 04:29):

Yeah, so from my limited understanding I hope that something like that will be possible. I imagine a bevy plugin that "manages" the roc plugin and can reload it at runtime. I feel like this should be possible because it is a dynamic library... but this is definitely beyond my level of expertise or knowledge.

peeps (Dec 21 2023 at 04:31):

Yeah I'm in the same boat there. I'd love to help but I can only observe and learn for now.

Luke Boswell (Dec 21 2023 at 04:32):

I do thinks it's important to note that the Roc plugin will only have access to the API that the game developer exposes to it. So it's only going to be able to change things in the world that have been implemented by the game developer

Luke Boswell (Dec 21 2023 at 04:35):

I'm currently just trying to get the bevy example running on its own... I've copied the source from the Bevy repository but trying to make sense of the dependencies so it compiles

Luke Boswell (Dec 21 2023 at 04:36):

peeps (Dec 21 2023 at 04:38):

What kind of errors/issues are you getting? (also, should we move this to a different chat?)

Brendan Hansknecht (Dec 21 2023 at 05:02):

Would be really interesting to see what the list of primitives would be needed from bevy to make a good generic roc platform that can use it (hopefully able to generate or be part of arbitrary systems in the ecs).

Cause I think a significant portion of the complexity will be figuring out an API. Like forget about roc for a second, imagine you were loading a rust or c or Lua shared library. What primitives would they need to access to make a nice Becky plugin.

Luke Boswell (Dec 21 2023 at 05:04):

Totally agree. I think the best way to explore that is just to starting building something. It looks like we have the Roc part and the Bevy part done for our plugin-example above. Now we can wire them together, next up is to add more capabilities and see how that works in practice.

Vladimir Zotov (Dec 21 2023 at 09:40):

We could have a zoom/hangouts/whatever chat. We just need to pick a time. I'm in UTC+1.

Luke Boswell (Dec 21 2023 at 09:44):

Sounds good. Christmas isn't a great time for me with travel interstate to see family etc. Here is a WhenToMeet with some date i think could work. We can put in our availability and see if there is a good time to talk.

Alexander Pyattaev (Dec 21 2023 at 13:57):

I am stuck with a stupid problem. When I have a binary in rust that wants to create an instance of RocStr, I need to link against roc standard library and define functions like roc_alloc. When I make a platform, I also need to define such functions. So what happens when I load a dll that defines roc_alloc into a host app that also defines it? Which version is called when? All of this is very sketchy. Somehow i feel like allocation should only be done by the game and not the plugin, but i am not sure how to handle linking in this case...

Brian Carroll (Dec 21 2023 at 14:06):

I guess that DLL needs to call the roc_alloc defined in the main program, treating it as an extern.

peeps (Dec 21 2023 at 15:02):

That's an interesting website, i shall bookmark it. I would like to be part of this call as well so I have provided my availability.

Alexander Pyattaev (Dec 21 2023 at 16:10):

Okay so there is an MVP in https://github.com/alexpyattaev/roc-plugin-example that seems to be able to load a .so made by roc and call stuff from there. It is a MASSIVE hack though, and I am still unable to actually provide more than one function from platform to the host. When I add multiple provides entries roc compiler just panics which is kinda sad. The message is :
thread 'main' panicked at 'There were still outstanding Arc references to module_ids', crates/compiler/load_internal/src/file.rs:1560:37
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

Brian Carroll (Dec 21 2023 at 17:03):

Yeah you can only call one Roc function from the platform. That should be fixed at some point, not sure if there are specific plans.

Brendan Hansknecht (Dec 21 2023 at 17:10):

As a hack workaround, you can use a record of functions (though not sure if glue will generate that correctly or if it needs to be done manually)

Luke Boswell (Dec 21 2023 at 17:39):

Alexander Pyattaev (Dec 21 2023 at 19:08):

Luke Boswell (Dec 21 2023 at 19:39):

Yeah, I suspect this is one of those things where the implementation is going to change soon to enable effect interpreters, and so it hasnt been a priority to fix this and then have to change it.

Alexander Pyattaev (Dec 21 2023 at 19:45):

Incidentally, I am trying to figure out this - how can I declare a function with no inputs? I know its sort of silly but my FP journey literally started with roc ...

Brendan Hansknecht (Dec 21 2023 at 20:22):

Brendan Hansknecht (Dec 21 2023 at 20:23):

fn = \{} -> ...

fn {}

Alexander Pyattaev (Dec 21 2023 at 20:24):

But then I still need to provide that as argument when I call it. How do I define a pure "constructor" function?

Brendan Hansknecht (Dec 21 2023 at 20:25):

Brendan Hansknecht (Dec 21 2023 at 20:26):

But probably if you are thinking of a constructor for a data structure, the answer is to use an empty record.

Alexander Pyattaev (Dec 21 2023 at 20:27):

Mmmm... that is really strange limitation. I'm so used to having "make me a thing" functions... I suppose still have some residual OO trauma

Brendan Hansknecht (Dec 21 2023 at 20:28):

If you give some concrete examples, I can try to give idiomatic answers, but I think my comment above is the general case in roc.

Alexander Pyattaev (Dec 21 2023 at 20:31):

Yes I understand. I was just trying to define a function like this
init_plugin: \ {} -> {
get_colors: \ Str -> RGBA,
get_object_size: \ Str -> Int,
}

Brendan Hansknecht (Dec 21 2023 at 21:10):

Ah, as the function exposed to the platform, yeah, probably need the empty record.

Brendan Hansknecht (Dec 21 2023 at 21:10):

Brendan Hansknecht (Dec 21 2023 at 21:11):

Alexander Pyattaev (Dec 21 2023 at 21:44):

I am dum dum... How do I define a struct that holds several functions?
This gets compiler very unhappy and all crashy...

EngineCallins:{
    init: U64 -> U64,
    fetch_color: ColoredThings -> RGBA,
    get_bounce_angle: {vx:F32,vy:F32}-> {vx:F32,vy:F32}
    }

Alexander Pyattaev (Dec 21 2023 at 21:45):

Luke Boswell (Dec 21 2023 at 22:02):

Luke Boswell (Dec 21 2023 at 22:03):

I only just updated this last night, theres a segfault I haven't had a chance to look into so it's not generally usable. But the platform and glue gen work ok which may help with your question here.

Steven Chen (Dec 21 2023 at 22:18):

FP DSL in a game engine (good fit for Roc)! Isn't that what Simon Peyton Jones does at Epic games :grinning:

Vladimir Zotov (Dec 21 2023 at 22:28):

Luke Boswell (Dec 22 2023 at 09:20):

Ok, @Alexander Pyattaev @peeps @Vladimir Zotov let's have a chat at (SEE NEW TIME BELOW). Here is a link for a google meet we can use. I'm keen to talk Roc plugin ideas for Bevy games. :smiley:

Alexander Pyattaev (Dec 22 2023 at 10:38):

peeps (Dec 22 2023 at 14:58):

Alexander Pyattaev (Dec 22 2023 at 19:50):

Ok so I have a plugin that returns a bunch of functions from its main that can then be called by engine. However, it turns out that the way roc represents functions is "interesting". I get this sort of generated code in roc glue:

  #[derive(Debug)]
  #[repr(C)]
  pub struct RocFunction_93 {
      closure_data: Vec<u8>,
  }

Problem is, Vec is incompatible with repr(C), and rustc gives me scary warnings about FFI stability. This all looks very suspicious and "wrong" in some sense. Am I missing something obvious?

Alexander Pyattaev (Dec 22 2023 at 19:53):

And using the resulting struct results in segfault, naturally. So FFI warning was 100% justified...

Brendan Hansknecht (Dec 22 2023 at 20:03):

Brendan Hansknecht (Dec 22 2023 at 20:04):

Brendan Hansknecht (Dec 22 2023 at 20:05):

pub struct ForHost {
    pub init: RocFunction_81,
    pub render: RocFunction_83,
    pub update: RocFunction_82,
}

pub fn mainForHost() -> ForHost {
    extern "C" {
        fn roc__mainForHost_1_exposed_generic(_: *mut ForHost);
    }

    let mut ret = core::mem::MaybeUninit::uninit();

    unsafe {
        roc__mainForHost_1_exposed_generic(ret.as_mut_ptr(), );

        ret.assume_init()
    }
}

Brendan Hansknecht (Dec 22 2023 at 20:05):

pub fn mainForHost() -> ForHost {
    extern "C" {
        fn roc__mainForHost_1_exposed_generic(_: *mut u8);
        fn roc__mainForHost_1_exposed_size() -> isize;
        fn roc__mainForHost_0_size() -> isize;
        fn roc__mainForHost_1_size() -> isize;
        fn roc__mainForHost_2_size() -> isize;
    }
    let size = unsafe { roc__mainForHost_1_exposed_size() } as usize;
    let mut captures = Vec::with_capacity(size);
    captures.resize(size, 0);

    unsafe {
        roc__mainForHost_1_exposed_generic(captures.as_mut_ptr());
    }

    let init_size = unsafe { roc__mainForHost_0_size() } as usize;
    let render_size = unsafe { roc__mainForHost_1_size() } as usize;
    let update_size = unsafe { roc__mainForHost_2_size() } as usize;

    let mut ret = ForHost {
        init: RocFunction_81 {
            closure_data: Vec::with_capacity(init_size),
        },
        render: RocFunction_83 {
            closure_data: Vec::with_capacity(render_size),
        },
        update: RocFunction_82 {
            closure_data: Vec::with_capacity(update_size),
        },
    };

    let mut data_slice = captures.as_slice();
    ret.init.closure_data.extend(&data_slice[..init_size]);
    data_slice = &data_slice[init_size..];
    ret.init.closure_data.extend(&data_slice[..render_size]);
    data_slice = &data_slice[render_size..];
    ret.init.closure_data.extend(&data_slice[..update_size]);

    ret
}

Brendan Hansknecht (Dec 22 2023 at 20:06):

Brendan Hansknecht (Dec 22 2023 at 20:07):

Technically, if you avoid captures, each of those functions could just be called by giving them a null closure data.

Alexander Pyattaev (Dec 22 2023 at 20:09):

Alexander Pyattaev (Dec 22 2023 at 20:12):

Interesting that the EngineCallins struct as returned contains some data in the Vec's, even though it probably should not.

src/main.rs:31] ret = EngineCallins {
    bounce: RocFunction_92 {
        closure_data: [
            224,
        ],
    },
    colors: RocFunction_91 {
        closure_data: [
[1]    16968 segmentation fault (core dumped)  cargo run ./plugin_logic.so

Brendan Hansknecht (Dec 22 2023 at 20:12):

There is no guarantee they don't have captures. So glue will always generate in a way that could support captures for a closure.

Brendan Hansknecht (Dec 22 2023 at 20:12):

Alexander Pyattaev (Dec 22 2023 at 20:16):

ok fair. but then why are vecs not empty? They really really have no captures, as per the code i linked...

Brendan Hansknecht (Dec 22 2023 at 20:23):

If you look at the second code example I sent, it just that closures have to be initialized in a special way. Roc does not return a vec at all.

Brendan Hansknecht (Dec 22 2023 at 20:23):

Brendan Hansknecht (Dec 22 2023 at 20:26):

Probably the simplest way to use things currently would be to skip calling mainForHost at all. Just assume no captures.

Brendan Hansknecht (Dec 22 2023 at 20:27):

Brendan Hansknecht (Dec 22 2023 at 20:28):

Also, I think you can name each underlying function as well with another as statement

Alexander Pyattaev (Dec 22 2023 at 20:29):

Hm.... but the whole point of a plugin is that we call a function (mainforhost in my case) that returns a bunch of function pointers that engine can use. And it kinda sorta works as well, just I'm really surprised there is some non-empty vecs attached to them.

Brendan Hansknecht (Dec 22 2023 at 20:38):

The vecs are all empty if you have no closure captures. If you use what glue generates currently, it is just totally wrong generation.

Alexander Pyattaev (Dec 22 2023 at 21:10):

Richard Feldman (Dec 22 2023 at 21:47):

Richard Feldman (Dec 22 2023 at 21:48):

Folkert is aiming to land it in early January, but it's a surprisingly difficult implementation

Brendan Hansknecht (Dec 22 2023 at 22:31):

Luke Boswell (Dec 23 2023 at 04:20):

Ok let's start 1 hours earlier, @Alexander Pyattaev @peeps @Vladimir Zotov updated time 2024-01-05T06:00:00+11:00. Here is the link for a google meet we can use (it's the same one)

Alexander Pyattaev (Dec 23 2023 at 18:41):

I think we are talking about slightly different cases here. Your code example works in case of static (or semi-static) linking, i.e. when linker runs before the host does. In case of a plugin, the linker runs when the plugin is loaded, so your example would not be able to compile. Given what I know about functions and closures in general, I think the glue code is indeed wrong, but in a much more fundamental way. It essentially assumes that this is a correct representation for a function returned from a function is

pub struct RocFunction_93 {
     closure_data: Vec<u8>,
}

and this can not possibly be correct (since I could be returning a different function depending on some runtime condition, so the correct (or, at least, plausible) representation for returned function should be something like

pub struct RocFunction_93 {
     entrypoint: fn(arg1: u64, closure_data: *mut u8,  output: *mut PluginState),
     closure_data: Vec<u8>,
}

In this case I'd be able to actually call entrypoint(...) and have that work while doing runtime linking. Where can I find how roc represents the function pointers in runtime? If I had this info, I should be able to make relevant glue code to at least test if I'm completely wrong or not.

Brendan Hansknecht (Dec 23 2023 at 19:12):

Just having data and no function pointer is correct for roc. Roc turns the n potential runtime functions into a single call site. The dispatch information would actually also be captured in the closure data.

Brendan Hansknecht (Dec 23 2023 at 19:13):

This avoids the need for virtual function tables in certain closure cases and makes it always static (which llvm is able to optimize much better)

Brendan Hansknecht (Dec 23 2023 at 19:14):

I guess for ffi boundary it is a stranger choice, but for all the closures within roc, it leads to a lot more inlining and branches being optimized away. So leads to high performance with llvm optimizations

Brendan Hansknecht (Dec 23 2023 at 19:16):

Also, within roc, we avoid the vectors and always use static size chunks of memory (I believe in the stack) cause we know that info at compile time. Only over ffi does we have to deal with dynamism (cause the platform doesn't know which app will be compiling for it)

Alexander Pyattaev (Dec 23 2023 at 20:15):

Ok I get it. However, this means there is no way to actually make a viable plugin in roc at the moment. There can not be more than one function listed in "provides" and trying to return function pointers is also impossible. Sure there are symbols available in the produced elf file for all the functions, but their names have nothing to do with function names defined in the code. For example, I get "roc__mainForHost_1_exposed_generic" even though actual function name is "reset". Or am I missing an opportunity somewhere?

Alexander Pyattaev (Dec 23 2023 at 20:16):

Also I've checked and calling the function that should return the struct with vec data does literally nothing (i.e. it does not even touch that memory, so whatever junk it had is the junk it ends up with). So it seems that the closure_data is indeed unused.

Alexander Pyattaev (Dec 23 2023 at 20:18):

Alexander Pyattaev (Dec 23 2023 at 20:50):

With this code, the reset function should be a pointer, FFI or not (as you can not know in advance which version would be in the returned record). Or am I missing something?

main: U64->EngineCallins
main = \arg ->
    when arg is
        0 -> {reset : reset, colors : selectColor, bounce : bounceAngle}
        _ -> {reset : reset2, colors : selectColor, bounce : bounceAngle}

Brendan Hansknecht (Dec 23 2023 at 22:40):

Everything functions correctly. It just functions differently in Roc than most other systems.

I am going to use the gui platform for my example, but the same would apply to your plugin.
Base main function (just a record of functions):

program = { init, update, render }

This has no captures and nothing potentially dynamic. The roc__mainForHost_1_exposed_generic function literally does nothing. The roc__mainForHost_1_exposed_size is 0. All captures are of zero size. The roc__mainForHost_0/1/2_caller functions just directly call the underlying init/update/render function directly.

program =
    capture = 2.0f32
    { init: \{height, width} -> init {height: height + capture, width}, update, render }

The roc__mainForHost_1_exposed_generic function stores a float into the captures for the init function. The update and render function still have no data. roc__mainForHost_1_exposed_size now has a size of 4 for the float being returned. The roc__mainForHost_0_caller which maps to the init function now requires closure data, other wise it will attempt to read invalid memory.

program =
    when List.first [7] is
        Ok 7 ->
            { init, update, render }
        _ ->
            { init: \{height, width} -> init {height: height + 1, width}, update, render }

The list has memory effects and does not get optimized away. As such, we have to conditionally pick which init function to run. roc__mainForHost_1_exposed_generic runs the condtional and based on the result, stores a 0 or 1 in memory. roc__mainForHost_1_exposed_size now has a size of 1. It is simply store a tag to know which version of the function to actually run. That value needs to be passed to roc__mainForHost_0_caller for the right version of the init function to execute.

Roc completely avoids function pointers. The platform is just making static calls with various closure data that roc knows how to interpret. So the plugin case works just fine.

Roc does this to all closures in the entire codebase. It enables a llvm to do a lot more compile time optimization of calls to closures instead of just giving up because it sees a call to a function pointer. Over FFI, this protocol is a bit strange, but within an llvm optmization context, it is a huge performance boon. That said, even over FFI, it enables the branch predictor to guess which closure is being called instead of just being blocked by a pointer.

Final note, naming. If glue just works, naming really doesn't matter and the platform author will only ever see the record field name. Since glue doesn't just work, a lot of these strange names leak out. ~~All that said, we have a way to enable nicer names here. Though the syntax only works when written in line with the function exposed to the platform.~~ I can't seem to get the naming syntax working. It may not be functional anymore.

Brendan Hansknecht (Dec 23 2023 at 22:40):

Alexander Pyattaev (Dec 24 2023 at 19:08):

Ok I see, beating imperative with functional requires some sacrifices to the optimizer gods =). Given this, it seems that getting roc to generate "normal" function pointers would be extremely painful. With these limitations, it appears that what we really should aim to have as a "proper fix" is multiple "provides" entries (at least for platforms that are ultimately built as shared libraries). With that, a roc library could expose several callable functions that would all have distinct names, which would make some sense & be compatible with what other languages expect on the FFI boundary.

Currently, the way I see this, roc is just not really set up to build standalone libraries (or maybe there are some magical flags we are not aware of). Specifically, for some reason the compiler insists I put an actual platform into the build, despite the fact that the platform itself will not be used for anything whatsoever in the final library. Further, the platform does not allow me to expose multiple unmangled functions for other people to be able to call. I'd say that anything labeled as "package" should be buildable into a shared library, there is really no reason not to support that as far as I can see. But that is not something fixable short term.

I suppose there are a couple of workarounds we can use for now to get plugin example off the ground:

1. "one function = 1 .so file". This is somewhat wasteful in terms of disk space, but should not be a major showstopper for game plugins (as those tend to be fairly small). This has the downside of getting cumbersome (especially for plugins with more complex life cycles which may benefit from init-work-teardown function bundles).
2. dispatching functions inside roc code. Only one function is exposed by the platform, but it has internal switching based on some sort of enum that is provided by the caller. This would be far more readable/maintainable in the short term, but would introduce runtime branching on every call (which might also get really bad branch prediction).

I'll try to follow with workaround 2 and see how ugly it gets. I imagine that a mispredicted branch is nowhere near as painful as python.

Brendan Hansknecht (Dec 24 2023 at 19:59):

Brendan Hansknecht (Dec 24 2023 at 20:00):

Alexander Pyattaev (Dec 24 2023 at 20:00):

Brendan Hansknecht (Dec 24 2023 at 20:00):

For your use case, you probably want to just always use --lib when building with roc

Brendan Hansknecht (Dec 24 2023 at 20:02):

I mean that roc currently requires the platform to build to a .o or .a file and then deals with the final linking itself when calling roc build ... on most platforms.

Brendan Hansknecht (Dec 24 2023 at 20:03):

So roc is the final linker of the binary and just attempts to hack things together.

Brendan Hansknecht (Dec 24 2023 at 20:03):

Brendan Hansknecht (Dec 24 2023 at 20:04):

Alexander Pyattaev (Dec 24 2023 at 20:05):

Yes, in this sense its true. I'd take it one step further and just allow building packages as standalone .so, no platform whatsoever. It would make for a really good start in the dynamic library story I think. There will be a limitation that it would be very hard to define platform-specific data types in this case.

Brendan Hansknecht (Dec 24 2023 at 20:06):

But there doesn't need to be any concrete platform or code in another language for that to work.

Alexander Pyattaev (Dec 24 2023 at 20:06):

--lib requires a platform. Even though it does not use any of it right now. And it is not documented at all.

Brendan Hansknecht (Dec 24 2023 at 20:06):

Brendan Hansknecht (Dec 24 2023 at 20:08):

Alexander Pyattaev (Dec 24 2023 at 20:09):

to run roc build --lib you need a platform file, which does absolutely nothing that a package or interface file would not achieve.

Alexander Pyattaev (Dec 24 2023 at 20:09):

Brendan Hansknecht (Dec 24 2023 at 20:10):

The platform file is required. It specifies the effects and the API that roc needs to expose to the host. So you need a platform/main.roc file with associate task and effect files.

Alexander Pyattaev (Dec 24 2023 at 20:12):

Brendan Hansknecht (Dec 24 2023 at 21:15):

Glue essentially uses a plugin and uses --lib, but it is built into the compiler, so not a self contained example.

Alexander Pyattaev (Dec 24 2023 at 21:29):

Brendan Hansknecht (Dec 24 2023 at 21:32):

Alexander Pyattaev (Dec 25 2023 at 13:23):

I think I have a plan how to get plugins working. The platform's main file can define static function pointers. And normal functions for library. I am surprised i did not see this earlier. We may not need to fix glue quite yet

Alexander Pyattaev (Dec 30 2023 at 22:49):

Overall, this basically works, and I could use code from @Brendan Hansknecht to replace the generated glue with something that does not just outright panic.

Brendan Hansknecht (Dec 30 2023 at 22:59):

Alexander Pyattaev (Dec 30 2023 at 23:06):

Ok yes it can... but I can not explain how to link with that to rustc... what is the flag to link against .o?

Brendan Hansknecht (Dec 30 2023 at 23:11):

You can convert it to a .a if you want. Just use ar, but yeah roc probably should emit a .a directly instead of a .o. a .a is more flexible and probably what we want long term. Also, you probably can tell rust to link the .o the same way as a .a or just rename the file. Generally that just work.

Alexander Pyattaev (Dec 30 2023 at 23:14):

hm... ar rc app.a libapp.o seems to do the converting trick... but i am stuck trying to explain this to rustc...
I have app.a in the project directory, and this in build.rs

println!("cargo:rustc-link-lib=static=app.a");

Brendan Hansknecht (Dec 30 2023 at 23:32):

Brendan Hansknecht (Dec 30 2023 at 23:33):

Alexander Pyattaev (Dec 31 2023 at 20:26):

Thanks! It works!
Rant follows:
I hate the silly name mangling with libraries. why can i not just say -L thing.so when i want to link a dynamic lib, or -L thing.a when i want a static? instead we do the libthing.a but specify -l thing in the flags... what a horrible abomination we have built...

Brendan Hansknecht (Dec 31 2023 at 20:37):

Alexander Pyattaev (Dec 31 2023 at 20:41):

ok so overall I think we are set for success. the wrapper library idea seems to work, so we can use it to isolate the roc compiler internals related to how roc represents closures from whoever will be calling the code from rust (or C for that matter).

Alexander Pyattaev (Dec 31 2023 at 20:41):

Luke Boswell (Jan 02 2024 at 10:19):

I'm excited to see the progress Alexander has been making with the above, and to discuss ideas/experiments for Roc plugins.

peeps (Jan 02 2024 at 16:16):

Vladimir Zotov (Jan 03 2024 at 08:10):

Alexander Pyattaev (Jan 04 2024 at 20:30):

peeps (Feb 17 2024 at 20:24):

Bevy 0.13 just dropped with Dynamic Queries! Been looking forward to this for the scripting use case. Might be useful to our endeavors with the roc-plugin stuff.

Luke Boswell (Feb 17 2024 at 20:45):

Awesome. I've been thinking about this experiment lately. I wish I had a little more time to work on it. It will be super cool to show how to use roc as a plugin.

peeps (Feb 17 2024 at 21:01):

Yeah i been dealing with a seemingly never ending queue of personal BS, so I have not been able to work on anything for like a month :sad:

Alexander Pyattaev (Feb 20 2024 at 11:18):

Same here, too much other work to do. Did some experiments with stabby
crate that would enable to make the plugin ABI less shit, but nothing
beyond that.

peeps (Feb 20 2024 at 13:01):

Oh I ran into stabby last year trying to use it with Bevy. Never managed to make it work lol. My fault tho.

Alexander Pyattaev (Feb 20 2024 at 13:40):

Alexander Pyattaev (Jun 04 2024 at 18:20):

Hi! I'm back at it... what did I miss? glue generation seems to be just as broken as before (or maybe more so)... If anyone can point me at a bunch of examples that work with relevant version of glue I'd appreciate it=)

Vladimir Zotov (Jun 04 2024 at 18:56):

I did have another look at it and yes, it didn't get better yet, so I'm just practising roc in hopes the glue topic gets more love :sparkles: later this year :cowboy:

Alexander Pyattaev (Jun 04 2024 at 18:57):

It is rather unfortunate that glue code is all in roc... I am basically unable to grasp it on the scale necessary to fix what needs to be fixed (and it is not actually a huge amount)

Alexander Pyattaev (Jun 04 2024 at 19:18):

Why does dbg command not work when I put it in roc glue code? It is as if some cruel person deliberately made it hard to debug that code

Brendan Hansknecht (Jun 04 2024 at 19:26):

Probably just that no one ever made it work. Debug was changed to make it work better, but that probably never got wired into the glue platform.

Alexander Pyattaev (Jun 04 2024 at 19:29):

Sorry I got far too much annoyed by it, should stay cool. its just that dbg command is supposed to print into stderr, but from inside roc glue impl it seems to not work, which makes figuring out what it does very tricky. No side-effects is nice but I see no mechanism to extract data from inside glue generation other than just spamming into the generated code

Brendan Hansknecht (Jun 04 2024 at 19:33):

Brendan Hansknecht (Jun 04 2024 at 19:34):

Alexander Pyattaev (Jun 04 2024 at 19:35):

yeah but for that I need to set up the build thing for the roc compiler, right? Last time I tried it it was not fun at all...

Brendan Hansknecht (Jun 04 2024 at 19:36):

I can try and do that tonight. Then it should be pulled into the following nightly.

Alexander Pyattaev (Jun 04 2024 at 19:36):

you'd be my hero! no need to rush it though, if it turns out to be a mess, it will not be very helpful.

Brendan Hansknecht (Jun 05 2024 at 15:27):

Ah....I know the problem here. It is bigger than I originally though, but doable if someone wants to attempt it.

Fundamentally, dbg is tied to expect in the roc compiler currently. Expect only works via direct execution with roc dev or roc run. It has to use one of the total roc internal execution methods. It does not work if we emit an executable file. (we want to change that so it simply calls roc_expect in the platform and can be run independent from roc).

That said, to make this work, the full roc_expect change is not required. For this, we simply have to allow for generating dbg statements in an executable. We already have roc_dbg in platforms. The compiler currently generates expect and dbg together. So those would need to be separated. I would assume this is mostly funneling a conditional through the compiler to allow for dbg without expect.

Alexander Pyattaev (Jun 05 2024 at 18:25):

Oh well. dbg is the least of the problems in that glue logic. I've dived a bit into it all, and the main problems, as I see them now, are as follows:

Brendan Hansknecht (Jun 05 2024 at 19:12):

Yeah, the fact we still have to return a struct of functions is really annoying. We should just let the user expose multiple functions and be done with it. Solves many problems by reducing complexity due to lambdas (which can capture arbitrary data).

Richard Feldman (Jun 05 2024 at 19:13):

Alexander Pyattaev (Jun 05 2024 at 19:14):

Also can someone explain what does HasClosure mean in context of glue code? I am asking cuz structs that clearly hold closures are marked as HasNoClosure...

Alexander Pyattaev (Jun 05 2024 at 19:19):

well it helps defining callins, yes, but the moment some enterprising user wants to return a struct with a callback we are back to square one. we might just as well get glue to work properly if possible.

Brendan Hansknecht (Jun 05 2024 at 19:26):

Brendan Hansknecht (Jun 05 2024 at 19:27):

Alexander Pyattaev (Jun 05 2024 at 19:27):

Alexander Pyattaev (Jun 05 2024 at 19:29):

It sorta works but when you are returning a struct with roc functions (not even actual closures) it will generate garbage and segfault

Alexander Pyattaev (Jun 05 2024 at 19:31):

#[derive(Clone, Debug, )]
#[repr(C)]
pub struct Callins {
    pub colors: RocFunction_88,
    pub reset: RocFunction_89,
}

pub fn mainForHost(arg0: u64) -> Callins {
    extern "C" {
        fn roc__mainForHost_1_exposed_generic(_: *mut Callins, _: u64);
    }

    let mut ret = core::mem::MaybeUninit::uninit();

    unsafe {
        roc__mainForHost_1_exposed_generic(ret.as_mut_ptr(), arg0);

        ret.assume_init()
    }
}

pub fn mainForHost(arg0: u64) -> EngineCallins {

    extern "C" {
        fn roc__mainForHost_1_exposed_size() -> isize;
        fn roc__mainForHost_1_exposed_generic(_: *mut u8, _: u64);
        fn roc__mainForHost_1_size() -> isize;
        fn roc__mainForHost_2_size() -> isize;

    }
    //figure out size of captures
    let size = unsafe { roc__mainForHost_1_exposed_size() } as usize;
    let mut captures = Vec::with_capacity(size);
    captures.resize(size, 0);

    unsafe {
        roc__mainForHost_1_exposed_generic(captures.as_mut_ptr(), arg0);
    }

    let colors_size = unsafe { roc__mainForHost_1_size() } as usize;
    let reset_size = unsafe { roc__mainForHost_2_size() } as usize;

    let mut ret = EngineCallins {
        colors: RocFunction_colors {
            closure_data: Vec::with_capacity(colors_size),
        },
        reset: RocFunction_reset {
            closure_data: Vec::with_capacity(reset_size),
        },
    };
    //let mut ret = core::mem::MaybeUninit::uninit();

    let mut data_slice = captures.as_slice();
    ret.bounce.closure_data.extend(&data_slice[..bounce_size]);
    data_slice = &data_slice[colors_size..];
    ret.reset.closure_data.extend(&data_slice[..reset_size]);

   ret
}

Alexander Pyattaev (Jun 05 2024 at 19:33):

As you can see in generated version the boilerplate to prepare the memory for roc code to be called is not generated, this results in troubles when calling this stuff (essentially UB)

Alexander Pyattaev (Jun 05 2024 at 19:34):

All of this is fairly straightforward once you know what's needed to be there, I'm just trying to wire it all into the current RustGlue.roc which sorta assumes none of this bs would be necessary for "normal functions"

Brendan Hansknecht (Jun 05 2024 at 19:38):

Brendan Hansknecht (Jun 05 2024 at 19:39):

Glue doesn't have any sense of this at all. So it passes I. The struct of closures and simply hopes for the best

Alexander Pyattaev (Jun 05 2024 at 19:40):

well what gets generated by roc is not really a problem imo. what is a problem is that we get enough info exposed to glue code to generate the necessary stuff. if that is handled, noone will ever need to look at that code ever again. it needs not be pretty.

Brendan Hansknecht (Jun 05 2024 at 19:42):

Sure, but splitting the closures like this requires extra allocations and isn't really efficient anyway. Plus, sometimes simplifying generation may simplify the glue code and make it easier to write

Brendan Hansknecht (Jun 05 2024 at 19:43):

But yeah, for this case, if a struct contains closures you have to generate it totally different. And a closure really mains anything that contains a rocfn

Alexander Pyattaev (Jun 05 2024 at 19:43):

Can you clarify exactly why rocFn needs a Vec to be called? Is it for the stack?

Alexander Pyattaev (Jun 05 2024 at 19:44):

More specifically, does it have to be a Vec, or can it be just a memory slice that is "large enough"?

Alexander Pyattaev (Jun 05 2024 at 19:44):

I was under impression that roc functions only do allocations via roc_alloc, is it different for the stack memory?

Brendan Hansknecht (Jun 05 2024 at 19:46):

Alexander Pyattaev (Jun 05 2024 at 19:47):

I see, so technically an actual pure function (not a closure) would never actually touch that vec, right?

Brendan Hansknecht (Jun 05 2024 at 19:47):

This can be stored anyway. In most cases, it probably should actually be stored on the stack ....but that doesn't play nice with the simplicity of glue gen.

Brendan Hansknecht (Jun 05 2024 at 19:47):

Alexander Pyattaev (Jun 05 2024 at 19:49):

Brendan Hansknecht (Jun 05 2024 at 19:52):

Alexander Pyattaev (Jun 05 2024 at 19:53):

right... so functions like something_something_size() are basically const, right? as in, the values they return for a given roc library build are never going to change, correct?

Alexander Pyattaev (Jun 05 2024 at 19:53):

Cuz if that is the case, I could conceivably cache the necessary allocations such that future calls into roc do not allocate memory for the captures

Alexander Pyattaev (Jun 05 2024 at 19:54):

Brendan Hansknecht (Jun 05 2024 at 19:54):

Brendan Hansknecht (Jun 05 2024 at 19:55):

Though you do have to be careful of refcounts. If you pass a closure capture to a function, anything refcounted might be freed.

Alexander Pyattaev (Jun 05 2024 at 19:56):

this simplifies the problem substantially, as in this case I can just make a factory for the structs that contain closures. And that factory could have a pool of those structs handy with preallocated memory

Alexander Pyattaev (Jun 05 2024 at 19:56):

Brendan Hansknecht (Jun 05 2024 at 19:58):

Just when you run a RocFn, it consumes the capture (will free refcounted things in the capture).

Brendan Hansknecht (Jun 05 2024 at 19:59):

So you can reuse the underlying memory, but you can't call the same RocFn repeated forever without regenerating it due to recounting.

Alexander Pyattaev (Jun 05 2024 at 20:02):

Ok I see, so whenever someone gives me a closure with that vec, the size of that vec might be different (and content, depending on what got captured by the returned closure). So I need to have a unique Vec of appropriate size ready to be filled in for every case where a RocFn might get returned. Now once I call that RocFn and have its result (and therefore am not going to call it again) I can reuse that Vec to receive another instance of that same RocFn with different closure, reusing the allocated buffer (but its contents will now be different)

Alexander Pyattaev (Jun 05 2024 at 20:03):

As a result, the safest mechanism is to just allocate a fresh Vec for every closure return from roc, as that guarantees owned memory (even though that technically is slower)

Alexander Pyattaev (Jun 05 2024 at 20:03):

Brendan Hansknecht (Jun 05 2024 at 20:10):

Alexander Pyattaev (Jun 05 2024 at 20:12):

Brendan Hansknecht (Jun 05 2024 at 20:14):

peeps (Oct 08 2024 at 06:14):

Hello! Its been awhile since checked in on this. I wanted to ask what the story looks like today for trying to use Roc as a plugin system, such as within a game? Last time i was here it looked really rough (i think the issue was with the glue generation).

Luke Boswell (Oct 08 2024 at 06:41):

Luke Boswell (Oct 08 2024 at 06:42):

I'd say it's still similar to when we last talked about it. There's a lot of breaking changes planned. But for building a proof of concept and exploring the ideas it's great fun! :smiley:

peeps (Oct 08 2024 at 06:42):

Alexander Pyattaev (Oct 08 2024 at 06:52):

I have been unable to find much time to move this forward. The glue code is a bit of a mess, and my roc skills are insufficient to improve it substantially. If you want, I can give you guidance as to where the stumbling blocks are.

Luke Boswell (Oct 08 2024 at 06:53):

For the glue code, @Sven van Caem is working on that as we speak. It's super cool to see the progress he has already made finding issues.

Alexander Pyattaev (Oct 08 2024 at 06:53):

Oh I suppose I need to get in touch with him then. Is there a zulip thread on that work?

Luke Boswell (Oct 08 2024 at 06:55):

Not really. If you have specific things you've discovered, I think it would be helpful to start a thread about it. Sven would be best placed to decide if it's in scope for the things he's working on. I know he has a bit of a plan he's tracking, but maybe he's looking for ideas too.

Alexander Pyattaev (Oct 08 2024 at 07:30):

Erlend Sogge Heggen (Oct 08 2024 at 16:49):

fyi we’d be happy to talk you through plugin setup for Bones, a Bevy-derived engine that’s made specifically to be scripting-friendly, currently supporting Lua (piccolo).

Erlend Sogge Heggen (Oct 08 2024 at 16:49):

peeps (Oct 08 2024 at 16:57):

Oh hi, i think i saw a video of someone talking about Bones in a Bevy talk/meetup a few days ago. Small world lol.

Alexander Pyattaev (Oct 08 2024 at 18:13):

I specifically do not want to use lua for scripting:) consider that a religious opposition to 1-indexed arrays.

Erlend Sogge Heggen (Oct 13 2024 at 13:10):

Yeah I’m talking about using Roc instead of Lua, e.g. on an engine like Bones. Bones isn’t Lua-specific and it’s more scripting-lang friendly than Bevy, so it could be a good testing ground for roc plugins, if anyone gets the itch, heh.

Alexander Pyattaev (Oct 13 2024 at 13:16):

How can an engine be scripting-unfriendly? Or scripting-friendly, for that matter?

Vladimir Zotov (Oct 13 2024 at 14:25):

Bevh has a neat DI setup, and it's the idiomatic way. But using a raw imperative machinery sitting behind the DI facade may be not so nice, especially with Roc. At least this is my naive vision right now.

Brendan Hansknecht (Oct 13 2024 at 17:35):

@Erlend Sogge Heggen any quick link to what your plugin api currently looks like? I assume you specify and speak c abi somewhere?

Alexander Pyattaev (Oct 13 2024 at 17:49):

The idea is to have ABI via hot reload crate or dextrous developer crate for bevy. So essentially the roc platform is a rust shim that on one hand has a api for the game engine, and on the other provides the needed machinery to call into roc. The reason it needs to happen this way is due to how roc linking works. Basically you either compile roc into a basic .so with C abi (which is unsafe), or you make a platform and link against that, and that can in turn provide a nice API for rust.
Current wip here https://github.com/alexpyattaev/roc-plugin-example

Brendan Hansknecht (Oct 13 2024 at 17:52):

Erlend Sogge Heggen (Oct 13 2024 at 18:30):

Erlend Sogge Heggen (Oct 13 2024 at 18:37):

Alexander Pyattaev (Oct 13 2024 at 18:43):

This will necessarily require the plugin to have no internal state, and expose exactly one pure function. But yes, that works, and youbcan even have the platform in rust if you want. But if you want to have multiple callins it does not work.

Zicklag (Oct 13 2024 at 21:16):

Bevy, when I was last involved in it was unfriendly to scripting because it made heavy use of the Rust type system and things like compile-time generated queries over it's ECS. It's also very complicated internally, which meant trying to make some of the normally static, compile-time-checked things, and convert them into runtime-only things, was pretty tricky.

It's something that Bevy is getting better at, and is probably much further along than it was when I last used it.

Bones, on the other hand, is really simple, and has a much smaller API surface. Also, everything in bones is designed to be able to be manipulated at runtime, so in Bones it is way easier to give a scripting language nearly 100% power over the engine.

Our Lua implementation is also written in Rust so integration was pretty easy. We didn't need to make an external C API, we just had to use the Rust Lua library to add Lua integrations with our engine's Rust functions.

We did make try to make Bones with the hope that we could make a C API for it in the future, just in case it was necessary to integrate with other scripting languages, but we haven't done it yet.

Scriptable components in the ECS are also stored with a C memory layout, so as long as you know the schema of the component you can soundly cast pointers to components to C structs if you wanted to write plugins in C. The idea was hopefully to avoid any unnecessary barriers to high-performance scripting solutions. In particular, we hope this might be better than Bevy's reflection system for certain scripting languages, because you can get direct access to the component data through pointers instead of having to make calls through the Bevy Reflection system's vtable for every field access.

I'm hardly familiar with Roc at all yet, but I'm very interested in it! I've wanted to get into some functional programming at some point after I discovered it, but hadn't had the chance to yet.

If this is how Roc can work, I think the ideal way to integrate with Bones would be to make a Bones plugin so that you can compile bones as a Roc platform, hopefully making easier to call into engine functionality since Roc and Bones are both written in Rust.

But it's also possible that we could make a C API for Bones, and then Roc could consume that.

There are little rough edges here and there, but things are mostly in place to allow experimentation with this kind of stuff in Bones already.

Zicklag (Oct 13 2024 at 21:18):

Oh, like, for one thing, UI in bones is done with Egui, and I'm not sure if there's an easy way to bind Egui into Roc. That doesn't stop us from making a Roc-specific/compatible UI integration, though.

Egui's epaint library is really easy to use, and we could basically make a scriptable Egui component that just ignores all of the egui helpers and does everything manually, without having to change anything about UI rendering on Bones.

Brendan Hansknecht (Oct 13 2024 at 22:29):

Thanks for all the info @Zicklag. Roc is still in it's early stages, but we definitely hope that in the long term it can be great as a plugin language. I think that today it definitely has a few interfacing pains, but it could be used. One thing I have noticed that is hard with at least the godot api is that it is very codegen heavy. To make that play nice with roc probably means we need to do a lot of code gen on the interface and platform side to map from the low level primitives godot expects to roc. While doable, it makes the project a lot more tedious.

On the other hand, something like raylib is really easy to integrate with roc (just takes time to wrap all the functions). Fundamentally, it is all static C calls with no codegen or overly dynamic interfacing to deal with.

I think for a lot of the plugin use cases, the amount of dynamism in the api will be a big decider in how complex it is to integrate with roc.

Alexander Pyattaev (Oct 13 2024 at 22:35):

Ok I understand. Yes, bevy does have these features, but in the end your script would never have access to the entire world state in any multiplayer game (for netsync reasons if nothing else). So I am not sure if that is indeed a problem. Also if you are so inclined you can run a system in bevy that locks the entire ecs and gets exclusive access to everything.

Dan G Knutson (Oct 14 2024 at 01:20):

If Bones gives direct pointers, does that mean that your Bones scripting component could be something like an opaque RocArena, so that you could edit Roc scripts without recompiling the rust project at all?

Dan G Knutson (Oct 14 2024 at 01:27):

Like you could have global roc heap in the equivalent of a Bevy Resource, and a bunch of entity-specific roc heaps so that you actually get (some of) the benefit of the ECS. I'm thinking you'd have some kind of standard default script system that would have an already-compiled-in way for roc to touch the rapier components.

Zicklag (Oct 14 2024 at 03:26):

I'm not very familiar with how Roc code is executed but roughly it sounds like that would work.

Zicklag (Oct 14 2024 at 03:26):

Zicklag (Oct 14 2024 at 03:27):

Then we we have systems that will go and run the lua code at certain times, such as update, pre-update, etc.

Brendan Hansknecht (Oct 14 2024 at 03:41):

Yeah, you would just be loading a shared library and calling into it most likely

Alexander Pyattaev (Oct 14 2024 at 04:46):

That is exactly how most such systrns work. Except you generally have a whole buch of things you may want to call in the plugin. And you generally want to allow the plugin to somehow generate effects in the engine and call helper functions there to perform compute work that may be specific to your app.

Brendan Hansknecht (Oct 14 2024 at 04:56):

Yeah, platform apis can be complex for plugins cause they may be overly dynamic. In roc, that may turn into talking via some sort of tagged type that can represent any data. Cause that would be needed for type safety. That or you have to expand glue to generate part of the platform based on the api that the roc plugin requests ( this is what I think will end up working best for godot)

Dan G Knutson (Oct 14 2024 at 05:24):

Coming from some minimal hobby experience with bevy, the thing I was wanting to avoid is generating/writing rust structs that have to be compiled when making changes roc-side (or giving up type safety). Which might be completely unreasonable. Roc can't do the "it's just an asset" thing really, but maybe you can make it feel like that if the stuff you usually want to do is all in the glue, and you don't have to like, run code gen or macros on a per-query-type basis.

Brendan Hansknecht (Oct 14 2024 at 05:37):

Yeah, I understand that sentiment. Due to being strongly typed (and not the most friendly api), I only can see 2 ways to really handle this in roc:

I guess 3 would be to manually deal with mapping, but then writing roc also means writing rust wrappers or whatever platform language for the plugin

Alexander Pyattaev (Oct 14 2024 at 06:17):

There is another option. You can slap a proc-macro on all the structs you need, which would produce roc signatures during compilation of the engine. Then a small build script to glue the whole thing into a coherent platform code. Finally, import all of that into roc plugin code and you are in business. Strict typing and full compatibility. The only issue will be that some rust datastructures may be mutable in a way that is not roc-friendly (such as hashmaps) so you would have to somehow work around that if you want mutable access.

Zicklag (Oct 14 2024 at 14:11):

In bones we already have a runtime reflection / schema system that is meant to let the scripting system bind to almost any structs stored in the ECS. You could use those runtime schemas to generate all of the Roc glue code that you need for each type.

Zicklag (Oct 14 2024 at 14:12):

In my recent Bevy meetup talk I gave an overview of how the schema system works:

Dan G Knutson (Oct 14 2024 at 15:05):

Does bones have a way to dynamically generate ECS queries? And is it possible to have the same component type go in different ECS storage based on some runtime thing? Like monsters with aggressive: true get moved to AggressiveMonsters to get packed together?

Zicklag (Oct 14 2024 at 15:19):

There is a way to generate ECS queries at runtime, but Bones is a rather simplistic ECS and doesn't have different kind of storages right now.

I think we will migrate to a sparse-set based storage later, but right now each component type has it's own storage which is ( sort of ) a sparse Vec<MaybeUninit<Component>>> and a bitmap that has a bit set for every entity ID that has that component.

Queries such as "all entities that have component A and not component B" can be created just by doing bitset and and not operations on the bitsets for the global Entities resource, the A component bitset and the B component bitset.

Dan G Knutson (Oct 14 2024 at 15:33):

Sweet! I'm thinking of how possible it is to get an ECS-focused Roc scripting API that could be the main home of a game (rather than just modding). Dynamic queries sounds like it means you could define new query types in Roc without a codegen step. Storage-wise, you could do what I said earlier, and have like a GlobalRocArena resourcey thing and an EntityRocArena component, and then query for everything with an EntityRocArena. The ECS storage based on flags was because I was thinking that an EntityRocArena could expose something like a set of roc type ids to bones for querying (so you wouldn't have to ask for all scriptable objects).

Dan G Knutson (Oct 14 2024 at 15:34):

Thanks for showing up and answering questions! I'm excited about Bones because it seems like a nice fit for the kind of games I want to make.

Zicklag (Oct 14 2024 at 15:34):

Zicklag (Oct 14 2024 at 15:35):

I've also been quite interested in the possibility of using Bones as an engine for games written completely using the scripting system.

Zicklag (Oct 14 2024 at 15:38):

The ECS and schema system is also designed so that you can create runtime defined components.

Dan G Knutson (Oct 14 2024 at 15:39):

Zicklag (Oct 14 2024 at 15:40):

I'm not sure exactly how Roc types / structs work, but the idea is that you could have a struct, and you create a description of it, like it's called Pos and has two f32 fields named x and y, and you register that Schema with the system and get back a unique-for-the-process SchemaId.

Zicklag (Oct 14 2024 at 15:40):

You are then allowed to add/read/write Pos components for any entities in the ECS.

Zicklag (Oct 14 2024 at 15:40):

Zicklag (Oct 14 2024 at 15:41):

Zicklag (Oct 14 2024 at 15:42):

So if you created a component in Roc, and added it to an entity, a Lua script could import that Roc component by name, and then access it's fields using it's knowledge of the schema to automatically resolve field names like x and y.

Zicklag (Oct 14 2024 at 15:43):

And that's how we expose the core types and assets that are written in Rust to scripts, too. Each Rust struct derive's the HasSchema trait, which allows it to be stored in the ECS, and if you additionally have the #[repr(C)] annotation on the Rust struct, then it includes the memory layout info so that scripts can access it's fields.

peeps (Oct 14 2024 at 15:46):

Bones seems very attractive but I just dont see myself not using Bevy at this point. :/

Dan G Knutson (Oct 14 2024 at 15:46):

I think exposing the fields from Roc structs/records to bones would be tricky, but you could make a lot of game without that.

Zicklag (Oct 14 2024 at 15:47):

That means, you say, "here's the size and alignment of the data I need to store in the ECS, and that's all I'm telling you about it".

Zicklag (Oct 14 2024 at 15:48):

This means that only Roc would be able to access the data, because only it knows what it is, but that can be perfectly fine for many use-cases.

Zicklag (Oct 14 2024 at 15:48):

Zicklag (Oct 14 2024 at 15:49):

This allows any Rust struct to implement HasSchema, because by default it can just be opaque, and the schema system just makes sure that you never cast it to anything other than it's correct Rust type. ( Assuming you don't use any unsafe APIs. )

Zicklag (Oct 14 2024 at 15:54):

So, without understanding Roc hardly at all, if I were to structure it kind of similar to how I did the Lua integration, I would think it'd be something like this:

Zicklag (Oct 14 2024 at 15:55):

You could come up with any other way that you wanted to control when the Roc scripts are run, though.

Dan G Knutson (Oct 14 2024 at 15:55):

Bevy is really cool! But the two kinds of games I'm interested in are 2D single-player strategy stuff, and the kinds of rollback-based 2D multiplayer that Bones is made for. If I want 2-player Celeste physics, then the bevy scheduler stuff feels like a net negative vs "write the steps in order". Very different than a AAAish thing where you'd have background tasks and GPU state or whatever to worry about. Like, bevy is giving up determinism to get parallelism, but I want the opposite.

Zicklag (Oct 14 2024 at 15:57):

Yeah, Bevy is much more advanced, which can be useful for some games. And Bones is a little rough around the edges.

Bones's super power is probably being relatively simple. We're also hoping to make it nicer to use as we find time, so that other people can get the most out of it without as much learning curve, too.

Zicklag (Oct 14 2024 at 15:59):

We recently had the first person outside of our core team start seriously making a game with it, and they've had some great ideas and input.

Zicklag (Oct 14 2024 at 15:59):

It looks like it's feasible for us to really zero in on the easy small-ish network-multiplayer enabled game niche.

peeps (Oct 14 2024 at 16:01):

Yeah i can definitely see its potential. im just a big fan boy and my current serious game is already significantly completed in Bevy so porting it to Bones for the scripting is just not gonna happen. I have no idea what kind of hurdles il face with Bones (especially since its a very new engine) so theres a chance id have to retreat back to Bevy anyway. But its definitely on my radar now and one day I will revisit it. :+1:

Zicklag (Oct 14 2024 at 16:03):

Oh, yeah, if you've already got any significant amount of stuff in Bevy, you most likely want to stay there.

There are things that we've left out of Bones for simplicity that we haven't found we've needed, but when you are already using them, might be difficult to get away from.

Zicklag (Oct 14 2024 at 16:03):

peeps (Oct 14 2024 at 16:03):

I think its really interesting and i look forward to seeing how it evolves! I have to admit tho, a part of me wished u had used ur energy to like fork Bevy or something so its still Bevy but with the same scripting powers, lol. Please dont take that the wrong way. Im not trying to imply u made a bad decision. Just would have been nice for me specifically :)

Zicklag (Oct 14 2024 at 16:04):

The lack of events was due to trying to avoid re-allocating event queues on snapshot / restore. We might find a way to do this efficiently later, events can be handy.

peeps (Oct 14 2024 at 16:04):

Dan G Knutson (Oct 14 2024 at 16:04):

Zicklag (Oct 14 2024 at 16:05):

In fact I added the support for the web platform to it and did some other contribution, too.

Zicklag (Oct 14 2024 at 16:06):

The internals were pretty complicated and I didn't understand them all, and I'm not sure if it works with latest Bevy anymore, though.

Zicklag (Oct 14 2024 at 16:07):

But I think Bevy will definitely have the abilities necessary for scripting in the future, if it's not already there and just waiting for an update to a scripting integration.

peeps (Oct 14 2024 at 16:12):

Yeaaaah i dont want to get into it but there were some pain points for me specifically that dissuaded me. Im also interested in no-perf-compromise solutions which i dont think that crate provides (but correct me if im wrong). I have imagined a game that has no "core vs mods" structure. Everything is a mod, including the base code, but the modding needs to be near-native performance to achieve that for every game, not just the small ones. I just find a fully modular game like that interesting and I want to try it for a future game I have in mind.

Dan G Knutson (Oct 14 2024 at 16:15):

Fair enough; I haven't really used it. I had a maybe-similar disappointment where I got excited about the johan helsing bevy rollback / matchbox tutorials, and then it seemed like his own project descended into a debugging-non-determinism hell.

Zicklag (Oct 14 2024 at 16:17):

I'm trying to think back to it. :thinking: I think that for native ( non-web ) it was pretty close to being as performant as you could get with Bevy's reflection system. The thing that seemed to me that might be a performance limitation was having to make an indirect function call for every field read or write.

But like, as far as using JS for a scripting language at all, I don't know that that's really that bad. JS might be more of your problem than anything else there.

Zicklag (Oct 14 2024 at 16:17):

Zicklag (Oct 14 2024 at 16:18):

Calling out of WASM into the Browser's JS engine was really slow and you had to do it constantly.

Zicklag (Oct 14 2024 at 16:19):

yeah, that was part of the motivation for making schemas instead of Box<dyn Reflect> objects like in Bevy.

Zicklag (Oct 14 2024 at 16:19):

If you needed maximum performance it seemed like the best way to do it would be to cast a raw pointer to a matching type in your script so that you could operate on it directly without calling functions.

Zicklag (Oct 14 2024 at 16:20):

But I'm not actually sure if that's truly going to be able to unlock the perf advantages I may have imagined. I just wanted to make sure I didn't build a bottleneck into the system that would hurt me later.

Zicklag (Oct 14 2024 at 16:21):

Scripting has been a big goal of mine since I got into making games after modding the old Star Wars Battlefront games.

Zicklag (Oct 14 2024 at 16:21):

And I couldn't stand that there was often a division between the "game" and the "mods", so that the mods could only work on the small pieces of the game that the game exposed to them.

peeps (Oct 14 2024 at 16:22):

Yeah its stuff like that that can turn off some people, i.e. someone trying to make a crazy 3D game with a big simulated world (with multiplayer co-op). In those cases everything starts to add up. But for a 2D turned based game? JS is perfectly fine. Bones is interesting cus its concerned with the scripting stuff right off the bat, so theres a potential there to achieve that dream lol.

Zicklag (Oct 14 2024 at 16:22):

This created a divide between the game makers who could work on the "deep arts" of the game, and on some games left the modders wishing the game developers could give them more access.

Zicklag (Oct 14 2024 at 16:23):

That's one reason I went to ECS. I was thinking, "having to make function call bindings manually to all aspects of the game is always going to make modders second class citizens".

Zicklag (Oct 14 2024 at 16:23):

Zicklag (Oct 14 2024 at 16:24):

So if we can generate automatic bindings to the data we can let mods do almost anything.

Zicklag (Oct 14 2024 at 16:24):

It's a lot better anyway, and doesn't take a large amount of effort by the game maker in many cases.

Zicklag (Oct 14 2024 at 16:25):

I like that mods can do things in the game that the game developer hasn't even thought of yet.

Zicklag (Oct 14 2024 at 16:25):

Just by allowing them to stick their hands right into the game and mess with all kinds of stuff. :grinning_face_with_smiling_eyes:

peeps (Oct 14 2024 at 16:26):

Yess exactly. This is the exact situation i been trying to avoid with my current game. I recently scrapped my last idea on how to mod it cus i just didnt like it, but it had the potential to avoid this problem by essentially exposing everything.

I actually realized something like that pretty recently. When i was thinking about the modding of my current game, i realized that were some things that bevy just took care care of for me. For example, i dont need to code anything to allow someone to insert a system into a schedule or before/after a specific system. Bevy just lets u do it. So part of the modding api was just bevy being bevy lol.

Zicklag (Oct 14 2024 at 16:31):

Yeah, there is some question as to how performant the Bones ECS itself can be, since it's really pretty simple, but we also realized that having the ECS itself be super parallel and performant may not really be the best way to make advanced performant games all the time anyway.

It's got very performance intensive simulations, but all that has to be parallelized and optimized independent of the ECS anyway. If you've got large simulation structures and such, you could possibly optimize it better by storing it in a data structure optimized for that simulation.

And even though the rest of bones isn't really focused on multi-threading, we did leave the possibility open for multi-threaded schedulers that work similar to Bevy.

And the renderer can be multi-threaded without the rest of the game being multi-threaded, too.

Zicklag (Oct 14 2024 at 16:33):

Bevy itself will undoubtedly be more performant that Bones in probably any metric ( other than maybe scripting ), but we're not yet sure if that actually means you can't make at least semi-performance-critical games in bones. :shrug:

Zicklag (Oct 14 2024 at 16:35):

And we're of course still interested in improving performance where it helps, without making things overly complicated.

Bones can be reasonably understood in completeness by one person right now, which is important to me.

peeps (Oct 14 2024 at 16:45):

Well based on all that, it seems you got ur head screwed on straight, lol. Bones seems to be in capable hands :+1:

Dan G Knutson (Nov 12 2024 at 03:15):

I'm curious what this plugin use-case looks like post-purity-inference. In particular, I'd like to find a way for a platform using the embedded/plugin workflow to load a compiled roc dynamic library (built with --no-link) at runtime. It looks like the rust crates people use for this kind of thing are libloading and pluginator, but I'm not sure how to go about making the roc_fx functions available to the loaded library.

Richard Feldman (Nov 12 2024 at 03:18):

we actually want to simplify how that works too, by changing it so that you pass those in as a struct of function pointers when you call the compiled Roc function

Richard Feldman (Nov 12 2024 at 03:19):

and then the compiled Roc code automatically passes a pointer to that struct of function pointers around to all the inner Roc calls so they can all access them

Richard Feldman (Nov 12 2024 at 03:19):

so then there's no two-way linking needed, you just link in the Roc function and call it passing a pointer to an ordinary struct of function pointers for roc_alloc and friends

Richard Feldman (Nov 12 2024 at 03:20):

it definitely simplifies that boundary a lot, but it's also a nontrivial project to implement :big_smile:

Zicklag (Nov 12 2024 at 03:22):

Oh, that would be so much nicer than having to double-link. Currently do you have to actually load the roc shared library, and then have the roc shared library load the host library ( or something like that )?

Richard Feldman (Nov 12 2024 at 03:28):

Dan G Knutson (Nov 12 2024 at 03:30):

Is there a way to do that double-linking at runtime today with libloading? Or would it boil down to some kind of manually-passed-around record of effectful functions?

Brendan Hansknecht (Nov 12 2024 at 03:31):

Currently, the plugin shared library just expects to be able to grab existing exposed functions in the host for each of the roc_ platform functions. Depending on your link settings, this may or may not work. It is kinda like taboo instead of proper shared library use. The shared library is just grabbing and calling functions out of the host. So a circular dependency between the host and shared library. It is bad form, but generally just works (though tends to break with musl build)

Brendan Hansknecht (Nov 12 2024 at 03:31):

The future will be a manual made record of function pointers passed from the host to roc

Dan G Knutson (Nov 12 2024 at 03:32):

ah, I'm learning things about linking. the current method is relying on name resolution/conventions, so we'd expect it to 'just work' (even loading a library at runtime) without passing the effectful functions around?

Dan G Knutson (Nov 12 2024 at 03:34):

like, does a platform loading a compiled roc dynamic library at runtime actually need to do much of anything differently?

Brendan Hansknecht (Nov 12 2024 at 03:38):

It just needs to make sure the functions that roc needs are dynamically exposed. Otherwise when roc calls them, name resolution will fail.

Brendan Hansknecht (Nov 12 2024 at 03:39):

In a lot of programming languages with default compiliation setup, this just means ensuring that the functions don't get cleaned up due to dead code elimination. In some cases, it means jumping through a few extra hoops to force the linker to keep the function dynamically exposed.

Luke Boswell (Nov 12 2024 at 06:08):

I thought the last time we discussed it, the main blocker for upgrading to have passed in allocators was module params and purity inference. Is this unblocked now if someone was motivated to implement it? It's basically just passing an extra parameter to every function in a module (a record with the roc_alloc, roc_dbg function etc).

Brendan Hansknecht (Nov 12 2024 at 06:14):

It would also include the available effects, but yeah, I think it could be done now if someone is motivated.

Brendan Hansknecht (Nov 12 2024 at 06:15):

Implicit param to all functions plus automatic dispatch on any effect call or builtin call to a special roc method like allocators

Zicklag (Nov 12 2024 at 14:39):

Ah, OK, cool, that doesn't sound as bad as I thought, then. I think that should be perfectly usable enough.

Stream: beginners

Topic: Plugins in roc

Alexander Pyattaev (Dec 15 2023 at 08:18):

Hannes (Dec 15 2023 at 10:35):

Hannes (Dec 15 2023 at 10:37):

Luke Boswell (Dec 15 2023 at 10:40):

Alexander Pyattaev (Dec 15 2023 at 11:50):

Richard Feldman (Dec 15 2023 at 11:55):

Richard Feldman (Dec 15 2023 at 11:56):

Richard Feldman (Dec 15 2023 at 12:01):

Alexander Pyattaev (Dec 15 2023 at 12:03):

Richard Feldman (Dec 15 2023 at 12:25):

Alexander Pyattaev (Dec 15 2023 at 12:33):

Alexander Pyattaev (Dec 15 2023 at 12:34):

Richard Feldman (Dec 15 2023 at 13:26):

Vladimir Zotov (Dec 15 2023 at 15:08):

Brendan Hansknecht (Dec 15 2023 at 15:59):

Alexander Pyattaev (Dec 16 2023 at 18:55):

Alexander Pyattaev (Dec 16 2023 at 18:57):

Luke Boswell (Dec 16 2023 at 21:17):

Luke Boswell (Dec 16 2023 at 22:28):

Alexander Pyattaev (Dec 17 2023 at 06:34):

Vladimir Zotov (Dec 19 2023 at 14:57):

Vladimir Zotov (Dec 19 2023 at 15:07):

Alexander Pyattaev (Dec 20 2023 at 21:59):

peeps (Dec 21 2023 at 01:42):

Luke Boswell (Dec 21 2023 at 04:04):

Luke Boswell (Dec 21 2023 at 04:07):

peeps (Dec 21 2023 at 04:25):

peeps (Dec 21 2023 at 04:28):

Luke Boswell (Dec 21 2023 at 04:29):

peeps (Dec 21 2023 at 04:31):

Luke Boswell (Dec 21 2023 at 04:32):

Luke Boswell (Dec 21 2023 at 04:35):

Luke Boswell (Dec 21 2023 at 04:36):

peeps (Dec 21 2023 at 04:38):

Brendan Hansknecht (Dec 21 2023 at 05:02):

Luke Boswell (Dec 21 2023 at 05:04):

Vladimir Zotov (Dec 21 2023 at 09:40):

Luke Boswell (Dec 21 2023 at 09:44):

Alexander Pyattaev (Dec 21 2023 at 13:57):

Brian Carroll (Dec 21 2023 at 14:06):

peeps (Dec 21 2023 at 15:02):

Alexander Pyattaev (Dec 21 2023 at 16:10):

Brian Carroll (Dec 21 2023 at 17:03):

Brendan Hansknecht (Dec 21 2023 at 17:10):

Luke Boswell (Dec 21 2023 at 17:39):

Alexander Pyattaev (Dec 21 2023 at 19:08):

Luke Boswell (Dec 21 2023 at 19:39):

Alexander Pyattaev (Dec 21 2023 at 19:45):

Brendan Hansknecht (Dec 21 2023 at 20:22):

Brendan Hansknecht (Dec 21 2023 at 20:23):

Alexander Pyattaev (Dec 21 2023 at 20:24):

Brendan Hansknecht (Dec 21 2023 at 20:25):

Brendan Hansknecht (Dec 21 2023 at 20:26):

Alexander Pyattaev (Dec 21 2023 at 20:27):

Brendan Hansknecht (Dec 21 2023 at 20:28):

Alexander Pyattaev (Dec 21 2023 at 20:31):

Brendan Hansknecht (Dec 21 2023 at 21:10):

Brendan Hansknecht (Dec 21 2023 at 21:10):

Brendan Hansknecht (Dec 21 2023 at 21:11):

Alexander Pyattaev (Dec 21 2023 at 21:44):

Alexander Pyattaev (Dec 21 2023 at 21:45):

Luke Boswell (Dec 21 2023 at 22:02):

Luke Boswell (Dec 21 2023 at 22:03):

Steven Chen (Dec 21 2023 at 22:18):

Vladimir Zotov (Dec 21 2023 at 22:28):

Luke Boswell (Dec 22 2023 at 09:20):

Alexander Pyattaev (Dec 22 2023 at 10:38):

peeps (Dec 22 2023 at 14:58):

Alexander Pyattaev (Dec 22 2023 at 19:50):

Alexander Pyattaev (Dec 22 2023 at 19:53):

Brendan Hansknecht (Dec 22 2023 at 20:03):

Brendan Hansknecht (Dec 22 2023 at 20:04):

Brendan Hansknecht (Dec 22 2023 at 20:05):

Brendan Hansknecht (Dec 22 2023 at 20:05):

Brendan Hansknecht (Dec 22 2023 at 20:05):

Brendan Hansknecht (Dec 22 2023 at 20:06):

Brendan Hansknecht (Dec 22 2023 at 20:07):

Alexander Pyattaev (Dec 22 2023 at 20:09):