Adding external libraries · beginners

Stream: beginners

Topic: Adding external libraries

Aurélien Geron (Jul 30 2024 at 00:35):

IIUC, adding external packages is done by simply adding a URL to the app header, right? Something like this:

app [main] {
    pf: platform "https://github.com/roc-lang/basic-cli/releases/download/0.10.0/vNe6s9hWzoTZtFmNkvEICPErI9ptji_ySjicO6CkucY.tar.br",
    weaver: "https://github.com/smores56/weaver/releases/download/0.3.0/GJtHjepVuDUSzWogFflwMiqPnCV5FKR81722WVmUC5E.tar.br",
    # ... other packages
}

A few related questions:

Is there a tool similar to pip or cargo, to help find, install, update, and remove libraries? If not, is there any plan for that?
Is there an official list of packages, similar to pypi.org or crates.io? I've found https://hasnep.github.io/roc-packages/ but it's not official. If not, any plan for that?
Suppose I want to write a program in Roc that will use a non-Roc library Foo (e.g., SDL). If I'm lucky someone has already written a platform for that, and if not I can write my own. So far so good. But then suppose I also want to use the basic-cli platform in the same program. Should I fork basic-cli and somehow merge it with the Foo platform? Basically, I'm confused as to how Roc code can access non-Roc libraries in a flexible way.

Perhaps the answers to the above questions could be added to the tutorial or the FAQ?
Thanks for your help.

Brendan Hansknecht (Jul 30 2024 at 01:42):

Is there a tool

Is there an official list of packages

nope

If not, any plan for that?

I think we may have docs with rough plans and overall it is definitely wanted, but nothing concrete to my knowledge

Brendan Hansknecht (Jul 30 2024 at 01:43):

Should I fork basic-cli and somehow merge it with the Foo platform?

Forking basic cli and adding new libraries is recommended. Merging platforms isn't necessarily a thing, but you definitely could be a platform with all the effects of two other platforms.

Trevor Settles (Jul 30 2024 at 01:45):

The closest thing we have to a full list is Roc Awesome

Brendan Hansknecht (Jul 30 2024 at 01:52):

I'm confused as to how Roc code can access non-Roc libraries in a flexible way.

It doesn't for the most part.

Part of the design for Roc is trying to enable platform authors to build elm like experiences. They are meant to be targeted to a specific audience. As such the author has to decide what to include and what to ban. Once that is decided, they only way to change it is to fork (or PR) the platform.

If platforms wanted to be composable, it would have to be done on the host language side. In rust, we could make a package that is the basis for many effects. I could be shared between basic-cli and basic-webserver and others for example.

Otherwise, a platform technically could allow for abitrary ffi via shared libraries and libffi. But that would be up to each individual platform to decide to do.

Brendan Hansknecht (Jul 30 2024 at 01:54):

As an extra note, roc never plans to add a native escape hatch. The library ecosystem will remain 100% roc code. That said, it is allowed to generate effects. So a postgres library might require TCP primitives and would create tasks to interact with a postgres server.

Brendan Hansknecht (Jul 30 2024 at 01:56):

So a lot of composibility for libraries in Roc will come from multiple platforms sharing a few basic primitives (like tcp or file io). Then the roc libraries will build more complex tasks off of the primitives.

Aurélien Geron (Jul 30 2024 at 02:12):

Thanks for the clear answers. The design decision of having no escape hatch for Roc code is really interesting. I can see the benefits but I think it must go hand in hand with super easy composability at the platform level. E.g., suppose I want to use TensorFlow in a Roc project: I would fork basic-cli (or other platform) and do the work of wrapping TF's C++ API (plus CUDA). It's probably quite a lot of work to wrap TF, however, so I would probably share my basic-cli-with-tf platform on Github. But then someone else comes along and needs TF + A + B + C, and someone else needs TF + C + D + E, and someone else needs TF + A + E + F, and so on. If there's too much work involved in this kind of composition, I think it might be a blocker for many people. That said, I'm completely new to Roc, so perhaps I'll have a different perspective once I use it more.

Brendan Hansknecht (Jul 30 2024 at 02:51):

Yeah, the combinatorial platform problem will definitely be interesting to see how it pans out in practice. My main thoughts today:

For more specialized platforms that are trying to be a full batteries included framework, I don't think this matters much
For super common platforms like basic-cli will likely support a handful of common configs
Platform development will get a lot easier over time: better glue support, libraries in host languages for specific effects (could have an SDL effect library that could be important into any platform).
Forking is probably fine for many use cases. Having basic-cli + A + E + F is just fine. Shouldn't even be a big deal to update to new version of basic-cli over time. Just pull in the update, your change is only expanding a tag union and adding some extra dispatch.
Roc is pretty fast an efficient. For many classes of work, code can be ported to roc on slim primitive making it easy to share between platforms.
Though roc doesn't have an escape hatch, a platform can choose to have an escape hatch. Assuming none of the above works out for certain use cases, a platform escape hatch might become common for specific subsets of roc platforms.

That said, things like TF will always be a hard problem. Using TF in rust is a hard problem and it has full ffi support. The tensorflow in rust library is 368,108 lines of code (hopefully essentially all autogenerated, but I didn't dig in).

Aurélien Geron (Jul 30 2024 at 03:06):

Great answer, thanks Brian. The key will be in the tooling, the ecosystem, and the documentation. The various cases you listed should be as frictionless as possible. To be honest, this is such a fundamental design choice, I think it should be addressed upfront on the website, the pros and cons should be laid out clearly, the various scenarios you listed should be presented clearly, and the plan towards better platform composability should be laid out, or else newbies like me will be scratching their heads, since it's quite unusual for a general purpose language. To be fair, some parts of this are clearly stated on the website, but I for one didn't grasp the importance of it until now. WDYT ?

Aurélien Geron (Jul 30 2024 at 03:06):

Happy to help btw

Brendan Hansknecht (Jul 30 2024 at 04:20):

Yeah, I definitely think platforms could use more details on the website. Specifically something less focused on how they function and more focused on the implications for the ecosystem (including the goal to only allow pure packages but how we have tools to make those generate tasks nonetheless).

I think part of the core is mossing cause a lot of this is still being figure out with major changes to platforms to come. Also, major tolling improvements. All a long journey.

Last updated: Jul 06 2025 at 12:14 UTC