Stream: beginners

Topic: Algebraic Effects

Aurora (Apr 11 2024 at 03:03):

so I remember seeing Richard's talk on using algebraic effects,, which was really exciting to see in a non-research language, I thought it was a really elegant way to incorporate this capabilities based paradigm with functional ideas.. but looking at the docs as they stand today it seems like that was moved away from? or that the task system is more opaque now? and proposals linked on the docs page seem to break the promises of the platform based architecture... if y'all did indeed move away from algebraic effects, what was the rationale behind that decision? what were the pain points of that API?

Brendan Hansknecht (Apr 11 2024 at 03:06):

You're talking about the 3 parameter version of Task that has a 3rd arg that tracks what effects can be run?

Task okType errType [Stdout, Stdin, EtcIO]

Brendan Hansknecht (Apr 11 2024 at 03:17):

The spiritual successor proposal to that is the module params proposal. Less verbose, though also less precise, but still affords many similar benefits. It is currently being implemented, but is not complete yet.

Aurora (Apr 11 2024 at 03:33):

I guess I was asking why module params is the choice over 3 param tasks, to me 3 param tasks (or 2 params with a return type that can be a result) are exactly what I want from an effect system, I want Effectful functions to be explicit, I want that verbosity (although that verbosity isn't mandatory with full type inference). I want the compiler to tell me "you're using IO in this function so your type signature is wrong". I'm sure y'all have probably been over these discussions hundreds of times already which is why I'm sure y'all have a reason for picking the module params option, but from an outsider perspective it seems like a much less elegant and expressive API?

Richard Feldman (Apr 11 2024 at 13:30):

a few thoughts!

Richard Feldman (Apr 11 2024 at 13:30):

we've never done algebraic effects in Roc - Unison does them but we've always used Task instead

Richard Feldman (Apr 11 2024 at 13:32):

Aurora said:

I want the compiler to tell me "you're using IO in this function so your type signature is wrong".

Task accomplishes this; there's no way to do I/O in Roc, other than to have a function return a Task - so there are no side effects in Roc (with the exception of the dbg and expect keywords, which can potentially print things out), just managed effects

Richard Feldman (Apr 11 2024 at 13:33):

an example of 2-arg Task is something like File.readBytes; its 2 type parameters are the type the Task gives you if it succeeds (List U8 in the case of File.readBytes) and the type it gives you if the task fails ([FileReadErr Path ReadErr] in the case of File.readBytes)

Richard Feldman (Apr 11 2024 at 13:34):

3-arg Task was something we tried out where we had a third type parameter which was just for tracking separate information about what specifically the effect was doing

Richard Feldman (Apr 11 2024 at 13:35):

for example, instead of File.readBytes returning a Task (List U8) ReadErr (2-arg design) it might instead return Task (List U8) ReadErr [FilesystemRead] (3-arg design)

Richard Feldman (Apr 11 2024 at 13:37):

so then if you made a HTTP request task return [NetworkAccess] instead of [FilesystemRead], then if you did both a HTTP request as well as a filesystem read, you'd end up with a Task whose third argument included both of those: [NetworkAccess, FilesystemRead] and that way you'd be able to see in the type that "this task is both accessing the network and also reading from the filesystem"

Richard Feldman (Apr 11 2024 at 13:37):

and you'd also know that the task is not writing to the filesystem, because if it were, then FilesystemWrite would have been a part of that type

Richard Feldman (Apr 11 2024 at 13:38):

so that's the upside of that design

Richard Feldman (Apr 11 2024 at 13:38):

a downside is that you end up having to thread that information through everywhere, even in places where you don't really care

Richard Feldman (Apr 11 2024 at 13:40):

it's easy to think "I want that everywhere!" but then in practice if you end up with a bunch of little helper funcions that are now returning Task Str [] [FilesystemRead, FilesystemWrite, NetworkAccess, Stdout, Stderr, Stdin] then pretty quickly you either start wanting to make a type alias or else just write Task Str [] _ so you don't have to see all that in the source code (although whenever you write _ the compiler keeps tracking the type, it's just hidden in that particular annotation)

Richard Feldman (Apr 11 2024 at 13:40):

another downside is that it has limited expressivity

Richard Feldman (Apr 11 2024 at 13:41):

for example, there's no way to say "this task only writes to subdirectories of ./some/path/blah on the filesystem"

Richard Feldman (Apr 11 2024 at 13:42):

or "this task only connects to the database I've named foo" or "this task only does network requests to my-error-reporting-service.com and my-analytics-service.com but it doesn't do network requests to any other domain"

Richard Feldman (Apr 11 2024 at 13:43):

the design we're moving towards gives us the ability to express those more granular constraints, but not using Task itself (which is why it no longer needs the third argument)

Richard Feldman (Apr 11 2024 at 13:46):

so first I'll explain the design without module parameters, and then explain how module parameters make the design nicer

Richard Feldman (Apr 11 2024 at 13:50):

the basic idea is to express all I/O operations (e.g. Http.getUtf8) in terms of some low-level I/O primitive (e.g. Http.getUtf8 is a wrapper around Http.request), and that functions like Http.getUtf8 don't actually internally know what that low-level operation is. Instead, wrapper functions like Http.getUtf8 all get an additional argument - so when you call Http.getUtf8 you have to pass in Http.request as an argument to it, because it doesn't actually know how to do a HTTP request otherwise; it just knows how to call the Http.request function you're providing it to configure it for UTF-8 and doing a HTTP GET

Richard Feldman (Apr 11 2024 at 13:50):

(that probably sounds very un-ergonomic, which it would be! Module params solve that ergonomics problem, as we'll see)

Richard Feldman (Apr 11 2024 at 13:52):

importantly, in the new design, the way you get access to I/O primitives is that the lowest-level ones essentially get passed in from the platform to the application's main as arguments

Richard Feldman (Apr 11 2024 at 13:52):

so like main actually receives the low-level Http.request (and other I/O primitives) as arguments at runtime

Richard Feldman (Apr 11 2024 at 13:52):

and - this is also very important - there is no way to just import Http.request directly

Richard Feldman (Apr 11 2024 at 13:53):

the only possible way to get access to any function that actually returns a Task is by asking for it as an argument

Richard Feldman (Apr 11 2024 at 13:53):

and then threading it through by argument passing

Richard Feldman (Apr 11 2024 at 13:53):

(again, the ergonomics sound terrible without module params!)

Richard Feldman (Apr 11 2024 at 13:53):

this has several nonobvious benefits

Richard Feldman (Apr 11 2024 at 13:53):

one is that, since all I/O operations are now threaded through the whole program as functions being passed as arguments, you are always free to pass different functions

Richard Feldman (Apr 11 2024 at 13:54):

for example, instead of passing a real Http.request, in a test I can pass in a fake one that simulates whatever I want it to simulate - e.g. that the server timed out, or returned a 500, etc. My test doesn't need to run any real I/O, try to contact a real server, etc.

Richard Feldman (Apr 11 2024 at 13:56):

and since all effects in the code base are necessarily being passed around as arguments (again, in this design there is no way, at a language level to directly import a real Task in an application - literally the only possible way to obtain one is by having it passed into main as an argument and then threaded through everything else, and the same is true of packages)

Richard Feldman (Apr 11 2024 at 13:56):

...we know that the simulation will work everywhere, and there will never be any real HTTP done in that test

Richard Feldman (Apr 11 2024 at 13:56):

(this is a difference between this design and, say, mocking/monkey patching in other languages)

Richard Feldman (Apr 11 2024 at 13:56):

another thing we get is that platform-agnostic packages become very straightforward

Richard Feldman (Apr 11 2024 at 13:57):

since packages all need to say "pass me in a Http.request function because otherwise I have no possible way to do HTTP"

Richard Feldman (Apr 11 2024 at 13:58):

you can have different platforms implement that function completely differently under the hood, but as long as they have the same type and work the same way, your application can pass them along to packages and the package neither knows nor cares that the particular Http.request it's receiving came from a different platform

Richard Feldman (Apr 11 2024 at 13:58):

I said earlier that 3-arg task design couldn't do this:

Richard Feldman said:

or "this task only connects to the database I've named foo" or "this task only does network requests to my-error-reporting-service.com and my-analytics-service.com but it doesn't do network requests to any other domain"

Richard Feldman (Apr 11 2024 at 13:58):

this design can absolutely do that!

Richard Feldman (Apr 11 2024 at 13:59):

the way it works is, if I have a function which asks for Http.request as an argument (which tells me at the type level that the function does HTTP, just like how [NetworkAccess] did in 3-arg task, except in a different part of the type)

Richard Feldman (Apr 11 2024 at 13:59):

I don't have to give that function a real Http.request

Richard Feldman (Apr 11 2024 at 14:00):

I can give it a wrapper around Http.request which only permits contacting a particular domain - essentially, I'm giving it "sandboxed access" to the network

Richard Feldman (Apr 11 2024 at 14:02):

so now if I'm using a 3rd-party error logging package for bugsnag.com, which requests Http.request so that it can do HTTP, I can give it a sandboxed function which only ever accepts URLs to bugsnag.com, which means that if someone malicious compromises that package and publishes a new release which sends data to stealyourdata.com instead of bugsnag.com, my data won't end up going there because the package has literally no way to do HTTP other than the wrapped Http.request I gave it which refuses to talk to anything but bugsnag.com

Richard Feldman (Apr 11 2024 at 14:03):

anyway, I gotta run, but - the linked doc from earlier explains how module params preserve all of these benefits without the ergonomics downside of having to pass these through as arguments to functions (instead they get passed through module imports and the functions themselves don't look like they're getting extra arguments)

Richard Feldman (Apr 11 2024 at 14:03):

but if you really wanted that granularity, you could choose to do it at the function level like I've described here - although personally I think it would be more ergonomic to use module params :big_smile:

Brendan Hansknecht (Apr 11 2024 at 14:34):

:point_up: this is a really detailed explanation. We should capture it somewhere for future reference.

Isaac Van Doren (Apr 11 2024 at 20:01):

Yes this was a great read!

Aurora (Apr 12 2024 at 01:50):

thanks! this explains the design decision in a very clear way for someone who's not familiar with the details of the implementation or the development history! :)

Brendan Hansknecht (Apr 12 2024 at 01:57):

One interesting side note:

Technically speaking, you could have both. You could have 3 arg tasks and still use module params. That said, we plan to make task a builtin module eventual and decouple it from individual platforms. When that happens, we will almost certainly lock in on 2 arg tasks, so the ecosystem would follow. A platform would have to roll their own task type to get 3 arg tasks (which a very security focused platform might).

Eli Dowling (Apr 23 2024 at 22:08):

I would love to hear a further look into the benifits and drawbacks of this system as opposed to an effects system like ocaml(once types effects arrives) or unison that you linked to?

A few basic arguments for I can think of:

To me it seems like you could implement the same pattern of http request limiting using:

let constrain_http func domain
try
func()
with
|(HttpRequest destination) as httpEfffect ->
   if destination|> Str.startswith domain then
       reraise HttpRequest httpEffect

(Wrote on my phone, syntax is probably slightly wrong but you get the idea)

It's also a lot more general than Task, meaning you can build your own amazing things on-top of effects systems like implementing a concurrency system as a library author rather than a compiler dev. (Eio and riot for Ocaml are both pretty wild examples of this.), or the dotnet Json parsing thing we discussed, recently where they halt and then resume parsing when more data is available. Dotnet does some manic storing and resuming a stack frame, but effects enable that easily.

It doesn't give you coloured functions. You don't have to constantly write async everywhere. This is largely solved by type inference for task, but it's even nicer because you don't even need Task.await or anything, you just call functions.

Effects allow embedding of stack traces in debug mode.

Some arguments for Task I can think of :

You can run simple functions on the errors like t|>Task.mapErr, whereas in an effect system you need a proper try with block around your statement.

I imagine it's easier to implement

As I said I'd love to hear some more reasons for Task .While I don't have a huge preference effect systems do seem slightly more powerful and ergonomic from my own (limited) useage.

Richard Feldman (Apr 25 2024 at 15:29):

I have various unstructured thoughts on this topic, but overall it pretty much boils down to:

algebraic effects are a new idea that's only very recently gone from research-only to being implemented in non-research languages, and unlike research-y behind the scenes optimizations (like how Roc compiles lambdas etc.) if it turns out to have practical problems that make it undesirable, taking it out of the language after the fact is a gigantic breaking change for every code base - whereas if a research-y behind-the-scenes optimization doesn't end up working out, you can always switch to a different optimization strategy (e.g. one that's not as fast but which is more battle-tested) without requiring a rewrite of the whole ecosystem

Richard Feldman (Apr 25 2024 at 15:30):

so basically, there might be benefits, but nobody knows what the unexpected surprises will be when you build an ecosystem on top of them, because they're so new that it has literally never happened before - there's no precedent to point at to say "that's what it looks like when an ecosystem is built on algebraic effects"

Richard Feldman (Apr 25 2024 at 15:30):

Unison's is the biggest in the world, by far

Richard Feldman (Apr 25 2024 at 15:31):

a second reason not to is that in general I have a strong preference for keeping the number of semantic primitives in Roc as small as possible.

Richard Feldman (Apr 25 2024 at 15:34):

the bar for introducing things that are effectively sugar for other things that already exist in the language (e.g. string interpolation, !, record builders, and module params are all language features that can be desugared into Roc code that compiles to the same thing; they improve ergonomics but don't fundamentally let you do something that couldn't be done before) is much lower than the bar for introducing things that are impossible to express in terms of something else

Richard Feldman (Apr 25 2024 at 15:34):

for example, Abilities were a new language primitive that couldn't be desugared into existing primitives, and the bar was really high for adding them to the language. We thought there would be enough benefits (e.g. automatic encoding/decoding, function equality becoming a compile-time error, Eq for custom data structures, more control over what opaque types do and don't support) to meet that bar

Richard Feldman (Apr 25 2024 at 15:35):

it seems like especially given ! the potential benefits of algebraic effects compared to Task are a lot less clear to me

Richard Feldman (Apr 25 2024 at 15:36):

Eli Dowling said:

It's also a lot more general than Task, meaning you can build your own amazing things on-top of effects systems like implementing a concurrency system as a library author rather than a compiler dev. (Eio and riot for Ocaml are both pretty wild examples of this.)

I don't think it's safe to assume this is a good thing :big_smile:

Richard Feldman (Apr 25 2024 at 15:36):

I don't have much familiarity with the OCaml ecosystem, but even before algebraic effects, a common complaint I'd heard is that there were multiple competing ways to do concurrency, and there's an ecosystem split around which ways to represent libraries which want concurrency. Same with Scala.

Richard Feldman (Apr 25 2024 at 15:37):

having one Task type that everyone uses, and where platform authors can implement domain-specific concurrency systems behind the scenes (as opposed to library authors doing the same) seems like it can get the performance benefits of the domain-specific concurrency system without the drawback of the ecosystem split. Or at least that's the theory! We'll see how it works out in practice.

Richard Feldman (Apr 25 2024 at 15:43):

Eli Dowling said:

You don't have to constantly write async everywhere. This is largely solved by type inference for task, but it's even nicer because you don't even need Task.await or anything, you just call functions.

when writing things like quick scripts without type annotations, I think having to write Stdout.line! compared to Stdout.line is an insignificant cost, and I actually like that it adds a visual marker of where all the effects are happening in that code!

Richard Feldman (Apr 25 2024 at 15:50):

another thing that falls under the category of "I'm not sure if that power is actually good" is that algebraic effects let you implement try/catch in userspace

Richard Feldman (Apr 25 2024 at 15:51):

which in turn means that now you can use algebraic effects for handling errors, or sum types (e.g. Result) and you may end up with a mix of them

Richard Feldman (Apr 25 2024 at 15:52):

at the end of that page, the Unison docs have a stylistic suggestion about how to resolve that:

As a rule of thumb we suggest you use the [algebraic effects] based approach for error handling

this seems like a reasonable stylistic guideline, but personally I don't like that it makes errors feel like effects

Richard Feldman (Apr 25 2024 at 15:53):

I personally prefer Result because it's not treated as an effect or exceptional, it's just "this operation can potentially fail, and we represent that using plain old data"

Richard Feldman (Apr 25 2024 at 15:54):

(I also like that Task is an opaque type just like Parser or Random.Generator, with the main difference between those and Result being that it's normal for them to store functions inside them whereas for Result that would be very uncommon)

Richard Feldman (Apr 25 2024 at 16:00):

anyway, to summarize these various thoughts:

• It's true that they let you do more things, but I think it's important to note that this power has both upsides and downsides, and algebraic effects haven't existed outside research languages for long enough that anyone can confidently know how those manifest in practice across an ecosystem (also worth noting that the upsides compared to Task seem mostly speculative - e.g. "____ is now possible, isn't that cool?" and less "here is a very specific thing that would definitely be an overall improvement")
• it would be a big risk for Roc to use them instead of Task because if we wanted to change that decision later, it would be a massive breaking change
• stylistically, I think it fits better with the "small number of simple language primitives" to set a high bar for features like this (ones which can't be expressed as sugar for existing features) and I don't think algebraic effects meet that bar

Eli Dowling (Apr 30 2024 at 23:46):

Thanks for writing out your ideas. I feel like that completes most of the common questions around "what is a task, and how does it relate to similar concepts" :).

A couple notes:
The reason I cited riot from Ocaml is that it implements the Erlang actor model which I'm unsure would be easily model-able with roc (though it would be interesting to try and would obviously be its own platform.)

Ocaml is very likely a much larger effects codebase than unison. They are used pretty widely, and I believe they were in janestreet's internal compiler for a long time.

Last updated: Jul 06 2025 at 12:14 UTC