Stream: ideas

Topic: Task example re-do

Luke Boswell (Nov 24 2023 at 00:32):

I want to re-write the Tasks example to be more helpful and to adress this Issue.

I am looking for feedback on my plan here before I submit a PR.

My plan is to

1. Re-write the example code to be one largish basic-cli app that kind of does the kitchen sink. Using something like the following template to show how you can easily bring all the tasks together.
2. Re-write the example README to be more like a tutorial and walk through the tasks one by one and explain what is happening with the Task and specifically focusing on the types success and error values.

app "task-usage"
    packages { pf: "https://github.com/roc-lang/basic-cli/releases/download/0.5.0/Cufzl36_SnJ4QbOoEmiJ5dIpUxBvdB3NEySvuH82Wio.tar.br" }
    imports [
        pf.Stdout,
        pf.Stderr,
        pf.Arg,
        pf.Task.{ Task },
    ]
    provides [main] to pf

main : Task {} *
main = run |> Task.onErr handleErr

AppError : [
    UnableToReadArguments,
    ArgumentNotProvided,
]

handleErr : AppError -> Task {} *
handleErr = \err ->
    msg = when err is
        ArgumentNotProvided -> "No argument provided, usage: roc run main.roc -- <argument>"
        UnableToReadArguments -> "Unable to read command line arguments"

    Stderr.line "ERROR: \(msg)"

run : Task {} AppError
run =
    # Read an argument
    arg <- readArgument |> Task.await

    # Print argument to stdout
    {} <- Stdout.line "Read argument: \(arg)" |> Task.await

    # Read an environment variable
    # Print environment variable to stdout
    # Read a file
    # Print file to stdout
    # Get UTC epoch
    # Print epoch to stdout
    # Fetch a website
    # Print website content to stdout
    # Read the contents of a directory
    # Print directory contents to stdout

    Task.ok {}

readArgument : Task Str AppError
readArgument =
    args <-
        Arg.list
        |> Task.onErr \_ -> Task.err UnableToReadArguments
        |> Task.await

    when args is
        [_, first, ..] -> Task.ok first
        _ -> Task.err ArgumentNotProvided

# TODO implement more tasks

Luke Boswell (Nov 24 2023 at 04:22):

So here is the WIP PR examples #112 :working_on_it: . I've completed the Task example code, here :octopus: .

Any feedback or comments would be most appreciated. :big_smile:

Luke Boswell (Nov 24 2023 at 04:24):

Note, the happy path without DEBUG=1 just prints "Completed" to keep the CI expect script simple.

Luke Boswell (Nov 24 2023 at 08:39):

One thing I am wondering about is what should the return type for the "tasks" be. I can see two viable options currently.

1. Common Type e.g. readUrlArg : Task { url: Str, path: Path } Error
2. Individual Tags e.g. readUrlArg : Task { url: Str, path: Path } [UnableToReadArgs]_

I've used the first option here as that was the first thing I thought of, but I'm not sure if that is the best option.

The second feels like it would be clearer and more testable if/when we can simulate Tasks in future, though I'm not sure if this is a good idea. It only works if I include the _ which seems strange, maybe that is just this issue and will be fixed in the future :face_with_diagonal_mouth: .

Luke Boswell (Nov 24 2023 at 08:41):

Maybe I should stick with the current option instead of introducing syntax which is complicated for an example, and we can update it in future when that it fixed.

Luke Boswell (Nov 24 2023 at 08:42):

But, still interested to know if that is a better way to write tasks (assuming the bug was fixed).

Brian Carroll (Nov 24 2023 at 09:05):

I assume that the Error here would be a union of all possible errors from any function in the platform?

Brian Carroll (Nov 24 2023 at 09:08):

I think option 2 sounds more aligned with how errors are designed to work in Roc.
Our type system automatically gathers up all possible tags that could be produced in any branch of your code, so that when you handle them in a when ... is somewhere near the top of your program, you are handling all the possible cases, and no impossible cases.

Brian Carroll (Nov 24 2023 at 09:10):

So there is no need to gather them up in a combined Error type, and in fact it's better if you don't, because the compiler will do that for you, and do it in a more detailed way.

Luke Boswell (Nov 24 2023 at 09:12):

I'm trying to teach people how Tasks work, I'm concerned that if I don't have type annotations that it might be too much magic for people to follow how the types flow through the program. I do like Option 2, but I think we should keep a common Error type that includes them all so that people can follow the example easier.

Does that sound reasonable?

Brian Carroll (Nov 24 2023 at 09:21):

Oh I didn't realise this was mainly for educational purposes, I thought it was for a real library.

Hmm I would have thought it was easy enough for people to follow how possible errors accumulate as you go further and further up the call stack. The tag union will get longer and longer as you go up, and you can annotate it at each point.
It's a key feature of the language that could attract people to using it.

Brian Carroll (Nov 24 2023 at 09:21):

Could it be a separate example? Maybe presented immediately after the first. "Here's a version with more precise error types".

Brian Carroll (Nov 24 2023 at 09:21):

By the way I think * should work as well as _ and might feel less weird.

Luke Boswell (Nov 24 2023 at 09:22):

Yeah, I just realised I can use * which is much clearer for "cannot fail" but for returning error tags I still need the _ character

Brian Carroll (Nov 24 2023 at 09:22):

Ah OK

Brian Carroll (Nov 24 2023 at 09:23):

Yeah on further reflection I can see how it might be too much to introduce both Tasks and the tag union behaviour at the same time in the same example.

Luke Boswell (Nov 24 2023 at 09:25):

I just pushed another round of polish to that example. If you have a minute to skim read I would really appreciate it.

Brian Carroll (Nov 24 2023 at 09:25):

But on the other hand, Task and Result are two great places to introduce this. So that's why I'm wondering if we can get two examples in there.

Luke Boswell (Nov 24 2023 at 09:25):

If you checkout that branch then run bash build-dev-local.sh and navigate to http://localhost:8080/Tasks/README.html

Brian Carroll (Nov 24 2023 at 09:27):

I just clicked on "view file" in GitHub!

Brian Carroll (Nov 24 2023 at 09:28):

Screenshot-2023-11-24-at-09.28.26.png

Luke Boswell (Nov 24 2023 at 09:30):

Love it.

Mine has pretty colors :smiley:
Screenshot-2023-11-24-at-20.29.39.png

Brian Carroll (Nov 24 2023 at 09:37):

OK I had a skim through and I understand the context better after clicking the link! I hadn't looked at this repo before.

It seems to be a place where we are really digging in and explaining everything in depth, and showing people how all the language features come together. So I definitely think this repo should contain an example of how to use tag unions for error handling!

But there could be another example specifically for "error handling". In the Task example program, all the effects are just done one-by-one in a straight line. In an error handling example we'd probably split it up into functions, with each function having a different set of possible error tags, and the top level function has to handle them all.

Brian Carroll (Nov 24 2023 at 09:37):

If we had both examples they could link to each other.

Brian Carroll (Nov 24 2023 at 09:40):

Or maybe after writing the error handling example, we'd realise that the two could just be merged.

Luke Boswell (Nov 24 2023 at 09:42):

Ok, that sounds reasonable. I think the approach I have here so far is a nice medium then. It uses a number of tasks in various different ways, but is really pretty linear, though touches on some amount of error handling.

I think another example focussed on error handling is a good idea, as we can show how it works for a more complicated flow and how things bubble up (or are captured) etc.

Brian Carroll (Nov 24 2023 at 09:42):

Yeah the PR looks like a great improvement from where we are now! We can always do more later if we think it makes sense.

Brian Carroll (Nov 24 2023 at 09:50):

PR still has "WIP" in the title so I don't know if you're still working on it but it seems ready to me, so I approved it. Ping me if you want me to re-review.

Luke Boswell (Nov 24 2023 at 09:50):

It is now, just completed spelling and grammar

Declan Joseph Maguire (Nov 24 2023 at 09:53):

Given I'm a dumbass with little prior experience with managed effects, I might look at this later to see how well you idiot-proofed it against me. If it's transparent to me, it should be transparent to most.

Luke Boswell (Nov 24 2023 at 09:56):

Oh dear

Luke Boswell (Nov 24 2023 at 10:04):

I definitely think we should aim for this to be approachable for someone with literally zero experience beyond working through the tutorial.

It's really difficult though because there is so much syntax that might be weird depending on someones background.

My goal is not so much that the reader totally groks it, but more they are mostly able to follow along and are left with an impression that if they wanted to, they could copy-paste it into a file, modify it, and make a start without too much hassle.

Declan Joseph Maguire (Nov 24 2023 at 10:05):

Yeah I'm not going to sweat every detail, so long as I get a sense of "I'll get that detail later" and it doesn't block my understanding of other bits

Declan Joseph Maguire (Nov 24 2023 at 10:23):

So I just re-read the tutorial bit on tasks just so I'd have it fresh in my mind, and I think it has a bit of an explanatory gap, which contributes to the separate issue of await being hard for newbies to grok. I feel like more time should be spent on the issue of "how does a Task that returns a string get turned into that string?" A task feels a bit like a function, a bit like a wrapper, a little bit magic. In the case of await, you just send two tasks into a magic box that seems to act like pseudo-function composition, sorta. Tasks feel like something I could probably start using pretty quickly, but only be 92% confident I was understanding correctly.

The longer I think about it, the more it starts to click, but it's requiring me to spend a lot of time thinking through examples and trying to find the right questions to ask, which is a good bit more than the other tutorial sections ask of the reader.

Declan Joseph Maguire (Nov 24 2023 at 10:25):

on to the actual thing I'm meant to be looking at

Luke Boswell (Nov 24 2023 at 10:29):

It took me a while to grok it.

One thing that helped me is when I heard Brendan describe them as a "future".

I still don't really know what that actually means (I've heard that word in passing in other languages), but what I took away from it was that Tasks aren't actually doing anything. They're just bits of data which describes an action.

We compose them and sequence them together to describe the behaviour we want our program to run, and pass them off to the host which will then take those task descriptions and execute them in the order we have prescribed.

Luke Boswell (Nov 24 2023 at 10:33):

I don't know if I have totally butchered that description, but it feels right.

I've written a few programs now and feel confident I understand how to compose Tasks in various ways and get them to do what I want. When I see an type mismatch error from the compiler I can usually see the issue pretty quickly.

Declan Joseph Maguire (Nov 24 2023 at 10:34):

I sorta already understood the way it describes an abstract process, it's more a matter of understanding how it gets actualised and how it then interacts with the atemporal world of functions. I think it's interesting that tasks (and their equivalents in others languages) often end up coming down to some hand waving and "you'll get it after you start using it a while". Which feels like our conceptual language is deficient? Because it doesn't seem like it's fundamentally arcane, but it ends up being like one of those pictures that looks concrete from a distance but a mass of blurry nothing up close.

Declan Joseph Maguire (Nov 24 2023 at 10:34):

Wheras most other things functional get sharper as you walk closer to the image, not blurrier

Luke Boswell (Nov 24 2023 at 10:37):

I definitely think the platform/application abstraction is super important here. Like when I think about a Task provided by the platform (and presumably it does something) it's just a black box, or an opaque interface at the boundary with the real world. I just think, oh the types tell me it will resolve with a thing or it wont.

Declan Joseph Maguire (Nov 24 2023 at 10:38):

Perhaps it would be good to figure out a way to describe tasks purely on their own terms, and then figure out how articulate what a function is within that language, as a restricted sort of task (if I'm right that a task generalises a function)(values are also functions with no inputs, up to isomorphism).

Declan Joseph Maguire (Nov 24 2023 at 10:40):

Back to the text of what you wrote, is "resolves" a jargon term specific to tasks, or just a generic term you would just as easily use when talking about a function output?

Luke Boswell (Nov 24 2023 at 10:41):

Declan Joseph Maguire said:

Perhaps it would be good to figure out a way to describe tasks purely on their own terms, and then figure out how articulate what a function is within that language, as a restricted sort of task (if I'm right that a task generalises a function)(values are also functions with no inputs, up to isomorphism).

What is your goal there?

Luke Boswell (Nov 24 2023 at 10:42):

Declan Joseph Maguire said:

Back to the text of what you wrote, is "resolves" a jargon term specific to tasks, or just a generic term you would just as easily use when talking about a function output?

I'm not sure where I got that from. But it feels like the most correct way to describe what a task does.

Luke Boswell (Nov 24 2023 at 10:43):

It's like a task is description of an action that will happen at some point in the future, and that action will either succeed and give me a thing or fail with another thing.

Declan Joseph Maguire (Nov 24 2023 at 10:44):

Just conceptual clarity. I guess my maths background is showing and I'm trying to figure out the axioms of tasks in the way I know the axioms of functions. I'm not talking about in the example I'm going through, I just mean in a general sense because I feel it would help us create better explanations.

Declan Joseph Maguire (Nov 24 2023 at 10:48):

Luke Boswell said:

It's like a task is description of an action that will happen at some point in the future, and that action will either succeed and give me a thing or fail with another thing.

Something odd about that is that everything that happens with a task, happens in the notional future. Using "resolves" in this way feels like a kind of double future to me. Both the input and the output of some task are equally "at some point during a process that will occur". Like, if the input can spoken of in the same way as a function's input, why not the output, same as some function whose return type is a union including an error, after a chunky block of thinking?

Luke Boswell (Nov 24 2023 at 10:51):

I think the best conceptual model for a task is that it is just data which describes a future action to be taken by the thing executing our program.

When we are writing our program we are simply ordering or sequencing these "tasks".

The actual execution of those tasks is done by the platform/host when it executes and runs the program.

So when we write a task we are thinking, "what do I want to do if this succeeds and gives me the thing, or fails"

Luke Boswell (Nov 24 2023 at 10:53):

I imagine you could draw a big graph showing the "flow" of a program from one task to another until it finishes.

Declan Joseph Maguire (Nov 24 2023 at 10:55):

Oh I understand that bit by now, because that's all jist and I get the jist. Where I get hung up is why that's not just a function where the input is provided at some later time by someone else.

Declan Joseph Maguire (Nov 24 2023 at 10:55):

Wait holy crap I think something clicked

Declan Joseph Maguire (Nov 24 2023 at 10:55):

It'll take me a second to articulate it

Declan Joseph Maguire (Nov 24 2023 at 10:57):

When you chain tasks together, outputs from earlier in the chain might later influence inputs elsewhere, even if not explicit in the syntax, because it happens outside of the inner functioning of the program

Luke Boswell (Nov 24 2023 at 10:59):

I'm not sure I follow this part

because it happens outside of the inner functioning of the program

Declan Joseph Maguire (Nov 24 2023 at 10:59):

Goddamit I did not expect my uni work on abstract causality in quantum systems to ever help me elsewhere

Luke Boswell (Nov 24 2023 at 10:59):

Well that escalated

Luke Boswell (Nov 24 2023 at 11:00):

I asked ChatGPT what that means, and it responded with an image :shrug:
image.png

Declan Joseph Maguire (Nov 24 2023 at 11:02):

A task abstractly defines a process that occurs in the future. However, so does a function (if you imagine it in the future: a ~\~future~\~ function). So what makes a task not a function? Pragmatically there seemed an obvious difference between the two, but I found it difficult to pin down conceptually.

Declan Joseph Maguire (Nov 24 2023 at 11:14):

Lets say we have a function fanficGenerator of type movie -> fanfic, and another fanficReviewerof type fanfic -> uint. There's basically two ways of conjoining them into a function - composition, or product, which is basically just having the two side by side. You hook the output of one up to the other, or you don't.

Declan Joseph Maguire (Nov 24 2023 at 11:18):

But if you replaced those functions with tasks, such that fanficGenerator then saves its fanfic to file, and fanficReviewer reads from disk, it is possible that the input to fanficReviewer might depend on what fanficGenerator wrote, or possibly not. Therefore as a process, you need to keep track of the relative order in which inputs and outputs are generated, you can't parallelise in the way you can with pure functions. That's where the causality comes in, the order matters even if the inputs and outputs of various tasks might not seem related according to the pure syntax.

Declan Joseph Maguire (Nov 24 2023 at 11:20):

And that's why a task isn't a function whose inputs will be created later when run, or whose outputs are just values that will be dealt with later.

Luke Boswell (Nov 24 2023 at 11:27):

Yeah that makes sense I think. Its the context around the task, like the sequencing or ordering that really matters.

Declan Joseph Maguire (Nov 24 2023 at 11:47):

I'm still reeling from the fact that my arcane thesis bullshit actually helped me elsewhere, so concretely

Declan Joseph Maguire (Nov 24 2023 at 12:32):

I am gonna spend some time thinking this over, because I suspect there might be a way to condense this down into a single small example that intuitively explains this defining feature, but it won't be easy to find.

Richard Feldman (Nov 24 2023 at 13:13):

I think maybe explaining what tasks compile to (or actually they will compile to in the future once we have effect interpreters working) could be helpful.

Basically an individual Task will end up compiling to a tag union like this:

Operation : [
    WriteFile {
        path : Path,
        contents : List U8,
        callback :
            Result {} WriteErr -> Operation,
    },
    ReadFile {
        path : Path
        callback :
            Result (List U8) ReadErr -> Operation
    },
    Done,
]

Richard Feldman (Nov 24 2023 at 13:14):

when the lower-level language calls main : Task {} [] it will get back one of these (there will be a way to go from Task to Operation)

Richard Feldman (Nov 24 2023 at 13:16):

then it basically looks at it and sees "oh ok this is a file read, so I'll call some C or Rust function to do that, possibly async, and then whenever I get that answer back (or an error), I'll call that callback function with it, which will give me back another Operation, at which point I do it all over again

Richard Feldman (Nov 24 2023 at 13:16):

add then eventually one of the callbacks returns Done, at which point we're done!

Richard Feldman (Nov 24 2023 at 13:20):

so it's not a metaphor that a task represents a description of what's going to be done - that's literally what it is in memory! :big_smile:

Declan Joseph Maguire (Nov 24 2023 at 13:20):

That may well help for other types of confusion, but for me it was about the higher level conceptual stuff, about why a task wasn't just a clever way of turning effects into functions (it took me a while to even figure out that this was the question I had). I actually understood pretty well how a lot of basic tasks worked in practice, as all the tutorial examples are basically just IO, plus some abstract idea of combining small tasks into bigger ones that eventually results in main.

Declan Joseph Maguire (Nov 24 2023 at 13:22):

It was one of those types of confusion you can only really explain once you've figured out the answer for yourself, annoyingly. I'm very good at discovering that type of confusion.

Declan Joseph Maguire (Nov 24 2023 at 13:24):

Trying to understand the inner logic of tasks based on what they compile to feels like trying to understand what a type system is by taking a program written in a strongly typed language, and analysing the assembly you get after compilation.

Last updated: Jun 16 2026 at 16:19 UTC