Stream: beginners

Topic: Manual decoding patterns

Agus Zubiaga (Mar 15 2023 at 15:39):

So I know Roc has Decode which is great. However, sometimes our types do not capture how something needs to be decoded. In those cases, have we come up with an idiomatic way to compose manual decoders?

These are some of the options I have thought about:

1. mapN

In Elm, it's common to use mapN functions which get passed the automatically generated record constructors:

type alias Person = { name: Str, age: Int }

decodePerson =
     map2 Person
         (field "name" string)
         (field "age" int)

Roc doesn't seem to generate a function like Person automatically, so you'd have to do this:

decodePerson =
     map2
         (field "name" str)
         (field "age" u8)
         (\name, age -> { name, age })

This is more verbose and will only get longer as you add more fields.

However, I prefer this overall because decodePerson can't depend on the order of the fields in the record alias, which can lead to errors if somebody decides to change it in the future and two fields have the same type.

That said, you can still mess up the order of decoders.

2. Applicatives

An alternative that is also common in Elm is applicative pipelines:

decodePerson =
     succeed (\name -> \age -> {name, age})
         |> andMap (field "name" str)
         |> andMap (field "age" u8)

These allow you to compose any number of decoders. But, besides that, they have the same downside as mapN regarding the order of decoders.

3. Backpassing / await

An option I quite like is having an await function and using backpassing:

decodePerson =
    name <- field "name" str |> await
    age <- field "age" u8 |> map
    {name, age}

What's cool about this is that the field's name appears right next to the argument with our decoded value, which makes it much easier to notice a mismatch.

The downside of this approach is that you cannot use it in cases where the library needs to know all the fields ahead of time. So, for example, you couldn't build something like dillonkearns/elm-graphql, which mixes decoding and selecting in one step.

--

Are there other options folks have considered?

Richard Feldman (Mar 15 2023 at 16:27):

yeah, I have a proposed syntax to address the downsides of #2: https://docs.google.com/document/d/1Jo9nZCekkoF6SaDcRqPqoPcgPaAAvlNZC7v3kgVQ3Tc/edit?usp=sharing

Richard Feldman (Mar 15 2023 at 16:27):

it's not a high priority though :big_smile:

Agus Zubiaga (Mar 15 2023 at 16:36):

Ooh, this is exactly what I was hoping you'd say :heart_eyes:

Agus Zubiaga (Mar 15 2023 at 16:37):

Can I have the definition of posts refer to users? It looks like it's in scope, so I should be able to, right? (No, applicatives don't let you do that. That's maybe easier to understand with this example where the whole point is for them to run in parallel, but the reason for why not would be less obvious with a CLI arg parser.)

This is great

Agus Zubiaga (Mar 15 2023 at 17:17):

How's the general feeling about this? I know it's low priority but does it look like something you'd want to implement in practice

Joshua Warner (Mar 15 2023 at 17:18):

I was actually musing about a more general version of this (I think...) - where any expression can be prefixed with <-, and the entire expression would be desugared into a sequence of backpassing steps.

So, e.g. this:

(<- getAnswer 5) + (<- getAnswer 6)

Would desugar to:

a <- getAnswer 5
b <- getAnswer 6
a + b

Agus Zubiaga (Mar 15 2023 at 17:20):

hm, that wouldn't work for parallel stuff though, would it?

Joshua Warner (Mar 15 2023 at 17:21):

True

Joshua Warner (Mar 15 2023 at 17:23):

There could be a separate syntax in the language that indicates you want parallism in otherwise normal backpassing syntax, without having to dip down to the level of Task.collect

Agus Zubiaga (Mar 15 2023 at 17:27):

What I like about Richard's proposal is that it's not specifically about task parallelism. Being able to collect all the dependencies ahead of time with a syntax that doesn't look crazy would be great for a lot of DSLs

Joshua Warner (Mar 15 2023 at 17:27):

Yeah that makes sense

Joshua Warner (Mar 15 2023 at 17:28):

I also like the idea of the more general syntax - where I can specify exactly how the results of the tasks are used (e.g. it doesn't have to be a record)

Agus Zubiaga (Mar 15 2023 at 17:31):

Something like this?

Task.collect (
  <- getAnswer 5,
  <- getAnswer 6
)

Joshua Warner (Mar 15 2023 at 17:32):

Yeah

Joshua Warner (Mar 15 2023 at 17:33):

So, let me modify my proposal to instead desugar to:

\wrap ->
  wrap (\a, b -> a + b)
  |> getAnswer 5
  |> getAnswer 6

(I think, If I read the proposal right?)

Agus Zubiaga (Mar 15 2023 at 17:33):

right

Brendan Hansknecht (Mar 15 2023 at 17:35):

@Richard Feldman what actually gets passed to the host with Task.batch? Cause I don't think it can work in roc alone (would be serial).

Joshua Warner (Mar 15 2023 at 17:35):

Actually the input would be more like:

Task.collect (
  (<- getAnswer 5) +
  (<- getAnswer 6)
)

i.e. just using a normal binary operator here

Joshua Warner (Mar 15 2023 at 17:35):

Not sure if those parens would be needed, depends on the predence of <-

Agus Zubiaga (Mar 15 2023 at 17:37):

if it supported all expressions generally, I guess the record example would look like this instead:

Task.collect {
    users: <- Http.get "/users" |> Task.batch,
    posts: <- Http.get "/posts" |> Task.batch
}

Brendan Hansknecht (Mar 15 2023 at 17:39):

These syntaxes are starting to look really strange and unclear. Especially "anonymous(unnamed?) backpassing".

Agus Zubiaga (Mar 15 2023 at 17:40):

I guess you could even do something like this:

Task.collect (
   if (<- getBool) then
       ...
)

Agus Zubiaga (Mar 15 2023 at 17:40):

Yeah, I can see how this could be powerful but I'd be more than happy with only the record option

Agus Zubiaga (Mar 15 2023 at 17:41):

I think it strikes the right balance of weirdness and convenience

Brendan Hansknecht (Mar 15 2023 at 17:43):

yeah, I would definitely perfer:

Task.collect (
    doThing <- getBool
    if doThing then
         ...
)

or something that uses a wraper function for the code. Or maybe getting a list of results back from Task.collect and using those.

Brendan Hansknecht (Mar 15 2023 at 18:18):

Oh, I just realized the original proposal is just for building a record. I definitely prefer that to these generic code alternatives.

Joshua Warner (Mar 15 2023 at 19:48):

The reason I like the general expression syntax is it provides a nice way to parallelize things, without having to clutter the code with a bunch of extra Task.whatever calls.

Brendan Hansknecht (Mar 15 2023 at 19:56):

Thats totally fair. I think I might like it with some different form of syntax.

Last updated: Jul 05 2025 at 12:14 UTC