Stream: ideas

Topic: map2-based record builders


view this post on Zulip Agus Zubiaga (Jun 22 2024 at 15:25):

map2-based record builders

The problem

Record Builders have some pretty cool applications already, but seem to be confusing to most people.

Take the example from the initial proposal:

 Task.ok {
    users: <- Task.batch (Http.get "/users"),
    posts: <- Task.batch (Http.get "/posts"),
    news: <- Task.batch (Http.get "/news"),
}

and the types of the functions involved:

Task.ok : ok -> Task ok err
Http.get : Str -> Task Http.Response Http.Err
Task.batch : Task a err -> (Task (a -> b) err -> Task b err)

There are quite a few pieces there, and from the surface, it's not clear how they all fit together. Why is Task.ok there? Task.batch returns a function that takes a Task with a function inside!?

If you're explaining it to someone, you first have to show the desugared version, so you can then introduce Applicatives, which are already a mind-bending concept.

The proposed solution

Applicatives are pretty convenient in a language like Elm where Record Builders do not exist and you can partially apply functions. However, if we're going to have sugar for this problem, we don't really need them.

What if Record Builders required only map2 and just paired results?

So if I have a map2-like function that combines two tasks into one:

Task.concurrent : Task a, Task b, (a, b -> c) -> Task c

I can combine N tasks into a record like this:

Task.concurrent <- {
    users: Http.get "/users",
    posts: Http.get "/posts",
    news: Http.get "/news",
}

Which would desugar to:

Task.concurrent
    (Http.get "/users")
    (Task.concurrent
        (Http.get "/posts")
        (Http.get "/news")
        \posts, news -> (posts, news)
    )
    \users, (posts, news) -> { users, posts, news }

:bangbang: Note: Even though we are making pairs of results, the task themselves would all run concurrently.

I think this is a lot more straightforward. Instead of storing the intermediary results in closures, we just use tuples. There are no mysterious apply/lift functions on the surface, and if you want to understand the desugaring, you only need to understand map2.

This also has the benefit that the module (e.g. Task) likely already provides map2 and it doesn't have to expose functions that are specific to Record Builders.

Nicer types

Finally, I think this will lead to more flexibile APIs. I noticed that basically all the applications of Record Builders expose convinence functions that do too much.

For example, roc-pg's query builder exposes a column function, so that instead of writing the following for each field:

Sql.into {
    id: <- Sql.apply (Sql.select products.id),
    price: <- Sql.apply (Sql.select products.price),
}

You can just write:

Sql.into {
    id: <- Sql.column products.id,
    price: <- Sql.column products.price,
}

That's shorter, but the type out of Sql.column is Selection (a -> b) -> Selection c, which is not very amenable to further transformation.

With the map2-based approach, the type at each field is simply Selection a, so I can just use Sql.map if I want to:

Sql.combine <- {
    id: Sql.select products.id,
    price: Sql.select products.price |> Sql.map fromCentsToDec,
}

And I can even have nested records without any extra work:

Sql.combine <- {
    id: Sql.select products.id,
    meta: Sql.combine <- {
        name: Sql.select products.name,
        manufacturer: Sql.select products.manufacturer,
    }
}

view this post on Zulip Ian McLerran (Jun 22 2024 at 17:16):

I love this proposal. I think the map2 design makes things a lot simpler to understand and, and as you said, the types are a lot nicer as well. Having the signature of the functions used be more generally applicable really seems like a nice benefit both for chaining in record builders and for building more modular code bases.

Kasper Møller Andersen said:

So I would take a good hard look at whether the sigil should actually be an arrow. My experience is that I tend to think of the arrow sigil as "input -> output".

My only thought is that in the vein of Kaspers comment here, maybe a different operator could be found for record building. Just a thought, but what about <+? That would be similar to the current operator, but the plus adds the connotation of combination.

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 18:38):

Hm, I get the input -> output idea, but I don't know if <+ is more descriptive of what's going on with Record Builders :thinking:

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 18:40):

Maybe something like this?

Sql.combine ~{
    id: Sql.select products.id,
    price: Sql.select products.price,
}

view this post on Zulip Kilian Vounckx (Jun 22 2024 at 18:41):

Maybe <= because it resembles the arrow but it takes more values?

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 18:42):

That might be weird because it's also "less than or equal"

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 18:44):

which code fonts with ligatures would render as

view this post on Zulip Kilian Vounckx (Jun 22 2024 at 18:46):

Oh yeah didn't think of that because I use ligatures :sweat_smile:

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 18:47):

The idea behind ~ is that builders sort of thread the function through the fields

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 18:49):

Throwing another option out there:

    Sql.combine <~ {
        id: Sql.select products.id,
        price: Sql.select products.price,
    }

view this post on Zulip Luke Boswell (Jun 22 2024 at 18:50):

I think the <- backwards arrow will be fine if we remove backpassing... there wont be anywhere else you see it then.

view this post on Zulip Kilian Vounckx (Jun 22 2024 at 18:50):

I honestly think the single arrow <- is best, but yeah some discussion should be had. I don't think we should bikeshed too much though

view this post on Zulip Luke Boswell (Jun 22 2024 at 18:51):

Can we add Task.map2 and just have it run things sequentially for now? Or does this proposal need to wait for Effect Interpreters?

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 18:51):

I'm not really concerned about the conflict with backpassing. I think even if we kept it, we could distinguish in the parser because a { on the RHS of the backpassing arrow doesn't make sense.

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 18:52):

It might be confusing if we have both and use the same symbol, though

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 18:54):

Luke Boswell said:

Can we add Task.map2 and just have it run things sequentially for now? Or does this proposal need to wait for Effect Interpreters?

We can, but I don't know if we would gain much. You can just use ! if you want to run things sequentially.

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 18:55):

I only used the Task example because it's a simple one. Record Builders are not tied to tasks.

view this post on Zulip Luke Boswell (Jun 22 2024 at 18:56):

Record Builders are not tied to tasks.

Oh nice, I didn't quite follow that. So anything with that shape can use record builders to build up a record?

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 18:57):

Yeah, I don't think I've seen any examples of using Record Builders with tasks yet

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 18:57):

They only really make sense for tasks if you can run them concurrently

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 18:58):

The examples I showed above with roc-pg are not task-based, and neither is weaver

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 19:00):

Agus Zubiaga said:

We can, but I don't know if we would gain much. You can just use ! if you want to run things sequentially.

By this I mean:

users = Http.get! "/users"
posts = Http.get! "/posts"
news = Http.get! "/news"

or if you want them in a record:

{
    users: Http.get! "/users",
    posts: Http.get! "/posts",
    news: Http.get! "/news",
}

view this post on Zulip Kasper Møller Andersen (Jun 22 2024 at 20:14):

Compared to the existing record builder syntax, these seem restricted to all fields having the same type, and the old one is not, right? That's not necessarily a problem, but just something worth highlighting.

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 20:16):

No, fields can have different types as long as the map2 function allows it.

view this post on Zulip Sam Mohr (Jun 22 2024 at 20:18):

@Kasper Møller Andersen I don't believe that's the case, it's just a bit trickier to use, maybe? Given the example Agus gave:

Task.concurrent
    (Http.get "/users")
    (Task.concurrent
        (Http.get "/posts")
        (Http.get "/news")
        \posts, news -> (posts, news)
    )
    \users, (posts, news) -> { users, posts, news }

We can see that it's giving us an eventual \a, (b, (c, (d, ...)))) -> output function. We just get the values in a different order than in the old approach

view this post on Zulip Sam Mohr (Jun 22 2024 at 20:19):

I'll try seeing if I can make Weaver work with the desugared version of this syntax, and if that works, then I'm all in on this syntax change! This should support both normal builders like Weaver, but also very simple stuff like parallel task awaits, or trying multiple results at once very conveniently

view this post on Zulip Kasper Møller Andersen (Jun 22 2024 at 20:20):

Okay, so doable in theory at least, but I guess the ordering of the keys versus the ordering of the map2 args becomes relevant? As in, if you don't order your keys such that they desugar in the order your map2 function expects, you'll get a compile error.

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 20:22):

I guess I don't really get what are you trying to do. The fact tuples are used internally is really just an implementation detail. From the perspective of map2, you don't really care.

view this post on Zulip Sam Mohr (Jun 22 2024 at 20:23):

The same thing is true of how the old way works, basically. If you order it differently, the functions change signature, but it's okay because the record fields change order in the final function as well

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 20:24):

@Sam Mohr The order of what part?

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 20:24):

The order of the fields shouldn't matter

view this post on Zulip Sam Mohr (Jun 22 2024 at 20:24):

the fields in the struct

view this post on Zulip Sam Mohr (Jun 22 2024 at 20:24):

Yeah, it doesn't

view this post on Zulip Sam Mohr (Jun 22 2024 at 20:24):

But, technically, the functions that desugar out are shaped differently

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 20:26):

Right, but as the author of the package/module, you don't provide the closure that destructures the tuples, it's generated automatically.

view this post on Zulip Kasper Møller Andersen (Jun 22 2024 at 20:36):

Okay, I'm trying to get the scenario written down, but I can't fully articulate it anyway, which is probably a sign that it's not super relevant :big_smile:

view this post on Zulip Kasper Møller Andersen (Jun 22 2024 at 21:00):

A different thought: it feels funny that this essentially makes map2 functions variadic, except the arguments don't become a list, but a record (so "named variadics" maybe?)

I guess the piece I feel is sort of missing is that it becomes a particular skill to look at a function signature and be able to read whether I can use this sugar with it or not. It's still way better than the existing syntax, but compared to other languages with variadic functions, the unclear signposting feels suboptimal.

Which I think is also why I was struggling with articulating my previous point: I understand the happy path of using this, but it's hard for me to grasp what the contract with map2 function is before I can use this sugar.

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 21:05):

Yeah, that's a valid point. Unfortunately, I don't really see a way around that without introducing HKTs.

view this post on Zulip Agus Zubiaga (Jun 22 2024 at 21:09):

We could maybe require that the function's type looks like:

_, _, (a, b -> c) -> _

This would at least prevent people from depending on the fact that we use tuples internally.

view this post on Zulip Kiryl Dziamura (Jun 23 2024 at 05:41):

In terms of language consistency, let’s assume we have the following syntax:

m2 ~{ x: fx a }

Should this new record builder work with pizza operator somehow?

~{ x: fx a } |> m2

Or, is it possible to pass the record as a variable?

r = { x: fx a }
m2 ~r

It feels like the sigil is more about modifying the function interface rather than the record, so it’s probably better to tie the sigil to the map2 function call as a modifier. It will also be simpler for parser to generate a meaningful error (e.g. “the interface of the function is not map2”). I didn’t think much about the beauty of the approach, just for example, I hope the idea is clear:

m2@mapRecord { x: fx a }

{ x: fx a } |> m2@mapRecord

r = { x: fx a }
m2@mapRecord r

notM2@mapRecord { x: fx a } # error: notM2 should be aligned with map2 interface

view this post on Zulip Kiryl Dziamura (Jun 23 2024 at 05:48):

However, my suggestion probably moves it from static “template-based” record building to dynamic “sugar-based” record transformation.

This also has an advantage, the records mapping becomes generic and straightforward. But it probably provides too much freedom? Maybe record mapping shouldn't be something easy to do and in general, considered as an antipattern?

view this post on Zulip Kasper Møller Andersen (Jun 23 2024 at 07:00):

Speaking of fitting with the existing language constructs, I guess this example:

Task.concurrent <- {
    users: Http.get "/users",
    posts: Http.get "/posts",
    news: Http.get "/news",
}

should actually be

Task.concurrent! <- {
    users: Http.get "/users",
    posts: Http.get "/posts",
    news: Http.get "/news",
}

(note the !)
in order to be useful, right?

view this post on Zulip Kiryl Dziamura (Jun 23 2024 at 07:03):

It would only mean that the resulting task is awaited, no?

view this post on Zulip Kasper Møller Andersen (Jun 23 2024 at 07:04):

That's my thinking, yes.

view this post on Zulip Kasper Møller Andersen (Jun 23 2024 at 07:28):

Also, just going off on something of a tangent here, but I promise it ties back into record builders :smiley:

We're talking about syntax sugar for something where you essentially describe how to build a record, using some input which also looks like a record, even though it isn't. But what would it take for the input to be an actual record?

The reason I bring it up is because I've been wanting a way to work with records more dynamically in Elm for some time, essentially being able to iterate over fields of a record, so I could use functions like Record.map, Record.walk, etc.

Having such functions would make you able to use records kind of like a Dict, and would potentially also make the record builder use case quite elegant I think. Aside from those functions, the part that's missing is being able to describe a record without describing its actual field names, but simply the types in it, like what you can in TypeScript with index signatures:

interface Foo {
   [key: string]: number;
}

or

type Index = 'a' | 'b' | 'c'
type FromIndex = { [k in Index]?: number }

Doing this would make the "variadic" use case much clearer too, as a function can now declare that it takes a record with varying fields.

In Elm, I've specifically wanted this every time I've had the urge to use a tagged union as keys in a Dict. That is, I know all the fields up front, but for one reason or another, I still need to be able to iterate through them.

Using an actual record like this would make the suggestion of ignored fields less nice though.

view this post on Zulip Kasper Møller Andersen (Jun 23 2024 at 07:30):

I also get that this goes into a level of dynamism that is something of a departure for Roc, so it may not fly at all, but I wanted to make sure it was brought up at least :smile:

view this post on Zulip Kiryl Dziamura (Jun 23 2024 at 08:22):

I like the std approach more than my function modifier proposal! It would only be a bit more verbose I think, but much more clear

view this post on Zulip Kasper Møller Andersen (Jun 23 2024 at 09:02):

Sorry, I'm a little confused. Is the "std approach" the one I outlined above? Or another one? :sweat_smile:

view this post on Zulip Kiryl Dziamura (Jun 23 2024 at 09:05):

I think the only place where this kind of Record.map can be implemented is std. So I was talking about your proposal

view this post on Zulip Kasper Møller Andersen (Jun 23 2024 at 09:13):

I also feel it's much clearer. A function can explicitly take a "dynamic record" as input, and apply the transformations here with operations we know quite well at this point.

One potential downside though is that I don't know if we can rely on the ordering of the fields in the record, and whether we need to be reliant on it.

view this post on Zulip Richard Feldman (Jun 23 2024 at 12:09):

I’d say open another #ideas thread about that idea :big_smile:

view this post on Zulip Kiryl Dziamura (Jun 24 2024 at 09:51):

@Agus Zubiaga have tuple builders been already discussed? Does it make sense to expand record builder to structure builder?

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 10:58):

Yeah, we did talk about it for the initial implementation.

IIRC one of the main issues was that the existing syntax didn’t really work because each field has an <-, so we’d have to support something like:

wrap (<- apply , <- apply )

If that works, it’d be reasonable to think this should also work:

wrap (
    if <- apply someBooleanProducingThing then
        <- apply 
    else
        <- apply 
)

Which is pretty wild, and makes it hard to reason about control flow, because unlike ! everything has to be aggregated ahead.

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 11:04):

That said, in the new proposed syntax, the <- is moved from each field to between the map2 function and the “record”, so something like this could work:

Task.concurrent <- (Http.get "/users", Http.get "/posts")

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 11:07):

It might make sense to support it for consistency, but it’s probably very rare

view this post on Zulip Kiryl Dziamura (Jun 24 2024 at 11:13):

Also, speaking of “the same type for all fields restriction” problem mentioned above, isn’t the following code a problem with this proposal?

Task.concurrent <- {
    id: id, # let’s say we already got it from somewhere
    users: Http.get "/users",
    posts: Http.get "/posts",
    news: Http.get "/news",
}

So what’s the type of id? I’m not very experienced in FP, but it looks like the proposed pattern is monadic, right? Meaning, Task.concurrent is for composing and there also should always be a wrapping operation, which is Task.ok in this case. So id should be a Task, right? Can it lead to wrappers hell when composition is a bit trickier?

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 11:16):

Not exactly monadic, but it’s true that’d you’d have to wrap a value in the type with some function like Task.ok, if you want it to go as is in the record.

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 11:19):

I much prefer not to have to specify the function for every single field that you do want to chain

view this post on Zulip Kasper Møller Andersen (Jun 24 2024 at 11:21):

Well, that answers my question from earlier I think, thanks for making it understandable @Kiryl Dziamura :big_smile:

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 11:22):

I think having to use Task.ok is fine, that’s how it works with !, and in my experience it’s quite rare with record builders

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 11:23):

Kasper Møller Andersen said:

Well, that answers my question from earlier Kiryl Dziamura :big_smile:

Oh, I thought you were talking about the types of the fields in the final record! :grinning:

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 11:32):

Another implication I just realized I didn’t mention is that a record builder would have to have at least 2 fields, because we wouldn’t be able to call the map2 function with fewer than that.

We would need to also take a map or a wrap function to support 0..1 builders, which does not seem worth it for a case where you’re not combining anything so you don’t really need builders.

view this post on Zulip Kiryl Dziamura (Jun 24 2024 at 11:46):

2 fields

It can be solved with ignore fields i think. Tho it will look weird

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 11:46):

Yeah, if you really need a record with one thing, you can map it afterwards:

Http.get "/users" |> Task.map \users -> { users }

or use ignored fields as you said:

Task.concurrent <- {
    users: Http.get "users",
    _: Task.ok {},
}

view this post on Zulip Kiryl Dziamura (Jun 24 2024 at 12:02):

Alternative syntax idea: meta fields for the record literal. I tend to put the chaining operation somewhere under the brackets to avoid ambiguity with regular records. Also, this way parser can generate more readable error for expected map2 compliance

{
    @with: Task.concurrent,
    users: Http.get "/users",
    posts: Http.get "/posts",
}

view this post on Zulip Kiryl Dziamura (Jun 24 2024 at 12:14):

And just for extreme verbosity, ignore fields: :grinning_face_with_smiling_eyes:

{
    @with: Parser.combine,
    year: Parser.int,
    @then: Parser.symbol "/",
    month: Parser.int,
    @then: Parser.symbol "/",
    day: Parser.int,
}

view this post on Zulip Kasper Møller Andersen (Jun 24 2024 at 12:22):

One of my issues with the map2 based syntax is that it's not clear that order actually matters, because it usually doesn't in records. So making it clearer that the ordering does matter is a win IMO :blush:

view this post on Zulip Kiryl Dziamura (Jun 24 2024 at 12:24):

It doesn’t matter for the memory layout, but during record building the order of calls is preserved. I’d assume always

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 12:27):

Order matters in the applicative approach the same

view this post on Zulip Kiryl Dziamura (Jun 24 2024 at 12:29):

I mean,

{
  b: 1 + 2,
  a: 3 * 4,
}

Summation is the first operation, multiplication is the second. I think records can be implied as named tuples so to speak. I mean, there’s no order ambiguity in the case of tuples, right?

view this post on Zulip Kasper Møller Andersen (Jun 24 2024 at 12:31):

Yes, sorry, it's not specific to map2 versus the existing syntax :blush:

view this post on Zulip Kasper Møller Andersen (Jun 24 2024 at 12:34):

I personally view records as unordered. With your example, if you had two records with the same content, but different order of fields, an == would say they are the same.

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 12:37):

That’s true. However, these are not records, they are builders that produce records. I think it’s important to preserve the order for some use cases such as parsers, and CLI arg docs.

view this post on Zulip Kasper Møller Andersen (Jun 24 2024 at 12:38):

I agree, though my concern is exactly that they look like records, implying record-like behavior. Which is why I like the idea of having them look less like normal records :blush:

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 12:48):

I see

view this post on Zulip Kiryl Dziamura (Jun 24 2024 at 12:57):

I mean, it’s not necessarily regular records are order agnostic. When it comes to effects, the order matters anyway. The following records are different:

{
    a: set! "x" 42,
    b: get! "x",
}
{
    b: get! "x",
    a: set! "x" 42,
}

Yes, after desugaring it becomes clear that the order matters but it still looks like a regular record.

I think it’s more a feature than a problem. It allows writing more compact code. In any case, what really matters is the order of calculations not the order of assignments.

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 13:27):

Yeah, records are order agnostic, record literal expressions aren’t

view this post on Zulip Kasper Møller Andersen (Jun 24 2024 at 13:34):

Okay, I tried to expand on how we might solve the issues that I personally think still remain with the record building in this proposal: https://roc.zulipchat.com/#narrow/stream/304641-ideas/topic/Record.20builder.20as.20a.20function/near/446616377

view this post on Zulip Kiryl Dziamura (Jun 24 2024 at 15:32):

@Kasper Møller Andersen @Agus Zubiaga what do you think about record literal meta fields proposal I wrote above? It’s still map2-based and distinguishes record builders by introducing meta-field @with (expected to be always top-level defined). I have no concerns about changing the naming.

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 15:38):

Personally, I don't love how the "map2" function appears as a field, it does something very different than all the other ones.

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 15:40):

For the parser, the <- option is a little easier because you know you're parsing a builder as soon as you encounter it. (In a world without backpassing)

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 15:41):

Not a big deal, though

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 15:42):

If we want the map2 function to appear inside {...}, what about this?

{ Task.concurrent <-
    users: Http.get "/users",
    posts: Http.get "/posts",
    news: Http.get "/news",
}

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 15:42):

It follows the record update syntax pattern

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 15:54):

I do like that if the function appears inside, there's no confusion about whether pipes should work or whether you can put the RHS of <- on a variable

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 15:57):

If we want to confuse every Elm developer, we can also go with :smiley:

{ Task.concurrent |
    users: Http.get "/users",
    posts: Http.get "/posts",
    news: Http.get "/news",
}

view this post on Zulip Kiryl Dziamura (Jun 24 2024 at 16:01):

Yeah, having the map2 inside produces less doubtful whitespaces, so to speak

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 16:05):

Not sure which symbol to use, but I quite like this direction the more I think about it

view this post on Zulip Brendan Hansknecht (Jun 24 2024 at 16:07):

I think putting it in the record with special syntax like these final proposal is definitely better.

view this post on Zulip Brendan Hansknecht (Jun 24 2024 at 16:07):

More clear the record isn't normal

view this post on Zulip Brendan Hansknecht (Jun 24 2024 at 16:08):

Yeah, records are order agnostic, record literal expressions aren’t

This feels like a compiler implementation detail rather than expected behaviour

view this post on Zulip Brendan Hansknecht (Jun 24 2024 at 16:08):

Order matters in the applicative approach the same

Never realized that, but it definitely worries me.... Just waiting for someone to sort a record and break all the parsing.

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 16:09):

It's an implementation detail unless ! is used like @Kiryl Dziamura pointed out

view this post on Zulip Brendan Hansknecht (Jun 24 2024 at 16:09):

Ah, I see, missed that ! already enforcing ordering here.

view this post on Zulip Brendan Hansknecht (Jun 24 2024 at 16:11):

But yeah, even straight line code in roc that could theoretically be run out of order are often ordered for understandability and side effect reasons.

view this post on Zulip Brendan Hansknecht (Jun 24 2024 at 16:26):

Probably would be a terrible idea as something implicit, but I'm pretty sure that we could allow this:

{ Task.concurrent <-
    id: id, # lets say we already got it from somewhere
    users: Http.get "/users",
    posts: Http.get "/posts",
    news: Http.get "/news",
}

id would be a type mismatch if we try to feed it through Task.concurrent. So instead build a special desugaring where anything that is a type mismatch is just passed through without any changes. So never passed into Task.concurrent.

Or better yet, make it explicit (all special fields require <-:

{ Task.concurrent <-
    id: id, # lets say we already got it from somewhere
    users <- Http.get "/users",
    posts <- Http.get "/posts",
    news <- Http.get "/news",
}

So I think I actually like this syntax a lot. First line kinda says that <- means it is getting passed into Task.concurrent. Later lines specify which things are getting passed in.
Could also do, but I think it is less clear. Though definitely more distinct.

  <- users: Http.get "/users",

This also further removes from record syntax while still having the basic shape of a record which owuld help ensure it doesn't get missed.

view this post on Zulip Richard Feldman (Jun 24 2024 at 16:28):

I think specifically having : is important for the intuition that these are record fields and not variables - which in turn is important for the intuition that fields can’t reference each other

view this post on Zulip Brendan Hansknecht (Jun 24 2024 at 16:30):

I feel like that is easily solved with a single error message. So I personally am not worried about that.

view this post on Zulip Brendan Hansknecht (Jun 24 2024 at 16:31):

Oh, though you could have a variable with the same name as a field and reference that, which would look like referencing a field if we don't use : where the expectation is already set...so maybe : has more merit

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 16:32):

I know being able to add “fixed” values is cool in theory, but I haven’t really needed it yet.

I’d prefer not having to write <- for 95% of the cases and just use Task.ok or equivalent in the rare cases that it’s needed.

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 16:35):

@Richard Feldman what do you think about the update-style syntax?

view this post on Zulip Kiryl Dziamura (Jun 24 2024 at 16:35):

Can record update syntax to be used here to define unwrapped fields?

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 16:36):

We currently disallow update in record builders, I think it’s too much :big_smile:

view this post on Zulip Brendan Hansknecht (Jun 24 2024 at 16:37):

One other general comment:

The one element record case feels like an easy bug for users to hit and get annoyed by. Especially if they have an existing record and are just commenting out a field (I bet many new users will hit this when they try a record builder for parsing and want to start with the simplest one element parsing). I wonder if there is a way to fix it.

I guess we could make it take both map and map2. Or would could make it take map1or2 that uses a variant....but of this don't feel great though.

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 16:39):

I think we can just make it a compile time error. Like tuples, record builders only make sense with at least 2 items.

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 16:40):

I don’t think you’d reach for a builder if you only have 1 thing. There’s nothing to batch, and you can map it later if you really need a record for some reason.

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 16:44):

Brendan Hansknecht said:

(I bet many new users will hit this when they try a record builder for parsing and want to start with the simplest one element parsing)

Oh, I missed that. This is a good point.

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 16:47):

Maybe we could make it a runtime error at least?

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 16:48):

I mean compiler error but can still run

view this post on Zulip Richard Feldman (Jun 24 2024 at 16:48):

I think a compile-time error message can address that

view this post on Zulip Richard Feldman (Jun 24 2024 at 16:48):

or a warning, yeah

view this post on Zulip Richard Feldman (Jun 24 2024 at 16:48):

like “this doesn’t do anything and will be ignored”

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 16:50):

Well, it would have to crash, right? Because there’s no way to call map2

view this post on Zulip Richard Feldman (Jun 24 2024 at 16:50):

and actually yeah we could have the semantics be “1 field is a no-op” plus we give a warning along the lines of “unused record builder”

view this post on Zulip Richard Feldman (Jun 24 2024 at 16:50):

yeah just don’t call it

view this post on Zulip Kiryl Dziamura (Jun 24 2024 at 16:50):

We still want to have type checking there right?

view this post on Zulip Richard Feldman (Jun 24 2024 at 16:50):

just replace the whole literal with the one field

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 16:50):

That doesn’t work because you’d have to wrap it in the record

view this post on Zulip Richard Feldman (Jun 24 2024 at 16:50):

oh true

view this post on Zulip Richard Feldman (Jun 24 2024 at 16:51):

yeah ok then runtime error :+1:

view this post on Zulip Richard Feldman (Jun 24 2024 at 16:53):

Agus Zubiaga said:

Richard Feldman what do you think about the update-style syntax?

I like the general idea! :+1:

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 16:53):

We could copy the one field and pass it to both arguments of map2, but that’d change behavior in some cases

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 16:55):

That’d be horrible nvm

view this post on Zulip Sam Mohr (Jun 24 2024 at 16:59):

I was under the assumption that a one-field record builder with the new syntax would simply not be called by map2, and would be just added to a single element record. If something like Weaver needs to have 2 elements per arg definition record, then usage is inconsistent between large and small records

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:01):

The problem is you cannot call map2 with less than 2 things, but you have to return a record instead of the value of the field directly

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:02):

Can you share an example of a one field record builder in weaver?

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:09):

Any time we just want to parse a subcommand and nothing else: https://github.com/roc-lang/basic-cli/blob/aa1a19802708fc32d418b6208ae4e4a18389dbde/examples/args.roc#L30

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:10):

I have a few examples of that exact use case, so I'll avoid linking them

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:12):

I could make it so that there's also a Cli.finishJustSubcommands alternative that doesn't take a builder, but then we're still working around a syntax that is inconsistent between large and small records, which seems weird

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:12):

It may be worth it for the simplicity of the map-2 based approach to allow this, but it's still important to understand what will break with this

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:13):

Also, roc-query will probably break with this...

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:13):

Yeah, the API will have to change for sure, but I don't think it'd make it worse

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:13):

Something like this should work:

    Subcommand.required [maxSubcommand, divideSubcommand],
    |> Cli.finish {
        name: "args-example",
        description: "A calculator example of the CLI platform argument parser.",
        version: "0.1.0",
    }

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:14):

Here's a query that only gets one field: link

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:15):

In that case, I think it'd look like:

    buildQuery "LoggedInQuery" \{ root } ->
        Query.field root.loggedIn

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:16):

The point is, the functions would have to change from applying to returning the "batch" type directly. So whether you use a builder or not, the types should work out

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:17):

Consider this larger query (link): you'd have to pass a single function for single fields, and a record for larger fields

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:17):

The types should work out yes

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:18):

Can't you just drop the top-level in that case?

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:18):

    buildQuery "UserQueryLarge" \{ root, user, address } ->
-        Query.start {
-            user: <-
                Query.start {

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:19):

The worry for roc-query is that it makes sure that the record you build matches the JSON structure of the API, so that when we decode using roc's built-in Decoding we get the right structure out

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:19):

That might break, though I think it would be fine

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:19):

Ohh, I didn't think roc-query would use Decoding at all

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:19):

I thought it would generate the decoders from the schema, I see

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:19):

GraphQL uses JSON, so we need to as well

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:20):

And Roc doesn't have raw JSON yet

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:20):

Gotcha

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:21):

I never thought about using Record Builders AND Decoding together :thinking:

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:24):

Also,

Can't you just drop the top-level in that case?

    buildQuery "UserQueryLarge" \{ root, user, address } ->
        Query.start {
            name: <- Query.field user.name,
            age: <- Query.field user.age,
            address: <-
                Query.start {
                    state: <- Query.field address.state,
                }
                |> Query.object user.address,
        }

now has to be this:

    buildQuery "UserQueryLarge" \{ root, user, address } ->
        Query.start {
            name: <- Query.field user.name,
            age: <- Query.field user.age,
            addressState: <- Query.object (address.state) user.address,
        }

Which at least breaks decoding

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:24):

What happens if I change the name of user? That'd break, right?

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:25):

Yeah, it currently breaks

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:26):

As you can see in the Query.finish type annotation, we ensure that a query can only be built if the type of the record and the resolved API type we got from the fields we added are the same object layout

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:27):

I think Decoding makes sense when you're just going off the inferred types, but if you're generating Roc code anyway, you can include the decoders for full safety

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:27):

Easier said than done, I know :smiley:

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:28):

The problem with that is we now have a 2-pass decoding step when we could have just one. We can decode to raw JSON and then to our actual data, but that's slower and more memory intensive. Users are now punished because they wanted a convenient syntax

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:29):

Not like Roc is a super performant, but it's pretty performant, so users that understand how this works will understand that this library is convenient to use, but slower than hand-written code for large JSON data returned

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:30):

Which isn't true for the current implementation: it's as fast as a hand written version

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:30):

Now

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:31):

I think all of these problems go away if we allow users to make a record builder with zero or one fields, and just don't call map2 on it

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:31):

If we allow that, Weaver and roc-query work the same way from the perspective of the user, assuming I can get the builder stuff to work, which should not be too bad

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:34):

Sam Mohr said:

The problem with that is we now have a 2-pass decoding step when we could have just one. We can decode to raw JSON and then to our actual data, but that's slower and more memory intensive. Users are now punished because they wanted a convenient syntax

Well, you don't really need the two passes. We can have a library that, like Decoding, incrementally parses the json and your generated decoders would use that.

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:35):

Sam Mohr said:

I think all of these problems go away if we allow users to make a record builder with zero or one fields, and just don't call map2 on it

The problem is that we cannot change the shape from the outside

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:36):

If we don't call map2 at all and just returned the value of the field, the produced value would not be a record with the field inside. It would just be the value of the field.

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:36):

So the types won't line up

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:39):

{ map2 <-
    id: getIdSomehow
}

If we desugar that to:

getIdSomehow

The type is Id, not { id: Id }

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:39):

That's how it currently works, right?

Cli.weave {
    verbosity: <- Opt.count { ... },
}
|> Cli.finish { ... }

translates to

Cli.weave (\verbosity -> { verbosity })
|> Opt.count { ... }
|> Cli.finish

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:41):

The new way, I'd expect, would be:

{ Cli.weave <-
    verbosity: <- Opt.count { ... },
}
|> Cli.finish { ... }

which would go to

(\verbosity -> { verbosity })
|> Opt.count { ... }
|> Cli.finish { ... }

right?

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:43):

{ Cli.weave <-
    verbosity: <- Opt.count { ... },
    somethingElse: <- Opt.count { ... },
}
|> Cli.finish { ... }

would desugar to:

Cli.weave
    (Opt.count {...})
    (Opt.count {...})
    \verbosity, somethingElse -> {verbosity, somethingElse}

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:44):

The one field version doesn't work because we cannot call weave

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:45):

If it's one field, can we just desugar to

(\verbosity -> { verbosity })
|> Opt.count { ... }

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:45):

That shouldn't break anything

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:45):

Opt.count doesn't return a function in this world

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:45):

Yeah

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:45):

So you cannot call it like that

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:46):

True

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:47):

Okay, yeah, I might have been thinking about how the new approach desugars incorrectly

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:47):

Let me go back and re-read, and then comment back here with my understanding

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:47):

In order to support 1-field record builders, we either need to pass map or wrap (what you currently call weave)

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:50):

Weaver was designed very specifically to work with the existing desugaring

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:50):

Task.concurrent
    (Http.get "/users")
    (Task.concurrent
        (Http.get "/posts")
        (Http.get "/news")
        \posts, news -> (posts, news)
    )
    \users, (posts, news) -> { users, posts, news }

This works if the type of Task.concurrent is Task.concurrent : Task a err, Task b err, (a, b -> out) -> Task out err, right

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:51):

And if stuff like Weaver breaks so that users have a better experience overall, I'm totally fine with that

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:51):

Same with roc-query

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:52):

I don't think it would break, but it'd look more like Elm packages that do stuff like this

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:52):

I do get why 1-field record builders are nice for things like Weaver

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:53):

Especially for starting with one thing, and easily adding more later

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:54):

This not being possible doesn't prevent you from doing anything, but it's more cumbersome initially

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:55):

I'm gonna see if I can make types that work for Weaver with the new syntax and then come back to this

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:56):

It seems like I keep running into the thought that "my answer here depends on whether this code I'm imagining would actually work"

view this post on Zulip Sam Mohr (Jun 24 2024 at 17:56):

So I'll just write that code to shore up my worries

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 17:58):

Yeah, I'd imagine it'd be hard to convert an existing API to this new idea. For me it's the opposite with roc-pg, because I started with the usual mapN functions and then added record builders (when I implemented them).

view this post on Zulip Sam Mohr (Jun 24 2024 at 18:00):

Haha, interesting

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 18:14):

Brendan Hansknecht said:

I guess we could make it take both map and map2. Or would could make it take map1or2 that uses a variant....but of this don't feel great though.

something like this?

build : [
    Zero out,
    One (Task a) (a -> out),
    Two (Task a) (Task b) (a, b -> out)
] -> Task out

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 18:15):

Maybe we don’t need zero, that could be a compiler error

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 18:17):

It preserves some of the original goals, but it’s not as nice as just passing map2

view this post on Zulip Kiryl Dziamura (Jun 24 2024 at 18:20):

An optional empty value? Which is an empty wrapper

{ map2, empty <-
    x: ...
}
{ Task.concurrent, Task.ok {} <-
  ...
}

Or ignored fileds for the rescue

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 18:22):

Yeah, that's the first thing I thought, but that seems like a worse user experience

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 18:24):

The Task module could expose a record with both things, but at that point, I rather go with the tags

view this post on Zulip Sam Mohr (Jun 24 2024 at 18:25):

In theory, we just need a way to get config from a record builder with zero or one fields to a function that takes said applicative and finishes everything, like Task.await or Cli.finish

view this post on Zulip Sam Mohr (Jun 24 2024 at 18:26):

Also, the person writing the record builder knows how many fields there are, so we don't need map2, empty, we just need map2 with 2+ fields and empty for 0/1 fields

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 18:27):

Yeah, it's just annoying to have to switch

view this post on Zulip Sam Mohr (Jun 24 2024 at 18:28):

I don't think so, compared to the alternative of not being able to make a record builder at all with just one field

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 18:28):

That's true

view this post on Zulip Sam Mohr (Jun 24 2024 at 18:28):

A little annoying yes

view this post on Zulip Sam Mohr (Jun 24 2024 at 18:29):

But it's a consistent means that lets stuff like Weaver and roc-query still be shaped the same, more or less

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 18:30):

So you'd start with something like:

{ Cli.map <-
    verbosity: Opt.count { ... },
}
|> Cli.finish { ... }

and then switch to:

{ Cli.map2 <-
    verbosity: Opt.count { ... },
    somethingElse:  Opt.count { ... },
}
|> Cli.finish { ... }

view this post on Zulip Sam Mohr (Jun 24 2024 at 18:30):

Yep!

view this post on Zulip Sam Mohr (Jun 24 2024 at 18:30):

That seems like the best option by a country mile to me, syntax pending of course

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 18:31):

Yeah, that might be a good tradeoff

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 18:32):

I like that it doesn't require the module to expose something specific to record builders

view this post on Zulip Sam Mohr (Jun 24 2024 at 18:33):

If that wasn't the case, we may as well stick with the current record builder approach

view this post on Zulip Sam Mohr (Jun 24 2024 at 18:33):

So this needs to support "regular" functions IMO

view this post on Zulip Sam Mohr (Jun 24 2024 at 18:33):

Which this does

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 18:33):

I think most importantly, it doesn't require you to understand applicatives

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 18:34):

which have those scary looking types with functions inside :smiley:

view this post on Zulip Sam Mohr (Jun 24 2024 at 18:34):

And for empty records, you just have a

Cli.empty
|> Cli.finish { ... }

So roc-query can still infer your data type because you can always pass your type

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 18:35):

I think for GQL you do need to select at least one thing, right?

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 18:35):

Well, I guess you can default to __typename if empty

view this post on Zulip Sam Mohr (Jun 24 2024 at 18:35):

Presumably yes

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 18:36):

Depending on the API, you might not even need empty, but yeah, I don't think we need to support empty record builders

view this post on Zulip Sam Mohr (Jun 24 2024 at 18:36):

You're right, but the important point is that between Task.concurrent, Task.map, and Task.ok {}, you can make a record with 0-n fields that all return the right type

view this post on Zulip Luke Boswell (Jun 24 2024 at 18:43):

This is looking good. :grinning:

view this post on Zulip Sam Mohr (Jun 24 2024 at 18:55):

Yes, with the map2 and map version (depending on record size), I have things working (without the desugaring, of course). It looks good to me!

https://gist.github.com/smores56/c3d6b9f333e4900fcea25edbc4ac8604

view this post on Zulip Sam Mohr (Jun 24 2024 at 18:56):

Should maybe rename cliMap to cliSingleArg or something, but the point is made

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 18:57):

Nice, that's exactly what I imagined!

view this post on Zulip Sam Mohr (Jun 24 2024 at 18:59):

So yeah, this new syntax:

I'm 100% onboard at this point

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 19:03):

Another cool thing you can do now is map the values directly in the builder:

{ cliWeave <-
    email : strParam { name: "email" } |> map toLower,
    ...
}

view this post on Zulip Sam Mohr (Jun 24 2024 at 19:04):

Wow, that's really nice

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 19:04):

That can be really useful with GQL queries too

view this post on Zulip Richard Feldman (Jun 24 2024 at 21:51):

can we see the side-by-side on the 1-field version vs if 1-field didn't exist?

view this post on Zulip Richard Feldman (Jun 24 2024 at 21:52):

it feels like a weirdly specific situation to have syntax sugar for haha

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 22:07):

With 1-field record builder:

{ Cli.map <-
    verbosity: Opt.count { ... },
}
|> Cli.finish { ... }

Without:

Opt.count { ... }
|> Cli.map \verbosity -> { verbosity }
|> Cli.finish { ... }

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 22:08):

Honestly, it’s hard to see the benefit when you’re just comparing them directly

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 22:09):

The motivation is about easily adding new fields

view this post on Zulip Agus Zubiaga (Jun 24 2024 at 22:15):

You still have to change the map function when going to and from 1-field builders, so the impact is debatable

view this post on Zulip Richard Feldman (Jun 25 2024 at 00:20):

yeah it seems like in either case you have to teach "here's what you do if there's exactly 1 thing you want" and then the question is becomes, "should there be special syntax sugar for that case, which also needs to be taught, but which makes it look more similar to the 2+ field case?"

view this post on Zulip Richard Feldman (Jun 25 2024 at 00:20):

and it doesn't really seem to me like that's worth a dedicated language feature

view this post on Zulip Richard Feldman (Jun 25 2024 at 00:21):

especially since for most uses of it (e.g. Parser, Task, Random) I don't think people would even use it in the 1-arg case

view this post on Zulip Richard Feldman (Jun 25 2024 at 00:21):

they'd just use .map directly instead of making a record with one field and then destructuring it

view this post on Zulip Kiryl Dziamura (Jun 25 2024 at 00:21):

I understand the weirdness, but let's exhaust the options. What if the chaining function is

build : (T a) ([Some T b, None]) (a, [Some b, None] -> out) -> T out

Or

build : [One (T a), Two (T a) (T b)] ([One a, Two a b] -> out) -> T out

view this post on Zulip Kiryl Dziamura (Jun 25 2024 at 00:26):

How often we might see similar code manually written?

Task.concurrent taskA (Some taskB)

view this post on Zulip Agus Zubiaga (Jun 25 2024 at 00:47):

Kiryl Dziamura said:

build : (Task a) ([Some Task b, None]) (a, [Some b, None] -> out) -> Task out

That signature doesn't really work because you need b to be always there when the second argument is Some. We need to include the function in the variant like this:

Agus Zubiaga said:

build : [
    Zero out,
    One (Task a) (a -> out),
    Two (Task a) (Task b) (a, b -> out)
] -> Task out

view this post on Zulip Agus Zubiaga (Jun 25 2024 at 00:49):

Richard Feldman said:

especially since for most uses of it (e.g. Parser, Task, Random) I don't think people would even use it in the 1-arg case

I think even when writing roc-pg selections, I'd flatten it if I only want to query one thing

view this post on Zulip Agus Zubiaga (Jun 25 2024 at 00:51):

The structure of the record doesn't affect the query shape, it will always do the right thing

view this post on Zulip Sam Mohr (Jun 25 2024 at 00:52):

Yeah, it's entirely a visual thing except for maybe Encoding or Decoding

view this post on Zulip Sam Mohr (Jun 25 2024 at 00:52):

Just because it's a visual thing, however, doesn't mean we shouldn't do it.

view this post on Zulip Agus Zubiaga (Jun 25 2024 at 00:53):

Yeah, that's where it gets tricky, because the structure in Roc affects the shape of the data

view this post on Zulip Agus Zubiaga (Jun 25 2024 at 00:56):

I know Decoding is the best thing we have for JSON decoding now, but I still think ideally we shouldn't use it for cases like roc-query

view this post on Zulip Agus Zubiaga (Jun 25 2024 at 00:57):

All the information is available, there's no need to resort to a inferred decoders

view this post on Zulip Sam Mohr (Jun 25 2024 at 00:58):

When RawJson gets implemented it's more achievable

view this post on Zulip Agus Zubiaga (Jun 25 2024 at 01:00):

Yeah, I imagine roc-json could expose its partial decoders directly in addition to the Decode implementation

view this post on Zulip Sam Mohr (Jun 25 2024 at 01:01):

That was how I started doing it, basically

view this post on Zulip Sam Mohr (Jun 25 2024 at 01:01):

You at the very least need to pull them out from just being usable for the @Json opaque type so you can use them for RawJson and Json

view this post on Zulip Sam Mohr (Jun 25 2024 at 01:02):

But being able to decode an object or a string in particular would be beneficial

view this post on Zulip Agus Zubiaga (Jun 25 2024 at 01:03):

You can totally make a elm/json-style decoder and use that to power these style of schema-based codegen tools

view this post on Zulip Sam Mohr (Jun 25 2024 at 01:06):

Yeah, that's how it would work under the hood

view this post on Zulip Kasper Møller Andersen (Jun 27 2024 at 10:12):

For my proposal of function-based record builders, it seems like there's lukewarm feelings on it because it seems to require higher kinded types (HKTs). But I don't really understand how the proposal here does not require HKTs.

That is, if we have a record builder syntax which requires users provide a function like

build : [
    Zero out,
    One (Task a) (a -> out),
    Two (Task a) (Task b) (a, b -> out)
] -> Task out

and users are expected to use that with syntax a la

{ Task.build <-
    users: Http.get "/users",
    ...
}

doesn't the compiler need to be able to verify that Task.build matches some type signature?

In other words, if a user writes

{ List.map <-
    users: Http.get "/users"
}

this is obviously wrong, and they need to get a compiler error. But what does that error say about having a wrong type signature on this function? The compiler still needs to tell users what signature it is expecting this function to have after all. Maybe the concrete types needed for the given build function can be inferred from the given record fields? That way the error message presented can have a concrete type signature that is specific to Task, or whatever thing the user is trying to build with, as opposed to being described in terms of HKTs.

Another aspect is teaching this. Where do users go to see the signature of the function they must obey when they want to use a builder like this? If we can't specify the signature generally, do we then push the complexity on to the user instead of keeping it in the language?

In other words, this proposal still seems to rely on HKTs conceptually, and I would worry that we end up merely hiding or moving this complexity as opposed to removing it. And if we can't remove it, I'm personally more a fan of being up front about it.

view this post on Zulip Kiryl Dziamura (Jun 27 2024 at 10:23):

It would be great to have type error for unexpected function signature. In the current proposal the invalid application error will be highlighted on the fields which could be confusing indeed

view this post on Zulip Agus Zubiaga (Jun 27 2024 at 10:34):

This proposal, like the current implementation, does not require HKTs. We basically get type checking for free because the syntax is desugared before constraining.

The difference is that function-based record builders would desugar to an intermediate RecordBuilder type that can later be applied by passing it and map2 to a generic function. That’s what requires HKTs.

view this post on Zulip Agus Zubiaga (Jun 27 2024 at 10:36):

Even though the type checking occurs on the desugared code, all the error messages will point to regions on the real source, and we can add metadata during desugaring so that we can provide more helpful messages (specific to record builders)

view this post on Zulip Agus Zubiaga (Jun 27 2024 at 10:42):

Here’s an example of an error customized because it originated from a record builder

view this post on Zulip Agus Zubiaga (Jun 27 2024 at 10:45):

I believe we can make our errors more helpful than the current implementation because there are more constraints

view this post on Zulip Kiryl Dziamura (Jun 27 2024 at 11:09):

The main concern is that if your build signature is not valid, you might think that it's a problem with fields. But maybe it's a rare case.

view this post on Zulip Agus Zubiaga (Jun 27 2024 at 11:15):

I see. Yeah, that’s a tricky one. Unfortunately, without HKTs, there’s a limit to how much we can constrain the function itself.

view this post on Zulip Agus Zubiaga (Jun 27 2024 at 11:16):

I think we’ll have to point out that this could be the case in the error message

view this post on Zulip Agus Zubiaga (Jun 27 2024 at 11:22):

In the end it’s like function application, if your arguments don’t match what the function excepts, they’ll get type errors, not the function

view this post on Zulip Agus Zubiaga (Jun 27 2024 at 11:23):

We can probably detect some cases like “the number of args doesn’t match” and highlight the function instead

view this post on Zulip Kiryl Dziamura (Jun 27 2024 at 11:40):

Forgive me my ignorance, wouldn’t it be beneficial to have HKT internally only for linting? I imagine it might be helpful also for potential ! to andThen expansion.

Not in the scope of this record builder proposal obv

view this post on Zulip Agus Zubiaga (Jun 27 2024 at 11:46):

Right. However, in order for HKTs to be useful, we need to expose them to user-space at least for implementation.

Fully internal HKTs would only work if the type had one type variable, because otherwise (e.g Task), how do you know which one to pick?

view this post on Zulip Sam Mohr (Jun 27 2024 at 12:12):

@Kiryl Dziamura HKT's are off the table largely for performance reasons (in addition to learnability), so even having them internally could make the "compile fast" goal that much harder to maintain

view this post on Zulip Kiryl Dziamura (Jun 27 2024 at 12:30):

which one to pick

Ah, of course. Just didn’t think thoroughly about it

view this post on Zulip Richard Feldman (Jun 27 2024 at 12:31):

Sam Mohr said:

Kiryl Dziamura HKT's are off the table largely for performance reasons (in addition to learnability), so even having them internally could make the "compile fast" goal that much harder to maintain

worth noting that this depends on how far we were to go with them...last year I learned that limited forms of HKTs don't have the same drawbacks that fully general HKTs do

view this post on Zulip Richard Feldman (Jun 27 2024 at 12:31):

that said, I don't think this syntax sugar needs to get the type checker involved :big_smile:

view this post on Zulip Agus Zubiaga (Jun 27 2024 at 12:33):

Agreed. Just reporting involved :grinning:

view this post on Zulip mainrs (Jul 02 2024 at 13:30):

I tried looking Task.concurrent up but couldn't find anything about this. Does roc nowadays allow for concurrent tasks to take place?

view this post on Zulip Anton (Jul 02 2024 at 13:37):

Not yet, I think we listed the required steps to land that somewhere but I can't find it

view this post on Zulip Sam Mohr (Jul 05 2024 at 06:35):

For syntax parsing of these, do we want to ensure that the map2 function is on its own line, or is it okay to have the whole builder on a single line? Would the following be readable to you all, or newcomers:

apiData = { Task.concurrent <- users: getField "users", events: getField "events" }

or should we force multiline builders:

apiData =
    { Task.concurrent <-
        users: getField "users",
        events: getField "events",
    }

I definitely think the multiline version is more clear, but I think the first version is clear enough that it's not worth preventing devs from writing builders on one line.

view this post on Zulip Kiryl Dziamura (Jul 05 2024 at 06:45):

I think oneliners are fine. They’d have even more sense for tuple builders, I’d assume we’ll have them eventually. But tuples deserve their own discussion ofc.

(users, events) = (Task.concurrent <- getField "users", getField "events")

view this post on Zulip Anton (Jul 05 2024 at 09:09):

I think we should be able to parse both, and perhaps roc format should make it multiline once there are more than two fields?

view this post on Zulip Sam Mohr (Jul 05 2024 at 09:11):

I'm structuring the parsing just like the record updater & syntax, so that's roughly how it will work

view this post on Zulip Sam Mohr (Jul 05 2024 at 09:12):

The difference being that formatting for record updates doesn't force multiline formatting. I actually think we should stick with that, since it's more consistent, and Roc doesn't have a max line limit in general

view this post on Zulip Sam Mohr (Jul 05 2024 at 09:18):

We do have a default line wrap for the old record builder syntax if the value of a field is somewhat complex

view this post on Zulip Sam Mohr (Jul 05 2024 at 09:19):

Though I personally prefer being given the option to decide myself, roc format is intentionally opinionated to avoid formatting discussions from needing to happen on dev teams, meaning we should always line break if there are more than 2 fields, probably.

view this post on Zulip Sam Mohr (Jul 08 2024 at 02:05):

So, interesting thought: I was planning on coming back here to request a re-consideration on allowing single field record builders because of a use case I wanted to make convenient, and that is giving argument context at a call-site with record builder-sourced data. What do I mean by this? If I have a function that builds a Task { foo: Str, bar: Str } and I consume that Task's data in a closure, I can write the function as \{ foo, bar} -> ... and even though both of my arguments are strings, I can't accidentally switch the arguments or forget which one is which.

This isn't true with a single variable, however. If I'm trying to pass a closure to Task.await for a Task Str, there are no structural restrictions on what I call the Str, I could call it foo or bar or anything else, so the type of the function doesn't tell me what data to expect.

However, @Agus Zubiaga mentioned that this new syntax allows us to map over record builder field values normally, since they're treated as normal values. That means that we can do something really simple:

multipleArgs =
    { Cli.weave <-
        foo: Arg.str,
        bar: Arg.num,
    }

singleArg =
    Arg.str |> Cli.map Foo

So if I wanted to consume the singleArg variant, I could just write a closure \Foo foo -> ..., and even if the dev doesn't name the pattern inside the Foo tag "foo", they still know what value they're dealing with besides it being a string. And since it's a tag union with a single variant, it's a zero-byte wrapper around the inner value, meaning this is a simple abstraction with no runtime cost!

All that said, unless this turns out to be a really annoying way to write Weaver parsers over one that uses single field record builders (I'd be surprised), I think I'm now convinced we don't need to support single field record builders.

view this post on Zulip Brendan Hansknecht (Jul 30 2024 at 15:21):

Do we have an update example for making a record builder now?

view this post on Zulip Brendan Hansknecht (Jul 30 2024 at 15:22):

I don't think the tutorial or example on the website got updated

view this post on Zulip Sam Mohr (Jul 30 2024 at 15:22):

No, you're right

view this post on Zulip Sam Mohr (Jul 30 2024 at 15:23):

I started writing a change, but I got stuck on giving the user the right mental model on when they would want to use a builder

view this post on Zulip Sam Mohr (Jul 30 2024 at 15:24):

I think just saying "when you want to combine complex data" or "you want a clean interface" is way too generic, and "you want to combine tasks in parallel" is too specific

view this post on Zulip Sam Mohr (Jul 30 2024 at 15:24):

I don't think the old example did a good job of this either, it more just explained how desugaring worked

view this post on Zulip Brendan Hansknecht (Jul 30 2024 at 18:31):

Do we have an example elsewhere? I want to try and port something to it.

view this post on Zulip Sam Mohr (Jul 30 2024 at 18:33):

Weaver now uses the new syntax, and uh

view this post on Zulip Sam Mohr (Jul 30 2024 at 18:33):

https://github.com/roc-lang/basic-cli/blob/main/examples/record-builder.roc

view this post on Zulip Brendan Hansknecht (Jul 30 2024 at 18:35):

:thank_you: Thanks!

view this post on Zulip Richard Feldman (Jul 30 2024 at 22:54):

regarding this snippet:

    myrecord : Task { apples : List Str, oranges : List Str } []_
    myrecord = { Task.concurrent <-
        apples: getFruit Apples,
        oranges: getFruit Oranges,
    }

    { apples, oranges } = myrecord!

view this post on Zulip Richard Feldman (Jul 30 2024 at 22:55):

seems like it'd be common to want to await the record builder, which I guess could be done like this, although it looks weird:

    { apples, oranges } = { Task.concurrent <-
        apples: getFruit Apples,
        oranges: getFruit Oranges,
    }!

view this post on Zulip Richard Feldman (Jul 30 2024 at 22:56):

what if we instead allowed moving the ! inside the builder, and having that await the combined task?

    { apples, oranges } = { Task.concurrent! <-
        apples: getFruit Apples,
        oranges: getFruit Oranges,
    }

view this post on Zulip Brendan Hansknecht (Jul 30 2024 at 23:54):

I wonder how common that will be in practice.
For example, in Sqlite3, the recorder builder is using tasks, but it is always passed to something that will run the task anyway:

SQLite3.execute!
    stmt
    { SQLite3.map2 <-
        id: SQLite3.i64 "id",
        task: SQLite3.str "task",
    }

view this post on Zulip Richard Feldman (Jul 31 2024 at 00:00):

I'm just thinking like "I want to run two http requests in parallel and then do something after they both finish"

view this post on Zulip Agus Zubiaga (Jul 31 2024 at 00:03):

Richard Feldman said:

    { apples, oranges } = { Task.concurrent <-
        apples: getFruit Apples,
        oranges: getFruit Oranges,
    }!

Honestly, I think that one is fine

view this post on Zulip Richard Feldman (Jul 31 2024 at 02:29):

}! looks weird to me haha

view this post on Zulip Richard Feldman (Jul 31 2024 at 02:30):

but I guess an argument for at least trying it that way is that it needs to at least be possible to do it that way, so there's value in not introducing another way to do it

view this post on Zulip Sam Mohr (Jul 31 2024 at 02:59):

I mean, all syntax sugar is a nicer way to do something you can already do. I'd say that even though you could put the bang in the record builder or after, if we always set the precedent of putting the bang inside the builder, it's better than }! being the standard


Last updated: Jun 16 2026 at 16:19 UTC