Record Builders have some pretty cool applications already, but seem to be confusing to most people.
Take the example from the initial proposal:
Task.ok {
users: <- Task.batch (Http.get "/users"),
posts: <- Task.batch (Http.get "/posts"),
news: <- Task.batch (Http.get "/news"),
}
and the types of the functions involved:
Task.ok : ok -> Task ok err
Http.get : Str -> Task Http.Response Http.Err
Task.batch : Task a err -> (Task (a -> b) err -> Task b err)
There are quite a few pieces there, and from the surface, it's not clear how they all fit together. Why is Task.ok there? Task.batch returns a function that takes a Task with a function inside!?
If you're explaining it to someone, you first have to show the desugared version, so you can then introduce Applicatives, which are already a mind-bending concept.
Applicatives are pretty convenient in a language like Elm where Record Builders do not exist and you can partially apply functions. However, if we're going to have sugar for this problem, we don't really need them.
What if Record Builders required only map2 and just paired results?
So if I have a map2-like function that combines two tasks into one:
Task.concurrent : Task a, Task b, (a, b -> c) -> Task c
I can combine N tasks into a record like this:
Task.concurrent <- {
users: Http.get "/users",
posts: Http.get "/posts",
news: Http.get "/news",
}
Which would desugar to:
Task.concurrent
(Http.get "/users")
(Task.concurrent
(Http.get "/posts")
(Http.get "/news")
\posts, news -> (posts, news)
)
\users, (posts, news) -> { users, posts, news }
:bangbang: Note: Even though we are making pairs of results, the task themselves would all run concurrently.
I think this is a lot more straightforward. Instead of storing the intermediary results in closures, we just use tuples. There are no mysterious apply/lift functions on the surface, and if you want to understand the desugaring, you only need to understand map2.
This also has the benefit that the module (e.g. Task) likely already provides map2 and it doesn't have to expose functions that are specific to Record Builders.
Finally, I think this will lead to more flexibile APIs. I noticed that basically all the applications of Record Builders expose convinence functions that do too much.
For example, roc-pg's query builder exposes a column function, so that instead of writing the following for each field:
Sql.into {
id: <- Sql.apply (Sql.select products.id),
price: <- Sql.apply (Sql.select products.price),
}
You can just write:
Sql.into {
id: <- Sql.column products.id,
price: <- Sql.column products.price,
}
That's shorter, but the type out of Sql.column is Selection (a -> b) -> Selection c, which is not very amenable to further transformation.
With the map2-based approach, the type at each field is simply Selection a, so I can just use Sql.map if I want to:
Sql.combine <- {
id: Sql.select products.id,
price: Sql.select products.price |> Sql.map fromCentsToDec,
}
And I can even have nested records without any extra work:
Sql.combine <- {
id: Sql.select products.id,
meta: Sql.combine <- {
name: Sql.select products.name,
manufacturer: Sql.select products.manufacturer,
}
}
I love this proposal. I think the map2 design makes things a lot simpler to understand and, and as you said, the types are a lot nicer as well. Having the signature of the functions used be more generally applicable really seems like a nice benefit both for chaining in record builders and for building more modular code bases.
Kasper Møller Andersen said:
So I would take a good hard look at whether the sigil should actually be an arrow. My experience is that I tend to think of the arrow sigil as "input -> output".
My only thought is that in the vein of Kaspers comment here, maybe a different operator could be found for record building. Just a thought, but what about <+? That would be similar to the current operator, but the plus adds the connotation of combination.
Hm, I get the input -> output idea, but I don't know if <+ is more descriptive of what's going on with Record Builders :thinking:
Maybe something like this?
Sql.combine ~{
id: Sql.select products.id,
price: Sql.select products.price,
}
Maybe <= because it resembles the arrow but it takes more values?
That might be weird because it's also "less than or equal"
which code fonts with ligatures would render as ≤
Oh yeah didn't think of that because I use ligatures :sweat_smile:
The idea behind ~ is that builders sort of thread the function through the fields
Throwing another option out there:
Sql.combine <~ {
id: Sql.select products.id,
price: Sql.select products.price,
}
I think the <- backwards arrow will be fine if we remove backpassing... there wont be anywhere else you see it then.
I honestly think the single arrow <- is best, but yeah some discussion should be had. I don't think we should bikeshed too much though
Can we add Task.map2 and just have it run things sequentially for now? Or does this proposal need to wait for Effect Interpreters?
I'm not really concerned about the conflict with backpassing. I think even if we kept it, we could distinguish in the parser because a { on the RHS of the backpassing arrow doesn't make sense.
It might be confusing if we have both and use the same symbol, though
Luke Boswell said:
Can we add
Task.map2and just have it run things sequentially for now? Or does this proposal need to wait for Effect Interpreters?
We can, but I don't know if we would gain much. You can just use ! if you want to run things sequentially.
I only used the Task example because it's a simple one. Record Builders are not tied to tasks.
Record Builders are not tied to tasks.
Oh nice, I didn't quite follow that. So anything with that shape can use record builders to build up a record?
Yeah, I don't think I've seen any examples of using Record Builders with tasks yet
They only really make sense for tasks if you can run them concurrently
The examples I showed above with roc-pg are not task-based, and neither is weaver
Agus Zubiaga said:
We can, but I don't know if we would gain much. You can just use
!if you want to run things sequentially.
By this I mean:
users = Http.get! "/users"
posts = Http.get! "/posts"
news = Http.get! "/news"
or if you want them in a record:
{
users: Http.get! "/users",
posts: Http.get! "/posts",
news: Http.get! "/news",
}
Compared to the existing record builder syntax, these seem restricted to all fields having the same type, and the old one is not, right? That's not necessarily a problem, but just something worth highlighting.
No, fields can have different types as long as the map2 function allows it.
@Kasper Møller Andersen I don't believe that's the case, it's just a bit trickier to use, maybe? Given the example Agus gave:
Task.concurrent
(Http.get "/users")
(Task.concurrent
(Http.get "/posts")
(Http.get "/news")
\posts, news -> (posts, news)
)
\users, (posts, news) -> { users, posts, news }
We can see that it's giving us an eventual \a, (b, (c, (d, ...)))) -> output function. We just get the values in a different order than in the old approach
I'll try seeing if I can make Weaver work with the desugared version of this syntax, and if that works, then I'm all in on this syntax change! This should support both normal builders like Weaver, but also very simple stuff like parallel task awaits, or trying multiple results at once very conveniently
Okay, so doable in theory at least, but I guess the ordering of the keys versus the ordering of the map2 args becomes relevant? As in, if you don't order your keys such that they desugar in the order your map2 function expects, you'll get a compile error.
I guess I don't really get what are you trying to do. The fact tuples are used internally is really just an implementation detail. From the perspective of map2, you don't really care.
The same thing is true of how the old way works, basically. If you order it differently, the functions change signature, but it's okay because the record fields change order in the final function as well
@Sam Mohr The order of what part?
The order of the fields shouldn't matter
the fields in the struct
Yeah, it doesn't
But, technically, the functions that desugar out are shaped differently
Right, but as the author of the package/module, you don't provide the closure that destructures the tuples, it's generated automatically.
Okay, I'm trying to get the scenario written down, but I can't fully articulate it anyway, which is probably a sign that it's not super relevant :big_smile:
A different thought: it feels funny that this essentially makes map2 functions variadic, except the arguments don't become a list, but a record (so "named variadics" maybe?)
I guess the piece I feel is sort of missing is that it becomes a particular skill to look at a function signature and be able to read whether I can use this sugar with it or not. It's still way better than the existing syntax, but compared to other languages with variadic functions, the unclear signposting feels suboptimal.
Which I think is also why I was struggling with articulating my previous point: I understand the happy path of using this, but it's hard for me to grasp what the contract with map2 function is before I can use this sugar.
Yeah, that's a valid point. Unfortunately, I don't really see a way around that without introducing HKTs.
We could maybe require that the function's type looks like:
_, _, (a, b -> c) -> _
This would at least prevent people from depending on the fact that we use tuples internally.
In terms of language consistency, let’s assume we have the following syntax:
m2 ~{ x: fx a }
Should this new record builder work with pizza operator somehow?
~{ x: fx a } |> m2
Or, is it possible to pass the record as a variable?
r = { x: fx a }
m2 ~r
It feels like the sigil is more about modifying the function interface rather than the record, so it’s probably better to tie the sigil to the map2 function call as a modifier. It will also be simpler for parser to generate a meaningful error (e.g. “the interface of the function is not map2”). I didn’t think much about the beauty of the approach, just for example, I hope the idea is clear:
m2@mapRecord { x: fx a }
{ x: fx a } |> m2@mapRecord
r = { x: fx a }
m2@mapRecord r
notM2@mapRecord { x: fx a } # error: notM2 should be aligned with map2 interface
However, my suggestion probably moves it from static “template-based” record building to dynamic “sugar-based” record transformation.
This also has an advantage, the records mapping becomes generic and straightforward. But it probably provides too much freedom? Maybe record mapping shouldn't be something easy to do and in general, considered as an antipattern?
Speaking of fitting with the existing language constructs, I guess this example:
Task.concurrent <- {
users: Http.get "/users",
posts: Http.get "/posts",
news: Http.get "/news",
}
should actually be
Task.concurrent! <- {
users: Http.get "/users",
posts: Http.get "/posts",
news: Http.get "/news",
}
(note the !)
in order to be useful, right?
It would only mean that the resulting task is awaited, no?
That's my thinking, yes.
Also, just going off on something of a tangent here, but I promise it ties back into record builders :smiley:
We're talking about syntax sugar for something where you essentially describe how to build a record, using some input which also looks like a record, even though it isn't. But what would it take for the input to be an actual record?
The reason I bring it up is because I've been wanting a way to work with records more dynamically in Elm for some time, essentially being able to iterate over fields of a record, so I could use functions like Record.map, Record.walk, etc.
Having such functions would make you able to use records kind of like a Dict, and would potentially also make the record builder use case quite elegant I think. Aside from those functions, the part that's missing is being able to describe a record without describing its actual field names, but simply the types in it, like what you can in TypeScript with index signatures:
interface Foo {
[key: string]: number;
}
or
type Index = 'a' | 'b' | 'c'
type FromIndex = { [k in Index]?: number }
Doing this would make the "variadic" use case much clearer too, as a function can now declare that it takes a record with varying fields.
In Elm, I've specifically wanted this every time I've had the urge to use a tagged union as keys in a Dict. That is, I know all the fields up front, but for one reason or another, I still need to be able to iterate through them.
Using an actual record like this would make the suggestion of ignored fields less nice though.
I also get that this goes into a level of dynamism that is something of a departure for Roc, so it may not fly at all, but I wanted to make sure it was brought up at least :smile:
I like the std approach more than my function modifier proposal! It would only be a bit more verbose I think, but much more clear
Sorry, I'm a little confused. Is the "std approach" the one I outlined above? Or another one? :sweat_smile:
I think the only place where this kind of Record.map can be implemented is std. So I was talking about your proposal
I also feel it's much clearer. A function can explicitly take a "dynamic record" as input, and apply the transformations here with operations we know quite well at this point.
One potential downside though is that I don't know if we can rely on the ordering of the fields in the record, and whether we need to be reliant on it.
I’d say open another #ideas thread about that idea :big_smile:
@Agus Zubiaga have tuple builders been already discussed? Does it make sense to expand record builder to structure builder?
Yeah, we did talk about it for the initial implementation.
IIRC one of the main issues was that the existing syntax didn’t really work because each field has an <-, so we’d have to support something like:
wrap (<- apply …, <- apply …)
If that works, it’d be reasonable to think this should also work:
wrap (
if <- apply someBooleanProducingThing then
<- apply …
else
<- apply …
)
Which is pretty wild, and makes it hard to reason about control flow, because unlike ! everything has to be aggregated ahead.
That said, in the new proposed syntax, the <- is moved from each field to between the map2 function and the “record”, so something like this could work:
Task.concurrent <- (Http.get "/users", Http.get "/posts")
It might make sense to support it for consistency, but it’s probably very rare
Also, speaking of “the same type for all fields restriction” problem mentioned above, isn’t the following code a problem with this proposal?
Task.concurrent <- {
id: id, # let’s say we already got it from somewhere
users: Http.get "/users",
posts: Http.get "/posts",
news: Http.get "/news",
}
So what’s the type of id? I’m not very experienced in FP, but it looks like the proposed pattern is monadic, right? Meaning, Task.concurrent is for composing and there also should always be a wrapping operation, which is Task.ok in this case. So id should be a Task, right? Can it lead to wrappers hell when composition is a bit trickier?
Not exactly monadic, but it’s true that’d you’d have to wrap a value in the type with some function like Task.ok, if you want it to go as is in the record.
I much prefer not to have to specify the function for every single field that you do want to chain
Well, that answers my question from earlier I think, thanks for making it understandable @Kiryl Dziamura :big_smile:
I think having to use Task.ok is fine, that’s how it works with !, and in my experience it’s quite rare with record builders
Kasper Møller Andersen said:
Well, that answers my question from earlier Kiryl Dziamura :big_smile:
Oh, I thought you were talking about the types of the fields in the final record! :grinning:
Another implication I just realized I didn’t mention is that a record builder would have to have at least 2 fields, because we wouldn’t be able to call the map2 function with fewer than that.
We would need to also take a map or a wrap function to support 0..1 builders, which does not seem worth it for a case where you’re not combining anything so you don’t really need builders.
2 fields
It can be solved with ignore fields i think. Tho it will look weird
Yeah, if you really need a record with one thing, you can map it afterwards:
Http.get "/users" |> Task.map \users -> { users }
or use ignored fields as you said:
Task.concurrent <- {
users: Http.get "users",
_: Task.ok {},
}
Alternative syntax idea: meta fields for the record literal. I tend to put the chaining operation somewhere under the brackets to avoid ambiguity with regular records. Also, this way parser can generate more readable error for expected map2 compliance
{
@with: Task.concurrent,
users: Http.get "/users",
posts: Http.get "/posts",
}
And just for extreme verbosity, ignore fields: :grinning_face_with_smiling_eyes:
{
@with: Parser.combine,
year: Parser.int,
@then: Parser.symbol "/",
month: Parser.int,
@then: Parser.symbol "/",
day: Parser.int,
}
One of my issues with the map2 based syntax is that it's not clear that order actually matters, because it usually doesn't in records. So making it clearer that the ordering does matter is a win IMO :blush:
It doesn’t matter for the memory layout, but during record building the order of calls is preserved. I’d assume always
Order matters in the applicative approach the same
I mean,
{
b: 1 + 2,
a: 3 * 4,
}
Summation is the first operation, multiplication is the second. I think records can be implied as named tuples so to speak. I mean, there’s no order ambiguity in the case of tuples, right?
Yes, sorry, it's not specific to map2 versus the existing syntax :blush:
I personally view records as unordered. With your example, if you had two records with the same content, but different order of fields, an == would say they are the same.
That’s true. However, these are not records, they are builders that produce records. I think it’s important to preserve the order for some use cases such as parsers, and CLI arg docs.
I agree, though my concern is exactly that they look like records, implying record-like behavior. Which is why I like the idea of having them look less like normal records :blush:
I see
I mean, it’s not necessarily regular records are order agnostic. When it comes to effects, the order matters anyway. The following records are different:
{
a: set! "x" 42,
b: get! "x",
}
{
b: get! "x",
a: set! "x" 42,
}
Yes, after desugaring it becomes clear that the order matters but it still looks like a regular record.
I think it’s more a feature than a problem. It allows writing more compact code. In any case, what really matters is the order of calculations not the order of assignments.
Yeah, records are order agnostic, record literal expressions aren’t
Okay, I tried to expand on how we might solve the issues that I personally think still remain with the record building in this proposal: https://roc.zulipchat.com/#narrow/stream/304641-ideas/topic/Record.20builder.20as.20a.20function/near/446616377
@Kasper Møller Andersen @Agus Zubiaga what do you think about record literal meta fields proposal I wrote above? It’s still map2-based and distinguishes record builders by introducing meta-field @with (expected to be always top-level defined). I have no concerns about changing the naming.
Personally, I don't love how the "map2" function appears as a field, it does something very different than all the other ones.
For the parser, the <- option is a little easier because you know you're parsing a builder as soon as you encounter it. (In a world without backpassing)
Not a big deal, though
If we want the map2 function to appear inside {...}, what about this?
{ Task.concurrent <-
users: Http.get "/users",
posts: Http.get "/posts",
news: Http.get "/news",
}
It follows the record update syntax pattern
I do like that if the function appears inside, there's no confusion about whether pipes should work or whether you can put the RHS of <- on a variable
If we want to confuse every Elm developer, we can also go with :smiley:
{ Task.concurrent |
users: Http.get "/users",
posts: Http.get "/posts",
news: Http.get "/news",
}
Yeah, having the map2 inside produces less doubtful whitespaces, so to speak
Not sure which symbol to use, but I quite like this direction the more I think about it
I think putting it in the record with special syntax like these final proposal is definitely better.
More clear the record isn't normal
Yeah, records are order agnostic, record literal expressions aren’t
This feels like a compiler implementation detail rather than expected behaviour
Order matters in the applicative approach the same
Never realized that, but it definitely worries me.... Just waiting for someone to sort a record and break all the parsing.
It's an implementation detail unless ! is used like @Kiryl Dziamura pointed out
Ah, I see, missed that ! already enforcing ordering here.
But yeah, even straight line code in roc that could theoretically be run out of order are often ordered for understandability and side effect reasons.
Probably would be a terrible idea as something implicit, but I'm pretty sure that we could allow this:
{ Task.concurrent <-
id: id, # let’s say we already got it from somewhere
users: Http.get "/users",
posts: Http.get "/posts",
news: Http.get "/news",
}
id would be a type mismatch if we try to feed it through Task.concurrent. So instead build a special desugaring where anything that is a type mismatch is just passed through without any changes. So never passed into Task.concurrent.
Or better yet, make it explicit (all special fields require <-:
{ Task.concurrent <-
id: id, # let’s say we already got it from somewhere
users <- Http.get "/users",
posts <- Http.get "/posts",
news <- Http.get "/news",
}
So I think I actually like this syntax a lot. First line kinda says that <- means it is getting passed into Task.concurrent. Later lines specify which things are getting passed in.
Could also do, but I think it is less clear. Though definitely more distinct.
<- users: Http.get "/users",
This also further removes from record syntax while still having the basic shape of a record which owuld help ensure it doesn't get missed.
I think specifically having : is important for the intuition that these are record fields and not variables - which in turn is important for the intuition that fields can’t reference each other
I feel like that is easily solved with a single error message. So I personally am not worried about that.
Oh, though you could have a variable with the same name as a field and reference that, which would look like referencing a field if we don't use : where the expectation is already set...so maybe : has more merit
I know being able to add “fixed” values is cool in theory, but I haven’t really needed it yet.
I’d prefer not having to write <- for 95% of the cases and just use Task.ok or equivalent in the rare cases that it’s needed.
@Richard Feldman what do you think about the update-style syntax?
Can record update syntax to be used here to define unwrapped fields?
We currently disallow update in record builders, I think it’s too much :big_smile:
One other general comment:
The one element record case feels like an easy bug for users to hit and get annoyed by. Especially if they have an existing record and are just commenting out a field (I bet many new users will hit this when they try a record builder for parsing and want to start with the simplest one element parsing). I wonder if there is a way to fix it.
I guess we could make it take both map and map2. Or would could make it take map1or2 that uses a variant....but of this don't feel great though.
I think we can just make it a compile time error. Like tuples, record builders only make sense with at least 2 items.
I don’t think you’d reach for a builder if you only have 1 thing. There’s nothing to batch, and you can map it later if you really need a record for some reason.
Brendan Hansknecht said:
(I bet many new users will hit this when they try a record builder for parsing and want to start with the simplest one element parsing)
Oh, I missed that. This is a good point.
Maybe we could make it a runtime error at least?
I mean compiler error but can still run
I think a compile-time error message can address that
or a warning, yeah
like “this doesn’t do anything and will be ignored”
Well, it would have to crash, right? Because there’s no way to call map2
and actually yeah we could have the semantics be “1 field is a no-op” plus we give a warning along the lines of “unused record builder”
yeah just don’t call it
We still want to have type checking there right?
just replace the whole literal with the one field
That doesn’t work because you’d have to wrap it in the record
oh true
yeah ok then runtime error :+1:
Agus Zubiaga said:
Richard Feldman what do you think about the update-style syntax?
I like the general idea! :+1:
We could copy the one field and pass it to both arguments of map2, but that’d change behavior in some cases
That’d be horrible nvm
I was under the assumption that a one-field record builder with the new syntax would simply not be called by map2, and would be just added to a single element record. If something like Weaver needs to have 2 elements per arg definition record, then usage is inconsistent between large and small records
The problem is you cannot call map2 with less than 2 things, but you have to return a record instead of the value of the field directly
Can you share an example of a one field record builder in weaver?
Any time we just want to parse a subcommand and nothing else: https://github.com/roc-lang/basic-cli/blob/aa1a19802708fc32d418b6208ae4e4a18389dbde/examples/args.roc#L30
I have a few examples of that exact use case, so I'll avoid linking them
I could make it so that there's also a Cli.finishJustSubcommands alternative that doesn't take a builder, but then we're still working around a syntax that is inconsistent between large and small records, which seems weird
It may be worth it for the simplicity of the map-2 based approach to allow this, but it's still important to understand what will break with this
Also, roc-query will probably break with this...
Yeah, the API will have to change for sure, but I don't think it'd make it worse
Something like this should work:
Subcommand.required [maxSubcommand, divideSubcommand],
|> Cli.finish {
name: "args-example",
description: "A calculator example of the CLI platform argument parser.",
version: "0.1.0",
}
Here's a query that only gets one field: link
In that case, I think it'd look like:
buildQuery "LoggedInQuery" \{ root } ->
Query.field root.loggedIn
The point is, the functions would have to change from applying to returning the "batch" type directly. So whether you use a builder or not, the types should work out
Consider this larger query (link): you'd have to pass a single function for single fields, and a record for larger fields
The types should work out yes
Can't you just drop the top-level in that case?
buildQuery "UserQueryLarge" \{ root, user, address } ->
- Query.start {
- user: <-
Query.start {
The worry for roc-query is that it makes sure that the record you build matches the JSON structure of the API, so that when we decode using roc's built-in Decoding we get the right structure out
That might break, though I think it would be fine
Ohh, I didn't think roc-query would use Decoding at all
I thought it would generate the decoders from the schema, I see
GraphQL uses JSON, so we need to as well
And Roc doesn't have raw JSON yet
Gotcha
I never thought about using Record Builders AND Decoding together :thinking:
Also,
Can't you just drop the top-level in that case?
buildQuery "UserQueryLarge" \{ root, user, address } ->
Query.start {
name: <- Query.field user.name,
age: <- Query.field user.age,
address: <-
Query.start {
state: <- Query.field address.state,
}
|> Query.object user.address,
}
now has to be this:
buildQuery "UserQueryLarge" \{ root, user, address } ->
Query.start {
name: <- Query.field user.name,
age: <- Query.field user.age,
addressState: <- Query.object (address.state) user.address,
}
Which at least breaks decoding
What happens if I change the name of user? That'd break, right?
Yeah, it currently breaks
As you can see in the Query.finish type annotation, we ensure that a query can only be built if the type of the record and the resolved API type we got from the fields we added are the same object layout
I think Decoding makes sense when you're just going off the inferred types, but if you're generating Roc code anyway, you can include the decoders for full safety
Easier said than done, I know :smiley:
The problem with that is we now have a 2-pass decoding step when we could have just one. We can decode to raw JSON and then to our actual data, but that's slower and more memory intensive. Users are now punished because they wanted a convenient syntax
Not like Roc is a super performant, but it's pretty performant, so users that understand how this works will understand that this library is convenient to use, but slower than hand-written code for large JSON data returned
Which isn't true for the current implementation: it's as fast as a hand written version
Now
I think all of these problems go away if we allow users to make a record builder with zero or one fields, and just don't call map2 on it
If we allow that, Weaver and roc-query work the same way from the perspective of the user, assuming I can get the builder stuff to work, which should not be too bad
Sam Mohr said:
The problem with that is we now have a 2-pass decoding step when we could have just one. We can decode to raw JSON and then to our actual data, but that's slower and more memory intensive. Users are now punished because they wanted a convenient syntax
Well, you don't really need the two passes. We can have a library that, like Decoding, incrementally parses the json and your generated decoders would use that.
Sam Mohr said:
I think all of these problems go away if we allow users to make a record builder with zero or one fields, and just don't call
map2on it
The problem is that we cannot change the shape from the outside
If we don't call map2 at all and just returned the value of the field, the produced value would not be a record with the field inside. It would just be the value of the field.
So the types won't line up
{ map2 <-
id: getIdSomehow
}
If we desugar that to:
getIdSomehow
The type is Id, not { id: Id }
That's how it currently works, right?
Cli.weave {
verbosity: <- Opt.count { ... },
}
|> Cli.finish { ... }
translates to
Cli.weave (\verbosity -> { verbosity })
|> Opt.count { ... }
|> Cli.finish
The new way, I'd expect, would be:
{ Cli.weave <-
verbosity: <- Opt.count { ... },
}
|> Cli.finish { ... }
which would go to
(\verbosity -> { verbosity })
|> Opt.count { ... }
|> Cli.finish { ... }
right?
{ Cli.weave <-
verbosity: <- Opt.count { ... },
somethingElse: <- Opt.count { ... },
}
|> Cli.finish { ... }
would desugar to:
Cli.weave
(Opt.count {...})
(Opt.count {...})
\verbosity, somethingElse -> {verbosity, somethingElse}
The one field version doesn't work because we cannot call weave
If it's one field, can we just desugar to
(\verbosity -> { verbosity })
|> Opt.count { ... }
That shouldn't break anything
Opt.count doesn't return a function in this world
Yeah
So you cannot call it like that
True
Okay, yeah, I might have been thinking about how the new approach desugars incorrectly
Let me go back and re-read, and then comment back here with my understanding
In order to support 1-field record builders, we either need to pass map or wrap (what you currently call weave)
Weaver was designed very specifically to work with the existing desugaring
Task.concurrent
(Http.get "/users")
(Task.concurrent
(Http.get "/posts")
(Http.get "/news")
\posts, news -> (posts, news)
)
\users, (posts, news) -> { users, posts, news }
This works if the type of Task.concurrent is Task.concurrent : Task a err, Task b err, (a, b -> out) -> Task out err, right
And if stuff like Weaver breaks so that users have a better experience overall, I'm totally fine with that
Same with roc-query
I don't think it would break, but it'd look more like Elm packages that do stuff like this
I do get why 1-field record builders are nice for things like Weaver
Especially for starting with one thing, and easily adding more later
This not being possible doesn't prevent you from doing anything, but it's more cumbersome initially
I'm gonna see if I can make types that work for Weaver with the new syntax and then come back to this
It seems like I keep running into the thought that "my answer here depends on whether this code I'm imagining would actually work"
So I'll just write that code to shore up my worries
Yeah, I'd imagine it'd be hard to convert an existing API to this new idea. For me it's the opposite with roc-pg, because I started with the usual mapN functions and then added record builders (when I implemented them).
Haha, interesting
Brendan Hansknecht said:
I guess we could make it take both map and map2. Or would could make it take map1or2 that uses a variant....but of this don't feel great though.
something like this?
build : [
Zero out,
One (Task a) (a -> out),
Two (Task a) (Task b) (a, b -> out)
] -> Task out
Maybe we don’t need zero, that could be a compiler error
It preserves some of the original goals, but it’s not as nice as just passing map2
An optional empty value? Which is an empty wrapper
{ map2, empty <-
x: ...
}
{ Task.concurrent, Task.ok {} <-
...
}
Or ignored fileds for the rescue
Yeah, that's the first thing I thought, but that seems like a worse user experience
The Task module could expose a record with both things, but at that point, I rather go with the tags
In theory, we just need a way to get config from a record builder with zero or one fields to a function that takes said applicative and finishes everything, like Task.await or Cli.finish
Also, the person writing the record builder knows how many fields there are, so we don't need map2, empty, we just need map2 with 2+ fields and empty for 0/1 fields
Yeah, it's just annoying to have to switch
I don't think so, compared to the alternative of not being able to make a record builder at all with just one field
That's true
A little annoying yes
But it's a consistent means that lets stuff like Weaver and roc-query still be shaped the same, more or less
So you'd start with something like:
{ Cli.map <-
verbosity: Opt.count { ... },
}
|> Cli.finish { ... }
and then switch to:
{ Cli.map2 <-
verbosity: Opt.count { ... },
somethingElse: Opt.count { ... },
}
|> Cli.finish { ... }
Yep!
That seems like the best option by a country mile to me, syntax pending of course
Yeah, that might be a good tradeoff
I like that it doesn't require the module to expose something specific to record builders
If that wasn't the case, we may as well stick with the current record builder approach
So this needs to support "regular" functions IMO
Which this does
I think most importantly, it doesn't require you to understand applicatives
which have those scary looking types with functions inside :smiley:
And for empty records, you just have a
Cli.empty
|> Cli.finish { ... }
So roc-query can still infer your data type because you can always pass your type
I think for GQL you do need to select at least one thing, right?
Well, I guess you can default to __typename if empty
Presumably yes
Depending on the API, you might not even need empty, but yeah, I don't think we need to support empty record builders
You're right, but the important point is that between Task.concurrent, Task.map, and Task.ok {}, you can make a record with 0-n fields that all return the right type
This is looking good. :grinning:
Yes, with the map2 and map version (depending on record size), I have things working (without the desugaring, of course). It looks good to me!
https://gist.github.com/smores56/c3d6b9f333e4900fcea25edbc4ac8604
Should maybe rename cliMap to cliSingleArg or something, but the point is made
Nice, that's exactly what I imagined!
So yeah, this new syntax:
I'm 100% onboard at this point
Another cool thing you can do now is map the values directly in the builder:
{ cliWeave <-
email : strParam { name: "email" } |> map toLower,
...
}
Wow, that's really nice
That can be really useful with GQL queries too
can we see the side-by-side on the 1-field version vs if 1-field didn't exist?
it feels like a weirdly specific situation to have syntax sugar for haha
With 1-field record builder:
{ Cli.map <-
verbosity: Opt.count { ... },
}
|> Cli.finish { ... }
Without:
Opt.count { ... }
|> Cli.map \verbosity -> { verbosity }
|> Cli.finish { ... }
Honestly, it’s hard to see the benefit when you’re just comparing them directly
The motivation is about easily adding new fields
You still have to change the map function when going to and from 1-field builders, so the impact is debatable
yeah it seems like in either case you have to teach "here's what you do if there's exactly 1 thing you want" and then the question is becomes, "should there be special syntax sugar for that case, which also needs to be taught, but which makes it look more similar to the 2+ field case?"
and it doesn't really seem to me like that's worth a dedicated language feature
especially since for most uses of it (e.g. Parser, Task, Random) I don't think people would even use it in the 1-arg case
they'd just use .map directly instead of making a record with one field and then destructuring it
I understand the weirdness, but let's exhaust the options. What if the chaining function is
build : (T a) ([Some T b, None]) (a, [Some b, None] -> out) -> T out
Or
build : [One (T a), Two (T a) (T b)] ([One a, Two a b] -> out) -> T out
How often we might see similar code manually written?
Task.concurrent taskA (Some taskB)
Kiryl Dziamura said:
build : (Task a) ([Some Task b, None]) (a, [Some b, None] -> out) -> Task out
That signature doesn't really work because you need b to be always there when the second argument is Some. We need to include the function in the variant like this:
Agus Zubiaga said:
build : [ Zero out, One (Task a) (a -> out), Two (Task a) (Task b) (a, b -> out) ] -> Task out
Richard Feldman said:
especially since for most uses of it (e.g.
Parser,Task,Random) I don't think people would even use it in the 1-arg case
I think even when writing roc-pg selections, I'd flatten it if I only want to query one thing
The structure of the record doesn't affect the query shape, it will always do the right thing
Yeah, it's entirely a visual thing except for maybe Encoding or Decoding
Just because it's a visual thing, however, doesn't mean we shouldn't do it.
Yeah, that's where it gets tricky, because the structure in Roc affects the shape of the data
I know Decoding is the best thing we have for JSON decoding now, but I still think ideally we shouldn't use it for cases like roc-query
All the information is available, there's no need to resort to a inferred decoders
When RawJson gets implemented it's more achievable
Yeah, I imagine roc-json could expose its partial decoders directly in addition to the Decode implementation
That was how I started doing it, basically
You at the very least need to pull them out from just being usable for the @Json opaque type so you can use them for RawJson and Json
But being able to decode an object or a string in particular would be beneficial
You can totally make a elm/json-style decoder and use that to power these style of schema-based codegen tools
Yeah, that's how it would work under the hood
For my proposal of function-based record builders, it seems like there's lukewarm feelings on it because it seems to require higher kinded types (HKTs). But I don't really understand how the proposal here does not require HKTs.
That is, if we have a record builder syntax which requires users provide a function like
build : [
Zero out,
One (Task a) (a -> out),
Two (Task a) (Task b) (a, b -> out)
] -> Task out
and users are expected to use that with syntax a la
{ Task.build <-
users: Http.get "/users",
...
}
doesn't the compiler need to be able to verify that Task.build matches some type signature?
In other words, if a user writes
{ List.map <-
users: Http.get "/users"
}
this is obviously wrong, and they need to get a compiler error. But what does that error say about having a wrong type signature on this function? The compiler still needs to tell users what signature it is expecting this function to have after all. Maybe the concrete types needed for the given build function can be inferred from the given record fields? That way the error message presented can have a concrete type signature that is specific to Task, or whatever thing the user is trying to build with, as opposed to being described in terms of HKTs.
Another aspect is teaching this. Where do users go to see the signature of the function they must obey when they want to use a builder like this? If we can't specify the signature generally, do we then push the complexity on to the user instead of keeping it in the language?
In other words, this proposal still seems to rely on HKTs conceptually, and I would worry that we end up merely hiding or moving this complexity as opposed to removing it. And if we can't remove it, I'm personally more a fan of being up front about it.
It would be great to have type error for unexpected function signature. In the current proposal the invalid application error will be highlighted on the fields which could be confusing indeed
This proposal, like the current implementation, does not require HKTs. We basically get type checking for free because the syntax is desugared before constraining.
The difference is that function-based record builders would desugar to an intermediate RecordBuilder type that can later be applied by passing it and map2 to a generic function. That’s what requires HKTs.
Even though the type checking occurs on the desugared code, all the error messages will point to regions on the real source, and we can add metadata during desugaring so that we can provide more helpful messages (specific to record builders)
Here’s an example of an error customized because it originated from a record builder
I believe we can make our errors more helpful than the current implementation because there are more constraints
The main concern is that if your build signature is not valid, you might think that it's a problem with fields. But maybe it's a rare case.
I see. Yeah, that’s a tricky one. Unfortunately, without HKTs, there’s a limit to how much we can constrain the function itself.
I think we’ll have to point out that this could be the case in the error message
In the end it’s like function application, if your arguments don’t match what the function excepts, they’ll get type errors, not the function
We can probably detect some cases like “the number of args doesn’t match” and highlight the function instead
Forgive me my ignorance, wouldn’t it be beneficial to have HKT internally only for linting? I imagine it might be helpful also for potential ! to andThen expansion.
Not in the scope of this record builder proposal obv
Right. However, in order for HKTs to be useful, we need to expose them to user-space at least for implementation.
Fully internal HKTs would only work if the type had one type variable, because otherwise (e.g Task), how do you know which one to pick?
@Kiryl Dziamura HKT's are off the table largely for performance reasons (in addition to learnability), so even having them internally could make the "compile fast" goal that much harder to maintain
which one to pick
Ah, of course. Just didn’t think thoroughly about it
Sam Mohr said:
Kiryl Dziamura HKT's are off the table largely for performance reasons (in addition to learnability), so even having them internally could make the "compile fast" goal that much harder to maintain
worth noting that this depends on how far we were to go with them...last year I learned that limited forms of HKTs don't have the same drawbacks that fully general HKTs do
that said, I don't think this syntax sugar needs to get the type checker involved :big_smile:
Agreed. Just reporting involved :grinning:
I tried looking Task.concurrent up but couldn't find anything about this. Does roc nowadays allow for concurrent tasks to take place?
Not yet, I think we listed the required steps to land that somewhere but I can't find it
For syntax parsing of these, do we want to ensure that the map2 function is on its own line, or is it okay to have the whole builder on a single line? Would the following be readable to you all, or newcomers:
apiData = { Task.concurrent <- users: getField "users", events: getField "events" }
or should we force multiline builders:
apiData =
{ Task.concurrent <-
users: getField "users",
events: getField "events",
}
I definitely think the multiline version is more clear, but I think the first version is clear enough that it's not worth preventing devs from writing builders on one line.
I think oneliners are fine. They’d have even more sense for tuple builders, I’d assume we’ll have them eventually. But tuples deserve their own discussion ofc.
(users, events) = (Task.concurrent <- getField "users", getField "events")
I think we should be able to parse both, and perhaps roc format should make it multiline once there are more than two fields?
I'm structuring the parsing just like the record updater & syntax, so that's roughly how it will work
The difference being that formatting for record updates doesn't force multiline formatting. I actually think we should stick with that, since it's more consistent, and Roc doesn't have a max line limit in general
We do have a default line wrap for the old record builder syntax if the value of a field is somewhat complex
Though I personally prefer being given the option to decide myself, roc format is intentionally opinionated to avoid formatting discussions from needing to happen on dev teams, meaning we should always line break if there are more than 2 fields, probably.
So, interesting thought: I was planning on coming back here to request a re-consideration on allowing single field record builders because of a use case I wanted to make convenient, and that is giving argument context at a call-site with record builder-sourced data. What do I mean by this? If I have a function that builds a Task { foo: Str, bar: Str } and I consume that Task's data in a closure, I can write the function as \{ foo, bar} -> ... and even though both of my arguments are strings, I can't accidentally switch the arguments or forget which one is which.
This isn't true with a single variable, however. If I'm trying to pass a closure to Task.await for a Task Str, there are no structural restrictions on what I call the Str, I could call it foo or bar or anything else, so the type of the function doesn't tell me what data to expect.
However, @Agus Zubiaga mentioned that this new syntax allows us to map over record builder field values normally, since they're treated as normal values. That means that we can do something really simple:
multipleArgs =
{ Cli.weave <-
foo: Arg.str,
bar: Arg.num,
}
singleArg =
Arg.str |> Cli.map Foo
So if I wanted to consume the singleArg variant, I could just write a closure \Foo foo -> ..., and even if the dev doesn't name the pattern inside the Foo tag "foo", they still know what value they're dealing with besides it being a string. And since it's a tag union with a single variant, it's a zero-byte wrapper around the inner value, meaning this is a simple abstraction with no runtime cost!
All that said, unless this turns out to be a really annoying way to write Weaver parsers over one that uses single field record builders (I'd be surprised), I think I'm now convinced we don't need to support single field record builders.
Do we have an update example for making a record builder now?
I don't think the tutorial or example on the website got updated
No, you're right
I started writing a change, but I got stuck on giving the user the right mental model on when they would want to use a builder
I think just saying "when you want to combine complex data" or "you want a clean interface" is way too generic, and "you want to combine tasks in parallel" is too specific
I don't think the old example did a good job of this either, it more just explained how desugaring worked
Do we have an example elsewhere? I want to try and port something to it.
Weaver now uses the new syntax, and uh
https://github.com/roc-lang/basic-cli/blob/main/examples/record-builder.roc
:thank_you: Thanks!
regarding this snippet:
myrecord : Task { apples : List Str, oranges : List Str } []_
myrecord = { Task.concurrent <-
apples: getFruit Apples,
oranges: getFruit Oranges,
}
{ apples, oranges } = myrecord!
seems like it'd be common to want to await the record builder, which I guess could be done like this, although it looks weird:
{ apples, oranges } = { Task.concurrent <-
apples: getFruit Apples,
oranges: getFruit Oranges,
}!
what if we instead allowed moving the ! inside the builder, and having that await the combined task?
{ apples, oranges } = { Task.concurrent! <-
apples: getFruit Apples,
oranges: getFruit Oranges,
}
I wonder how common that will be in practice.
For example, in Sqlite3, the recorder builder is using tasks, but it is always passed to something that will run the task anyway:
SQLite3.execute!
stmt
{ SQLite3.map2 <-
id: SQLite3.i64 "id",
task: SQLite3.str "task",
}
I'm just thinking like "I want to run two http requests in parallel and then do something after they both finish"
Richard Feldman said:
{ apples, oranges } = { Task.concurrent <- apples: getFruit Apples, oranges: getFruit Oranges, }!
Honestly, I think that one is fine
}! looks weird to me haha
but I guess an argument for at least trying it that way is that it needs to at least be possible to do it that way, so there's value in not introducing another way to do it
I mean, all syntax sugar is a nicer way to do something you can already do. I'd say that even though you could put the bang in the record builder or after, if we always set the precedent of putting the bang inside the builder, it's better than }! being the standard
Last updated: Jun 16 2026 at 16:19 UTC