Stream: ideas

Topic: record field operations

Richard Feldman (Jul 18 2026 at 21:31):

I've been thinking about this topic for a long time, and I think I finally ended up with a coherent design for the following things:

• adding and removing record fields on the fly
• making some record fields actually optional

I thought we could get away with not having optional record fields, but in the end I think we actually do want them for serialization - among other use cases.

Richard Feldman (Jul 18 2026 at 21:32):

here's an example with all of the operations:

y_optional : { x : U32, y ?: U32, z : U32 }
y_optional = if foo { x: 1, y: 2, z: 7 } else { x: 1, z: 7 }

y_overridden : { x : U32, y : U32, z : U32 }
y_overridden = { ..y_optional, { y: 4 } }

y_defaulted : { x : U32, y : U32, z : U32 }
y_defaulted = { ..y_optional, { y:  y_optional.?y ?? 0 } }

y_removed : { x : U32, z : U32 }
{ y: _, ..y_removed } = y_optional

Richard Feldman (Jul 18 2026 at 21:37):

compared to today, this would let you say:

• "evaluate to this record except with the following field(s) added: ______"
• "evaluate to this record except with the following field(s) removed: ______"
• "some of this record's fields may be missing at runtime; they must be accessed with my_record.?maybe_missing_field", which returns a Try(_, [FieldMissing])
• you can chain multiple .? together, e.g. my_record.?maybe_another_record.?maybe_another_field and the whole thing would return one Try(_, [FieldMissing]) instead of nested Trys

Richard Feldman (Jul 18 2026 at 21:38):

originally we used to have a "default fields" idea specifically for optional arguments in configuration records being passed into functions. That idea had various type problems in practice, and was confusing to learn, and so in the new compiler we decided to just go with static dispatch and "builder" config values instead.

Richard Feldman (Jul 18 2026 at 21:39):

this would allow for the "config record with optional fields" again, but it's not really what I'm thinking of here - what I'm actually thinking about is use cases more like serialization

Richard Feldman (Jul 18 2026 at 21:41):

@Niclas Ahden I'm curious how much of this would replace the need for roc-maybe btw

Richard Feldman (Jul 18 2026 at 21:42):

on the serialization side, right now it's not great trying to serialize and deserialize "optional fields" or for that matter "default fields" - all of which come up pretty often

Richard Feldman (Jul 18 2026 at 21:46):

my thinking is that we can combine this with nominal record fields allowing for default values, e.g.

MyNominalRecord := {
    required_thing : U64,
    optional_thing ?: U64,
    optional_thing_with_default ?: U64 ?? 0,
}

Richard Feldman (Jul 18 2026 at 21:50):

and then in the same way that we already have structural records unifying with nominal records, when you unify a structural record with this and the field is missing (or just use nominal syntax to define it without mentioning the field) then we use the default

Richard Feldman (Jul 18 2026 at 21:52):

so this means we have a way to specify "I want to deserialize into this shape, and these fields can be missing, and these other fields have a default value" - which comes up all the time, and which I really really do not want to introduce arbitrary field annotations to solve that problem

Niclas Ahden (Jul 18 2026 at 21:53):

Lovin' it! It could probably replace a lot of roc-maybe! I'd have to try it out once I get to that level to see how it plays out :) Would you recommend against using optional fields for config/args for sweeter APIs? I used that quite a bit when it was available and liked it

I'd probably also be tempted to use the record extension for some stuff where I have to manually type out records to merge/extend them today. Do you think the perf of that is still a footgun like mentioned before when this was brought up?

Richard Feldman (Jul 18 2026 at 21:53):

I think both of those use cases are fine

Niclas Ahden (Jul 18 2026 at 21:53):

Will you adopt me? :star_struck:

Richard Feldman (Jul 18 2026 at 21:54):

I strongly suspect the perf concern will be a non-issue in practice

Richard Feldman (Jul 18 2026 at 21:55):

and I don't think it's enough of a concern to justify not trying it and seeing what patterns we see in practice, especially after LLVM optimizations

Richard Feldman (Jul 18 2026 at 22:07):

also I chose ?: for the "optional field" syntax just because there's prior art (TypeScript uses that to mean the same thing) and it seems reasonable enough to me.

Richard Feldman (Jul 18 2026 at 22:10):

also @Jared Ramirez I couldn't think of any issues this would have with type inference (since we've already proved out the nominal/structural unification stuff in general, the original Daan Leijen extensible records paper already supported even fancier adding/removing than this, and in this design the "maybe missing" design can basically follow what we do for tag unions - when we unify across branches, if any branches are missing the field then we unify it as ?: and then unify all the types on the other side of the ?: together)

Richard Feldman (Jul 18 2026 at 22:11):

anyway, any feedback on all this welcome!

Richard Feldman (Jul 18 2026 at 22:11):

I know @Tommy Graves had thoughts on this back in the day :smile:

Richard Feldman (Jul 18 2026 at 22:22):

oh yeah, I forgot to mention inferred type signatures for field removals:

|my_rec| {
   { x: _, ..without_x } = rec
   without_x
}

the inferred type of this would be:

{ x: _a, ..others } -> others

so we wouldn't have any special "type with field removed" syntax like in the Daan Leijen paper or in Elm's original record update types (before 0.12 or 0.13 removed them)

Jared Ramirez (Jul 18 2026 at 22:27):

yeah agree — should be fine regarding types!

we should make sure to add a nice hint to src/check/snapshot/diff.zig, to make error messages extra clear! (eg a call out on type mismatch where both records have the field, but one is optional but the other is required — could see that being confusing)

Karl (Jul 19 2026 at 00:58):

Not directly relevant but at least somewhat related, but I'd like a Typescript style Omit and Partial for web request inputs. Retyping is fine but they're convenient.

Eric Rogstad (Jul 19 2026 at 04:19):

Karl said:

Not directly relevant but at least somewhat related, but I'd like a Typescript style Omit and Partial for web request inputs. Retyping is fine but they're convenient.

I like them for the results of DB queries.

Jared Ramirez (Jul 19 2026 at 04:19):

one thing this does not cover is defaulting a field, but having that field not be optional. providing it on construction would overwrite the default, with the upside of being able to use the field directly without handling optionality (ie the Try)

Richard Feldman (Jul 19 2026 at 04:20):

@Jared Ramirez sorry, I don't follow :sweat_smile:

can you give an example?

Jared Ramirez (Jul 19 2026 at 04:23):

ya like

JobConfig := { name : Str, num_threads : U64 ?? 4 }

a = JobConfig.{ name = “a” }
b = JobConfig.{ name = “b”, num_threads = 6 }

in both cases you can do a.num_threads/b.num_threads and it resolves to U64, not Try(U64, [MissingField])

Jared Ramirez (Jul 19 2026 at 04:24):

default without optionality

Richard Feldman (Jul 19 2026 at 04:24):

ohh I see

Richard Feldman (Jul 19 2026 at 04:24):

yeah that makes sense!

Richard Feldman (Jul 19 2026 at 04:25):

and actually now I'm wondering whether it makes sense to be both optional and defaulted at the same time :laughing:

Jared Ramirez (Jul 19 2026 at 04:25):

feels straight forward to do that with explicit nominal construction, eg JobConfig.{ name: “a” }. but less straight forward for structural -> nominal lifting/coercion

Jared Ramirez (Jul 19 2026 at 04:26):

Richard Feldman said:

and actually now I'm wondering whether it makes sense to be both optional and defaulted at the same time :laughing:

yeah haha, imo it does not!

Eric Rogstad (Jul 19 2026 at 04:26):

In postgres you could make a field be default 0 or not null default 0. The difference is that the first one lets you explicitly pass in a null for that field. But Roc doesn't have null, so it seems like you'd only ever want the not null version.

Richard Feldman (Jul 19 2026 at 04:26):

yeah like if it's "defaulted but not optional" then you just check at runtime if it's missing, and if so, use the default, and otherwise just unify in the non-missing one

Richard Feldman (Jul 19 2026 at 04:26):

yeah but our version of null is a tag union

Richard Feldman (Jul 19 2026 at 04:27):

so that distinction is already covered

Eric Rogstad (Jul 19 2026 at 04:33):

Richard Feldman said:

yeah but our version of null is a tag union

Sorry, hit send too soon. Edited my comment. Just meant to say that it seems to me that default + optional does not make sense in Roc (unless I'm missing something).

Eric Rogstad (Jul 19 2026 at 04:42):

In other words, "this field is optional" and "this field has a default value" both mean that you do not have to pass in a value for that field when you create the record, so in that way they feel the same. But once you already have an instance of the record, then they should be different, because the version with a default value for the field should never have the field be missing.

And actually, for that reason I'd humbly suggest that the syntax for fields with defaults should be like this:

MyNominalRecord := {
    required_thing : U64,
    optional_thing ?: U64,
    thing_with_default : U64 ?? 0,
}

rather than this:

MyNominalRecord := {
    required_thing : U64,
    optional_thing ?: U64,
    optional_thing_with_default ?: U64 ?? 0,
}

(Using : rather than ?: before the type annotation.)

Richard Feldman (Jul 19 2026 at 05:05):

agreed!

Jared Ramirez (Jul 21 2026 at 02:27):

working on this

Jared Ramirez (Jul 21 2026 at 02:31):

there are several distinct pieces to this:

• adding fields on the fly
• removing fields on the fly
• adding optional fields
• addng field defaults

Jared Ramirez (Jul 21 2026 at 02:35):

starting with the first one, a question:

currently the fact that you cannot add fields on the fly means we can give you nice error messages about typos. for example:

a = { hello: "world" }
b = { ..a, hllo: "universe" }

currently results in:

This record does not have a `hllo` field.
**test:5:7:5:8:**

  { ..a, hllo: "goodbye" }
      ^

This is often due to a typo. The most similar fields are:

    - `hello`

So maybe `hllo` should be `hello`?

__Note:__ You cannot add new fields to a record with the record update syntax.

if we allow fields to be added arbitarily, then b's type would be { hello : Str, hllo: Str } – most certainly not what the user intended!

Luke Boswell (Jul 21 2026 at 02:39):

Riffing off the idea for optional fields ?:... what if we introduced a +: field assignment which means "add this new field"

a = { hello: "world" }
b = { ..a, hllo+: "universe" }

Jared Ramirez (Jul 21 2026 at 02:39):

so my question is: is this desired semantics?

IMO, easily adding fields seems not worth the cost! but curious if other see it differently!

note you can add fields by copying manually

a = { hello: "world" }
b =  { ..a, goodbye: "baby" } # fails like above
c = { hello: a.hello, goodbye: "baby" } # passes

this is more cumbersome, but avois the issue above. this is the same decision elm made, back in the day (no 100% sure Elm iss it for for this exact reason, but i know Evan did because it was confusing, and i suspect this kind of thing is why!)

Jared Ramirez (Jul 21 2026 at 02:40):

Luke Boswell said:

Riffing off the idea for optional fields ?:... what if we introduced a +: field assignment which means "add this new field"

a = { hello: "world" }
b = { ..a, hllo+: "universe" }

this would work fine i think! (if we want to add new syntax)

Luke Boswell (Jul 21 2026 at 02:42):

It's a little weird because everywhere else the assignment for fields on records are simply : and it's only in the type annotation we see :?

Luke Boswell (Jul 21 2026 at 02:47):

Is there a way to zero out or erase an optional field? like I could imagine we introduced something like using _ -- in which case you might want to use the ?: in the assignment too so it's clear this is optional.

MyNominalRecord := {
    required_thing : U64,
    optional_thing ?: U64,
    thing_with_default : U64 ?? 0,
}

my_record : MyNominalRecord
my_record = {
    required_thing : 12,
    optional_thing ?: _,
}

I'm not sure if "erasing" an optional thing is something that would every come up in practice.

Richard Feldman (Jul 21 2026 at 03:00):

here's an idea that might seem a bit strange, but could solve that problem: what if you had to do this to add a field?

a = { hello: "world" }
b = { ..a, ..{ hllo: "universe" } }

Richard Feldman (Jul 21 2026 at 03:00):

so basically if you say "merge this record into this other record" then we union the fields, and that can include adding fields

Richard Feldman (Jul 21 2026 at 03:00):

(which I expect to be rare in practice)

Richard Feldman (Jul 21 2026 at 03:01):

whereas the more concise case is for updating, and may not add new fields - so this would give the same nice error you get today:

a = { hello: "world" }
b = { ..a, hllo: "universe" }

Richard Feldman (Jul 21 2026 at 03:02):

that said, I haven't thought through whether "union these two records together" is representable in our type system :laughing:

Richard Feldman (Jul 21 2026 at 03:02):

like I'm not sure what the inferred type would be if you put |a, b| { ..a, ..b } into the repl

Richard Feldman (Jul 21 2026 at 03:03):

oh maybe just a, b -> { ..a, ..b } :thinking:

Richard Feldman (Jul 21 2026 at 03:03):

like both become extensions on an empty record

Bryce Miller (Jul 21 2026 at 03:05):

I do like the idea of having separate syntax for update vs add if that’s what we need to maintain the current nice error messages

Richard Feldman (Jul 21 2026 at 03:05):

anyway, the reason I like that idea is that if we're going to support unioning records together like that, it has to add fields

Richard Feldman (Jul 21 2026 at 03:05):

so there already has to be a dedicated "must add fields" syntax if we want to support unioning records together (which I vaguely recall years ago someone asked for)

Richard Feldman (Jul 21 2026 at 03:06):

and if we already have a syntax for that, then we can keep the existing syntax working the way it does today, including error messages, and it works out nicely that it's more concise since it's more common

Richard Feldman (Jul 21 2026 at 03:09):

and then the rule for updates like { ..a, ..d, hllo: "universe", ..e, ..f } would be that update field(s) must be present in one of the preceding ..record entries

Norbert Hajagos (Jul 21 2026 at 06:59):

Could the add field syntax also update the current fields? If so, typos have the same problem there, but maybe that's fine, since you have a place to look for those, since elsewhere it's not possible. Would it be unergonomic to disallow updating with that syntax, unlike the js spread operator?

Jared Ramirez (Jul 21 2026 at 13:02):

Richard Feldman said:

so basically if you say "merge this record into this other record" then we union the fields, and that can include adding fields

i think this should work! i like this more than adding += or the like! will play with this today

Jared Ramirez (Jul 21 2026 at 14:55):

removing fields on the fly already works on current main, fyi!

Jared Ramirez (Jul 21 2026 at 14:56):

eg { foo, ..rest } = bar can be used to strip foo away from rest

Romain Lepert (Jul 21 2026 at 17:29):

I feel that these b1 and b2 expressions should mean the same thing. That is how I tend to internalize destructuring.

a = { hello: "world" }
b1 = { ..a, ..{ hllo: "universe" } }
b2 = { ..a, hllo: "universe" }

how about using | for union ?

a = { hello: "world" }
b = { hllo: "universe" }
c = a | b  # { hello: "world", hllo: "universe" }

It has precedence in python and has connections with logical OR

# sets
{1, 2} | {2, 3}  # {1, 2, 3}

# dicts
{"x": 1, "y": 2} | {"y": 20, "z": 3}  # {"x": 1, "y": 20, "z": 3}

Jared Ramirez (Jul 21 2026 at 19:50):

yeah looking at:

a = { hello: "world" }
b1 = { ..a, ..{ hllo: "universe" } }
b2 = { ..a, hllo: "universe" }

i agree. using { ..a, ..{ hllo: "universe" } } to attach new fields feels overloaded

Jared Ramirez (Jul 21 2026 at 20:03):

in any case, i've been looking into this and i'm pretty sure merging records will require a new type primative and be somewhat complicated

Eric Rogstad (Jul 21 2026 at 20:47):

Coming from TypeScript, I think it'd be nice if:

a = { hello: "world" }
b = { ..a, hello: "universe" }

just worked, even at the expense of accidentally allowing:

c = { ..a, hllo: "universe" }

since wouldn't that typo likely be caught when you try to do something with c, like passing it to a function that expects a record with only a hello field?

Please correct me if I'm wrong, but I'd have guessed it's considered good Roc practice to add type annotations to your functions, even if technically inference could handle it. So would expect the hllo typo to be cause when crossing a function boundary.

I suppose that leaves unaddressed the case where you use c.hello within the same function, but TBH I don't remember running across that kind of bug in practice in TypeScript. Though maybe that's because I defensively add type annotations all over the place, and perhaps we don't want Roc'ers to have to do that?

Luke Boswell (Jul 21 2026 at 22:04):

This conversation has me thinking we should not rush into this feature... and consider kicking it down the road to a post 0.1 milestone. I put it in the mental category of nice to have, and we will gather a lot of real examples without it that would motivate the change. I'm not sure if it fundamentally changes our serialization approach though... but if it's only a minor change there, I see no reason to push for this now.

Luke Boswell (Jul 21 2026 at 22:05):

I don't want to stop the discussion here... I'm just expressing a thought around how we might prioritise this.

Richard Feldman (Jul 22 2026 at 01:41):

I think this is worth prioritizing...the main reason is because of ?: which is blocked on it, and I do think that's important for the reasons outlined at the start of the thread

Luke Boswell (Jul 22 2026 at 01:51):

Richard Feldman said:

on the serialization side, right now it's not great trying to serialize and deserialize "optional fields" or for that matter "default fields" - all of which come up pretty often

So we can avoid having Try in the type for things we want to serialize etc if I follow that correctly?

But could we live with that for a short while if we needed to? Or would implementing this proposal be changing the builtins etc and therefore we should get that done sooner than later?

Richard Feldman (Jul 22 2026 at 01:52):

yeah, not only changing the builtins but also changing the types people are using with them

Jared Ramirez (Jul 22 2026 at 15:50):

so a smattering of thoughts:

• i spent a good chunk yday looking into { ..a, ..b, ..{ c: d } }, and merging records like this is not trivial, especially if we want to support the generic case (eg |a, b| { ..a, ..b }). given that this may also be a perf footgun, IMO it would make sense to wait and ensure that there are compelling use-cases for this. (this is also not blocking useland things, as you can manually copy all fields elm-style)

• given the above, i think we need to:

  1. revisit syntax on how to add a new field (eg luke's +: idea, or something else)
  2. allow arbitrary fields to be added and accept the consequences (eg no helpful typo error messages)

• ^ I lean towards 1

• in any case, today i'm doing a similar investigation into optional record fields. on its surface, i think this should be straightforward.
• then, will look into defaulting values (i do not have an intuition on how easy/hard this will be)

but to Richard's last point, IMO optional fields + default values feel like the builtin API-impacting changes that are worth prioritizing, whereas adding record fields feel more punt-able

Last updated: Jul 23 2026 at 13:15 UTC