Richard Feldman (Apr 03 2024 at 19:25): Richard Feldman (Apr 03 2024 at 19:25):so we've talked in the past about how optional record fields were originally designed essentially to be optional named parameters (and that the way they're implemented is mostly in order to fit in well with the rest of Roc's design), but because of their name it seems like they're something more like "nullable" in other languages
Richard Feldman (Apr 03 2024 at 19:26):we talked about various alternative designs and their tradeoffs in another thread, including the idea of taking them out of the language (which has its own downsides):
Richard Feldman said:
I think it's reasonable to start new threads to discuss alternative designs to optional record fields.
Richard Feldman (Apr 03 2024 at 19:31):one option I'm curious to explore more is the idea of making optional record fields more first-class, specifically in the following way:
{ email ? Str } which means "a record which might or might not have the email field specified, but if it is specified, then it must be a Str"? suffix operator that I can use in expressions, e.g. result = myRecord.email? means that result has the type Result Str [FieldWasMissing] if myRecord has the type { email ? Str } - this would mean you can now delay handling the optional field until later, rather than having to destructure it right away. In particular it would mean that you could do conditionals based on whether the field was present, which is impossible the way the feature works today. Richard Feldman (Apr 03 2024 at 19:34):if we had this, then it would mean we could allow decoding into optional fields
Richard Feldman (Apr 03 2024 at 19:35):e.g. I could have a { name : Str, email ? Str } record and run decoding on it, and the decoder could say "if email is present in the serialized data, populate the field, but otherwise leave it off"
Richard Feldman (Apr 03 2024 at 19:35):and then because I can run conditionals on it, I can more easily later on handle the possibility of that field being missing
Richard Feldman (Apr 03 2024 at 19:47):one of my original concerns with having a builtin Option/Maybe/Optional type is that in a lot of ecosystems these sometimes get used as return values instead of Result, and I think it's better to consistently return Result for things that can fail, including a description of what the failure condition is (even if there's no more information to share)
e.g. I think List.first : List elem -> Result elem [ListWasEmpty] is a nicer API than List.first : List elem -> Option elem partly because it's more self-descriptive about what the problem was (and you can do a when branch of Err ListWasEmpty -> to make that more self-descriptive as well), and also because it means Result errors can accumulate automatically with Result.try, as well as it being more obvious that you're supposed to deal with the Result before storing the value in your data model
Richard Feldman (Apr 03 2024 at 19:48):a nice thing about a more "first-class" optional record fields in comparison to Option/Maybe/Optional is that they only work on records, so it wouldn't really even make sense to try to use them instead of Result for a return value like in List.get
Richard Feldman (Apr 03 2024 at 19:53):so, some relevant questions for discussion here:
myRecord.email? suffix operator (which evaluates to a Result and lets you do conditionals on optional record fields) would make the feature more useful for use cases like decoding when fields are missing from the serialized data and we want to handle that later rather than failing decoding? In other words, would it actually solve relevant problems?Option/Maybe/Optional builtin), but then again it might be a useful onboarding tool for anyone coming from a language that doesn't have sum types, because it's a way to model data in a familiar way at first, and then later you can learn other techniques as you spend more time with the language. Richard Feldman (Apr 03 2024 at 19:54):cc @Brendan Hansknecht @Eli Dowling @witoldsz based on your comments in other threads!
Brendan Hansknecht (Apr 03 2024 at 20:18):Decoding or encoding?
Brendan Hansknecht (Apr 03 2024 at 20:24):As in arbitrary bytes to record with optional (decoding) it record with optional to arbitrary bytes (encoding)?
Richard Feldman (Apr 03 2024 at 20:39):hm, I think it only makes sense for decoding
Richard Feldman (Apr 03 2024 at 20:40):like if I write:
record : { name : Str, email ? Str }
record = decode blah
...we can introduce a new scenario in the decoder API that handles "this is an optional record field we're decoding into" which allows the input serialized data to either have the field or to omit it
Richard Feldman (Apr 03 2024 at 20:42):I guess for encoding, if I passed a { name : Str, email ? Str } record to an encoding function, it could just mean it checks to see if it was provided, and if not, omit the field
Eli Dowling (Apr 03 2024 at 20:46):I think we would need some way to dynamically express a field should be omitted, otherwise you just end up with this, and the Json library defining Option and I do hope we can agree, that is definitely the worst of all worlds :sweat_smile:.
Richard Feldman (Apr 03 2024 at 20:49):random idea that just occurred to me: allow if without else in record fields:
{ email: if someCondition then blah }
so there, if someCondition is false then the field gets omitted
Richard Feldman (Apr 03 2024 at 20:50):if it's true then you get email: blah
Richard Feldman (Apr 03 2024 at 20:50):and the type of that record would be { email ? Str } (assuming blah is a Str)
Brendan Hansknecht (Apr 03 2024 at 21:02):I think that the ? suffix is fine.
I think the core concept of making ? a runtime piece of information instead of compile time is a fundamental mistake, and we should not support it. Instead of being able to monomorphize and get many optimizations, we are locking ourselves into runtime branching. I think that is a bad idea. As such, I do no think that decoding should be allowed to interact with types with ?. With decoding, a user needs to explicitly model their data. That includes optionality at runtime with a tag union or result.
Fundamentally in memory, all records should be equivalent to a type without ?. This is enough information to generate the result of myRecord.email? at compile time without any runtime representation or introspection. This enables a user to check if a field was set without specifying an explicit default value through destructuring. That could be a convenient addition.
Brendan Hansknecht (Apr 03 2024 at 21:04):Personally, I think that default value record fields are a fine concept and decent feature. They just have a bad name and a number of bugs. (can't specialize to two different versions, can't define as stand alone).
Eli Dowling (Apr 03 2024 at 21:05):The if without else is an interesting idea, I think my main reservation is just adding more syntax and oddity. However if it's useful maybe it's worth it.
What I think needs to happen with all of this discussion is to get some substantial examples of option heavy encoding and decoding and try modelling them with various proposals and compare the ergonomics. What you said above about not being convinced it's worth it is totally reasonable.
Luckily I'm writing a bunch of language server things, and oh boy does LSP love an optional field. So In the next couple of weeks I'll have some big examples I can try with a few different methods.
@witoldsz also sounds like he might have some good examples.
So I propose we play with these ideas in the wild and see how they go.
I'll make sure all my Option type stuff is available in one branch of roc-json so folks can just use the optional decoding fork of roc and experiment with Option and ResultOption then we can convert that to use the question mark option and any other alternatives
Brendan Hansknecht (Apr 03 2024 at 21:06):@witoldsz also sounds like he might have some good examples.
IIUC, his example just hit bugs but otherwise is a config that would be destructured with defaults.
Brendan Hansknecht (Apr 03 2024 at 21:09):
{ email: if someCondition then blah }
This feels like trying to avoid modeling the data and computation correctly. We are in a functional programming language. Tags are meant for this use case. I don't get why we want to avoid being explicit here:
{ email: if someCondition then NotNull blah else Null }
Richard Feldman (Apr 03 2024 at 21:46):it's specifically for the case where you want to conditionally omit the field instead of saying it's null (because some APIs you may be using draw a distinction between those two)
Brendan Hansknecht (Apr 04 2024 at 00:59):I still don't think this belongs in the roc type system. Especially not in a halfbaked form that requires runtime conditionals. If this is 100% comptime decision, I think that is fine. If it bleeds into runtime, I strongly feel we should be looking for a different solution.
Brendan Hansknecht (Apr 04 2024 at 01:01):I don't think this requires runtime conditionals for encode. For decode, you hit a 2^#optionalFields problem. So I don't think it is reasonable at compile time.
Brendan Hansknecht (Apr 04 2024 at 01:02):I would still prefer {email : [Some a, Null, Omitted]} over pulling this into the roc type system.
Last updated: Jun 16 2026 at 16:19 UTC