Stream: beginners

Topic: optional field problem

witoldsz (Apr 02 2024 at 19:06):

I have declared a type alias with optional field, but I am not able to satisfy the type system, see below.
Is it a compiler bug or what do I do wrong?

package "testing" exposes [] packages {}

EventSystemAlertsCreate : {
    submitId : Str,
    refs ? Str,
}

event1: EventSystemAlertsCreate
event1 = {
    submitId: "123",
    refs: "456"
}

event2: EventSystemAlertsCreate
event2 = {
    submitId: "123",

}

$ roc check test.roc

── TYPE MISMATCH in test.roc ───────────────────────────────────────────────────

Something is off with the body of the event1 definition:

 8│   event1: EventSystemAlertsCreate
 9│>  event1 = {
10│>      submitId: "123",
11│>      refs: "456"
12│>  }

The body is a record of type:

    {
        refs : Str,
        submitId : Str,
    }

But the type annotation on event1 says it should be:

    {
        refs ? Str,
        submitId : Str,
    }

Tip: To extract the .refs field it must be non-optional, but the type
says this field is optional. Learn more about optional fields at TODO.


── TYPE MISMATCH in test.roc ───────────────────────────────────────────────────

Something is off with the body of the event2 definition:

14│   event2: EventSystemAlertsCreate
15│>  event2 = {
16│>      submitId: "123",
17│>
18│>  }

The body is a record of type:

    { submitId : Str }

But the type annotation on event2 says it should be:

    {
        refs ? Str,
        submitId : Str,
    }

Tip: Looks like the refs field is missing.

────────────────────────────────────────────────────────────────────────────────

2 errors and 2 warnings found in 30 ms

$ roc version
roc nightly pre-release, built from commit 62cc19c on Di 02 Apr 2024 09:01:38 UTC

Brendan Hansknecht (Apr 02 2024 at 19:50):

Optional fields are named poorly. They are really default value fields. They require a value. The value will be specified during a destructuring. This generally happens when passing a record into a function and wanted default values.

Brendan Hansknecht (Apr 02 2024 at 19:51):

You probably want to directly represent an optional in the type system

Brendan Hansknecht (Apr 02 2024 at 19:52):

Something like refs: Result Str [NoRefSpecified]

witoldsz (Apr 02 2024 at 23:35):

Ok, so how do I use optional fields? I know I have to destructure to access them, but in my example I was unable just to create an object.

Brendan Hansknecht (Apr 03 2024 at 00:04):

Generally as a function input, something like:

someFn: EventSystemAlertsCreate -> ...
someFn = \{ submitId, refs ? "default value" } ->
    ...

someFn { submitId: "123", refs: "456" }
someFn { submitId: "123" }

witoldsz (Apr 03 2024 at 07:17):

OK, but how do I assign to a variable? How do I pass it around?

Hristo (Apr 03 2024 at 08:35):

@witoldsz, my understanding is that those are not intended to be passed around as named values.

From the corresponding tutorial section (which is quite helpful in its entirety, as it discusses example use-cases where such fields are applicable; I believe Brendan's example was an instance of one class of possible applications):

Destructuring is the only way to implement a record with optional fields. For example, if you write the expression config.title and title is an optional field, you'll get a compile error.

This means it's never possible to end up with an optional value that exists outside a record field. Optionality is a concept that exists only in record fields, and it's intended for the use case of config records like this. The ergonomics of destructuring mean this wouldn't be a good fit for data modeling, consider using a Result type instead.

witoldsz (Apr 03 2024 at 09:26):

As I said, I got the message "Destructuring is the only way to implement a record with optional fields". I have literally studied that tutorial section before asking this question. I still do not understand how a value (I mean a record with an optional field) cannot be passed around. This seems very strange.

Luke Boswell (Apr 03 2024 at 10:09):

They're not optional in that they are either there or they arent.

The field is always there. Just when you pass a record to a function that field is optional. So you can pass a record that doesn't have that field, and it will be given the default value.

Hristo (Apr 03 2024 at 10:28):

@witoldsz, first - my apologies, I definitely didn't mean to imply that you hadn't read the tutorial before asking your question :pray:

If you're familiar with Python (or equivalent), perhaps this example might be of help, conceptually speaking only:

def add3(a, b, c=5):
    return a + b + c

add3(1, 2) # ~> 8
add3(1, 2, 15) # ~> 18
add3(1, 2, c=15) # ~> 18

a, b and c are values that can be passed around and Roc's optional fields enable you to achieve similar functionality. But c = 5 isn't a value in itself (it's an assignment; it's not something that signifies "I'm giving you something that may optionally be equal to 5, if you're to pass it to add3 as a value which you'd previously stored; Python doesn't allow that either) and can't be passed around (of course, you can do d = c = 5 but it has a completely different meaning). It's used in the signature to indicate a default value, which may be optionally overriden.

Of course this isn't a perfect analogy in terms of language specifics, but - as mentioned above - this isn't the goal of the example.

Perhaps record types with optional fields could be viewed as what are referred to as "interfaces" in some languages. And the corresponding records with non-optional fields, which are subset of the former, are concrete instances conforming to those "interfaces". The "interfaces" facilitate substituting in default values when the desired behaviour is being able to specify the corresponding fields optionally only.

If your question is pertaining to the language-design level and programming language theory (e.g., why Roc doesn't implement this functionality) - then I'm afraid I cannot help at that kind of level of abstraction. I know that I couldn't think of any use-cases why I'd need this kind of language feature, but that's about it.

witoldsz (Apr 03 2024 at 11:12):

@Hristo no apologies needed, I do not feel offended in any way :smile:
My only problem is that I have no idea how can I use the type as in my example:

EventSystemAlertsCreate : {
    submitId : Str,
    refs ? Str,
}

event1: EventSystemAlertsCreate
event1 = {
    submitId: "123",
    refs: "456", # <----- error
}

event2: EventSystemAlertsCreate
event2 = {
    submitId: "123", # <----- error
}

# now I could use event1 or event2 to pass to a function.

Neither of that :point_up: works. From my perspective, whenever I use an optional field in record, there is no way to create an instance and pass it around.

My only way out of it is to stop using optional fields in a record, but there is no a "default" Option type, so I have to either come up with a custom tag union or use something like Result a {}?

Hristo (Apr 03 2024 at 11:14):

whenever I use an optional field in record, there is no way to create an instance and pass it around.

Yes, indeed. That's expected behaviour, by design.

so I have to either come up with a custom tag union

Yes, my understanding is that was essentially Brendan's suggestion above:

You probably want to directly represent an optional in the type system
Something like refs: Result Str [NoRefSpecified]

That would be the idiomatic way to achieve the desired functionality in Roc.

witoldsz (Apr 03 2024 at 11:14):

P.S.
I am looking hopefully at this thread: https://roc.zulipchat.com/#narrow/stream/304641-ideas/topic/Builtin.20Option.20type.20for.20decoding

Hristo (Apr 03 2024 at 11:15):

I am looking hopefully at this thread: https://roc.zulipchat.com/#narrow/stream/304641-ideas/topic/Builtin.20Option.20type.20for.20decoding

Good point - yes, I was meaning to reference that as a related as well.

witoldsz (Apr 03 2024 at 11:21):

One little problem with Result is that it does not feel idiomatic. Result sounds like a… result of something. The upside though is there are ready-to-use functions to deal with results. Using custom tag union seems more idiomatic, but then I have to create all the map/join/etc functions just for this…

witoldsz (Apr 03 2024 at 11:24):

I have a feeling that everything is moving towards the solution of introducing Option anyway. Even if this is going to be an alias for Result * {}. I don't know…

Hristo (Apr 03 2024 at 11:30):

So, I'll share a pattern that I've been using in my own Roc code and I'm not sure how relevant that might be to your use case, but it might potentially be of help.

• So, I've got models which, upon initialisation, take parameters (a record, containing optional fields), then I store the parameters in association with the models (I've got a parameters record field in a model record).
• Since I don't want to have to specify all parameters upon initialisation, and instead I want to be able to provide default values, I define one record type that contains optional fields and another record type that contains the same subset of fields, but none of them are optional. This way I'm passing in a subset of the parameters as a record (not explicitly typed), destructuring the input to utilise the defaults and storing this as an instance of the non-optional-field parameters type.

There might be a better and/or more idiomatic Roc way to achieve the same, but in terms of API readability and associated experience, this has been working particularly well in my use-case.

witoldsz (Apr 03 2024 at 11:35):

Could you provide a small example? Does it work well in a scenario where you have to deal with massive data structures with dozens of (possibly nested) fields?

Hristo (Apr 03 2024 at 11:37):

My use-case doesn't involve yet more than 2-3 levels of nesting, so I'm yet to see how that might hold up eventually, if/when such necessity arises.

I'll be able to share some minimal example later today after work, yes.

Eli Dowling (Apr 03 2024 at 11:40):

I sway back and forth, @witoldsz, but I do have some reasons to like using results for Options. So maybe as a fellow sceptic I can give you some food for thought :
In the thread you linked discussing Option as a builtin @Brendan Hansknecht mentioned using a result and I was initially skeptical but after some consideration I came around to it.

1. No duplication of functionality/more simplicity. This is obvious, if you can have less things you should, less choices, better interop, less complexity
2. For decoding it's super convenient because of error accumulation.
   Usually decoding looks like this:

• check if the field exists
• Perform some validation and maybe a transformation, eg: check if a filepath exists, if a string matches some regex etc
• output the final validated happy path object
  Because results accumulate Errors your final return for that field might be `Result val [Null, NotValidEmail] instead of Option (Result val [NotValidEmail])

Brendan Hansknecht (Apr 03 2024 at 15:19):

Going back to the original example here:

EventSystemAlertsCreate : {
    submitId : Str,
    refs ? Str,
}

event1: EventSystemAlertsCreate
event1 = {
    submitId: "123",
    refs: "456"
}

event2: EventSystemAlertsCreate
event2 = {
    submitId: "123",

}

I think this should compile as is. I think it not working is a bug.

Note: the refs field would not be useable until destructing happens to initialize the field with a default value:

{ submitId, refs ? "default" } = event1

The reason why, both of these types are clearly fine to be EventSystemAlertsCreate because both of these compile just fine:

event1 = {
    submitId: "123",
    refs: "456"
}

eventFn : EventSystemAlertsCreate -> Str
eventFn = \{submitId, refs? "default"} ->
    Str.concat submitId refs

event1Str = eventFn event1

and

event2 = {
    submitId: "123",
}

eventFn : EventSystemAlertsCreate -> Str
eventFn = \{submitId, refs? "default"} ->
    Str.concat submitId refs

event2Str = eventFn event2

Note, we currently have another bug in specialization that breaks this:

event1 = {
    submitId: "123",
    refs: "456"
}

event2 = {
    submitId: "123",
}

eventFn : EventSystemAlertsCreate -> Str
eventFn = \{submitId, refs? "default"} ->
    Str.concat submitId refs

event1Str = eventFn event1
event2Str = eventFn event2

Hristo (Apr 03 2024 at 21:45):

Coincidentally, I've just run into what seems to be exactly the same issue (although, I thought I didn't expect it to fail in this new use-case of mine). The minimum reproducible example is as follows (please, note the difference between OptionalFields1.roc and OptionalFields2.roc and their corresponding compilation results):

> cat OptionalFields2.roc
interface OptionalFields2
    exposes []
    imports []

RecordWithOptional : { abc : U64, def ? U64 }

sumRecord : RecordWithOptional -> U64
sumRecord = \{ abc, def ? 5 } -> abc + def

expect
    sumRecord { abc : 3 } == 8

expect
    sumRecord { abc : 3, def : 8 } == 11 # <------- this is the only effective difference between the two files

> roc test OptionalFields2.roc

── TYPE MISMATCH in OptionalFields.roc ─────────────────────────────────────────

This 1st argument to sumRecord has an unexpected type:

11│      sumRecord { abc : 3 } == 8
                   ^^^^^^^^^^^

The argument is a record of type:

    { … }

But sumRecord needs its 1st argument to be:

    { def : Int Unsigned64, … }

Tip: Looks like the def field is missing.
────────────────────────────────────────────────────────────────────────────────

1 error and 0 warnings found in 34 ms

> cat OptionalFields1.roc
interface OptionalFields1
    exposes []
    imports []

RecordWithOptional : { abc : U64, def ? U64 }

sumRecord : RecordWithOptional -> U64
sumRecord = \{ abc, def ? 5 } -> abc + def

expect
    sumRecord { abc : 3 } == 8

> roc test OptionalFields1.roc

0 failed and 1 passed in 147 ms.

Please, note that if we stick to records containing the same subset of non-optional fields, then the compiler is happy in such instances:

> cat OptionalFields3.roc
interface OptionalFields3
    exposes []
    imports []

RecordWithOptional : { abc : U64, def ? U64 }

sumRecord : RecordWithOptional -> U64
sumRecord = \{ abc, def ? 5 } -> abc + def

expect
    sumRecord { abc : 3 } == 8

expect
    sumRecord { abc : 5 } == 10  # <---------- same subset of fields as in the first `expect` block

> roc test OptionalFields3.roc

0 failed and 2 passed in 143 ms.

Brendan Hansknecht (Apr 03 2024 at 22:03):

Yep, that is the specialization bug.

Last updated: Jul 05 2025 at 12:14 UTC