Stream: ideas

Topic: `for` and `var`

Richard Feldman (Sep 20 2024 at 02:33):

this is the last of a batch of related proposals over the last few weeks - any feedback welcome! :smiley:

https://docs.google.com/document/d/1Ly5Cp_Z7dY8KLQkkDYZlGCldxQj4jLzZ0vIeB-F8lJI/edit?usp=sharing

Johan Lövgren (Sep 20 2024 at 09:42):

Can I pass a var as a module param? Or is that top level within the module?

Richard Feldman (Sep 20 2024 at 11:02):

yeah I'd say we shouldn't allow that...could probably make it work, but would probably be confusing :big_smile:

Johan Lövgren (Sep 20 2024 at 13:27):

That sounds like a wise choice :octopus:

Agus Zubiaga (Sep 20 2024 at 13:28):

Would the _ suffix for names of vars be enforced by a warning?

Agus Zubiaga (Sep 20 2024 at 13:29):

"enforced" might not be the right word there, I'm just wondering if there would be a warning

Johan Lövgren (Sep 20 2024 at 13:30):

In general, I like how readable the code is here. I am somewhat worried that I do not have a clear mental model for what it means for a variable to be "reassignable". From the proposal I gather that it is different enough from actually being mutable to preserve the important pure functional guarantees, but I don't quite grok why.

Agus Zubiaga (Sep 20 2024 at 13:32):

IIUC the main difference is that the mutation is only local to the current function. You cannot pass a mutable reference and have that function mutate it, so there's no "spooky action at a distance".

Richard Feldman (Sep 20 2024 at 13:32):

Agus Zubiaga said:

Would the _ suffix for names of vars be enforced by a warning?

yep, that's the idea!

Johan Lövgren (Sep 20 2024 at 13:34):

Should there be a compiler warning for a var that is not reused?

Agus Zubiaga (Sep 20 2024 at 13:35):

I would say yes, and also for a name with an _ suffix that is not a var.

Kilian Vounckx (Sep 20 2024 at 13:36):

So having a var function argument would be like having a normal argument which you can alter in the function, but the caller can't see the changes?
Like are these equivalent?

foo = \var x_ ->
    ...

foo = \x ->
    var x_ = x
    ...

Kilian Vounckx (Sep 20 2024 at 13:37):

But the type for this function would still be e.g. foo : Str -> Str, not foo : var Str -> Str, because an outside user can't detect that it changes?

Richard Feldman (Sep 20 2024 at 13:41):

right, there's no var in types

Kilian Vounckx (Sep 20 2024 at 13:47):

The proposal says this though?

In the uncommon situation that you want to annotate a var's type, you'd use the var keyword in the annotation too:

var name_ : Str
var name_ = "(name goes here)"

So this is only for variables declared locally then?

Richard Feldman (Sep 20 2024 at 13:51):

ah, so to clarify the distinction there:

• var only ever appears right before a name
• var never appears inside the type itself

Richard Feldman (Sep 20 2024 at 13:52):

it's like var vs const in Zig, or perhaps var (or let) vs const in JavaScript

Richard Feldman (Sep 20 2024 at 13:52):

it's really about "can I have this name refer to something different later on"

Richard Feldman (Sep 20 2024 at 13:52):

and says nothing at all about mutating values in place

Agus Zubiaga (Sep 20 2024 at 14:01):

At first I though the _ suffix was redundant and maybe giving more meaning to a name than it might be expected, but this changed my mind:

any time you see blah_ being passed to a function, you know to be aware that the value is changing

That's a really valuable guarantee that I don't think any language has.

Agus Zubiaga (Sep 20 2024 at 14:01):

take-my-money.gif

Notification Bot (Sep 20 2024 at 14:20):

3 messages were moved from this topic to #ideas > Augmented assignment operators for var by Agus Zubiaga.

Isaac Van Doren (Sep 20 2024 at 14:46):

Could we omit the var keyword and instead require the trailing underscore to identify a def as shadowable?

Sam Mohr (Sep 20 2024 at 14:57):

You kind of need both, because var lets you know where the first definition is. You don't need it for normal definitions because they can only be defined once anyway

Sam Mohr (Sep 20 2024 at 14:58):

But you don't have that for "shadowable" variables. Not important for short function bodies, definitely for long ones

Isaac Van Doren (Sep 20 2024 at 15:15):

Is that necessary though? It seems like it would be about as easy to scan down a function looking for the first use of a particular name as it would be to find it when var is included.

Sam Mohr (Sep 20 2024 at 15:18):

Not necessary for sure. But I think readability is better, and communicates more clearly to users what we mean, especially for function args

Isaac Van Doren (Sep 20 2024 at 15:31):

I don’t know that it is necessarily more readable. I’m not a big fan of the fact that both var and trailing underscore are used to communicate the same thing and that they are completely coupled. It seems like two things to learn and explain when only one is necessary.

Sam Mohr (Sep 20 2024 at 15:32):

I agree that the most important thing is the underscore suffix, and without the var keyword this proposal is still good to go basically

Sam Mohr (Sep 20 2024 at 15:32):

I just don't think that duplication is always bad.

Sam Mohr (Sep 20 2024 at 15:33):

Though Roc at its core is built on concision between not needing to put let x = ... and the fully decidable type inference

Sam Mohr (Sep 20 2024 at 15:34):

Meaning the var prefix is unusual

Sam Mohr (Sep 20 2024 at 15:34):

Which added to its pairing with the underscore suffix makes it more unnecessary

Isaac Van Doren (Sep 20 2024 at 15:35):

Of note, you could use var in front of any pattern, not just = patterns - e.g. in function arguments, when branches, or destructures:

What happens if I use var when destructuring a record? Would I have to do something like this? {foo: foo_} if I want to shadow foo?

Isaac Van Doren (Sep 20 2024 at 15:36):

Also, would var be allowed when pattern matching on tags with no payload even though there would be nothing to shadow?

Agus Zubiaga (Sep 20 2024 at 15:46):

Isaac Van Doren said:

Of note, you could use var in front of any pattern, not just = patterns - e.g. in function arguments, when branches, or destructures:

What happens if I use var when destructuring a record? Would I have to do something like this? {foo: foo_} if I want to shadow foo?

I think it'd be:

{ foo: var foo_ }

Agus Zubiaga (Sep 20 2024 at 15:48):

Isaac Van Doren said:

Also, would var be allowed when pattern matching on tags with no payload even though there would be nothing to shadow?

I think var would just be a qualifier for Identifier patterns, not just any pattern. So putting it before a tag name would be syntax error.

Isaac Van Doren (Sep 20 2024 at 15:55):

Okay cool, glad it would be a syntax error there

Sam Mohr (Sep 20 2024 at 16:09):

Another benefit of requiring var is that it makes defining mutable variables more awkward than immutable ones, incentivizing pure code where possible

Ayaz Hafiz (Sep 20 2024 at 16:10):

shared this in DMs but i would consider renaming var to mut - this isn't really shadowing per-se because you can't change the type of a var-bound name, and so it is better to think of it as mutation. Also, using var might overload what it means for something to be a "variable" (is it any bound name or a var-bound name?)

Richard Feldman (Sep 20 2024 at 16:10):

the reason var is important is for the following scenario:

• let's suppose there is no var, and the underscore suffix means that it's reassignable
• I have some existing code written that uses foo_ and reassigns it, as normal.
• In a nested conditional where foo_ is in scope, I start writing some new code. I happen to also use the name foo_ for this new code, not realizing that I am referring to a foo_ that already exists in the current scope.
• Bugs! I am now changing how the code in the outer scope works even though I didn't intend to.
• There is no possible compiler error to help me out, because I'm doing something that's totally allowed.

var fixes that because now if I'm introducing (what I believe is) a new usage of foo_ in the nested scope, I'll start by writing var foo_ and immediately get a shadowing error

Richard Feldman (Sep 20 2024 at 16:11):

Ayaz Hafiz said:

shared this in DMs but i would consider renaming var to mut - this isn't really shadowing per-se because you can't change the type of a var-bound name, and so it is better to think of it as mutation. Also, using var might overload what it means for something to be a "variable" (is it any bound name or a var-bound name?)

I think this would be confusing because mutation usually refers to changing values in-place. Like if I say fn = \arg1, mut list -> - then I expect to be able to mutate that list in-place, like I can in Rust, e.g. by calling push on it. But I wouldn't be able to! list is still an immutable value, even though I put mut in front of it.

Sam Mohr (Sep 20 2024 at 16:12):

Ayaz Hafiz said:

shared this in DMs but i would consider renaming var to mut - this isn't really shadowing per-se because you can't change the type of a var-bound name, and so it is better to think of it as mutation. Also, using var might overload what it means for something to be a "variable" (is it any bound name or a var-bound name?)

If you look at our "shadowing" in loops, it really doesn't look like shadowing because you're re-shadowing the same var in the same code location multiple times. That implies mutability to me

Ayaz Hafiz (Sep 20 2024 at 16:18):

yeah that's fair. Maybe something like rebind or another word like that. i think you're not going to want to use this too often so a longer name that better communicates the intent may be better.

Richard Feldman (Sep 20 2024 at 16:21):

reass, nailed it :100:

Richard Feldman (Sep 20 2024 at 16:23):

I could see an argument for let

Richard Feldman (Sep 20 2024 at 16:23):

on the grounds that it follows the JS definition of let, but it's weird for an ML family language to have let mean "I'm opting into reassignment" :sweat_smile:

Richard Feldman (Sep 20 2024 at 16:32):

Ayaz Hafiz said:

using var might overload what it means for something to be a "variable" (is it any bound name or a var-bound name?)

:thinking: does this problem come up in JS or Zig?

Richard Feldman (Sep 20 2024 at 16:33):

like in both languages you have a cultural norm to default to const foo = ... but do people refer to foo in that situation as a "variable" or "constant"?

Richard Feldman (Sep 20 2024 at 16:33):

I'd think whatever they do could be what we do too

Richard Feldman (Sep 20 2024 at 16:34):

in the sense that either people say "variable" and mean both, and it's not confusing in practice, or else they say "constant" and "variable" and the distinction is clear

Ayaz Hafiz (Sep 20 2024 at 16:37):

let would probably be more confusing since the ML family is let-bound immutable variables

Richard Feldman (Sep 20 2024 at 16:46):

yeah for sure

Isaac Van Doren (Sep 20 2024 at 17:13):

Could also use def or shadow

Jasper Woudenberg (Sep 20 2024 at 18:17):

Alternative syntax idea:

each List.toIter paths is
    path -> write! path "file contents"

What I like about this is it's similarity to the when .. is expression, that the expression iterated over appears before the pattern which feels more natural to me, and that there's more space for the 'iteration expression', which the examples in the proposal demonstrate can get pretty long.

Downside is that it looks less like a for loop so I imagine there's a higher learning cost.

Jasper Woudenberg (Sep 20 2024 at 18:34):

I think var is a super cool idea, this combination of it being reassignable but not across function boundaries, seems super useful for loops and shadowing while at the same time hard to abuse in ways that would lead to readability problems. The one thing I'm wondering is whether it will feel artificially limiting to use, particularly to new folks coming from non-functional languages, resulting in frustrating rather than delightful experiences. One way to find out!

Niclas Ahden (Sep 20 2024 at 21:15):

Is there a feasible way to sugar-away the List.toIter in for?

# Before
for path in List.toIter paths do
    write! path "file contents"

# After (this reads nicer, removes one thing a beginner has to learn, and probably covers the majority of loops)
for path in paths do
    write! path "file contents"

# Before
for (index, path) in List.toIter paths |> Iter.enumerate |> Iter.rev do
    try write! path "stuff"

# After
for (index, path) in paths |> Iter.enumerate |> Iter.rev do
    try write! path "stuff"

Whenever I have to type into_iter and collect in Rust I go "Come on compiler, just figure it out. What else do you think I meant?" This is probably me being a spoiled brat, but I can't help but reminisce about the good old Ruby days when iteration was clean and simple. I know I'll forever trip up on this and go "oh, right, and List.toIter" :clown:

Agus Zubiaga (Sep 21 2024 at 20:54):

So the current joins API in roc-pg's query builder looks like this:

products <- from Public.products
orderProducts <- join Public.orderProducts (on .productId products.id)

...

The reason backpassing is used is that a State monad is used internally to track things like used table aliases (in case you join the same one twice).

Of course, I'm going to have to change the API once we remove it. We could just use regular closures, but that'd result in very nested queries when there are many joins.

The solution is probably going to be to explicitly thread the state. However, that can be annoying without shadowing. Would var allow me to do something like this?

(products, var s_) = from Public.products
(orderProducts, s_) = join Public.orderProducts (on .productId products.id) s_

Agus Zubiaga (Sep 21 2024 at 20:54):

I don't know if the second bind to s_ would be allowed

Agus Zubiaga (Sep 21 2024 at 20:58):

Of course, this is not specific to roc-pg, we'd also want that for e.g. random generator seeds

Richard Feldman (Sep 21 2024 at 21:06):

yeah that's the idea!

Richard Feldman (Sep 21 2024 at 21:07):

destructuring record and tuple patterns before = work the same way as using = and dot accessors

Richard Feldman (Sep 21 2024 at 21:09):

I think for cases like this and like random seeds we could consider allowing standalone var s_ with no = as long as all subsequent code paths assign it before it gets read

Richard Feldman (Sep 21 2024 at 21:10):

Rust supports that with standalone let

Agus Zubiaga (Sep 21 2024 at 21:14):

Nice!

Agus Zubiaga (Sep 21 2024 at 21:14):

Richard Feldman said:

I think for cases like this and like random seeds we could consider allowing standalone var s_ with no = as long as all subsequent code paths assign it before it gets read

I don't know if standalone var makes much of a difference for this use case, though

Agus Zubiaga (Sep 21 2024 at 21:15):

That said, I can see it being useful for when you set it under if / when

Peter Marreck (Sep 22 2024 at 14:09):

I recently saw this tweet and when I saw the loop proposal in the google doc, I was immediately reminded of it: https://x.com/ChShersh/status/1837413739265413248
I have to say that I was a bit shocked to see this (I guess I've long since swallowed the "functional kool-aid"), but perhaps there is a point, if this is continually a sticking-point for newcomers to functional languages or to Roc. And if you can keep the ease of use of an imperative-looking way while preserving functional semantics underneath, then why not? Sounds like all-win, no-drawbacks to me (and those ARE occasionally possible!)

Richard Feldman (Sep 22 2024 at 15:12):

yeah one way to interpret that image is:

• if you learn this one thing, you can do a lot of things with it!
• there are a ton of common patterns of ways of using this, and once you've learned them you can write code that's both more concise and also no longer prone to subtle mistakes where it looks like it fits one of the common patterns but actually behaves slightly but relevantly differently
• having both means you can learn the first thing first and then the others at your own pace

Richard Feldman (Sep 22 2024 at 15:15):

or maybe to say it another way, the things on the left are helper functions for common uses of the thing on the right

Anton (Sep 22 2024 at 16:59):

also no longer prone to subtle mistakes

Yeah, I definitely remember making lots of mistakes with for loop indices

Brendan Hansknecht (Sep 24 2024 at 05:47):

I'm definitely a fan of this proposal in general

Brendan Hansknecht (Sep 24 2024 at 05:47):

Really like the clear suffix that will be seen everywhere

Brendan Hansknecht (Sep 24 2024 at 05:48):

Also think it is nicer than shadowing in general

Brendan Hansknecht (Sep 24 2024 at 05:49):

Explicit instead of implicit

Kilian Vounckx (Sep 24 2024 at 09:13):

One situation where I would prefer shadowing is for example when assigning the result of List.walk. If you do the following sum = List.walk list 0 \sum, x -> sum + x, it does not compile because sum is shadowed, even though the result isn't used until after the List.walk is done. I think this has to do with making nested functions automatically recursive, but in this case it has been annoying to me.

Kasper Møller Andersen (Sep 26 2024 at 08:09):

Kilian Vounckx sagde:

One situation where I would prefer shadowing is for example when assigning the result of List.walk. If you do the following sum = List.walk list 0 \sum, x -> sum + x, it does not compile because sum is shadowed, even though the result isn't used until after the List.walk is done. I think this has to do with making nested functions automatically recursive, but in this case it has been annoying to me.

Isn't the real problem that the shadowing analysis sees the two sums as the same, when the left-hand sum should only really exist in the analysis after the right-hand side has been executed? Logically, I think the analysis should detect these two sums as completely distinct.

Kilian Vounckx (Sep 26 2024 at 09:12):

I think this is there because the left-hand sum could be a function. In this case, it needs to be in scope to be able to call itself recursively. Not sure if this is the reason though. And it could be fixed by only doing this for functions I think

Richard Feldman (Sep 26 2024 at 14:12):

you can still have recursion by way of a higher-order function, e.g.

foo = fn \arg -> foo (arg + 1)

...where fn returns a function

Francois Green (Oct 04 2024 at 03:17):

I'm having a tough time picturing what living with underscore would look like vs actual shadowing. Below I have the same code in both Roc and F# and I would like to know how the Roc code would change if this feature is added.

Edge : { src : U64, dst : U64 }

parseInput : Str -> List Edge
parseInput = \string ->
    List.walk (Text.lines string) ([], Dict.empty {}) \(edges, map), line  ->
        (vertex, nodes) = Text.bisectOn line ": " |> \(x, y) -> (x, Text.words y)
        (src, aMap) = Dictx.computeIfAbsent map vertex \_ -> Dict.len map
        List.walk nodes (edges, aMap) \(rEdges, rMap), node ->
            (dst, sMap) = Dictx.computeIfAbsent rMap node \_ -> Dict.len rMap
            (List.append rEdges { src, dst }, sMap)
    |> fst

type Edge = { src : int; dst : int }

let parseInput str : list<Edge> =
    Seq.fold (fun (edges, map) line ->
        let (vertex, nodes) = String.split ": " line |> (fun [|x; y|] -> (x, String.words y))
        let (src, map) = Map.computeIfAbsent (fun _ -> Map.count map) vertex map
        Seq.fold (fun (edges, map) node ->
            let (dst, map) = Map.computeIfAbsent (fun _ -> Map.count map) node map
            edges @ [{ src = src; dst = dst }], map
        ) (edges, map) nodes
    ) ([], Map.empty<string, int>) (String.lines str)
    |> fst

Last updated: Jun 16 2026 at 16:19 UTC