Stream: ideas

Topic: backpassing -> lambda destructuring?

Richard Feldman (Dec 26 2024 at 13:34):

a random thought I had this morning: with the | syntax for functions, we could try out keeping backpassing in a form like this: (an example from roc-pg)

query =
    |orders| = from(Public.orders)
    |customers| = join(Public.customers(on(.id, orders.customer_id)))

    # ...

I'm not saying we should or shouldn't do this, just that:

• the main problem with backpassing has always been that you had to learn it right up front in order to use it with Task in order to do any I/O at all. Purity inference has removed it from the learning curve, so the tradeoffs are different if it's only an advanced concept.
• From a parsing perspective, a, b <- caused a lot of problems. I don't think |a, b| = would have the same problems, but @Joshua Warner would know better than I would. :big_smile:
• people have requested keeping backpassing for advanced use cases like parsing, random number generation, and roc-pg, and this could be a potential way to keep it for those use cases without exposing beginners to it

Richard Feldman (Dec 26 2024 at 13:34):

anyway, not saying we should do this or anything, just an interesting thought I figured I'd share! Curious what others think.

Joshua Warner (Dec 26 2024 at 13:49):

Ooh! Yeah, that gets around the parsing concerns.

Curious what the use cases would be now? Just “any time the body of a multi-line lambda takes up the rest of the scope?” Ie just avoiding some rightward drift?

Norbert Hajagos (Dec 26 2024 at 13:54):

I love this syntax. Don't know how often I would use it now that error handling is done differently. Would be better to try it now than after the 0.1 release. I think it's (more unfamiliar, but) better than Kotli's trailing labda syntax that people have suggested:

val product = items.fold(1) { acc, e -> acc * e }

Anthony Bullard (Dec 26 2024 at 15:16):

So this would desugar to:

query =
    from(Public.orders, |orders|
        join(Public.customers(on(.id, orders.customer_id)), |customers|
            # ...
        )
    )

and therefore just is useful in places where you might have a Pyramid of Doom? Is this truly more readable? Maybe it is, but I guess this is sugar that really hides some POTENTIAL performance penalties. And is that the kind of sugar we'd want in the language in a v1.0 of Roc? I don't really know

Sam Mohr (Dec 26 2024 at 15:48):

This is useful anywhere use is useful in Gleam, as it's the exact same feature

Sam Mohr (Dec 26 2024 at 15:50):

Which is generally for monadic APIs, context adding statements (time this block of code, or log its result), or defer alternatives (withOpen that gives a file handle and closes the file at the end of the block)

Sam Mohr (Dec 26 2024 at 15:50):

So it's probably the closest Roc can get to RAII, which I think is a good paradigm to be able to deploy

Brendan Hansknecht (Dec 26 2024 at 15:56):

Personally, I would advise not trying this. Backpassing is essentially guaranteed to be a convenience at the cost of the weirdness budget. It also leads to designing more nested code (which isn't great).

If the community can live without backpassing, I think that is strictly a better state for roc to be in. If we can't, we will hit similar pain points to what led to backpassing in the first place.

We can always add it later. It is a convenience that I think hurts beginners and consistency the most. I think we also can look at pain points one at a time and likely come up with better solutions than backpassing for specific cases

-A former backpassing addict

Anthony Bullard (Dec 26 2024 at 16:00):

Yeah, I'd need to see an RAII-like example. Their examples are a little silly:

import gleam/io
import gleam/result

pub fn main() {
  let _ = io.debug(without_use())
  let _ = io.debug(with_use())
}

pub fn without_use() {
  result.try(get_username(), fn(username) {
    result.try(get_password(), fn(password) {
      result.map(log_in(username, password), fn(greeting) {
        greeting <> ", " <> username
      })
    })
  })
}

pub fn with_use() {
  use username <- result.try(get_username())
  use password <- result.try(get_password())
  use greeting <- result.map(log_in(username, password))
  greeting <> ", " <> username
}

// Here are some pretend functions for this example:

fn get_username() {
  Ok("alice")
}

fn get_password() {
  Ok("hunter2")
}

fn log_in(_username: String, _password: String) {
  Ok("Welcome")
}

which in Roc (with PNC/SD) would be:

main = |_|
  dbg(without_richards_idea())
  dbg(with_richards_idea())


without_richards_idea = ||
    get_username().map(|username|
        get_password()map(|password|
          log_in(username, password).map(|greeting|
            "$(greeting),$(username)"
          )
        )
    )

with_richards_idea = ||
  |username| = get_username().map()
  |password| = get_password().map()
  |greeting| = log_in(username, password).map()
  greeting <> ", " <> username
}

// Here are some pretend functions for this example:
get_username = ||
  Ok("alice")

get_password = ||
  Ok("hunter2")

log_in : String, String -> Result String _
log_in = |_username, _password|
  Ok("Welcome")

Brendan Hansknecht (Dec 26 2024 at 16:00):

Sam Mohr said:

So it's probably the closest Roc can get to RAII, which I think is a good paradigm to be able to deploy

We already have a platform level solution that fits roc even better and just depends on our refcounting to cleanup resources. Even when we had withOpen style apis and backpassing, they were rarely used due to the limitations of scope. Just calling open and letting the refcount close is a nicer API that sees far more use. I would rather see boxes with destructors in roc as a proper solution to RAII than to try and use scope based RAII with backpassing. Scope based is not enough.

Sam Mohr (Dec 26 2024 at 16:01):

https://erikarow.land/notes/using-use-gleam

Here's an article on use cases. Interestingly, the only one I think is relevant is the last case, "context management". We now have proper early returns, which we can use for managing results way better than with backpassing, and using use for unnesting the callback in List.map is just confusing.

Anthony Bullard (Dec 26 2024 at 16:02):

I agree

Anthony Bullard (Dec 26 2024 at 16:02):

And with for, and other error handling improvements, we'll see a lot less callbacks in general

Sam Mohr (Dec 26 2024 at 16:04):

I agree with @Brendan Hansknecht that having a destructor for Box is a cleaner solution for RAII. Generally I think Roc does well with a few, powerful primitives, but backpassing probably can be replaced by better, more numerous tools that don't lead to beginners needing to engage with a complex concept.

Anthony Bullard (Dec 26 2024 at 16:06):

Yeah even the Result mapping case is better today with try or ?

with_question = ||
    username = get_username()?
    password = get_password()?
    greeting = log_in(username, password)?
    "$(greeting),$(username)"

with_try = ||
    username = try get_username()
    password = try get_password()
    greeting = try log_in(username, password)
    "$(greeting),$(username)"

Which is essentially the same

Anthony Bullard (Dec 26 2024 at 16:08):

And if try desugars to when, probably better perf than callbacks

Jasper Woudenberg (Dec 26 2024 at 16:09):

Adding another +1 for not adding this. I think it might be tricky to limit use of this feature to advanced use cases only, as it's super tempting to use it for designing APIs (I know at least one situation for a CI library where I'd be tempted myself).

Sam Mohr (Dec 26 2024 at 16:10):

The RAII that we had been doing before:

File := U64

withFile : Path, (File => Result {} err) => Result {} [OpenErr, ..err]
withFile = |path, useFile|
    file = openFile!(path)?
    res = useFile(file)
    closeFile!(path)?

    res

listUsers = ||
    |userFile| = withFile("users.json")

    Ok(usersFile.read!()?.decode()?.len())

Sam Mohr (Dec 26 2024 at 16:11):

File can't be created without withFile, so there's no way for it to escape our resource management

Anthony Bullard (Dec 26 2024 at 16:13):

And now, since it's native, we can have a destructor for File right? to clean it up when refcount goes to 0?

Sam Mohr (Dec 26 2024 at 16:15):

With what I'd call refcounting BS, yes, but I think a "real" destructor solution would be preferable

Anthony Bullard (Dec 26 2024 at 16:15):

I guess now you could do silly stuff like put it in a map or some other long-lasting data structure

Anthony Bullard (Dec 26 2024 at 16:15):

Sam Mohr said:

With what I'd call refcounting BS, yes, but I think a "real" destructor solution would be preferable

What do you mean by "real" here?

Anthony Bullard (Dec 26 2024 at 16:16):

Writing C++?

Sam Mohr (Dec 26 2024 at 16:19):

2 minutes

Anthony Bullard (Dec 26 2024 at 16:45):

Sam is writing a treatise on RAII in functional languages right now :rofl:

Sam Mohr (Dec 26 2024 at 16:45):

Okay, so what I meant by "real" was that the File type is defined and primarily used from the high-level Roc side of things, and in my mind, the platform mainly handles passing that structural data to the OS in the way we expected. The platform having to manage refcounting for Roc concepts felt more to me like a requirement imposed by List refcounting than a tool introduced for its own sake to platform authors. Since Roc primarily owns File and primarily interacts with it, it would make more sense to me that we handle cleanup of File on the Roc side.

However, on reflection, I know we are doing lots of stuff on the host side with the owned data like read-only lists that manage the refcount of their RocList's accordingly, on top of the normal refcounting stuff. So I guess as long as this refcounting mechanism is considered a feature and not just an implementation detail, it makes sense to handle destructors from the host side.

In addition, the host is the only one that would really know what the actual resource is (in this case, a std::fs::File), so it makes sense for it to manage resources.

Sam Mohr (Dec 26 2024 at 16:47):

I wish I was haha, but I'm not an expert. It's more of an attempt to distinguish between the use of a feature for its intended purpose vs. hijacking that feature to achieve some different behavior. The feature in question here is refcounting and its exposure to the host instead of just &str or "a C struct with fields".

Georges Boris (Dec 26 2024 at 16:54):

Just a note on Gleam as reference - the use thing really spread across usages that I wouldn't expect over there and now it is not uncommon to see it being used like:

fn sum(xs: List(Int)) -> Int {
  use x, acc <- list.foldl(xs, 0)

  acc + x
}

I think it is worth keeping that in mind as reference that when something is possible, people will use it in unplanned ways (tbh this is used like this by core folks on the gleam community so it would be taken as "best/common practice"?)

Anthony Bullard (Dec 26 2024 at 16:55):

Georges Boris said:

Just a note on Gleam as reference - the use thing really spread across usages that I wouldn't expect over there and now it is not uncommon to see it being used like:

fn sum(xs: List(Int)) -> Int {
  use x, acc <- list.foldl(xs, 0)

  acc + x
}

I think it is worth keeping that in mind as reference that when something is possible, people will use it in unplanned ways (tbh this is used like this by core folks on the gleam community so it would be taken as "best/common practice"?)

This is so true. If you give a mouse a cookie...

Richard Feldman (Dec 26 2024 at 17:19):

yeah backpassing used to be used in that way too sometimes. I didn't personally like that style, but it did see use! :big_smile:

Eli Dowling (Dec 27 2024 at 06:28):

I think an added convenience for monadic APIs would be nice.

I have an ocaml codebase that uses a reactive TUI library that lets you compose your UI using incremental computations.
Super nice way to build a UI. But you are constantly mapping and map2ing data when transforming say a reactive number into a reactive UI element showing that number.
That library exposes "let$" which essentially is backpassing but always calling a particular map function.

I don't think I'd even consider using it without the nice monadic syntax.
Otherwise the code would just be an incomprehensible mess of lambdas.

But yeah, I agree that we should wait till people notice meaningful suffering

Kilian Vounckx (Dec 27 2024 at 09:00):

One nice way I saw backpassing/use used in gleam was as a middleware API in the webserver library wisp. But there are other ways to get a similar API without backpassing. I really liked it when it was there, but I sadly think it is confusing for newcomers and hard to keep to advanced use cases only

Sam Mohr (Dec 27 2024 at 11:36):

@Eli Dowling could you link that to me here or in a DM or wherever? I'm constantly looking for ways to manage nested state for GUIs in functional languages

Eli Dowling (Dec 27 2024 at 11:44):

Sure
Here is the original project:
https://github.com/let-def/lwd

Here is my quite heavily modified fork:
https://github.com/faldor20/nottui

Here is the project I built with it:
https://github.com/faldor20/jj_tui

If you are curious about nottui. I wrote a really quick and dirty crash course:
https://github.com/faldor20/nottui/blob/main/tutorial%2Fhackernews%2Ftutorial.md

And I wrote a blog post going into how to think about writing nottui UIs too:
https://elidowling.com/blog/nottui_intro/

I'd like to get it into a more user friendly state one day... But you know... I'm busy :sweat_smile: so many side projects so little time

Eli Dowling (Dec 27 2024 at 11:45):

The GitHub version of my fork probably isn't totally up to date with the changes actually made in jj_tui. But it' all work. I'm just slowly getting it to be a bit more happy with Async executors and multi threading.

Sam Mohr (Dec 27 2024 at 11:47):

Thank you!

Eli Dowling (Dec 27 2024 at 11:54):

I'll warn you, whilst technically evaluating the incremental UI is pure I don't really worry at all about maintaining any kind of functional purity in the rest of the code.
Updating all the states is done by doing a couple checks and then writing to a ref.

Sam Mohr (Dec 27 2024 at 11:54):

I'll see what I can manage :wink:

Eli Dowling (Dec 27 2024 at 11:54):

Although thinking about it now... I could actually model all state updates as effects which might be really useful...

Sam Mohr (Dec 27 2024 at 11:55):

That would only work if Store guiState was allowed outside of tests, right?

Eli Dowling (Dec 27 2024 at 11:55):

Oh I meant Ocaml effects :sweat_smile:

Sam Mohr (Dec 27 2024 at 11:56):

Or if your platform provided a store_state!

Eli Dowling (Dec 27 2024 at 11:56):

But yeah if roc had arbitrary effects it would totally be possible.

Eli Dowling (Dec 27 2024 at 11:58):

There are just a couple of cases right now where you can deadlock the program by trying to set the program state while evaluating the UI. We have ways around it but they have their own confusing consequences.

It's not something you hit often but it's a rough edge.

Eli Dowling (Dec 27 2024 at 11:59):

Anyway let me know if you have any thoughts or questions if you check it out. I think it is an architecture that could fit well into the roc world too.

Sam Mohr (Dec 27 2024 at 11:59):

Will do! I've starred the message

Sam Mohr (Jan 06 2025 at 16:11):

I can't remember where I saw this, I think it was in a recent post for a Gleam release, but one reason why backpassing doesn't fit well in current Roc is a reason why Gleam can't have return, because it doesn't work well with use.

Sam Mohr (Jan 06 2025 at 16:13):

check_contents! = |path|
    full_path = get_full_path(path)
    |file| = with_file!(full_path)

    if full_path.is_dir()
        return "early"

    contents = file.read!()?
    ...

Sam Mohr (Jan 06 2025 at 16:14):

That early return and the ? are going to return to with_file!, not to check_contents!

Sam Mohr (Jan 06 2025 at 16:16):

We could put a warning on early returns within backpassing contexts, but it can otherwise lead to some surprising behavior

Last updated: Jun 16 2026 at 16:19 UTC