Stream: ideas

Topic: fs API using static dispatch

Richard Feldman (Jul 11 2025 at 23:03):

Zig's new IO thing reminded me of a plan I had for basic-cli etc. after static dispatch, and I wanted to write it down!

Richard Feldman (Jul 11 2025 at 23:04):

the basic idea is that you pass around a Fs value and then call things on it:

my_fn : Fs, Path => Result({}, _)
my_fn = |fs, path {
    a = fs.read!("filename.txt", Json.utf8)?
    b = fs.read_path!(path, Json.utf8)?
    ...etc
}

Richard Feldman (Jul 11 2025 at 23:07):

you can obtain one of these in one of two ways:

# use in production
fs = Fs.real()

# use in testing
fs =
    Fs.simulated()
    .read(|state, path_str, fmt| { ... })
    .read_path(|state, path, fmt| { ... })

Richard Feldman (Jul 11 2025 at 23:07):

the cool part about this is that there's no runtime cost in production

Richard Feldman (Jul 11 2025 at 23:08):

because if you have this:

Fs := {
    Real,
    Sim(Box({
        read : ...,
        read_path : ...,
        etc...
    }))
}

...but you only ever construct the Real variant in the whole program, LLVM will completely optimize away all the Sim stuff

Richard Feldman (Jul 11 2025 at 23:09):

and Fs will become zero-sized at runtime and the argument will get eliminated

Brendan Hansknecht (Jul 11 2025 at 23:40):

:thinking:

Brendan Hansknecht (Jul 11 2025 at 23:40):

Don't we plan to enable testing effects generically

Brendan Hansknecht (Jul 11 2025 at 23:41):

If so, this feels like a detour, if not, it feels like a fundamental missing feature.

Brendan Hansknecht (Jul 11 2025 at 23:42):

Like that works, but you don't want to have to do it for all effects period to enable testing

Richard Feldman (Jul 11 2025 at 23:46):

I think it has the nicest ergonomics for testing effects

Richard Feldman (Jul 11 2025 at 23:47):

it's how we've always done it at Zed in Rust and it's been nice, which is why Zig switching to it seems fine to me, and Roc doing the same also seems fine

Richard Feldman (Jul 11 2025 at 23:48):

and if you don't care about testing or sandboxing we can always have like a File module that offers drop-in replacements for all the Fs functions except it automatically passes Fs.real() as the first argument for you

Richard Feldman (Jul 11 2025 at 23:50):

which would be fine for scripting etc.

Richard Feldman (Jul 11 2025 at 23:52):

the second-best alternative I'm aware of for this would be module params, but honestly they just don't seem necessary to avoid an arg, especially when a function taking Fs makes it more self-documenting regarding what type of effects it does (in the same way that it taking a db connection tells you it's doing db things)

Brendan Hansknecht (Jul 12 2025 at 00:21):

In my mind, all platform effects are known and sandboxed anyway. So theoretically we should be able to automatically do this (though sometimes at a lower level of abstraction due to effect primitives)

I don't see why we need to make the user change API at all.

Richard Feldman (Jul 12 2025 at 00:29):

that's certainly possible too, although it's a more complicated testing design

Richard Feldman (Jul 12 2025 at 00:32):

and even then you'd still need something like a Fs value when using third-party packages (I don't think that's avoidable) so it would be an inconsistent UX when using packages vs not

Richard Feldman (Jul 12 2025 at 00:34):

at least in terms of filesystem stuff specifically; as previously discussed, most packages that do I/O will need a value anyway for something like an API key or database connection or config record, so they'll be passing values around regardless

Brendan Hansknecht (Jul 12 2025 at 00:50):

Richard Feldman said:

and even then you'd still need something like a Fs value when using third-party packages (I don't think that's avoidable) so it would be an inconsistent UX when using packages vs not

I don't understand this. Why?

Brendan Hansknecht (Jul 12 2025 at 00:51):

Oh, nvm

Brendan Hansknecht (Jul 12 2025 at 00:51):

You mean passed to the libraries, not specifically for testing the libraries

Brendan Hansknecht (Jul 12 2025 at 00:51):

Ok...yeah

Brendan Hansknecht (Jul 12 2025 at 00:52):

Cool. Then this sounds good

Brendan Hansknecht (Jul 12 2025 at 00:52):

I'm not sure llvm will optimize it away though

Brendan Hansknecht (Jul 12 2025 at 00:54):

It should in local scope, but not sure how well it will do if this is passed 10 functions down the stack

Luke Boswell (Jul 12 2025 at 01:04):

Whats the cost? an extra pointer for each IO call?

Luke Boswell (Jul 12 2025 at 01:04):

In the worst case

Richard Feldman (Jul 12 2025 at 01:40):

Brendan Hansknecht said:

I'm not sure llvm will optimize it away though

I tried it on Godbolt with Rust and it optimized it away, so I assume it will for us too! :grinning_face_with_smiling_eyes:

Brendan Hansknecht (Jul 12 2025 at 01:40):

Oh, cool

Richard Feldman (Jul 12 2025 at 01:40):

I think the optimization is based on the enum variant never being constructed

Richard Feldman (Jul 12 2025 at 01:41):

at which point​ it gets dropped, leaving the union with 1 variant and which is zero-sized etc

Brendan Hansknecht (Jul 12 2025 at 01:53):

Got it. I'm kinda surprised that is an llvm optimization that works for a tagged union

Brendan Hansknecht (Jul 12 2025 at 01:53):

I would expect that to work for enums in llvm, but I didn't think llvm understand tag unions well enough. So I would have guessed a rust level optimization

Richard Feldman (Jul 12 2025 at 02:02):

I suppose it's possible Rust is doing something fancy. We could certainly do the same for nominal types if so; would be very easy to tell "this variant is only ever instantiated in tests" for example :smile:

Tobias Steckenborn (Jul 12 2025 at 03:18):

I'm not yet sure if I like that? How would that look like for a more modular program with multiple such things? E.g. in addition http calls and maybe a higher level thing such as some replaceable thing offered by some imported "library"? Do we then start passing 4 parameters across the whole chain of functions down to the lowest level?

Tobias Steckenborn (Jul 12 2025 at 03:21):

I somewhat prefer the approach taken e.g. here, where you provide them once, can reference them and then only need to resolve at the top level, yet where it shows you the requirements the invidual function has (https://effect.website/docs/requirements-management/services/

Richard Feldman (Jul 12 2025 at 03:21):

nah just put them in a record along with other relevant pieces of state and pass that around

Richard Feldman (Jul 12 2025 at 03:24):

Dependency Declaration: You specify what services a function needs directly in its type, pushing the complexity of dependency management into the type system.

my experience with "you just say what type you need and then something else provides it for you" systems has been way worse than my experience with "just pass an argument" systems :sweat_smile:

Richard Feldman (Jul 12 2025 at 03:25):

there's this whole new layer of "injectors" that always seem to end up being discussed in meetings because either they're not behaving the way we expected or we're trying to figure out how to get to them to do what we want

Richard Feldman (Jul 12 2025 at 03:26):

it's this category of problems I'm glad to not have in the "just pass an argument" world :smile:

Tobias Steckenborn (Jul 12 2025 at 03:35):

Okay, not yet sure on the ergonomics of that as that’s then likely a thing carried around from top level to a lot of places. If there‘s a library, e.g. for creating PDFs would they also include FS then in every function they offer for testing reasons?

Luke Boswell (Jul 12 2025 at 03:36):

One relevant consideration is that Roc is a pure FP language, so I think a pure core is more likely to naturally to occur with IO pushed to the edges of the program. So I don't think it will be a problem in practice that these are passed super deep into the program.

Richard Feldman (Jul 12 2025 at 03:42):

right, I'd expect a library for creating PDFs to have like maybe 1 or 2 functions that take it? (e.g. one for reading existing PDFs from disk and one to write the final one to disk) and then a ton of pure functions for manipulating pdf data structures before finally calling the 1 function that needs fs to write the completed pdf to disk :smiley:

Tobias Steckenborn (Jul 12 2025 at 03:45):

Okay, then likely just me not being used to that. If I imagine something where you also have billing and then maybe in some settings profile -> invoices a possibility to download currently I wouldn’t structure that near top level :sweat_smile:

Brendan Hansknecht (Jul 12 2025 at 04:40):

Of note, for the application itself, you can always just directly import many things. So no need to pass it around. It is just the library boundary where this is required. And when you reach the library boundary, often most things will be pure...of course, for mocking, you need to use this setup, but depending on your use case, you may not need to mock that much in your application

Brendan Hansknecht (Jul 12 2025 at 04:40):

I do think a lack of platform level integration testing is a form of a gap that roc currently has though....but mocking cleanly is probably of better taste anyway.

Anton (Jul 12 2025 at 09:17):

The trade-offs seem pretty good :)

kris (Jul 12 2025 at 11:40):

This is also an interesting possibility to think about capabilities, we could forbid instanciations of "real" Fs and just have the platform provide it to the entrypoint

This combined with deriving Sub-fs's that you can hand out would allow for a very powerful security model.

Say i have a plugin system, and the plugin wants to do io, i could derive a Fs from my capability that is chrooted to ./plugins/x/ and the plugin would be able to use only that directory.

kris (Jul 12 2025 at 11:41):

I believe the vale language had some designs like this, but i cannot find them, so i might be thinking of something else.

Richard Feldman (Jul 12 2025 at 12:26):

yeah any platform could choose to do that! :thumbs_up:

Sky Rose (Jul 12 2025 at 13:31):

Extending on this idea:

It would be nice to be able to define higher level abstractions. So maybe you have a function that takes a Tcp param for doing effects, a different one that takes an Http param, and a function that takes a MyCustomApiService param, depending on what abstraction layer you want to mock in tests. I think that's possible with this design.

Sky Rose (Jul 12 2025 at 13:33):

It'd also be nice to define new implementations. Like, can I write a "AwsS3" module with read and write methods, and pass it to my_fn? That'd require using interfaces (from the other megathread) rather than nominal types, and has a runtime cost (but small compared to IO costs).

Sky Rose (Jul 12 2025 at 13:39):

But: This feels like a system that you could bolt on to any language without managed effects (and I have used systems a lot like this in other languages, they work well). It feels very entwined with the type system but not with the effect system.

To play to Roc's strengths, I would expect to be able to mock any effect without a new param cuz the compiler knows what they are already.

But then you're stuck at a lower abstraction level, so I see the tradeoff.

Jasper Woudenberg (Jul 13 2025 at 17:57):

I'm really excited to see this pattern, both in Zig and in Roc. I've worked with a similar pattern in Haskell for a bit and really enjoyed it there. I think most of the work in that code was related to writing the "fake" versions of io/fs args for use in tests. Getting all that with the platform sounds amazing.

Last updated: Jun 16 2026 at 16:19 UTC