Stream: beginners

Topic: It's Impossible to make generic abilities right?

Eli Dowling (Dec 07 2024 at 06:58):

I've had a few times recently when I've wanted to make abilities in roc that are generic. Normally I've just given up and not used abilities, but this time I cared a bit more about it actually working.

I was just finalising the improvements to the basic-cli file reader, and I realised the tcp stream and file api is basically the same.
Ofcourse in both cases we're just talking about a stream of bytes. This is a classic example kind of thing that I might want to write a general function around without worrying where the bytes are coming from.

So I go ahead and make an ability, but I immediately get stuck:
Reading from a file or a tcp socket or a pipe have some common error states, but also have their own unique errors.
You could probably try to coerce them into a common interface if you squint a little, but you'd be loosing information.
I was imagining something like

ByteReader a implements
    read!: reader => Result (List U8)  a  where reader implements ByteReader
    readLine! : reader=> Result (List U8) a where reader implements ByteReader

ByteWriter a implements
    write! : writer,  List U8 => Result {} a  where reader implements Byterwriter

I can imagine a bytewriter for file might return DiskFull which would be nonsensical for a tcp stream or a broken pipe which would be silly for a file.
I'm just looking for confirmation that roc doesn't support anything like this at present.

Eli Dowling (Dec 07 2024 at 06:59):

Also If it's not possible I'd love to hear suggestions for workarounds?
I guess we could just make a huge catch all IOErr type?

Eli Dowling (Dec 07 2024 at 07:02):

Then group them logically:

StreamErr : [
    StreamNotFound,
    PermissionDenied,
    ConnectionRefused,
    ConnectionReset,
    Interrupted,
    OutOfMemory,
    BrokenPipe,
    Unrecognized Str,
]

WriteErr:[FIleErr[DiskFull],TcpErr StreamErr ]`

Ayaz Hafiz (Dec 07 2024 at 07:05):

yeah it's not supported atm. Making the error type concrete is the only way.

Eli Dowling (Dec 07 2024 at 07:06):

My main complaint with this is that you've essentially just created exceptions again.
Like you have some large set of errors that a function may or may not throw, and you are never going to catch them all, so you just have to re-raise them mostly.
You then just hope the author of your function documents which of the errors this function throws so you can catch and handle those ones.

Eli Dowling (Dec 07 2024 at 07:06):

Ayaz Hafiz said:

yeah it's not supported atm. Making the error type concrete is the only way.

Out of curiosity, do you think that's likely to change?

Ayaz Hafiz (Dec 07 2024 at 07:07):

I have no idea, I'm pretty far removed these days

Eli Dowling (Dec 07 2024 at 07:07):

haha, fair. Thanks anyway :)

Brendan Hansknecht (Dec 07 2024 at 07:13):

Yeah, can make an ability, but can make an interface that uses generic higher order functions

Brendan Hansknecht (Dec 07 2024 at 07:13):

By interface I mean adhoc instead of abilities but all types confirm

Brendan Hansknecht (Dec 07 2024 at 07:14):

So you would still be sharing code

Brendan Hansknecht (Dec 07 2024 at 07:14):

I think static dispatch would fix this though.

Brendan Hansknecht (Dec 07 2024 at 07:15):

Hmm, though it would still depend on higher order relationships, so I'm not fully sure. But feels like you could specify what you want with static dispatch (just not 100% sure it would work)

Eli Dowling (Dec 07 2024 at 07:18):

oh, really, that's interesting.
I'm not sure I totally follow what you mean by interface that isn't an ability @Brendan Hansknecht ?

Say I wanted to write a function Tokenize: a-> List Tokens where a implements ByteStream
do you just mean writing Tokenize: ({}->List U8)->List Tokens?
then wrapping like this? reader=\{}->file |>readFile!

Eli Dowling (Dec 07 2024 at 07:23):

I guess you could always just invent crappy OOP:

ByteReader a:{
    read!:{}=> Result (List U8) a,
    readToBuf!:List U8=> Result (List U8) a
}

:joy:

Brendan Hansknecht (Dec 07 2024 at 07:23):

Yes, crappy oop

Brendan Hansknecht (Dec 07 2024 at 07:23):

That is what I was talking about

Brendan Hansknecht (Dec 07 2024 at 07:23):

But static dispatch makes it first class

Eli Dowling (Dec 07 2024 at 07:25):

That's cool. I think standardising on some of these interfaces Is going to be super important for making code shareable between platforms and stopping the ecosystem from fragmenting all over the place

Brendan Hansknecht (Dec 07 2024 at 07:25):

Yeah. And I think it will require static dispatch or more flexible abilities to work well

Eli Dowling (Dec 07 2024 at 07:31):

Good, well I'll shelve that grand plan for now and come back to it once one of those two things are implemented.

Brendan Hansknecht (Dec 07 2024 at 07:33):

If you just want to share code, maybe module params for all of those functions are enough?

Eli Dowling (Dec 07 2024 at 07:38):

It was more about being able to let the user easily and naturally write code that works with both TCP and file streams.

But yeah module Params would definitely work too. But overuse starts getting towards the functor nightmare you have in very generic Ocaml code, where modules start taking a gazillion parameters because they need Params for their children which need Params for their children.

Luke Boswell (Dec 07 2024 at 08:11):

I know it's not the question directly. But the API I'm moving towards in basic-cli uses a common internal IO error tag. So we could have this, but we'd just lose the top level StdinErr, or FileReadErr...

Luke Boswell (Dec 07 2024 at 08:12):

Here thr error in the host https://github.com/roc-lang/basic-cli/blob/08467a2447c5f27f3c32fac6144ebe60a691810d/crates/roc_host/src/glue.rs#L7

Luke Boswell (Dec 07 2024 at 08:13):

And in the platform https://github.com/roc-lang/basic-cli/blob/08467a2447c5f27f3c32fac6144ebe60a691810d/platform/PlatformTasks.roc#L54

Eli Dowling (Dec 07 2024 at 08:15):

This isn't intended to be public facing though right?
Like if I as a user of basic CLI call read I'm not going to get that am I?

Eli Dowling (Dec 07 2024 at 08:16):

@Luke Boswell ^

Luke Boswell (Dec 07 2024 at 08:31):

We could. @Richard Feldman had a separate tag for each, but they're all the same internally. Maybe for the sake of an ability if it makes things nicer, (or to experiment with it) we could try this.

Luke Boswell (Dec 07 2024 at 08:32):

Well almost identical internally... I am definitely keen to remove the RocStr and use a proper tag and the InternalIOErr was the plan.

Eli Dowling (Dec 07 2024 at 10:11):

We could, but I must say, I just really hate the idea that if we do that, every time I want to match on read related errors I have to put Err a -> Err a at the bottom, to handle all the cases which I know can't occur but the compiler says I need to handle.

Eli Dowling (Dec 07 2024 at 10:13):

But, I suppose if we add an ability as well as keep the reader specific API we could enable the generic reader with crappy errors temporarily without adversely affecting normal use. and then move to just using the ability once either parameterized abilities or static dispatch gets added

Eli Dowling (Dec 07 2024 at 10:14):

I would do a handy little x-> x at the end, but that seems to invariably lead to compiler crashes :sweat_smile:

Richard Feldman (Dec 07 2024 at 13:17):

hm, couldn't this be a concrete ByteReader opaque type that you can just create and then call things on? :thinking:

Richard Feldman (Dec 07 2024 at 13:18):

similarly to how Iter would be a concrete type and not an Ability

Eli Dowling (Dec 07 2024 at 14:48):

The main reason that is slightly less desirable is that it prevents my file from having any other methods.
What if I want a file reader that also has the ability to be closed, abilities ( good name choice :sweat_smile:) are a bit nicer there because I can have it be a "reader" and also a file with file specific stuff like closing .

But ofcourse the file type could just contain a reader and have a get reader function.

So yeah, I don't see why that wouldn't be a reasable solution though. Even if it is a touch less ergonomic.

What do you think @Luke Boswell would that feel like a good fit for the API to you?

Richard Feldman (Dec 07 2024 at 15:51):

ah, so I think we should probably not support manual closing (as opposed to automatically closing it when its refcount hits zero and it gets deallocated - there's a technique the host can use to make this happen) because by default it opens the door to "double-free" problems

Richard Feldman (Dec 07 2024 at 15:52):

where you open a file, close it, then the OS hands out that file descriptor again, you call close on the original file a second time, and it closes the other one by mistake

Richard Feldman (Dec 07 2024 at 15:53):

if only automatic closing is possible, it rules out that whole category of problems - just like how automatic memory management rules out actual double-free bugs

Eli Dowling (Dec 07 2024 at 16:34):

Oh, haha, I wasn't actually proposing adding a close function, It was just the first operation related to file reads that's not general to all reading. :sweat_smile:
But I agree, better to just make it clear dereferencing will auto close and not have to worry about double closing.

Last updated: Jul 06 2025 at 12:14 UTC