Stream: show and tell

Topic: roc-pg new features and examples

Agus Zubiaga (Apr 15 2023 at 18:38):

roc-pg has a nicer API for expecting results and can now do Prepared Statements and Pipelining.

I added examples to the README:
https://github.com/agu-z/roc-pg

Rene Mailaender (Apr 15 2023 at 19:14):

wow. very cool. i'll try this some time in the near future. :smile:

Rene Mailaender (Apr 15 2023 at 19:19):

i see you reference Task and Tcp in your Pg.Client: pf.Task.{ Task, await, fail },
is there any mechanism to assure a certian type exists on a platform? (or plans for such a mechanism?) otherwise using this package on a different platform would crash right? :thinking:

Agus Zubiaga (Apr 15 2023 at 19:21):

Correct. For now, this only works with basic-cli. The plan is to make it platform agnostic but the current model of hosted modules and effects doesn't make that very convenient.

Agus Zubiaga (Apr 15 2023 at 19:26):

If I understood correctly, that will change relatively soon :)

Agus Zubiaga (Apr 15 2023 at 19:34):

#ideas > effects in packages I think is the relevant conversation

Rene Mailaender (Apr 15 2023 at 19:42):

i see. and thanks for the link. :)

Richard Feldman (Apr 15 2023 at 19:55):

yooooo, this is amazing!!! :heart_eyes:

Luke Boswell (Apr 16 2023 at 00:52):

@Agus Zubiaga this is super cool.

Are you referring to having glue support Op tag union for effects before you modify to make it platform agnostic? Do you think it will be difficult to achieve? I assume you need sockets support in a platform, what would the interface look between the package and platform? is that going to be more generic like a List U8?

You mention SSL and Connection pooling will require new platform primitives, what did you have in mind here. Is this also going to be achievable platform agnostic?

Agus Zubiaga (Apr 16 2023 at 22:22):

From what I heard from Richard, the idea was to have a common Task type with a 3rd type variable for effects. Which in this case would be something like [ Tcp TcpEffect ], where TcpEffect encodes all the operations such as connect, read, and write.
That should work as long as the platform either supports the exact type or there's a way to map it to its own type.

Agus Zubiaga (Apr 16 2023 at 22:31):

You could also provide these as functions to Pg.Client.withConnect and make it platform-agnostic that way. That'd work, but it would make the types complex and it means the user needs to take care of establishing the TCP connection and doing the SSL stuff, which might not be great for beginners.

Agus Zubiaga (Apr 16 2023 at 22:32):

I could be persuaded into that direction though

Agus Zubiaga (Apr 16 2023 at 22:35):

I haven't given much thought to Pooling because there isn't really a concurrency story yet. Although, I wonder if that could be provided by the platform or by a package that isn't specific to PostgreSQL.

Agus Zubiaga (Apr 16 2023 at 22:37):

I imagine a platform like Folkert's server might want to be involved in/manage that

Agus Zubiaga (Apr 16 2023 at 22:42):

It's a little early to know the right answer for all this stuff, but in the meantime I'm focusing on implementing the protocol, the API, and building things on top of it :)

Georges Boris (Apr 16 2023 at 23:40):

I'm so happy for this package to exist so soon :heart:️

Just a nit if you don't mind. On elixir's ecto we use all and one for the select functions instead of expect1 and expectN. It is a bit unusual for me coming from Elm to see functions ending with 1/N.

I think the difference is that the expect is matching the return statement right? so it would work the same for insert/update/delete with returning?

just throwing random naming refactors here:

archiveUser : Str -> Task {} PgErrors
archiveUser userId =
  query "update users set archived = true where id == $1"
  |> withParams [ userId ]
  |> returningNothing

archiveUser : Str -> Task Str PgErrors
archiveUser userId =
  query "update users set archived = true where id == $1 returning id"
  |> withParams [ userId ]
  |> returningOne

archiveUsers : List Str -> Task Str PgErrors
archiveUsers userIds =
  query "update users set archived = true where id in $1 returning id"
  |> withParams [ userIds ]
  |> returningMany

(btw - for this last use case we should probably have some way to bind lists without turning them into strings first? don't know if this is something already in the package! :grinning_face_with_smiling_eyes:)

Agus Zubiaga (Apr 16 2023 at 23:47):

Oh, calling them “params” instead of “bindings” is a great suggestion! I went a bit too technical there

Agus Zubiaga (Apr 16 2023 at 23:50):

As per query, that’s what I started with and I don’t love it because I don’t think of update or insert as queries. So I went with “command” which is what Postgres uses in their docs.

Agus Zubiaga (Apr 16 2023 at 23:51):

The passing lists as parameters is something I’d like to have but I haven't decided how I want to do it. We could expand a binding into many, but that'd require parsing the SQL in Roc. Although, we might want to do that anyway so we can also have named parameters in addition to positional ones (which are the only type natively supported by PostgreSQL).

Agus Zubiaga (Apr 16 2023 at 23:54):

I’ll have to think about the returning suggestion. I kinda prefer expect because I don’t want people to think that roc-pg is adding a returning clause for them.

Agus Zubiaga (Apr 16 2023 at 23:56):

I agree the N and 1 suffixes are a little uncommon. I did start with Many and One as you suggested. However, since expectOne asserts that at least one row is returned, it felt like a beginner might think expectMany would fail if <2 rows were returned.

Agus Zubiaga (Apr 16 2023 at 23:59):

expectOne / expectAll might work though

Agus Zubiaga (Apr 17 2023 at 00:06):

expectAll makes sense because the argument it takes is a result decoder

Agus Zubiaga (Apr 17 2023 at 00:08):

We don’t need something like returningNothing because if you execute a command without an expect, you’ll just get the “undecoded” Pg.Result type which you can use for dynamic use cases

Brendan Hansknecht (Apr 17 2023 at 00:38):

Just a note: Fundamentally concurrency must be provided by the platform. So the story really has to be built out on platforms and not as part of the roc language. Of course, roc languages changes can make apis more convenient to use.

Agus Zubiaga (Apr 17 2023 at 00:42):

Agreed. I meant I haven't seen examples of platforms providing concurrency.

Brendan Hansknecht (Apr 17 2023 at 00:43):

Yeah, i think all i have seen is a webserver platform that i made a long time ago. It called into a roc request handling function from many threads, but the roc code was still fundamentally unaware of threading. I think we should soon have a lot more power with async rust hosts and the new formulation for hosts.

Brendan Hansknecht (Apr 17 2023 at 00:43):

But yeah, needs to be built out

Agus Zubiaga (Apr 17 2023 at 00:47):

Yeah, looking forward to all that!

Brendan Hansknecht (Apr 17 2023 at 00:53):

I might try to work on transforming a few platforms soon. Was debating between that or maybe zig glue. Updating platforms so exploration into multi threading and async can be done sounds likely more important. Though might need to expand rust glue before some of it works.

Richard Feldman (Apr 17 2023 at 01:09):

I think Rust glue is there now, although there may still be a bug relevant to effect interpreters? I forget what the current status of that is, but I think @Folkert de Vries knows!

Brendan Hansknecht (Apr 17 2023 at 01:29):

Did we finish porting all the tag stuff?

Georges Boris (Apr 17 2023 at 09:18):

Agus Zubiaga said:

We don’t need something like returningNothing because if you execute a command without an expect, you’ll just get the “undecoded” Pg.Result type which you can use for dynamic use cases

Oh! So in this case the expect is more like a map over the success response?

Folkert de Vries (Apr 17 2023 at 09:33):

@Brendan Hansknecht "the tag stuff" is all there, but there is a bug in how we generate the data for glue which means it fails for more complex recursive structures. We also lack glue gen for unsized values. Not sure how big an issue that is right now though, I think we can already do a lot of task stuff without it

Agus Zubiaga (Apr 17 2023 at 13:56):

Georges Boris said:

Oh! So in this case the expect is more like a map over the success response?

Yeah, somewhat like map but not quite.

Pg.Cmd.expect1 fails the Task if less than 1 row is returned and instructs the database to return at most one row even if you didn't add a LIMIT clause (which we can do at the protocol level).

They also explicitly take Pg.Result.Decode, not just any function, I do have Pg.Cmd.map for that but I don't expect people to reach for that often.

The original idea came from elm/http's expect but adapted to the needs of reading database query results.

Georges Boris (Apr 17 2023 at 14:07):

got it! the only that is weird about expect is that in Roc this word is also used for tests so the meaning becomes a bit tied to that imo.

Agus Zubiaga (Apr 17 2023 at 14:13):

Yeah, good point. I wonder which other word we could use:

• returning makes it seem like it adds the RETURNING clause
• decode doesn't quite capture the special logic for expect1

Agus Zubiaga (Apr 17 2023 at 14:16):

Maybe Pg.Cmd.takeOne / Pg.Cmd.takeAll?

Agus Zubiaga (Apr 17 2023 at 14:18):

Side note: I wonder if we should have an #api design stream

Richard Feldman (Apr 17 2023 at 14:37):

good idea! Now we do: #API Design

Last updated: Jul 06 2025 at 12:14 UTC