Richard Feldman (Jan 02 2025 at 06:29): Richard Feldman (Jan 02 2025 at 06:29):over the holiday break, I pretended a bunch of the design things we've been talking about already existed and built out this demo app (based on the realworld spec) to see what the code would look like.
I didn't end up completely finishing it, but I learned a lot from trying all these things out, and I've exceeded my timebox...so I'm sharing where I ended up with it! :big_smile:
I made a little screen recording talking through the code, which ended up taking a lot longer than I was anticipating...but here it is:
https://drive.google.com/file/d/17fKjW91tb-rtkxv3xGTfUhvfWAuuNqth/view?usp=sharing
Richard Feldman (Jan 02 2025 at 06:29):happy to talk about any of this, although I'm going to bed now, so may be awhile!
Tobias Steckenborn (Jan 02 2025 at 08:26):Looks interesting! Given you showed the time module, as a reference also https://effect.website/docs/data-types/duration/, https://effect-ts.github.io/effect/effect/Duration.ts.html#duration-overview and https://effect-ts.github.io/effect/effect/DateTime.ts.html.
It's sometimes really convenient to being able to simply use "3 hours" or the like and then being able to work with whatever unit you want at the place utilizing it.
Patrick Wierer (Jan 02 2025 at 09:42):Very cool! I was already going through your repos days before hoping to find it there already :big_smile:
Luke Boswell (Jan 02 2025 at 10:06):Thank you for taking the time to explain these ideas. I found it really helpful to catch up on the future design direction for roc. Overall it's looking very exciting :smiley:
I've got some random thoughts while watching the video...
Looks like a nice pattern storing effectful functions in a Custom Record. You mentioned this replaced Module Params... the thought was "do we still need Module Params"? I'm interested to try and update my AoC template package with a similar pattern using an Opaque Type and an effectul function, and see if that works today.
How is gen implemented? It looks a little like zig's comptime feature. I assume these are builtins somehow and not something that can be written oustide the compiler?
I don't love the zero argument functions... I think I prefer making it explicit like it currently is and requiring the unit {} arg. It's much easier to read at a glance and see it's a function. At the call site, can we have Time.now!() be sugar for Time.now!({})?
Re Instant... one of the things I was debating in basic-cli was what size to make the nanos since epoch, rust uses a u128 doc.rust-lang.org/nightly/std/time/struct.Duration.html. I went with I128 to simplify maths using these... but that's probably less of an issue now with static dispatch. Just making an observation that you chose U64 in this video. It's probably nothing significant, but just wondering if there was much thought in that design?
Static dispatch looks so much more concise and less viausl noise :chef's kiss:
My thoughts on integration tests is to duplicate main.roc and having a test.roc, which means I can use different dependencies if I want etc.
Jasper Woudenberg (Jan 02 2025 at 12:15):That was a great watch! Loved how the code looked, a couple of things that stood out to me:
request => route logic behind an abstraction, and then needing a lot of additional API to configure that abstraction (middelwares, hooks, ...). Jasper Woudenberg (Jan 02 2025 at 12:24):For testing against a real database, I think the main difficulty I've encountered writing tests against real external services in (non-Roc) work is the amount of effort required to setup a test harness for these types of tests with some nice features:
It'd be amazing if some part of the ecosystem took responsibility for providing a harness for the external service out of the box. I guess an opionated platform could, if it for instance only worked with a certain selection for databases and added built-in support for these. I guess Rails pushes in that direction a bit.
In the case of a platform-agnostic library though, say a Postgres client, I guess it'd make most sense for the library author to also provide test helpers, specifically in this case a test harness for writing tests against a real Postgresql database.
For a Postgres library, here's some random feature ideas for a test harness:
I'm not super certain about any of these ideas, the broader point I want to make is that the test harness might require very different and more comprehensive effects than the database client itself. In part such a test harness might require effects that the platform does not provide because it'd be bad idea in a production environment. For instance, a web platform might not want to provide any file-system access, but an ideal postgres harness might make use of the filesystem.
Not sure what the answer is. One thought is that maybe get! and set! are the tip of the iceberg, and that we'd potentially have additional test-only APIs. Creating an in-memory and sandboxed test directory for instance seems like something that could be usable in multiple scenarios, Zig offers a test helper for this which I make frequent use of.
Anthony Bullard (Jan 02 2025 at 14:17):I really love how the abstraction of effectful functions and packages not being able to rely on platform implementations is that we are basically created packages that are capabilities secure by design. And also possible to create easily testable effects!
Anthony Bullard (Jan 02 2025 at 14:24):I think depending on where we resolve static dispatch, we can have static dispatch method tear-offs. Especially easy if we do it in can.
Anthony Bullard (Jan 02 2025 at 14:24):Tear-offs would allow you to write the Ok(router.handle_req!)
Anthony Bullard (Jan 02 2025 at 14:25):But this is a form of "partial application"
Anthony Bullard (Jan 02 2025 at 14:26):If router's type is a custom type with a handle_req! method, you can desugar this to a closure that takes the rest of the args of the handle_req! method and then applies them to the method in static dispatch form (or a plain method if that desugaring has already happened).
Anthony Bullard (Jan 02 2025 at 14:33):I still think that the Str builtin should have ascii_* namespaced functions for capitalizing strings that are understood to be written using only ASCII representation. But we should also have a strong unicode package as well with great localization support.
Richard Feldman (Jan 02 2025 at 14:44):Anthony Bullard said:
I think depending on where we resolve static dispatch, we can have static dispatch method tear-offs. Especially easy if we do it in
can.
Tear-offs would allow you to write theOk(router.handle_req!)
I'm interested to explore this general topic - want to start an #ideas thread about it?
Anthony Bullard (Jan 02 2025 at 14:45):@Richard Feldman I would but I don't have the privileges to move to another channel
Richard Feldman (Jan 02 2025 at 14:45):oh I mean just start one from scratch
Anthony Bullard (Jan 02 2025 at 14:45):Ah, OK
Richard Feldman (Jan 02 2025 at 14:56):Luke Boswell said:
- How is
genimplemented? It looks a little like zig'scomptimefeature. I assume these are builtins somehow and not something that can be written oustide the compiler?
yeah I was thinking we'd have some hardcoded list of supported ones in the compiler, just like we do today with Abilities
- I don't love the zero argument functions... I think I prefer making it explicit like it currently is and requiring the unit
{}arg. It's much easier to read at a glance and see it's a function. At the call site, can we haveTime.now!()be sugar forTime.now!({})?
want to start an #ideas thread about zero-arg function syntax? I'd like to explore ideas!
- Re
Instant... one of the things I was debating in basic-cli was what size to make the nanos since epoch, rust uses au128doc.rust-lang.org/nightly/std/time/struct.Duration.html. I went withI128to simplify maths using these... but that's probably less of an issue now with static dispatch. Just making an observation that you choseU64in this video. It's probably nothing significant, but just wondering if there was much thought in that design?
it's a great question...also worth its own thread I think! :smiley:
Anthony Bullard (Jan 02 2025 at 14:58):Here's the thread: #ideas > Static Dispatch and method tear-offs
Anthony Bullard (Jan 02 2025 at 15:00):One last thing I'd like to say is: I may be in the minority (and I don't think this would be a "business move"), but I think videos like this would be interesting on a "Roc" YouTube channel, as well as the virtual meetups and such.
Agus Zubiaga (Jan 02 2025 at 17:39):This looks great! I really like the discovery that static dispatch can also replace module params. People are very familiar with this pattern from other languages—the difference is just that all packages would have to use this approach for any I/O which gives us all the nice ecosystem properties we want. Also a nice saving for the weirdness budget :smiley:
Dawid Danieluk (Jan 02 2025 at 17:41):Great video and I think that this dummy realworld example turned out great!
I've had some trouble understanding expect |get!, set!| part (1:46:25), was there some sort of proposal/discussion on zulip that I could dive into to get that better? I don't understand what set! is doing if we're passing manually created effects into Route.new anyway.
EDIT. nvm, I've rewatched the video second time and I think I get it. Packages would create dummy clients, so we could use set! in our expect and package could use get! to get value back.
Agus Zubiaga (Jan 02 2025 at 17:42):Re roc-pg connect function, I was actually already exploring making the stream opaque in the connect function sometime ago. All it needs is a way to read and write from a stream. This would enable the package to be used with Unix Domain sockets or mocked/in-memory servers like Jasper suggested.
Kilian Vounckx (Jan 03 2025 at 14:25):Just about a quarter of the way though. Very interesting so far! I was really surprised by how much I liked the logging API. I didn't think such a minimal design could still be so concise in user land
Kilian Vounckx (Jan 03 2025 at 14:32):One question about the gen from_str_case_insensitive part. (I am there now, so sorry if it gets answered later)
What about accents? (diacritics? Idk the difference). I guess in Turkish all the i's might be the same, but in many languages, there are words where putting an accent somewhere changes the meaning.
Kilian Vounckx (Jan 03 2025 at 14:34):After a search online, even swapping Turkish i's changes meaning of words. (As well as slightly different pronunciation? I'm not Turkish so not sure. But they are different letters altogether)
Richard Feldman (Jan 03 2025 at 15:08):interesting! Maybe this is a case where from_ascii_case_insensitive would make more sense
Richard Feldman (Jan 03 2025 at 15:08):assuming we add ASCII case-related things to Str
Richard Feldman (Jan 03 2025 at 15:08):like Rust has
Richard Feldman (Jan 03 2025 at 15:08):but keep full Unicode casing out of scope (unlike Rust)
Agus Zubiaga (Jan 03 2025 at 15:34):I think that makes more sense for an autoderived function working on enum variants that can only be ASCII
Agus Zubiaga (Jan 03 2025 at 15:36):Also, only supporting ASCII is probably better for the perf of the generated function
Anton (Jan 10 2025 at 15:36):based on the roc-realworld repo, do we plan to remove try @Richard Feldman ?
Anthony Bullard (Jan 10 2025 at 15:43):Part of me wishes we could keep try for function level early returns on errors, and recover ? as expression level recovery from errors
Anthony Bullard (Jan 10 2025 at 15:43):But that's my Javascript/Typescript/Dart brain probably
Sam Mohr (Jan 10 2025 at 15:43):I think we should get rid of it to always prefer having one way to do things, unless we really need it.
Sam Mohr (Jan 10 2025 at 15:44):But I see its value
Anthony Bullard (Jan 10 2025 at 15:44):Have we discussed what I just said above before?
Anthony Bullard (Jan 10 2025 at 15:45):Basically have ? postfix be a flatmap of the right side over the left side. I'm thinking of when we have SD
Sam Mohr (Jan 10 2025 at 15:45):We had a big ol' discussion about it
Anthony Bullard (Jan 10 2025 at 15:45):Ok, I'll look for it
Sam Mohr (Jan 10 2025 at 15:45):#ideas > `try` keyword instead of `?` suffix being one of them
Sam Mohr (Jan 10 2025 at 15:46):The outcome is that it's really hard to make expression level work
Sam Mohr (Jan 10 2025 at 15:46):In a way that doesn't have a lot of confusing rules
Sam Mohr (Jan 10 2025 at 15:46):But "? returns errors from the function" is brain-dead
Sam Mohr (Jan 10 2025 at 15:50):I will say, expression-level try actually is pretty simple with methods and Result.try:
decoded = Http.get!("api.com").try(.decode())
yeah I think going back to try being a function in the Result module is the way to go
Eli Dowling (Jan 11 2025 at 16:40):This was a great watch.
I'm very excited for destructuring and field access for custom records, It's going to make decoding and encoding data super smooth.
I'm certainly sad to see Module qualified function calls go, I do agree they are clearer to read, but overall I definitely agree that the advantages of static dispatch outweigh the loss.
Krzysztof Skowronek (Jan 15 2025 at 11:15):I haven't watched the whole codebase walkthrough yet, but on the topic of string interpolation, I think C# could be a nice inspiration: https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/tokens/interpolated
They solved the "keep raw JSON in string literal" problem by... just adding more " around the string literal. You indicate that the string is interpolated by doing $"some string {someVariable}" - and you can add more $ in the beginning to indicate how many { you for interpolation.
That means that the following is possible:
var json = $$"""
{
"age": 5,
"name": {{userName}}
}
""";
It also takes care of the leading whitespace - it's really AWESOME. You can also just have """" (four) around the literal, so that you can use """ inside of your string.
https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/proposals/csharp-10.0/improved-interpolated-strings#the-handler-pattern - there is also a way to hook into the interpolated string and for example use interpolated string for SQL commands with proper parameter handling, or for structured logging.
Dawid Danieluk (Jan 15 2025 at 12:21):I think rust also does the same thing, which imo is very flexible and easy to understand. Just make sure that you put more surrounding quotes than there are in a string and you're good to go.
Krzysztof Skowronek (Jan 15 2025 at 12:53):I just finished the whole things and wow - I can't wait for Roc to be done :)
as for tests, I again think that C# has this figured out :) with nUnit you can decorate methods with [Setup] and [TearDown] attributes, they will be run before and after each test in your test class (yes, you need a separate class for tests, a separate project to be fully honest).
There is also [OneTimeSetup] and [OneTimeTearDown], and the coolest thing is that they are discoverable upwards. The discovery starts at the test class in a namespace MyApp.BlogApi.Tests, then looks in MyApp.BlogApi, then in MyApp, then outside of all namespaces. They are also corresponding to directory structure (usually :)). All the setups (both one time and normal) are run from top to bottom, and teardowns from bottom to top.
So for example, you can have top-level class that starts TestContainer with PostgreSQL in [OneTimeSetup], and saves the connection string in a static variable. In the same class you can have [Setup] that takes a snapshot of the db, and [TearDown] that restores it.
Then at any level you spin up your api app using WebApplicationFactory, which basically runs your whole app. You get an HttpClient to it.
Cool thing is, you can even pass that client to Playwright runner, so you can have UI tests running on in-memory app! Even better, when you debug the test, you can debug both the API and the UI tests, both written in C#.
When I did AoC in Rust, I remebered reading that "Rust has testing built-in!". Ok, how do I run the same setup for for 5 tests? Ok, I just write a function that returns everything I need. But how do I run it once? Oh, there is a package that can run a thing once...
This doesn't feel like "proper" testing.
Krzysztof Skowronek (Jan 15 2025 at 13:07):Another thing C# is doing better than others (IMO) is ORM for database access. EF Core is just awesome.
Main feature that powers it (and many other cool things) is Expression - basically a way to introspect code at runtime.
We all know the cool functional style making collections easy:
var myProductIds = products.Where(x => x.Product.Owner.Id == currentUser.Id).Select(x => x.Id).ToList();
The best thing about EF Core is that you can just use the same thing, but against the db:
var myProductIds =
dbContext // that's the whole db
.Products // DbSet<Product>, represents a table
.Where(x => x.Product.Owner.Id == currentUser.Id) // filter
.Select(x => x.Id) // project
.ToList()
(you can also put the query in a separate method and mark it with attribute to become a prepared SQL statement)
the above will generate
select Id from Products where OwnerId = ($1)
How does it work? Where and Select (and every other method like this, Count, ToDictionary etc) doesn't take Func<Product>, it takes Expression<Func<Product>>. What's the difference? Expression "knows" the code.
It's a bit complicated, but you can easily parse the function you passed into there and go "oh, we are comparing the Id property to something", and just spit out proper SQL.
It build's a description in IQueryable - basically like a fancy iterator, that converts the expressions into a request to underlying tech, and enumerates the results. It's just awesome - no new syntax, you get compile-time errors if the expression cannot be converted to SQL (rarely comes up in reality).
I am not proposing that you actually write an ORM, but if the language had something like Expression<Func> - all of this would be possible also in Roc.
Last updated: Jul 06 2025 at 12:14 UTC