notramo (Oct 16 2026 at 15:40): notramo (Oct 16 2026 at 15:40):sqlite sucks.
It saddens me that every time I see some cool Roc web or app framework, the storage layer is sqlite. I instantly lose interest in any Roc project where I see SQL, especially in the form of sqlite. Migrations are terrible, it has a dynamic schema, and a huge C library that needs platform support.
And SQL query language is so different, compared to Roc. This is the worst thing. Most of the logic is about conversion between the two paradigms.
SurrealDB is quite good, but it's not supported in any Roc framework, and it's more like a Postgres alternative, not an sqlite alternative. It's much better for web projects than Postgres. I would never use Postgres for anything, if I can use SurrealDB instead. But for simple apps, where basic object persistence is required, a separate database process is overkill.
I think, a new, pure-Roc in-process object persistence framework could be written in about ~6000 lines of Roc code. (Maybe even in ~4000 lines.) It could work with any platform that provides basic file I/O, and it could provide a much more ergonomic query language, i.e. everything that's possible in Roc, because it's pure Roc. And the whole Roc typesystem, (including tag unions and other niceties) could be used for storing data.
Basic feature set that would be useful:
I have some architecture ideas, and I'll write them here soon, but meanwhile if anyone is interested in this, dump ideas here.
Let's design the in-process database for the next generation of Roc apps together!
Zak Kohler (Oct 16 2026 at 16:02):Sqlite is pretty awesome. Watch some of the creators talks on the technical aspects of testing. It's kind of legendary. Also you have been enforce a strict schema for a long time now.
Other directions I'm really interested in thinking about besides sequel light is something like Rich Hickey's datomic
notramo (Oct 16 2026 at 16:07):Oh, I didn't meant to disregard the engineering achievements of the project. There's so much to learn from it. But SQL (any dialect) is a terrible language to work with.
notramo (Oct 16 2026 at 16:08):I used EdgeDB, which was quite good, but still limited by how Postgres arranged data in 2 dimensions.
notramo (Oct 16 2026 at 16:08):But EdgeQL was much better than SQL.
notramo (Oct 16 2026 at 16:09):When the project got abandoned, I moved to SurrealDB, which allows tree-like data, and ADT.
notramo (Oct 16 2026 at 16:10):But I'm not satisfied with it either. (Tooling is weak.)
notramo (Oct 16 2026 at 16:11):It's nice until it has to be used in a different language. It requires so much plumbing.
notramo (Oct 16 2026 at 16:11):(Just like SQL.)
notramo (Oct 16 2026 at 16:12):Type-checking the queries should require multiple GiB of RAM, yet it does with SQL:
notramo (Oct 16 2026 at 16:12):https://github.com/niclas-ahden/roc-pg-insane-in-the-membrane
notramo (Oct 16 2026 at 16:16):Datomic is interesting, I have not heard of it until now.
notramo (Oct 16 2026 at 16:58):Checked it out, but it's quite different from how I would imagine a Roc in-memory database.
Steve Howell (Oct 16 2026 at 17:25):I have fond memories of dBase haha. (It had its own procedural language.)
Romain Lepert (Oct 16 2026 at 17:50):SpacetimeDB is a really fast in-memory db (with persistence) providing a redux-like functional API
notramo (Oct 16 2026 at 17:58):A database that treats data in rows and columns instead of a tree-like structure is not an option.
notramo (Oct 16 2026 at 17:59):In my experience, 2 dimensional layout often requires a few other tables, which increases complexity, and reduces readability.
notramo (Oct 16 2026 at 18:01):My goal with this idea is not to replicate other databases in Roc, but to find the most ergonomic way to manage persisted data in Roc.
Zak Kohler (Oct 16 2026 at 19:32):Why not just a typed EdgeQL style front end, in Roc, on a known acid compliant engine?
Also I don't know how you would get both strict typing and also avoid migrations. most migrationless systems just force the client to handle nulls all over and you get bad user experience with lots of bugs. (or you limit schema changes like protobuf does.
notramo (Oct 16 2026 at 20:50):Also I don't know how you would get both strict typing and also avoid migrations.
I want typed migrations with mandatory error handling. For example, most ORMs allow you to write a migration that changes an optional text field to a required one. Then, if the production DB has one record where that field is null, the migration fails. But it should be possible to type this field modification as returning Try({}, [IncompatibleValue]), so it's obvious when writing the migrations that it can fail, and has to be explicitly handled. This is a very simple example, but as the schema gets more complex, there are many more ways to fail, and if it's not tracked in the type system, migrations can quickly become a bad experience.
notramo (Oct 16 2026 at 20:53):Why not just a typed EdgeQL style front end, in Roc, on a known acid compliant engine?
If the whole data is in memory, queries can be very readable. So it's more like an object persistence library than a DB engine, as the “query optimizer” would be the Roc compiler itself, because the queries are pure Roc functions.
notramo (Oct 16 2026 at 20:58):If the data is elsewhere, and Roc code only builds queries, it's just indirection.
notramo (Oct 16 2026 at 21:17):How would you implement an type-safe EdgeQL-style front end in Roc? Comptime parse the queries?
Zak Kohler (Oct 17 2026 at 15:29):if the "whole data is in memory" how do you get the "D" in acid that "D" part is a huge huge portion of what make writing a db hard.
notramo (Oct 17 2026 at 18:53):Not only in memory. More like a full in-memory copy of the disk content, not only for caching, but also for a nicer query interface. Sorry, I'm not really good at describing this idea.
Krzysztof Skowronek (Oct 27 2026 at 19:15):@notramo what do you think about ORMs like Entity Framework Core for C#? for the most part you can forget what database you are using
soon you are reminded wich one it is once you have to set up indexes, edit the partitioning and so on
#ideas > Reflection - typed function bodies as Roc values (reifying) but this could "be the way"
notramo (Oct 27 2026 at 19:44):I don't know C# good enough, but I think it's generally a bad idea to reuse language syntax for queries if it's not a fluent builder.
notramo (Oct 27 2026 at 19:45):And I don't like fluent builders either.
notramo (Oct 27 2026 at 19:45):I want to either use the query language of the database, like how roc-pg works, or pure Roc without extra steps.
notramo (Oct 27 2026 at 19:47):for the most part you can forget what database you are using
This indicates how bad SQL is. Multiple database implementations, that are mostly the same, with just a few unique features.
Karakatiza (Oct 27 2026 at 19:59):or pure Roc without extra steps.
Would you prefer a.and(b).is_eq(c) or a and b == c?
notramo (Oct 27 2026 at 20:01):a.and(b).is_eq(c) if it's the same behavior as any other parts of Roc.
notramo (Oct 27 2026 at 20:01):But if it's not a query builder, but the actual data structure, we could get a and b == c without changing the behavior.
notramo (Oct 27 2026 at 20:02):This would cover all the necessary features for my idea: #ideas > Type reflection — Zig `@typeInfo()` equivalent
notramo (Oct 27 2026 at 20:03):I think, for an in-process object persistence framework, there's no need for a query builder, since everything is in the process, so it can be accessed like normal Roc values.
notramo (Oct 27 2026 at 20:04):Because it actually works with normal Roc values and only provides constraints and storage.
Isaac Van Doren (Nov 02 2026 at 19:24):What about roc being immutable do you think would help with implementing transactions?
notramo (Nov 02 2026 at 19:25):Easy rollback.
Isaac Van Doren (Nov 02 2026 at 19:25):Say more?
notramo (Nov 02 2026 at 19:33):It's much easier to save original database state than in languages that allow mutation.
notramo (Nov 02 2026 at 19:39):Currently I'm waiting for #ideas > Type reflection — Zig `@typeInfo()` equivalent.
Isaac Van Doren (Nov 02 2026 at 19:57):The state in your DB has to be written to disk to be persistent though. So it can't be a tree of in-memory Roc objects. Because the state has to be separate, whether or not the client language allows mutation doesn't really seem relevant
notramo (Nov 02 2026 at 20:00):So it can't be a tree of in-memory Roc objects.
Why? Redis works similarly.
Isaac Van Doren (Nov 02 2026 at 20:34):Redis doesn't give you durable persistence. It looks like it can write periodic snapshots to disk but I don't think there's a proper way to configure it where you can guarantee that every write you make to it will be saved to disk
Zak Kohler (Nov 02 2026 at 21:02):Look at datomic. it basically relys on external kv for duribility, with engine and all data as a stateless data structure.
notramo (Nov 02 2026 at 21:03):SurrealDB also supports external KV engines.
Last updated: Nov 04 2026 at 18:54 UTC