Ghislain (Feb 03 2025 at 11:04): Ghislain (Feb 03 2025 at 11:04):Sam Mohr said:
Long story short, Richard proposed moving to Zig while we're rewriting so much of the compiler
I may have missed the message, but is there a more developed explanation about the move from Rust to Zig? As it is an important decision, I would be very interested to understand it more deeply.
Richard Feldman (Feb 03 2025 at 12:17):yeah I should write something up
Richard Feldman (Feb 04 2025 at 00:59):I wrote something up! https://gist.github.com/rtfeldman/77fb430ee57b42f5f2ca973a3992532f
happy to answer any questions about it
Isaac Van Doren (Feb 04 2025 at 01:04):While we're at it, we also want to convert it to use recursive descent; it was originally written using parser combinators
Aren't parser combinators just a particular way to do recursive descent?
Brendan Hansknecht (Feb 04 2025 at 01:07):Maybe technically, but not in practice in terms of what the end code looks like and how readable it is (note, I personally am not a fan of parser combinators for anything medium sized or bigger. So I personally find it less readable for something like roc, but I'm sure some others disagree).
Anthony Bullard (Feb 04 2025 at 02:20):That’s funny. I was taught RD via Parser Combinators
Anthony Bullard (Feb 04 2025 at 02:21):But I never went to college, and wasn’t interested in compilers until I started doing FP
Joshua Warner (Feb 04 2025 at 02:26):I haven’t seen “traditional” recursive descent work in an FP language (yet!). I think combinators make that feasible for FP.
Joshua Warner (Feb 04 2025 at 02:27):I guess maybe it could be reasonable with the state monad? I only have minimal Haskell experience tho.
Anthony Bullard (Feb 04 2025 at 02:29):I thought Chakras parser was very readable, but it was at least a step function more simple than Rocs grammar
Isaac Van Doren (Feb 04 2025 at 02:49):Clearly, if you want to join in the long tradition of successful compilers that have been rewritten at some point, you can pick whatever target language suits you.
For some reason rewriting for purposes of self hosting has always sat in a different bucket in my head than other total rewrites. So It's very interesting to realize that so many compilers have been completely rewritten and it really does make it seem like a much more reasonable of a thing to do.
Pit Capitain (Feb 04 2025 at 17:02):Richard Feldman said:
I wrote something up! https://gist.github.com/rtfeldman/77fb430ee57b42f5f2ca973a3992532f
Richard, can I share this link in the Elm slack? There's a "language-implementation" channel, and I'm sure the folks there would be very interested.
Richard Feldman (Feb 04 2025 at 18:53):sure!
Isaac Van Doren (Feb 04 2025 at 21:50):I wrote something up! https://gist.github.com/rtfeldman/77fb430ee57b42f5f2ca973a3992532f
This is on the front page of hacker news :big_smile:
Isaac Van Doren (Feb 04 2025 at 21:54):At #3
Richard Feldman (Feb 04 2025 at 22:11):welp, I guess the cat is out of the bag now :laughing:
Agus Zubiaga (Feb 04 2025 at 22:56):Glad to know we already rewrote it
Richard Feldman (Feb 04 2025 at 22:59):comments, for posterity: https://news.ycombinator.com/item?id=42935516
Brendan Hansknecht (Feb 04 2025 at 23:07):Mostly positive vibes. A number of debates on rust compile times and rust vs zig vs c++ for safety. A surprising number of questions on parser combinators vs recursive descent. A few PNC haters.
Richard Feldman (Feb 04 2025 at 23:07):
Richard Feldman (Feb 04 2025 at 23:08):I thought it was funny that one commenter was like "I like what I'm reading about this language except I can't stand the backslash for lambda syntax" and then someone else responded "you're in luck, that syntax has changed!" and then someone else responded "I hate that it's not a backslash anymore"
Richard Feldman (Feb 04 2025 at 23:08):never change, Internet :joy:
Brendan Hansknecht (Feb 04 2025 at 23:09):Yeah, saw that too.
Brendan Hansknecht (Feb 04 2025 at 23:09):High quality opinions
Brendan Hansknecht (Feb 04 2025 at 23:10):Also, I think the first reply to the "you're in luck" comment was just "Booooo"
Andrew Kelley (Feb 04 2025 at 23:10):all the people with good opinions are busy hacking instead of reading hn
Richard Feldman (Feb 04 2025 at 23:48):btw I updated the wording around compile times - I mistakenly thought the "1-2 second compile times" was for the entire 300K LoC Zig code base all at once, but actually it was more like 4-5 seconds for rebuilding a subset; the 1-2 second build time was what Andrew mentioned we should be able to maintain for Roc's compiler, even as it gets big, and even with the current Zig compiler (and none of Zig's dev backend or incremental recompiles having landed yet; they are WIP but not ready for general use yet) - it's just that right now getting those numbers might require using build commands which manually exclude some of our code base.
Zig's goal is to make all of that automatic, so that with any luck by the time our Zig code base gets big we're still seeing 1-2 second rebuilds (or faster) without having to issue any special commands, but status quo is that as the code base gets bigger, we might need to pass slightly different CLI args when building or running tests to filter out parts of the code base we aren't working on currently :big_smile:
Richard Feldman (Feb 04 2025 at 23:58):also @Andrew Kelley graciously offered to help if we're ever seeing more than 1-2 seconds in our dev rebuild loop, and safe to say he knows all the tricks! :smiley:
Ben Cohen (Feb 05 2025 at 00:01):Hey vague follower of roc but fan of some of the ideas, the podcast and the general community / didactic qualities of the project communication. I'm stoked to see what comes from the rewrite in zig -- fast compile times for the win!!
In the sprit of learning -- there's a reference to abilities vs static dispatch in the why roc write up -- Im a person who completely doesn't understand how the various dynamic dispatch mechanisms became so prevalent and a fanboy for the phrase "static dispatch" in general -- so preferring it to something else sounds great to me!!
Buy I'm also not familiar with abilities and not fully understanding the context of what that particular tradeoff bullet point is talking about. I'd love to see this part of the why roc argument explained and expanded further ay some point (doesn't have to be here and now -- but on podcast or any other channel) - I'd like to hear whether the explanation generally aligns with my own general sensibilities about static dispatch (or whether this is really more about the weeds of compiler development).
Brendan Hansknecht (Feb 05 2025 at 00:20):Abilities are equivalent to traits in rust if that helps with understanding.
Brendan Hansknecht (Feb 05 2025 at 00:20):Static dispatch gets us the same thing but in a very different form.
Richard Feldman (Feb 05 2025 at 18:59):perfect example of why one of the use cases I do not trust LLMs for is summarizing (e.g. meetings, blog posts, etc.) - this one summarized the gist as "Richard is rewriting Roc's compiler in Roc" which is wrong on basically every level.
Screenshot 2025-02-05 at 10.57.47 AM.png
Brendan Hansknecht (Feb 05 2025 at 19:00):Literally missed the key point
Richard Feldman (Feb 05 2025 at 19:00):Yeah and also acts like I'm solo authoring the code, which is almost exactly the opposite of what's actually happening.
Richard Feldman (Feb 05 2025 at 19:00):the Rust compiler started out as just me, but so far the Zig compiler has a bunch of contributors but not me (yet!)
Richard Feldman (Feb 05 2025 at 19:02):also, if I told myself in 2013 that this would happen in 2025 I never would have guessed it :joy:
Screenshot 2025-02-05 at 11.01.34 AM.png
Anton (Feb 05 2025 at 19:08):Richard Feldman said:
perfect example of why one of the use cases I do not trust LLMs for is summarizing (e.g. meetings, blog posts, etc.) - this one summarized the gist as "Richard is rewriting Roc's compiler in Roc" which is wrong on basically every level.
I believe that does not come from a current top model, but yeah hallucination is definitely not solved
Richard Feldman (Feb 05 2025 at 19:14):more generally, I like LLMs a lot for use cases where I can verify their outputs - e.g. they generate some code instead of my writing it by hand, but I can read what they generated and see if it's doing what I want (and then make changes if it's not)
Richard Feldman (Feb 05 2025 at 19:14):with a summary, I'm trusting the LLM's output as a source of truth because I didn't read the whole thing and therefore have no way to verify the accuracy of the summary (and fix it if there are problems)
Richard Feldman (Feb 05 2025 at 19:15):leading to scenarios like the above, where anyone who reads that summary takes away the exact wrong thing but has no way to find out that this happened without reading the full thing instead of the LLM summary :big_smile:
Richard Feldman (Feb 05 2025 at 19:16):also for what it's worth I wouldn't call that mistake a hallucination
Richard Feldman (Feb 05 2025 at 19:18):the gist does talk about self-hosting a lot, and it does mention how I originally authored the (beginning of the original Rust) code base, so all the things it's claiming are present in the text and weren't made up out of thin air, it just didn't present them correctly
Brendan Hansknecht (Feb 06 2025 at 03:20):I wrote probably too long of a reply here to a simple question on lobsters, but thought others might find my answer interesting: https://lobste.rs/s/0jknbl/roc_rewrites_compiler_zig#c_zt8qdn
Sam Mohr (Feb 06 2025 at 03:22):My AI summary: Brendan thinks we'll have a perfect compiler, saying "We are going to write the compiler flawlessly in a new langauge with no issues at all."
Brendan Hansknecht (Feb 06 2025 at 03:42):Also, at this point, I think a lot of the hacker news thread has really devolved to many people complaining just to complain
Richard Feldman (Feb 06 2025 at 18:42):I'm gonna chat with Loris about our reasoning and the history of how we got here etc. on Zig Showtime next Monday! https://x.com/croloris/status/1887569862219731198
Richard Feldman (Feb 06 2025 at 18:42):it'll be live on Twitch in case anyone wants to hang out
jan kili (Feb 06 2025 at 19:08):the techyap-sphere
“double citizenship”: Rust in the sheets (the compiler), Zig in the streets (the stdlib)
:laughter_tears:
Nikolaus Wettstein (Feb 07 2025 at 15:04):It is probably very difficult, if not impossible to estimate when the rewrite and then the release 0.1.0 will be completed or is it more of a progressive/incremental switch from rust to zig?. 300k LOC contains so much of valuable knowledge from the previous years that have been incorporated into the project. Are we talking about a year or several years? I have no idea because compilers are not my area of expertise. Nevertheless thank you for your time and energy to develop roc, i'm very much looking forward to the new compiler and then the first numbered release! Very exciting!
Brendan Hansknecht (Feb 07 2025 at 16:36):I'm hoping the new version will be many less lines of code.
Brendan Hansknecht (Feb 07 2025 at 16:37):Also, it is a full switch and not incremental. Though we hope to get a slim slice of the compiler and then slowly expand
Brendan Hansknecht (Feb 07 2025 at 16:37):Timelines are exceptionally hard in open source.
Brendan Hansknecht (Feb 07 2025 at 16:38):Anyone can disappear (or show up) for any number of reasons.
Nikolaus Wettstein (Feb 08 2025 at 08:19):Thank you for your answer!
jan kili (Feb 10 2025 at 18:00):Richard Feldman said:
I'm gonna chat with Loris about our reasoning and the history of how we got here etc. on Zig Showtime next Monday! https://x.com/croloris/status/1887569862219731198
Richard's going live!
Loris Cro (Feb 10 2025 at 20:07):The recording is up on YouTube
Last updated: Jul 06 2025 at 12:14 UTC