Brendan Hansknecht (Jan 24 2025 at 03:03): Brendan Hansknecht (Jan 24 2025 at 03:03):Yeah, thinks like morphic will remain slow for non allocation related reasons (though morphic also doesn't do arena allocation which maybe could help its perf a bit).
Sam Mohr (Jan 24 2025 at 03:04):Is morphic just alias analysis?
Brendan Hansknecht (Jan 24 2025 at 03:06):I think morphic does analysis on uniqueness to enable inplace mutation without checking the refcount. It might also enable removing some refcount increments/decrements. That said, I don't think it currently works. I had to disable a chunk of its analysis due to causing bugs and it basically had no impact on perf for most benchmarks.
Anthony Bullard (Jan 24 2025 at 03:06):Cool, that would allow us to remove a dependency :smile:
Sam Mohr (Jan 24 2025 at 03:07):Once Roc works consistently, we can re-enable it
Sam Mohr (Jan 24 2025 at 03:07):But we are definitely not there right now
Luke Boswell (Jan 24 2025 at 03:08):This is probably the biggest roc app I know of https://github.com/lukewilliamboswell/roc-htmx-tailwindcss-demo
Luke Boswell (Jan 24 2025 at 03:10):I had one more advanced that this I was messing around with for a work project/prototype... I should probably back-port features from that into this demo.
Brendan Hansknecht (Jan 24 2025 at 03:10):I think it would be great to double check with someone who understands what it is supposed to do better before just removing it.
To be fair, it may be the case they we aren't leaning on it enough. Potentially more zig biultins could be using it's inplace vs not indicator at comptime. On top of that, it may not show huge gains until hit in a hot loop where removing the refcounting completely could be huge....but I don't know enough about it to be sure.
From what I know of it currently, it is bugged, it really slows down compile time (all the worst compile times I have ever seen have been due to morphic), and it doesn't generally affect perf much.
So that does push towards removing it assuming we can't easily fix it and make it algorithmically sound.
EDIT: all that said, not trying to be mean to the morphic folks. It might be a great piece of tech we are holding wrong or that has some minor bugs that happen to lead to scary roc issues.
Anthony Bullard (Jan 24 2025 at 03:10):If you did a wc -l **/*.roc, what do you get?
Luke Boswell (Jan 24 2025 at 03:11):$ wc -l **/*.roc
49 src/Controllers/Product.roc
50 src/Controllers/User.roc
130 src/Helpers.roc
14 src/Models/Product.roc
16 src/Models/Session.roc
12 src/Models/User.roc
51 src/Sql/Product.roc
67 src/Sql/Session.roc
53 src/Sql/User.roc
49 src/Views/Layout.roc
3343 src/Views/Pages.roc
234 src/main.roc
4068 total
Yeah, that's big-ish. But I have Go lang apps that are 10k+ (and still compile in the low 100s of ms)
Luke Boswell (Jan 24 2025 at 03:12):My other one is 5908 total
Sam Mohr (Jan 24 2025 at 03:12):I don't know who understands Morphic...
Anthony Bullard (Jan 24 2025 at 03:12):I'm assuming you haven't migrated this app
Anthony Bullard (Jan 24 2025 at 03:13):I'd love to see a time roc build on it after you do
Brendan Hansknecht (Jan 24 2025 at 03:13):@William Brandon I think is one of the morphic authors
Brendan Hansknecht (Jan 24 2025 at 03:13):And I think @Folkert de Vries had solid understanding of morphic at a high level at least.
Brendan Hansknecht (Jan 24 2025 at 03:15):They also have a published paper and talk, but I think it is mostly on lambasets and not on the smarter inplace mutation without checking refcounts.
Sam Mohr (Jan 24 2025 at 03:15):With the new pipeline design, lambda sets won't exist at that part of the compiler
Sam Mohr (Jan 24 2025 at 03:16):Just concretely-typed first-order functions and values
Brendan Hansknecht (Jan 24 2025 at 03:16):Given checking a refcount is 1 is generally pretty cheap, the inplace smarts may not really ever give us gains (except in really hot mutate in place loops maybe). As opposed to lambasets which hugely help llvm understand and optimize our code.
Anthony Bullard (Jan 24 2025 at 03:16):Sam Mohr said:
Just concretely-typed first-order functions and values
So happy for this.
Sam Mohr (Jan 24 2025 at 03:16):It's more important than any other work we're doing in the next 6 months IMO
Notification Bot (Jan 24 2025 at 04:43):24 messages were moved here from #off topic > Bootstrapping Roc? by Richard Feldman.
Richard Feldman (Jan 24 2025 at 04:44):I like the idea of disabling (or even removing) Morphic before 0.1.0
Brendan Hansknecht (Jan 24 2025 at 04:45):I think it is already set to trivial analysis due to bugs. So it does the same amount of work for both debug and release builds (which is pretty minimal)
Brendan Hansknecht (Jan 24 2025 at 04:45):So it is kinda already disabled
Richard Feldman (Jan 24 2025 at 04:45):right, but even in that mode it's been a source of bugs, right?
Brendan Hansknecht (Jan 24 2025 at 04:45):Not to my knowledge, but I'm not sure
Richard Feldman (Jan 24 2025 at 04:46):hm, I could be misremembering then
Brendan Hansknecht (Jan 24 2025 at 04:46):This is all the context I have:
// TODO(#7367): Change this back to `morphic_lib::solve`.
// For now, using solve_trivial to avoid bug with loops.
// Note: when disabling this, there was not much of a change in performance.
// Notably, NQueens was about 5% slower. False interpreter was 0-5% faster (depending on input).
// cFold and derive saw minor gains ~1.5%. rBTreeCk saw a big gain of ~4%.
// This feels wrong, morphic should not really be able to slow down code.
// Likely, noise or the bug and wrong inplace mutation lead to these perf changes.
// When re-enabling this, we should analysis the perf and inplace mutations of a few apps.
// It might be the case that our current benchmarks just aren't affected by morphic much.
gotcha
Brendan Hansknecht (Jan 24 2025 at 04:47):I don't know of any bugs since we changed it to solve_trivial
Richard Feldman (Jan 24 2025 at 04:47):I forget, does it operate on traits?
Richard Feldman (Jan 24 2025 at 04:47):or do we actually build an IR for it
Brendan Hansknecht (Jan 24 2025 at 04:47):build an ir for it
Richard Feldman (Jan 24 2025 at 04:48):and that happens after specialization, so that's definitely a compiler perf downside :sweat_smile:
Richard Feldman (Jan 24 2025 at 04:48):but yeah maybe it's not urgent to actually skip that work
Richard Feldman (Jan 24 2025 at 04:48):if there aren't bugs in the trivial solver
Brendan Hansknecht (Jan 24 2025 at 04:50):Given we haven't noticed any measurable perf gains from the alias analysis, I am for just ripping it out. Might be worth trying to figure out a worst case example first just to see if it every makes a meaningful difference.
Brendan Hansknecht (Jan 24 2025 at 04:52):Theoretically, that would be some sort of hot loop with inplace updates. One version where morphic leads to inplace mutation without checking the refcount and one that just checks the refcount. Maybe I can just recursively loop a list calling List.set?
Even without morphic, I can probably force the inplace flag just to make sure it is getting set ignoring any potential morphic bugs.
Richard Feldman (Jan 24 2025 at 04:53):the thing is, I just don't think we're in a position to make it reliable anytime soon
Brendan Hansknecht (Jan 24 2025 at 04:53):Yeah, but if it could lead to large gains, may be worth just leaving it wired up in trivial mode. If not, probably worth ripping out.
Richard Feldman (Jan 24 2025 at 04:54):I'm just not sure it's worth the dependency, the extra IR generation, the concerns about bugs, the build time, etc.
Richard Feldman (Jan 24 2025 at 04:54):to be able to experiment with it cheaply
Richard Feldman (Jan 24 2025 at 04:55):because in my mind, the real question is "can we someday have a version of it that runs reliably and quickly?"
Richard Feldman (Jan 24 2025 at 04:55):if we can, then sure we might as well enable that because we already know we're set up for it
Richard Feldman (Jan 24 2025 at 04:56):but that version of it has to exist first, and right now that is a completely hypothetical future thing :sweat_smile:
Richard Feldman (Jan 24 2025 at 04:56):because the original authors haven't touched it in years, and none of us know how it works well enough to fix or rewrite it
Richard Feldman (Jan 24 2025 at 04:57):so it's not like even if we were able to experimentally confirm that it was really great in some scenario, that we'd want to enable it in the compiler in general - because we know it'd just start causing bugs again
Richard Feldman (Jan 24 2025 at 04:57):in other words, knowing that it has big benefits isn't actionable for us until we have a version that we can actually use long-term
Richard Feldman (Jan 24 2025 at 04:58):so being able to experiment on it is almost closer to a curiosity - like "hey, cool, someday a hypothetical future version of Morphic that we could use for real will get some benefits in this one scenario!"
Richard Feldman (Jan 24 2025 at 05:00):I guess if we started discovering some really big benefits it might be motivating to try to learn how it works in order to build out a real version, but that also seems like a very hypothetical scenario right now haha
Brendan Hansknecht (Jan 24 2025 at 05:01):Lambdasets are also from morphic, but we definitely want to keep those despite not 100% being sure we can make the compile in a reasonable about of time. Only recently with some of Ayaz's work did we feel we could compile them without bugs after a rewrite. I don't see how the rest of morphic is any different (assuming it can show similar perf gains)
Richard Feldman (Jan 24 2025 at 05:02):oh for sure
Richard Feldman (Jan 24 2025 at 05:02):I mean the Morphic solver specifically
Richard Feldman (Jan 24 2025 at 05:03):the difference is that we actually understand how to do lambda sets
Richard Feldman (Jan 24 2025 at 05:03):and we don't understand the rest of the solver
Richard Feldman (Jan 24 2025 at 05:04):so we have problems but no way to fix them, and also no way in sight for getting to a point where we can fix them
Richard Feldman (Jan 24 2025 at 05:05):(and also a payoff of unclear magnitude for what would definitely be a big project no matter what)
Richard Feldman (Jan 24 2025 at 05:05):(big project being getting it to run fast and without bugs)
Ayaz Hafiz (Jan 24 2025 at 05:06):morphic and lambda sets are two different things
Ayaz Hafiz (Jan 24 2025 at 05:06):morphic is about specializing borrows
Richard Feldman (Jan 24 2025 at 05:06):I'm not saying we should never revisit it or anything—there's a paper and a reference implementation, so it's certainly possible that we could figure out how it works someday—just that I think if we didn't have it in the compiler right now, I don't think we'd add it
Ayaz Hafiz (Jan 24 2025 at 05:07):lambda sets are about specializing indirect jumps
Richard Feldman (Jan 24 2025 at 05:07):that's a cool way of putting it! :smiley:
Brendan Hansknecht (Jan 24 2025 at 05:08):I thought both came from the morphic research?
Ayaz Hafiz (Jan 24 2025 at 05:08):yes but the technologies are distinct
Ayaz Hafiz (Jan 24 2025 at 05:08):they implemented both in the language
Brendan Hansknecht (Jan 24 2025 at 05:08):yes
Ayaz Hafiz (Jan 24 2025 at 05:09):the morphic pass in roc does nothing wrt lambda sets
Ayaz Hafiz (Jan 24 2025 at 05:09):it could but it doesn't
Richard Feldman (Jan 24 2025 at 05:10):Morphic does need all functions to be first-order, and lambda sets are a way to get that
Richard Feldman (Jan 24 2025 at 05:10):which I believe was what led the authors of Morphic to come up with the idea of lambda sets
Richard Feldman (Jan 24 2025 at 05:10):(which turned out to be valuable on their own to us)
Richard Feldman (Jan 24 2025 at 05:10):because of LLVM etc.
Ayaz Hafiz (Jan 24 2025 at 05:11):well, it doesn't need first order functions. it just guarantees full borrow specialization if you have first order functions
Ayaz Hafiz (Jan 24 2025 at 05:11):you can run morphic over programs with indirect jumps, that's not a problem
Ayaz Hafiz (Jan 24 2025 at 05:12):i would think of them as fully orthogonal
Richard Feldman (Jan 24 2025 at 05:13):cool
Richard Feldman (Jan 24 2025 at 05:13):Ayaz, what are your thoughts about keeping it in the code base in its current state?
Ayaz Hafiz (Jan 24 2025 at 05:14):i would remove it if it's possible
Ayaz Hafiz (Jan 24 2025 at 05:14):idk if stuff breaks
Ayaz Hafiz (Jan 24 2025 at 05:14):but you know my stance on simplification haha
Brendan Hansknecht (Jan 24 2025 at 05:15):So this is a benchmark trying to trigger morphic. So this is probably about the best possible gain from morphic in general:
Benchmark 1: ./check-refcount
Time (mean ± σ): 68.6 ms ± 0.3 ms [User: 65.6 ms, System: 2.2 ms]
Range (min … max): 67.9 ms … 69.6 ms 100 runs
Benchmark 2: ./no-check-refcount
Time (mean ± σ): 21.9 ms ± 0.6 ms [User: 19.0 ms, System: 2.2 ms]
Range (min … max): 20.8 ms … 23.2 ms 100 runs
Summary
./no-check-refcount ran
3.13 ± 0.08 times faster than ./check-refcount
So morphic can be a big win.
Brendan Hansknecht (Jan 24 2025 at 05:15):Not sure what numbers could realistically come up in practice though.
Brendan Hansknecht (Jan 24 2025 at 05:16):The above code is from a hot loop that only calls List.set. One version uses morphic to be statically inplace. The other is doing refcount checks for uniqueness.
Brendan Hansknecht (Jan 24 2025 at 05:17):Not saying this makes morphic worth keeping, might be too slow or buggy in practice (or simply have no one to actually own and fix it up eventually).
Brendan Hansknecht (Jan 24 2025 at 05:19):I always just worry that ripping something like this out will make it exceptionally unlike to ever be re-added. Leaving it (even in the trivial only state), may lead to us eventually looping back to it when the language has otherwise settled, fix it up, and see nice perf gains.
Brendan Hansknecht (Jan 24 2025 at 05:21):But I'm a sucker for things that go fast... :shrug:
Ayaz Hafiz (Jan 24 2025 at 05:21):i think a helpful thing to think about would be what performance ceiling are you all comfortable for a 0.1 release
Ayaz Hafiz (Jan 24 2025 at 05:22):if it is a hard requirement that it is the fastest thing out there sans ref counting that is one thing
Ayaz Hafiz (Jan 24 2025 at 05:22):if it needs to be comparable to existing functional languages/python/other web server players/etc maybe a different story
Richard Feldman (Jan 24 2025 at 05:23):yeah I'm definitely comfortable with our current performance for 0.1
Ayaz Hafiz (Jan 24 2025 at 05:23):and how important is performance relative to stability
Brendan Hansknecht (Jan 24 2025 at 05:24):Part of the wierd reality that we live in is that ocaml exists and is reasonably performant yet essentially no one uses it. So perf likely isn't the big selling point to most people.
Richard Feldman (Jan 24 2025 at 05:24):I'm very not comfortable with our current stability other than the parser (thanks to its fuzzing giving a lot of confidence, and setting aside the fact that we know it's temporarily in a more volatile state at this exact moment than it had been for a long time prior)
Ayaz Hafiz (Jan 24 2025 at 05:26):Brendan Hansknecht said:
Part of the wierd reality that we live in is that ocaml exists and is reasonably performant yet essentially no one uses it. So perf likely isn't the big selling point to most people.
well, 10% of the equities market relies on ocaml ;)
Brendan Hansknecht (Jan 24 2025 at 05:27):Another simple fact is that essentially no one who will use roc v0.1.0 will be using it for perf. They will be using it for the rest of roc with perf maybe being a nice afterthought.
So yeah, always compiling without crashes/segfaults, not having cffi bugs, having good errors almost all the time, having robust features will be much much more important for v0.1.0 and likely for many releases after that.
Brendan Hansknecht (Jan 24 2025 at 05:28):I personally care a ton about perf, but I am the exception who would be using rust or c++ instead of roc. Most folks will come from the other side (js, python, maybe go or ocaml) instead of roc.
Sam Mohr (Jan 24 2025 at 05:28):I don't use Ocaml because it feels like a kitchen sink language. Most of the code I read for it is a different flavor than every other example I see. It's nice, but it's somewhat C++-y in that way. Not much experience, tho
Richard Feldman (Jan 24 2025 at 05:28):I also personally care a ton about Roc's performance :grinning_face_with_smiling_eyes:
Brendan Hansknecht (Jan 24 2025 at 05:29):Aside how is ocamls perf in practice? Roughly in the same grouping as go?
Ayaz Hafiz (Jan 24 2025 at 05:29):yes ocaml is similar to go ime
Ayaz Hafiz (Jan 24 2025 at 05:29):compiler perf and runtime
Ayaz Hafiz (Jan 24 2025 at 05:31):i will also say ime performance doesn't really matter until you are at some scale. even then, most often performance is due to IO (can be solved separately), load (can be solved separately, horizontal or vertical scaling), or algorithmically (fix it, or if you need compute, easy hack is shell out). I love that Roc keeps in mind as a priority, but at the same time I have used many python webservers and CLI apps that were fine - and I'm sure Roc will be a better experience than that from day one
Brendan Hansknecht (Jan 24 2025 at 05:34):Yeah, perf is normally more about how much you have to scale (so cost, but servers are cheap) rather than language implementation. That and saving energy/battery life (kinda the reverse of scale: phone, embedded).
Ayaz Hafiz (Jan 24 2025 at 05:34):embedded is a good point
Ayaz Hafiz (Jan 24 2025 at 05:35):i don't know the distribution of realtime constraints in embedded, but if you need hard realtime roc probably isn't the language
Brendan Hansknecht (Jan 24 2025 at 05:36):Otherwise, it is mostly design rather than language (good io scheme, good algorithm).
Unless you need/want to be scrappy for some reason (like rewriting games from python to c++ so that they can simulate faster for alphazero implementation that can be trained on a single laptop). Most of the time perf is kinda optional, but nice to have a reasonable baseline for.
Brendan Hansknecht (Jan 24 2025 at 05:37):but if you need hard realtime roc probably isn't the language
Yeah, though I guess you could theoretically make guarantees about timing due to refcounting instead of gc.
Brendan Hansknecht (Jan 24 2025 at 05:37):That said, a lot of embedded is not true hard realtime. I mean a lot of embedded is running linux which even in real time mode really stretches the definition of real time.
Ayaz Hafiz (Jan 24 2025 at 05:38):fair
Brendan Hansknecht (Jan 24 2025 at 05:40):As a note, morphic is still being maintained. So there is a chance we could get someone interested from their team to help us.
Otherwise, yeah, probably should just rip out morphic. We don't have any idea why it is broke (could litterally just be a silly mistake in the ir we generate) and even when it does work, the core algorithm can get way too slow to be worth running during a roc build.
Ayaz Hafiz (Jan 24 2025 at 05:54):there are definitely bugs in the ir generation
Ayaz Hafiz (Jan 24 2025 at 05:54):there is at least one i remember
Brendan Hansknecht (Jan 25 2025 at 01:38):I am lightly looking into what it would take to remove this. I think it is only tangled into the llvm backend
Brendan Hansknecht (Jan 25 2025 at 01:39):That said, due to generating lots of extra specializations, it is tangled in reasonably deep. So it won't be simple to remove. That said, I would expect most of it to just be pretty manual, not necessarily hard.
Brendan Hansknecht (Jan 25 2025 at 01:40):I do think a solid amount of mapping exist that would be best to cleanup, but that will be harder to untangle than just removing morphic.
Last updated: Jul 06 2025 at 12:14 UTC