Richard Feldman (Oct 21 2026 at 05:18): Richard Feldman (Oct 21 2026 at 05:18):I'm working on a blog post about some recent roc-deflate performance numbers...I'm hoping to publish it in the next few days, so any feedback would be welcome!
here's the first draft: https://gist.github.com/rtfeldman/040f18de33a17848fdd10891f7419aca (I haven't yet written the last section, about roc-specific performance optimizations, but the main point of the post is to share the result and explain why I'm excited about it - and that's the part I'm looking for feedback on!)
Luke Boswell (Oct 21 2026 at 05:32):Looking good.
Karl (Oct 21 2026 at 05:58):I didn't see any problems with the article. I do think that any discussion of roc perf should mention the very common quadratic from having a non-unique reference. It's pretty much always the first perf problem for me to overcome when doing an optimization effort.
Luke Boswell (Oct 21 2026 at 06:05):It might help to be clear about optimisations that live upstream in the roc compiler itself, and then those refactoring the pure roc code in the package because to achieve these results you have needed both.
For package authors currently there are some roc specific tools available to assist like roc-fuzz :wink: or test platforms which track allocations etc.
Aurélien Geron (Oct 21 2026 at 06:32):Nice post! Just one thing: it isn't clear why "No side effects" is a benefit, in and of itself (beyond security, but that's the next point in the list). Perhaps add something like: "Pure functions unlock compiler optimizations, much easier testing, local reasoning, safe refactoring, deterministic debugging, and fearless concurrency. Most importantly, it unlocks security guarantess."
Richard Feldman (Oct 21 2026 at 12:11):Karl said:
I do think that any discussion of roc perf should mention the very common quadratic from having a non-unique reference. It's pretty much always the first perf problem for me to overcome when doing an optimization effort.
oh yeah, that's the first heading ("Avoiding Unnecessary Sharing") under the "Roc-Specific Optimizations" section at the end :smile:
Richard Feldman (Oct 21 2026 at 12:12):Luke Boswell said:
It might help to be clear about optimisations that live upstream in the roc compiler itself, and then those refactoring the pure roc code in the package because to achieve these results you have needed both.
yeah I mentioned in passing that there were compiler optimizations, but I wasn't sure if it made sense to focus on those here because...well, they now exist in the compiler, so everyone can take advantage of them! :smile:
Richard Feldman (Oct 21 2026 at 12:13):so it's not like a similar optimization project would necessarily need that part of the process, now that the compiler improvements have already landed etc.
Richard Feldman (Oct 21 2026 at 12:14):in other news, I ran Opus 5.5 overnight with a prompt that basically said "hey look at the commit history for optimization techniques and see if you can close the gap on this x64 machine compared to what we got on the ARM Mac" and when I got up this morning it had closed the gap from 19.7% to 11.9%, with two more follow-ups that might get it under 10% :smiley:
Zak Kohler (Oct 21 2026 at 13:37):This is kinda a tangent, but I see an oppourtunity to use agent review for continuously auditing against being backed into a performance corner. What I mean is that every feature (including the current set) constrain current, but also _future_ performance improvements or other features. This means that performace regression alone testing can't catch the case of a feature becoming a liability/block because it closes other doors implicitly, sometimes in sneaky/unobvious way.
Perhaps like this:
TeaDrinkingProgrammer (Oct 22 2026 at 06:10):Really great writing as always Richard! One thing I noticed: the type system and what it's advantages are (no exceptions, understanding a function from just it's signature, ...) is a bit hidden. It is not something unique to Roc, but it is unique among languages that have friendlyness as a goal, as far as I know. Maybe something like "A place where Roc is unique among other languages, is that it combines all of this with a friendly syntax (think Python) and a full (?) typesystem"
pyfisch (Oct 22 2026 at 06:20):https://kim-em.github.io/blog/2026-7-24-why-lean-is-faster-than-rust/
This post where someone used AI to optimize a deflate implementation in Lean for performance and compression comes to mind.
Richard Feldman (Oct 22 2026 at 13:18):lean-zip is certainly not the best here: libdeflate unsurprisingly blows it out of the water (unsurprisingly because this is a very carefully tuned implementation using architecture-specific SIMD, that we can't touch in Lean).
:grinning_face_with_smiling_eyes::grinning_face_with_smiling_eyes::grinning_face_with_smiling_eyes:
Eric Rogstad (Oct 22 2026 at 15:34):Since Roc has always been a functional language that aspired to both build fast and run fast
Grammatically, I think "aspired" should be "aspires" (or "has aspired").
notramo (Oct 22 2026 at 16:49):A quick LLM review before publishing is useful. Some typos it found:
Richard Feldman (Oct 22 2026 at 18:11):thanks! I published a revised DRAFT (please don't share it around yet - I want to keep revising it today and share it tomorrow morning!) here so you can see the actual graph and everything else:
any final feedback welcome! :smiley:
Anton (Oct 22 2026 at 18:27):but then again, the last time a skipped bounds check in C compression code led to a major Remote Code Execution vulnerability was yesterday.
:smile:
Anton (Oct 22 2026 at 18:28):I would put "roc-deflate ran these 144 benchmarks slower on average than libdeflate, in the median case 8.9% slower on the Ryzen 3900X, and 2.6% on the M1 MacBook Pro." above the chart. Once the reader is excited they are motivated to check out the details in the chart.
Anton (Oct 22 2026 at 18:42):The chart feels a bit dense. "gap with ..." seems like an unconventional thing in charts, perhaps it's common in deep perf optimization of software? One suggestion is to just show median perf per compression level in the chart and have tabs for the other proper perf expert charts.
MagicMan (Oct 22 2026 at 18:44):Love the article. One suggestion of mine for the chart is to add a little blurb somewhere that says "higher is better" to help reader interpret the plot.
Anton (Oct 22 2026 at 18:46):In the top right there already is the "↑ faster" but I can see how it can be missed
MagicMan (Oct 22 2026 at 18:46):I totally did miss that! :sweat_smile:
Richard Feldman (Oct 22 2026 at 18:46):Anton said:
The chart feels a bit dense. "gap with ..." seems like an unconventional thing in charts, perhaps it's common in deep perf optimization of software? One suggestion is to just show median perf per compression level in the chart and have tabs for the other proper perf expert charts.
it's not common, I just wanted to show both the fastest and slowest cases, both so you could see the range and also so that you if you look at either the highest ones or the lowest ones, you can see they're neck-and-neck across the board
Richard Feldman (Oct 22 2026 at 18:47):technically "gap with" accurately describes the darker part, which is how legends are supposed to work, although I agree it's not immediately obvious what it means...that said, if you hover over any bar, you can see title text that explains concisely what the bar means
Richard Feldman (Oct 22 2026 at 18:48):my inclination is to keep it as-is because I think it's the most useful - the overall shape of the bars gets the point across even if you aren't trying to dig into the details, and if you do dig into the details, there's a lot of useful information to be found in this design :smile:
Zak Kohler (Oct 22 2026 at 18:49):maybe add another filter "fastest/slowest" or fastest/slowest/gap?
Karl (Oct 22 2026 at 18:49):its nascent package ecosystem has had almost no optimization work done on it
I put effort into optimizing my stuff.
Richard Feldman (Oct 22 2026 at 18:50):very true! But I think we're outliers across the ecosystem as a whole :laughing:
Richard Feldman (Oct 22 2026 at 18:50):at least for the time being
Richard Feldman (Oct 22 2026 at 18:50):basically I wanted to set really clear expectations that if people grab a random roc package off the shelf, they shouldn't expect this level of optimization to have gone into it :sweat_smile:
MagicMan (Oct 22 2026 at 18:59):One final note, "also" seems overused in the bullet point about Portability.
Luke Boswell (Oct 22 2026 at 19:12):Anton said:
The chart feels a bit dense. "gap with ..." seems like an unconventional thing in charts, perhaps it's common in deep perf optimization of software? One suggestion is to just show median perf per compression level in the chart and have tabs for the other proper perf expert charts.
I second this -- I've had to stare at the graph for a while and I'm not sure I really comprehend all the details. It feels too dense at first glance for a an article. I like Anton's ideas here of retaining the detail but behind Tabs instead.
I would prefer the side by side be the two different architectures.
Richard Feldman (Oct 22 2026 at 19:20):oh that's interesting, I didn't think of side by side cpu architectures :thinking:
Luke Boswell (Oct 22 2026 at 19:20):I think the problem I have with showing the slowest and then a gap number, is there is no median or average, so I can't see that roc-deflate is almost as fast.
Also without a control or baseline for the numbers it's harder for me to follow because I'm not familiar with deflate library performance.
I'd love to know what kind of speed lean-zip and miniz_oxide get on this benchmark corpus for example if that's possible to capture?
Richard Feldman (Oct 22 2026 at 19:22):oh that's a cool idea, I can try to repro their numbers too
Luke Boswell (Oct 22 2026 at 19:24):Another idea, which might be less visually "cool" is to show a box and whisker plot if you do want all the juicy statisitcal details like slowest, fastest, median, and quartiles.
Luke Boswell (Oct 22 2026 at 19:26):Another idea, you could use lines instead of bars, then you could have an upper (fastest) and lower (slowest) and then a middle (median) line. it's probably getting too cluttered
Richard Feldman (Oct 22 2026 at 19:31):yeah I basically wanted a simpler box and whiskers plot
Richard Feldman (Oct 22 2026 at 19:32):maybe I could do it as a box without the whiskers :thinking:
Richard Feldman (Oct 22 2026 at 19:32):instead of a light vs dark stack
Richard Feldman (Oct 22 2026 at 19:32):so it's kinda floating
Richard Feldman (Oct 22 2026 at 19:33):part of the reason I wanted to show fastest and slowest instead of just median is so you can see the median isn't concealing any outliers
Anton (Oct 22 2026 at 19:42):I wanted to suggest box and whiskers as well, but I didn't because I was doubting the average reader would be familiar with that format.
Luke Boswell (Oct 22 2026 at 19:43):There might be too many dimensions and data points to include all in one graph and keep it visually simple and easy to understand, maybe pick a single architecture and compression level and have a switch and sliders for those?
Luke Boswell (Oct 22 2026 at 19:44):Or you could have a couple of simpler graphs using just median for example, and then include an Annex at the end of the article with more detailed graphs
Luke Boswell (Oct 22 2026 at 19:44):I read the article draft on my phone, and it was quite dense.
Romain Lepert (Oct 22 2026 at 19:50): Anton (Oct 22 2026 at 19:51):"Under 10% Slower" -> "Within 10% of C" has a more positive spin
:plus:
Zak Kohler (Oct 22 2026 at 19:53):
i show the max speed, and use a single whisker to show run to run variance. you have to assume variance is to do contention, so min/max is more reproducible than mean/median.
Anton (Oct 22 2026 at 19:56):Yeah, max seems reasonable indeed
Richard Feldman (Oct 22 2026 at 19:58):Anton said:
"Under 10% Slower" -> "Within 10% of C" has a more positive spin
:plus:
I had this at one point, but I don't think that's accurate in this case - because all the results are within 20% of C, but some of them are 18% slower and others are over 18% faster :sweat_smile:
Richard Feldman (Oct 22 2026 at 19:59):the median is under 10% slower, but "within" to me suggests you're always going to get that on all runs
Richard Feldman (Oct 22 2026 at 19:59):
here's a graph design that shows the spreads and also M1 vs. Ryzen - thoughts?
Niclas Ahden (Oct 22 2026 at 20:05):I find it harder to read than the first one. My 2c on simple changes that make the first graph clearer:
Anton (Oct 22 2026 at 20:08):As Luke said earlier, it seems like you could go with one architecture for the chart. Multiple architectures is valuable from a scientific perspective but it does not seem like it is worth the added cognitive load in the chart.
Richard Feldman (Oct 22 2026 at 20:09):how about this?
Anton (Oct 22 2026 at 20:11):Definitely best one so far for me
Niclas Ahden (Oct 22 2026 at 20:14):I like this one too! The only improvement I can suggest is adding a small callout of "fastest run"/"slowest run" at the top/bottom of the first bar, but that's going quite far
Aurélien Geron (Oct 22 2026 at 20:15):Anton said:
I wanted to suggest box and whiskers as well, but I didn't because I was doubting the average reader would be familiar with that format.
I vote for a box plot in this case. My kids learned that at school at age ~10, and the audience is quite technical, plus it's a pretty intuitive format, so I very much doubt that a significant percentage of readers would be confused. If that's a concern, a quick link to the box plot Wikipedia page would suffice imho.
Richard Feldman (Oct 22 2026 at 20:18):my problem with a box plot here is that I think it adds visual noise for no benefit
Richard Feldman (Oct 22 2026 at 20:18):the range is interesting, but the quartiles aren't
Richard Feldman (Oct 22 2026 at 20:18):the per-compression-level median isn't very interesting either
Richard Feldman (Oct 22 2026 at 20:19):the value of the ranges is that they answer the natural question "ok it's under 10% on average but how wildly does it vary?" with "oh I see, not very much"
Aurélien Geron (Oct 22 2026 at 20:19):I guess it depends on the data: if the data is uniformly distributed in the range, then sure, but if it's not, the box plot adds useful info, especially if one outlier really extends the range.
Richard Feldman (Oct 22 2026 at 20:23):another revision:
Niclas Ahden (Oct 22 2026 at 20:23):Don't know how it could be clearer than that :thumbs_up:
Zak Kohler (Oct 22 2026 at 20:32):if you rerun the bench a few times, is the ranking of the "slowest runs" stable for each compression level? e.g. cause by workload differences? or is it all just run to run variation? if they are not stable i would consider "slowest run" just noise.
Richard Feldman (Oct 22 2026 at 20:34):yeah maybe I should explain that...in the benchmark, each level runs 12 different inputs. within each of those 12 inputs we measure multiple times and take the median measurement.
Richard Feldman (Oct 22 2026 at 20:34):so it's showing the range between which of the 12 inputs ran fastest at that compression level vs which of the 12 inputs ran slowest at that compression level
Richard Feldman (Oct 22 2026 at 20:35):but each one had multiple measurements and only the average of those measurements ends up in the graph, to avoid noise
Richard Feldman (Oct 22 2026 at 20:35):but I'm not sure how to explain that concisely and non-awkwardly :laughing:
Zak Kohler (Oct 22 2026 at 20:35):That makes sense, but take max instead of avg of each group.
Zak Kohler (Oct 22 2026 at 20:37):one stall in a run can bring down avg unfairly and make overall trends look noiser.
Richard Feldman (Oct 22 2026 at 20:39):sorry, average as in median
Richard Feldman (Oct 22 2026 at 20:39):you think we should take the best from each?
Richard Feldman (Oct 22 2026 at 20:42):I feel like median avoids the "abnormally slow run" problem while also giving a more accurate picture of the MB/s you could actually expect on that hardware
notramo (Oct 22 2026 at 20:42):I noticed the HTML title attribute was used for the chart items. It's a very bad web development practice.
https://www.24a11y.com/2017/the-trials-and-tribulations-of-the-title-attribute/
Matthieu Pizenberg (Oct 22 2026 at 20:42):Aurélien Geron said:
I guess it depends on the data: if the data is uniformly distributed in the range, then sure, but if it's not, the box plot adds useful info, especially if one outlier really extends the range.
Yep, agreed with that. Regular box plots are fine. Violin plots even for the fun XD. Or a scatter 2D plot with libdeflate on one axis, and roc on the other, with everything in 1 plot instead of separating each compression level. This way you see roughly a diagonal and the density of the plot shows you both visual standard deviation, and outliers.
Zak Kohler (Oct 22 2026 at 20:44):Richard Feldman said:
I feel like median avoids the "abnormally slow run" problem while also giving a more accurate picture of the MB/s you could actually expect on that hardware
For a lot of benchmarks, you can't have an "accidently too fast" run. It is case by case, but for comparisons sake, I like to think of trying to search for the ceiling, because that is reproducible, espcially on noise personal laptop/desktop.
Richard Feldman (Oct 22 2026 at 20:46):ok fair enough but I don't have that granularity saved from the Ryzen machine, and I don't want to go redo all of that just to switch from median to max :laughing:
Zak Kohler (Oct 22 2026 at 20:52):just an idea, lines?
Zak Kohler (Oct 22 2026 at 20:52):favors implementation comparison over best-worse gap.
Richard Feldman (Oct 22 2026 at 20:54):true, although with lines like this I always look at it and immediately think "main point is that it's trending down" :sweat_smile:
Richard Feldman (Oct 22 2026 at 20:55):which is not at all the point in this case
Matthieu Pizenberg (Oct 22 2026 at 20:55):same with the bars going down to be fair
Matthieu Pizenberg (Oct 22 2026 at 20:57):To get rid of the going down, you could do relative speed. But you lose the absolute speed info. If that was ever important, considering that changes hardware to hardware with mem/cpu/arch/etc.
notramo (Oct 22 2026 at 21:04):Or just reverse the compression level, so L12 is on the left and L1 on the right.
Richard Feldman (Oct 22 2026 at 21:10):
new hovers
notramo (Oct 22 2026 at 21:36):This bar format is somewhat confusing, because I compare the bar length, not the positions.
notramo (Oct 22 2026 at 21:36):If it were a line, it would be obvious to compare to the axis.
notramo (Oct 22 2026 at 21:36):Or if there were a placeholder with a muted color.
notramo (Oct 22 2026 at 21:37):Also, it's not obvious that the "fastest/slowest run" arrows point to the ends of the bar, as it seems like they point to the bar itself.
Richard Feldman (Oct 22 2026 at 22:16):I dunno, I think I'm at the point of diminishing returns on revising this graph :sweat_smile:
Richard Feldman (Oct 22 2026 at 22:16):all of the options we've discussed have problems, and this seems like a significant improvement over the original
Romain Lepert (Oct 22 2026 at 22:21):isn't it missing the median (horizontal line in the bar) ? The median is the main claim of the post but it does not show up in the plot
Matthieu Pizenberg (Oct 22 2026 at 22:23):Can we know why there is so much variance on the L1 plot? Like why do we see numbers going from 150MB/s to 600MB/s? This seems like the biggest question in my mind when I see that graph. More than comparing gray and purple.
Matthieu Pizenberg (Oct 22 2026 at 22:24):And I don’t know if one data point was 150 in one impl and 600 in the other, or if they are all close to each other. Which are 2 very different situations to be in. And also why I was advocating for the scatter plot
Matthieu Pizenberg (Oct 22 2026 at 22:28):I’m exagerating a bit maybe because with both the minimum and maximum at each level being close for both implementations, the likelihood of that distribution being all over the place is very low.
Matthieu Pizenberg (Oct 22 2026 at 22:30):Can we know why there is so much variance on the L1 plot?
Is it just that some files are much more deflate-friendly than others? Or is there also variance in the same files in that plot?
notramo (Oct 22 2026 at 22:46):By the way, the root path returns 404: https://rtfeldman.com/
Richard Feldman (Oct 22 2026 at 23:27):the files are different sizes, it's not variance :smile:
Richard Feldman (Oct 22 2026 at 23:29):oh I see what you mean now - scatter plot of all 12 points at each level?
Richard Feldman (Oct 22 2026 at 23:36):@Matthieu Pizenberg did you mean something like this?
Jonathan (Oct 23 2026 at 00:11):When I've done plots like that, it's always been difficult to intuit the distribution because it's not too clear from opacity how dense an area is (especially L5 onwards), so I either make them much smaller dots, or add an overlay box and whiskers. I also think a violin plot is pretty intuitive imo, but it is probably clear enough from this scatter what the skew and rough median is.
MagicMan (Oct 23 2026 at 01:01):I like the dot a lot better than the bargraph
MagicMan (Oct 23 2026 at 01:01):In my opinion, a bar graph that does not start at the x axis looks strange
MagicMan (Oct 23 2026 at 01:23):Claude suggested something like this because it highlights better where Roc is faster. Graph
Richard Feldman (Oct 23 2026 at 01:30):revised dots:
Matthieu Pizenberg (Oct 23 2026 at 06:43):I meant something like one of these (absolute log, or relative) but I like your plot with dots too!
Matthieu Pizenberg (Oct 23 2026 at 06:46):I’d probably also ditch compression levels L8 to L12 from these plots because the speed change is such that a linear scale plots are not super useful, and even the log scale plots are meh. Makes the graph less easily interpretable
Richard Feldman (Oct 23 2026 at 10:57):posted publicly now at https://rtfeldman.com/fp-vs-c - thanks for all the feedback! :smiley:
notramo (Oct 23 2026 at 11:51):@Richard Feldman, when I open the root path on your site, it's 404. Open this link directly, not a sub-path:
https://rtfeldman.com/
Richard Feldman (Oct 23 2026 at 11:58):I know, I've never set up a homepage for it :laughing:
Richard Feldman (Oct 23 2026 at 11:58):I just use the domain to host individual posts
notramo (Oct 23 2026 at 12:03):Maybe a simple list of the posts?
notramo (Oct 23 2026 at 12:08):
From an older post.
notramo (Oct 23 2026 at 12:08):Maybe a <meta charset="utf-8"> missing?
notramo (Oct 23 2026 at 12:09):
And the <head> is not closed properly.
notramo (Oct 23 2026 at 12:12):Adding a lang attribute may help with SEO.
notramo (Oct 23 2026 at 12:14):A <title> element in the <head> is also missing. (That sets document.title and the search.)
notramo (Oct 23 2026 at 12:15):I see some of these are only occur on the linked older post.
Richard Feldman (Oct 24 2026 at 11:56):for future reference, this is #8 on HN right now (comments) and yesterday peaked at #4 on Lobsters (comments) and #2 on /r/programming (comments)
Richard Feldman (Oct 24 2026 at 11:58):thanks for all the feedback, everyone! :smiley:
Jay (Oct 24 2026 at 16:35):Be honest, guys, is Roc fast because you've been using LLM backend instead of LLVM backend this whole time?
Last updated: Nov 04 2026 at 18:54 UTC