Karl (Sep 03 2026 at 02:24): Karl (Sep 03 2026 at 02:24):Richard Feldman said:
we have SIMD builtins right now - are they not sufficient?
They are sufficient for the narrower 128 bit variant. I'm porting this from the Rust implementation which includes the 256 bit version so the Roc version is going to be slower. The LLM missed the builtins and I didn't catch it because I'd read a thread about missing SIMD float support .
Karl (Sep 03 2026 at 02:26):Initial benchmarks pass has be 100x-5300x slower than Rust. The high end is special case code I don't think I have a path to reach from Roc but even the baseline is excessively slow so I'm working on the reason.
Edit: 75% memory management. Typical reason.
Richard Feldman (Sep 03 2026 at 02:48):ok cool! so I've been working on a (probably?) similar project where I'm trying to get roc-deflate to within 10% of libdeflate's perf
Richard Feldman (Sep 03 2026 at 02:50):when I started out, decompression and compression both had hundreds of percents of slowdown, but eventually I got decompression down to 6-12% slowdown on all libdeflate's benchmarks, and I'm still working on compression - it's now under 100% on average, but not yet down to the 10% range
Karl (Sep 03 2026 at 02:50):I wanted to try the Regex because it's a pure function so in theory the machinery can be folded at compile time and the runtime is just a matcher.
Richard Feldman (Sep 03 2026 at 02:51):yeah I absolutely love it! I think this is a really good case where I'd love to push the boundaries and try really hard to get it within 10% of Rust's perf :smiley:
Richard Feldman (Sep 03 2026 at 02:52):so the approach I've taken with libdeflate has been to iterate a lot with Fable on it, both in terms of getting the Roc implementation closer to libdeflate, and also using that as a way to discover opportunities for improving our own compiler optimizations (while being wary of not overfitting to the benchmark)
Richard Feldman (Sep 03 2026 at 02:52):I'd say it's been mostly changes to the library itself, e.g. at first compression wasn't using our simd builtins at all because the C one wasn't, but it turned out LLVM wasn't autovectorizing ours as well as it was the C ones because we needed to do some bounds checks or something (I forget exactly what the difference was) and switching our implementation to do explicit simd made a difference
Richard Feldman (Sep 03 2026 at 02:54):but I will say that at least on the LLM side, there was a lot of coaxing to get it to the point where the implementation was actually a faithful port - it's so, so prone to doing a "close enough" port, and then trying to go off and optimize something that's fundamentally just doing something much less efficient than the already-optimized C (or in this case Rust) implementation
Richard Feldman (Sep 03 2026 at 02:54):regarding 128-bit vs 256-bit, I get nervous about enabling 256-bit SIMD
Karl (Sep 03 2026 at 02:54):I don't think I can get to within 10% of Rust when it's on the fast paths. The 256 bit wide SIMD is scanning 32 characters versus my 16. Getting to a significant fraction is probably doable on the paths where the matching engine is actually doing work.
Richard Feldman (Sep 03 2026 at 02:55):yeah I'm just nervous about hardware support and increasing the complexity of the targets we support etc.
Richard Feldman (Sep 03 2026 at 02:55):128-bit has very broad support today
Karl (Sep 03 2026 at 02:55):I think the perf will be totally acceptable. It's not like we need 4GB/s scanning to make it work.
Richard Feldman (Sep 03 2026 at 02:55):ok cool
Richard Feldman (Sep 03 2026 at 02:56):in that case I'd say you might consider temporarily changing your local rust impl to use 128-bit too, so you're comparing apples to apples
Richard Feldman (Sep 03 2026 at 02:57):and then if you can get within 10% of that, maybe you zoom out and say ok it's (whatever)% compared to the 256-bit Rust one, and if Roc ever adds 256-bit SIMD (e.g. when hardware support is more widespread, or maybe we just figure out some nice way to offer it in a way that doesn't break when the hardware doesn't support it) then we can presumably close that gap
Last updated: Sep 03 2026 at 15:16 UTC