Stream: show and tell

Topic: Decoder composition performance benchmark


view this post on Zulip Eli Dowling (Apr 11 2024 at 03:40):

This isn't super exciting, but I've been doing lots of custom decoders and so I started working on some better functions for composing decoders. I realised I could either make many decoders inside each other, or one decoder with many decodes, but I wondered if there was much performance difference between the two. So I made a benchmark to compare:

Turns out there was no difference whatsoever, which is nice because as you can see, one API is much nicer than the other

benchmark.roc

view this post on Zulip Brendan Hansknecht (Apr 11 2024 at 04:01):

Two questions:

  1. can you share DecodeUtils?
  2. Are you sure this doesn't optimize away?
\_ ->
        _ = fn {}
        {}

view this post on Zulip Eli Dowling (Apr 11 2024 at 04:02):

I'm pretty sure because it takes ages, and if I add more wrapOnErrors it takes longer.

view this post on Zulip Brendan Hansknecht (Apr 11 2024 at 04:03):

Oh, and one more question:
If you have any idea, how does it compare to what you would expect? Would it be comparable to other languages?

view this post on Zulip Eli Dowling (Apr 11 2024 at 04:04):

Oh I forgot that DecodeUtils was still included. I have inlined all the functions I needed from it

view this post on Zulip Brendan Hansknecht (Apr 11 2024 at 04:04):

Ah, and Core.roc. That is the other piece needed to run locally.

view this post on Zulip Eli Dowling (Apr 11 2024 at 04:04):

Unfortunately I have absolutely no clue about that. Benchmarking against others is a much bigger task

view this post on Zulip Brendan Hansknecht (Apr 11 2024 at 04:05):

Yeah, no worries. I am just always curious around these kinds of implementations. They tend to build a nest of closures for llvm to unravel.

view this post on Zulip Eli Dowling (Apr 11 2024 at 04:09):

Okay, I updated it to use the roc-json package. Everything is running off local branches because of this null decoding business :man_facepalming: Thanks for the catch @Brendan Hansknecht

view this post on Zulip Eli Dowling (Apr 18 2024 at 05:38):

@Brendan Hansknecht I have benchmarks!
I did some testing of decoding a large json file (24MB):
JsonVal is recursive tag union that can represent all the json primatives for dynamic json decoding.
jsonVal: Completed 1 runs in 1950ms
This test takes on a single field from each item in the long json list
json id only: Completed 1 runs in 1054ms

Dotnet:

| Method             | Mean      | Error    | StdDev   | Median    |
|------------------- |----------:|---------:|---------:|----------:|
| IdOnly                |  93.16 ms | 1.728 ms | 4.239 ms |  91.63 ms |
| JsonVal              | 184.21 ms | 1.518 ms | 1.420 ms | 184.50 ms |

So we are around 10x slower than the inbuilt decoder in dotnet

view this post on Zulip Brendan Hansknecht (Apr 18 2024 at 05:41):

Rough. Will be interesting to see a breakdown at some point and figure out the bugs. Almost certainly bad copying and refcounting going on.

view this post on Zulip Eli Dowling (Apr 18 2024 at 05:42):

How would you recommend approaching that?

view this post on Zulip Luke Boswell (Apr 18 2024 at 06:29):

Flame graphs :hearts: :fire:

view this post on Zulip Eli Dowling (Apr 18 2024 at 06:59):

Okay, I have some answers. Thankyou Luke!:
20% of the time is spent just doing List.get! So i guess I was correct to have concerns there @Brendan Hansknecht :)

view this post on Zulip Eli Dowling (Apr 18 2024 at 08:55):

Well I exposed getUnchecked, and tried to make more stuff tail recursive and it's a little better but not that much. About 20-25%... Not really the 10x improvement I'm looking for.


Last updated: Jul 06 2025 at 12:14 UTC