Isaac Van Doren (Mar 29 2024 at 02:17): Isaac Van Doren (Mar 29 2024 at 02:17):It would be nice to be able to compress responses with gzip in basic-webserver. I know this could be implemented by the platform, but it seems like it would be nice to have a pure Roc implementation of gzip so it could be reused and would not have to be exposed across multiple platforms. Does that sound right? Is there anything else I should be thinking about?
Luke Boswell (Mar 29 2024 at 02:48):That sounds right to me. I have no idea how complicated that would be to implement though
Brendan Hansknecht (Mar 29 2024 at 02:49):Sounds pretty reasonable. Always a little concerned about fine grain byte twiddling that might happen. Cause this should be solid perf in roc, but if it is running on every single web request, the extra perf from platform langauges feels nice too.
Brendan Hansknecht (Mar 29 2024 at 02:50):Like 3% slower web requests can actually lead to a much earlier slump under load.
Isaac Van Doren (Mar 29 2024 at 02:51):Right that makes sense. Even if some platforms choose to expose an implementation from another language, it would still be nice to have one implemented in Roc for other use cases
Luke Boswell (Mar 29 2024 at 02:51):I think it's totally worth it for a lot of applications. If perf is a concern you can always pass that off to the platform.
Maybe this would be a good package to think about for roc performance tooling?
Brendan Hansknecht (Mar 29 2024 at 02:54):Yeah, I definitely think we can close most of the gap. Just still can be a high cost when thinking about scale...the fun of scale.
Brendan Hansknecht (Mar 29 2024 at 02:54):But yeah would be useful.
Brendan Hansknecht (Mar 29 2024 at 02:54):Also, basic compression might be useful for the book, but idk.
Isaac Van Doren (Mar 29 2024 at 02:55):Also, basic compression might be useful for the book, but idk.
I was thinking the same thing. Do you know of any good algorithms that would be reasonable in scope?
Brendan Hansknecht (Mar 29 2024 at 02:57):Probably best would be one of the super simple cores of the lz algorithms that is basically a windowed lookup.
Brendan Hansknecht (Mar 29 2024 at 02:57):Though maybe that is less applicable to general users than parsers and things of that nature
Isaac Van Doren (Mar 29 2024 at 02:58):I think it could still definitely fit because devs use compression all the time but I would guess frequently don't know that much about how it works under the hood
Brendan Hansknecht (Mar 29 2024 at 03:01):fair
Brendan Hansknecht (Mar 29 2024 at 03:01):I would probably target LZ77
Brendan Hansknecht (Mar 29 2024 at 03:01):maybe slightly simplified if needed
Isaac Van Doren (Mar 29 2024 at 03:02):I believe gzip uses LZ77 so that's nice too
Brendan Hansknecht (Mar 29 2024 at 03:03):IIRC, It uses deflate which has a ton more on top of it.
Brendan Hansknecht (Mar 29 2024 at 03:04):eh, not a ton, but definitely a solid bit more complex
Isaac Van Doren (Mar 29 2024 at 03:05):Right, I'm just saying its a component of it
Luke Boswell (Mar 29 2024 at 03:08):If it is compressed in roc using LZ77, could another program detect that and decompress it?
Isaac Van Doren (Mar 29 2024 at 03:21):I don't think you would be able to detect it automatically, but another program could definitely decompress it if it knows the format from the get go
Isaac Van Doren (Mar 29 2024 at 03:21):I created an issue for the book here: https://github.com/roc-lang/book-of-examples/issues/48
Brendan Hansknecht (Mar 29 2024 at 03:23):With a header, you probably could lie about Huffman encoding and some how get gzip to do it....otherwise a decode for lz77 is also very simple
Luke Boswell (Mar 29 2024 at 03:37):Just found @Folkert de Vries's elm-tiny-inflate which implements deflate.
Luke Boswell (Mar 29 2024 at 03:39):Also maybe relevant, @Shritesh Bhattarai implemented the CRC32 and Adler32 in roc-image.
Isaac Van Doren (Mar 29 2024 at 03:40):Cool! I know crc32 is also needed for gzip
Folkert de Vries (Mar 29 2024 at 09:30):I've also been working on https://github.com/memorysafety/zlib-rs recently, so I'm quite familiar with the implementation details
Folkert de Vries (Mar 29 2024 at 09:32):though it's hard to strike a balance here between simplicity and efficiency
Last updated: Jul 06 2025 at 12:14 UTC