SQLite owned buffer · performance

Stream: performance

Topic: SQLite owned buffer

Richard Feldman (Sep 02 2026 at 21:07):

Karl said:

the SQLite buffer is (obviously) host side and getting it into Roc requires a copy

:thinking: why would that require a copy?

Richard Feldman (Sep 02 2026 at 21:08):

it's just memory, right? you can make a List in the host with reserved capacity for the buffer and then hand that directly into ordinary Roc code, which should be able to write into it as long as its refcount remains at 1

Karl (Sep 02 2026 at 21:09):

you can make a List in the host with reserved capacity for the buffer

SQLite owns the memory. As far as I know there isn't a way to do a slice on a host owned buffer

Richard Feldman (Sep 02 2026 at 21:10):

hm, "owns the memory" in what sense? :thinking:

Richard Feldman (Sep 02 2026 at 21:10):

like it gives you an address, but it reserves the right to write to it or something?

Karl (Sep 02 2026 at 21:10):

This is SQLite's cursor/results buffer. Owns as in allocates/deallocates.

The encoders+decoders are arranged so that the bytes go from SQLite's cursor to the output buffer with no intermediate allocation.

Karl (Sep 02 2026 at 21:13):

Let me see if it's compiling/working and push it to github.

Richard Feldman (Sep 02 2026 at 21:14):

I see, so it sounds like there are two missing pieces here:

We don't currently have a way to say "encode into this buffer" or "decode into this buffer" (I think this would be easy to add, but I haven't thought about it before)
if it's a List, we can't allow it to be resized because that would reallocate and SQLite wouldn't be okay with that

Richard Feldman (Sep 02 2026 at 21:14):

does that sound right?

Karl (Sep 02 2026 at 21:36):

Trying for a coherent explanation:

I have a web platform that's mostly a sandbox for ideas. After last month's discussion about encoders/decoders I decided to see how far I could push the encoder/decoder APIs. The one that's relevant for this discussion is that I'm implementing a decoder for SQLite query row results wired up to the encoder for Json. For a query SQLite copies data from its btree into (my term) the results buffer and provides a cursor for walking through the rows. The data also needs to be formatted/copied into the send buffer since MacOS doesn't have something like io_uring. So my goal is to get down to the minimum so the response is faster. I have it working but it required some massaging.

Problems:

All non-small RocStr/RocList are refcounted and the code for decref is:

alloc_ptr = get_allocation_ptr()        // == bytes, for a big (non-slice) string
rc = *(alloc_ptr - 8)                    // the refcount word
if rc == 0 { return }                    // REFCOUNT_STATIC_DATA: never free
if fetch_sub(rc) == 1 { host.dealloc(alloc_ptr - 8) }

The alloc_ptr on the host is in SQLite's memory space and control so there's no refcount at that position. There's a rc=0 variant that's used for static strings but that also requires the byte at ptr-8 to be the right value. There would need to be some variant of List/Str that knows it's host owned and doesn't do the refcounting.

The other factor for this particular problem is that the memory is only valid until the cursor moves so there needs to be some sort of lifetime feature. I have a hacky non-effectful host function sql_col_str_json_into that copies the bytes from the cursor into the output.

Luke Boswell (Sep 02 2026 at 21:38):

can you use a seamless slice? from the hosts sqlite buffer

Karl (Sep 02 2026 at 21:39):

seamless slice sets requires a backing Roc allocation and shares its refcount

Karl (Sep 02 2026 at 21:42):

It's hacky but the perf is super fast. Total roc portion of the handler time is 16.4µs instead of 39.2µs.

Richard Feldman (Sep 03 2026 at 02:07):

I think we had

Karl said:

seamless slice sets requires a backing Roc allocation and shares its refcount

this part is actually fine - we designed for this situation (although the lifetime is a separate question) - the three fields in a str or list are:

bytes_ptr
capacity_or_alloc_ptr
length

so what you can do is set the alloc ptr to be totally unrelated to where the bytes_ptr is, and just have your refcount be right there - e.g. you allocate 16B yourself, and use it to store (refcount, NULL) and just have capacity_or_alloc_ptr point to the NULL, so the refcount is right in front of it

Richard Feldman (Sep 03 2026 at 02:08):

(I think you can maybe even get away with not bothering to allocate the NULL, but there might be some super weird edge case where you're right at the end of the address space or something, and I also think most allocators will allocate a minimum of 16B anyway so it's probably zero-cost anyway)

Richard Feldman (Sep 03 2026 at 02:10):

anyway, that doesn't address the lifetime part of it

Karl said:

The other factor for this particular problem is that the memory is only valid until the cursor moves so there needs to be some sort of lifetime feature.

when does the cursor actually move? like is the platform fully in charge of when it moves, or do you expose the cursor to the app author to move directly?

Karl (Sep 03 2026 at 02:16):

I'll give the seamless slice a go. Not an approach I would have come up with.

The cursor moves by a call to sql_step_row! which returns a token that has to be passed to sql_col_str_json_into which asserts the allocation is valid is the lifetime soundness of the system.

Karl (Sep 03 2026 at 02:18):

I'm not committed to any particulars for this design. I'm just trying to eliminate copies via encoders/decoders.

Richard Feldman (Sep 03 2026 at 02:18):

yeah I think @Brendan Hansknecht thought to design seamless slices for this use case a couple years ago - full credit to him for the payoff here! :smile:

Last updated: Sep 03 2026 at 15:16 UTC