Stream: compiler development

Topic: List layout


view this post on Zulip Ayaz Hafiz (Jun 22 2024 at 23:18):

What is the layout for List pointers? I I know the struct layout is (ptr, size, capacity) but what is ptr exactly? is it (<refcount=ptr-size bytes><data>) or something else?

view this post on Zulip Ayaz Hafiz (Jun 22 2024 at 23:19):

and what exactly is the layout with seamless slices?

view this post on Zulip Richard Feldman (Jun 23 2024 at 01:23):

@Brendan Hansknecht is the seamless slices expert! :smiley:

view this post on Zulip Brendan Hansknecht (Jun 23 2024 at 01:26):

Pointer is directly to the data

view this post on Zulip Brendan Hansknecht (Jun 23 2024 at 01:27):

For normal lists, the refcount is directly before that (farther away if needed for alignment, this only comes up for lists with data that have alignment greater than 8, so just u128/i128 in roc)

view this post on Zulip Brendan Hansknecht (Jun 23 2024 at 01:28):

For seamless slices, the capacity is not a capacity. Instead, it is a pointer to the first data element in the underlying referenced list.

view this post on Zulip Ayaz Hafiz (Jun 23 2024 at 01:44):

https://github.com/roc-lang/roc/pull/6832 fixes an llvm IR blowup issue that causes huge compile times @Luke Boswell ran into. however i likely won’t have the time to take this over the finish line. can someone help me push it over? i’m running into some kind of bug, maybe due to a bad ref count i set in the code or maybe this implementation only works for List U8 atm

view this post on Zulip Luke Boswell (Jun 23 2024 at 02:06):

Wow, I must say this fix is a big improvement. Even running the roc-lang/examples CI tests are really snappy now, like 100ms for each test. I'm not sure what it was before, I'll switch back and check.

view this post on Zulip Luke Boswell (Jun 23 2024 at 02:36):

Yeah more like 200-400ms on current main

view this post on Zulip Brendan Hansknecht (Jun 23 2024 at 04:07):

First glance this looks roughly correct to me, though also think it may break the surgical linker. I don't think it currently supports lists that contain pointers in the constant sections (so lists of lists, lists of strings, and lists of recursive tags, or anything that contains those types).

view this post on Zulip Brendan Hansknecht (Jun 23 2024 at 04:07):

I think this is likely wrong: let element_type = element_type.into_int_type();

view this post on Zulip Brendan Hansknecht (Jun 23 2024 at 04:07):

You allowed any constants, so there is no guarantee it is an int type

view this post on Zulip Ayaz Hafiz (Jun 23 2024 at 04:36):

yes that is true, i just put the first implementation i could up - unfortunately i dont really have the bandwidth to take it over the line, it's more of a draft rn.

view this post on Zulip Brendan Hansknecht (Aug 10 2024 at 03:23):

I fixed up @Ayaz Hafiz's PR. It is passing CI now. #6832

One thing that is no longer handled is partially constant lists. Not sure how much they matter, but probably could add them back if we really want them. (I'm not sure we actually do want them though)

Also, looking into the support element types in list literals, this only supports primitives. It doesn't support any aggregate types like records or tag unions. Those both should be possible to support, but we would have to capture them differently in the ast such that they can be constants/literals in general.

view this post on Zulip Brendan Hansknecht (Aug 10 2024 at 03:24):

@Luke Boswell , can you double check that you still get the large compile time speed up?

view this post on Zulip Luke Boswell (Aug 12 2024 at 05:59):

Can confirm I'm still seeing the massive compile time speed up on 6b04318e27359404421cd99d8208b6ea6944ce07

view this post on Zulip Luke Boswell (Aug 12 2024 at 06:00):

Current 19,714 ms
Branch 1,523 ms

view this post on Zulip Luke Boswell (Aug 12 2024 at 06:02):

Tested using https://github.com/lukewilliamboswell/roc-htmx-playground/tree/upgrade-server


Last updated: Jul 06 2025 at 12:14 UTC