Stream: compiler development

Topic: addFile extra data

Will Hawkins (Jul 03 2026 at 03:09):

In addFile in the NodeStore, the file.header is stored in extra_data.

    try store.extra_data.append(store.gpa, @intFromEnum(file.header));

However, I cannot for the life of me figure out where the just-allocated index is stored for later retrieval. In fact, in add file, that index is calculated from

    const header_ed_idx = @as(usize, @intCast(node.data.lhs + node.data.rhs));

but, upon insertion, the data's lhs and rhs are values from the span of tokens.

I am sorry that I am missing something obvious. If anyone can help, I would really appreciate it!

Will Hawkins (Jul 03 2026 at 03:11):

Will Hawkins said:

In addFile in the NodeStore, the file.header is stored in extra_data.

    try store.extra_data.append(store.gpa, @intFromEnum(file.header));

However, I cannot for the life of me figure out where the just-allocated index is stored for later retrieval. In fact, in add file, that index is calculated from

    const header_ed_idx = @as(usize, @intCast(node.data.lhs + node.data.rhs));

but, upon insertion, the data's lhs and rhs are values from the span of tokens.

I am sorry that I am missing something obvious. If anyone can help, I would really appreciate it!

I logged some of the indexes that are being used. I get 32179 from calls to getFile during zig build roc. When I see a number like that, I get very suspicious.

(Sorry for replying to myself!)

Will Hawkins (Jul 03 2026 at 11:34):

Will Hawkins said:

Will Hawkins said:

In addFile in the NodeStore, the file.header is stored in extra_data.

    try store.extra_data.append(store.gpa, @intFromEnum(file.header));

However, I cannot for the life of me figure out where the just-allocated index is stored for later retrieval. In fact, in add file, that index is calculated from

    const header_ed_idx = @as(usize, @intCast(node.data.lhs + node.data.rhs));

but, upon insertion, the data's lhs and rhs are values from the span of tokens.

I am sorry that I am missing something obvious. If anyone can help, I would really appreciate it!

I logged some of the indexes that are being used. I get 32179 from calls to getFile during zig build roc. When I see a number like that, I get very suspicious.

(Sorry for replying to myself!)

Sorry, again, for replying to myself but, after further investigation, I now see the connection between the node data and the index to the top of the extra_data when addFile is invoked.

For anyone playing the "Will Talks To Himself In Zulip All Night" home game, the key to unlocking the mystery for me was looking at the implementation of statementSpanFrom (which does housekeeping on extra_data and creates a span that can be used to calculate the top index of the extra_data).

I am sorry for the messages but hope that the information helps someone in the future if they run into the same conundrum.

Aurélien Geron (Jul 03 2026 at 12:19):

Glad you found the solution! :slight_smile:

Anton (Jul 03 2026 at 13:05):

I am sorry for the messages

No need to apologize :)

Last updated: Jul 23 2026 at 13:15 UTC