Stream: compiler development

Topic: Multiple interners or single interner


view this post on Zulip Sam Mohr (Feb 12 2025 at 07:02):

Currently the new compiler has multiple interners based roughly on the outline Agus made for the Rust-based compiler's specialize_types pass. There's an array of tag_names, field_names, idents, etc. I thought that copying this would be a good pattern to help get the type safety that we got in Rust, e.g. an Index<TagName> in Rust can only be used for looking up tag names, mirrored in the Zig rewrite with a TagName.Idx.

However, I'm not sure if this is the right move. For one, the interners for everything but string literals are deduplicated, meaning we can't deduplicate variables names and record field names with the same names, taking more space. More importantly, if we want to look up a name, we can only look for things with that intern kind, meaning we can't do constant-time lookup of a tag name using a type name. That shouldn't be too much of a problem though: the only place I think the same token represents two different interned types is name punning, AKA records like { field_name }. That just means we'd need to convert that to { field_name: field_name } during desugaring and potentially copy the name field_name from the record field interner to the ident one.

So with that in mind, I'm not seeing one option as definitely better but I'd rather have type safety than a little space conservation.

For the parsing team, would there be any problems with interning into different interners based on the token type? I presume we'd be good to do that. If so, I'll probably make a change to break into more groups of types, e.g. one for module names, one for type names, etc.

view this post on Zulip Sam Mohr (Feb 12 2025 at 07:03):

And to those outside of @Joshua Warner and @Anthony Bullard , do you think there's a strong reason to prefer one or multiple interners?

view this post on Zulip Joshua Warner (Feb 12 2025 at 07:07):

My thinking right now is to have interning happen in canonicalization, not parsing

view this post on Zulip Sam Mohr (Feb 12 2025 at 07:08):

Hmm. Anthony suggested otherwise, but it'd be okay

view this post on Zulip Joshua Warner (Feb 12 2025 at 07:09):

I don't think there's a strong benefit to doing it earlier, since the ast needs to be carrying around references/indices into the text anyway - and if that's all it's carrying around we can maybe keep the ast smaller.

view this post on Zulip Sam Mohr (Feb 12 2025 at 07:09):

The benefit of doing it during parsing is that it's an O(n) pass over the data which allows us to do mostly constant time string comparisons (outside of checking imports) and also carry around less data starting with canonicalization

view this post on Zulip Sam Mohr (Feb 12 2025 at 07:10):

I guess it's a question if dropping the full text earlier during compilation would be considered beneficial

view this post on Zulip Joshua Warner (Feb 12 2025 at 07:10):

I definitely buy constant time string comparisons as a benefit

view this post on Zulip Sam Mohr (Feb 12 2025 at 07:10):

That happens as soon as interning happens

view this post on Zulip Joshua Warner (Feb 12 2025 at 07:10):

... but I don't want the parser to have to talk to some complicated multithreaded interning system

view this post on Zulip Sam Mohr (Feb 12 2025 at 07:11):

It's not multithreaded

view this post on Zulip Joshua Warner (Feb 12 2025 at 07:11):

That's going to slow down the parser

view this post on Zulip Sam Mohr (Feb 12 2025 at 07:11):

There's an interner per module

view this post on Zulip Joshua Warner (Feb 12 2025 at 07:11):

Ahhh; ok; that's somewhat less concerning

view this post on Zulip Sam Mohr (Feb 12 2025 at 07:11):

Specifically one of these: https://github.com/roc-lang/roc/blob/main/src/base/ModuleEnv.zig

view this post on Zulip Sam Mohr (Feb 12 2025 at 07:12):

The other thought is that we could check for ident attributes and problems in the same pass as we tokenize, or at least as we parse

view this post on Zulip Joshua Warner (Feb 12 2025 at 07:12):

Still, I'd rather have more work in can than in parsing (supposing we're just shifting it around, and not making lots more/less work). That way, things like formatting can be faster (very common to do across a whole file or even whole codebase!)

view this post on Zulip Sam Mohr (Feb 12 2025 at 07:13):

Because if we intern during canonicalization, we have to read over each ident in full (not just the starting and ending chars) to check for warning info, like multiple underscores in a row or uppercase letters in a lowercase ident (we want to warn on camelCase)

view this post on Zulip Sam Mohr (Feb 12 2025 at 07:13):

You're right, this would make formatting faster...

view this post on Zulip Joshua Warner (Feb 12 2025 at 07:14):

(re)building an ast after saving a file, too

view this post on Zulip Sam Mohr (Feb 12 2025 at 07:14):

It'd be nice to have some estimate whether interning during parsing would be faster for overall compilation time

view this post on Zulip Sam Mohr (Feb 12 2025 at 07:14):

Because if it's not, then it should definitely be done during can

view this post on Zulip Sam Mohr (Feb 12 2025 at 07:15):

I only suggested it because I thought it would speed up overall compilation, even if it makes formatting slower

view this post on Zulip Joshua Warner (Feb 12 2025 at 07:15):

If anything, I'd intern during tokenization rather than parsing

view this post on Zulip Sam Mohr (Feb 12 2025 at 07:15):

Yeah, that was my thought too

view this post on Zulip Joshua Warner (Feb 12 2025 at 07:15):

That way the parser never has to look at the source

view this post on Zulip Sam Mohr (Feb 12 2025 at 07:15):

yep

view this post on Zulip Joshua Warner (Feb 12 2025 at 07:15):

(just the tokens)

view this post on Zulip Joshua Warner (Feb 12 2025 at 07:15):

I'd still like to push that later, until/unless we have some evidence to the contrary

view this post on Zulip Sam Mohr (Feb 12 2025 at 07:16):

A reasonable decision

view this post on Zulip Sam Mohr (Feb 12 2025 at 07:16):

Not sure how to get that evidence

view this post on Zulip Sam Mohr (Feb 12 2025 at 07:17):

Let me ask in a separate thread about regions if you have a plan for those

view this post on Zulip Sam Mohr (Feb 12 2025 at 07:20):

For other readers, if you have an opinion on the root question of this thread, I'd still love to hear it

view this post on Zulip Richard Feldman (Feb 12 2025 at 13:09):

if we're doing roc check, doing it during tokenization makes the most sense because the whole string will be in L1 cache right then

view this post on Zulip Richard Feldman (Feb 12 2025 at 13:09):

if we're doing roc format, not doing it at all makes the most sense :big_smile:

view this post on Zulip Richard Feldman (Feb 12 2025 at 13:10):

deduplicating string literals only comes up if we're doing roc build and not using the interpreter as the backend

view this post on Zulip Richard Feldman (Feb 12 2025 at 13:29):

I think doing it per-module instead of globally makes the most sense

view this post on Zulip Richard Feldman (Feb 12 2025 at 13:29):

it avoids contention when we get to parallelizing things, because the interns are only used by the current module

view this post on Zulip Richard Feldman (Feb 12 2025 at 13:31):

(or if they're being used by other modules it's because we're translating IDs from one module to another, but at that point they aren't being written to)

view this post on Zulip Richard Feldman (Feb 12 2025 at 13:33):

I think we'll also need to serialize them to disk as part of caching, which of course we want to do on a per-module basis instead of globally

view this post on Zulip Richard Feldman (Feb 12 2025 at 13:35):

also having them per-module makes it easier for the language server to manage their memory, since otherwise we'd need to refcount each individual interned string to know when it was safe to free, compared to just resetting a particular module's arena and redoing parsing etc

view this post on Zulip Richard Feldman (Feb 12 2025 at 13:36):

oh and also per-module means there's more cache coherence, so fewer cache misses when using interns because there aren't a bunch of gaps from indentifiers that aren't even used in the module and won't come up

view this post on Zulip Richard Feldman (Feb 12 2025 at 13:47):

as an aside, I think there are some potentially very interesting ways we can reduce memory usage in interns by packing them into size and capitalization categories and using simd to find matches, but all of that is definitely not stuff we should do for 0.1.0 :big_smile:

view this post on Zulip Joshua Warner (Feb 12 2025 at 15:10):

Ok, if I’m reading things right, the lexer for Carbon (for which I know that team has been very performance-focused), does ident interning. Here: https://github.com/carbon-language/carbon-lang/blob/trunk/toolchain/lex/lex.cpp

view this post on Zulip Joshua Warner (Feb 12 2025 at 15:10):

That’s good enough reason to me

view this post on Zulip Joshua Warner (Feb 12 2025 at 15:11):

At least we can be relatively confident it’s not a serious perf problem

view this post on Zulip Richard Feldman (Feb 12 2025 at 15:26):

yeah and in the future we can potentially have roc format tell it to use a "null interner" which just returns 0 for every ident (or something) because we know it's never going to look them up again

view this post on Zulip Sam Mohr (Feb 12 2025 at 16:23):

Richard Feldman said:

deduplicating string literals only comes up if we're doing roc build and not using the interpreter as the backend

During canonicalization, we can check if the same variable/type/tag/etc. name is used while scope checking in constant time if all strings are interned, so I think it's beneficial before interpreting too

view this post on Zulip Richard Feldman (Feb 12 2025 at 16:44):

I mean string literals specifically

view this post on Zulip Richard Feldman (Feb 12 2025 at 16:44):

like "foo" in the Roc code cli_arg_name = "foo"

view this post on Zulip Richard Feldman (Feb 12 2025 at 16:45):

as opposed to identifier names in code

view this post on Zulip Joshua Warner (Feb 12 2025 at 16:45):

Ah, do we need to unique those?

view this post on Zulip Richard Feldman (Feb 12 2025 at 16:45):

we want to deduplicate those when compiling to a static binary, so that we don't put the same string in the binary's readonly section more than once

view this post on Zulip Richard Feldman (Feb 12 2025 at 16:46):

there's no benefit to having duplicates since they're readonly

view this post on Zulip Joshua Warner (Feb 12 2025 at 16:46):

Ah true; but I would assume that happens later

view this post on Zulip Richard Feldman (Feb 12 2025 at 16:46):

yeah that's what I mean

view this post on Zulip Richard Feldman (Feb 12 2025 at 16:46):

we only need to dedupe those if we're building a binary

view this post on Zulip Richard Feldman (Feb 12 2025 at 16:46):

but check and format have no reason to do that work

view this post on Zulip Richard Feldman (Feb 12 2025 at 16:46):

(and those strings can be pretty big!)

view this post on Zulip Sam Mohr (Feb 12 2025 at 16:47):

Yep, those are not deduped right now

view this post on Zulip Sam Mohr (Feb 12 2025 at 16:47):

Because big and unique

view this post on Zulip Richard Feldman (Feb 12 2025 at 16:47):

yeah, so I don't think we should use interns for those (at least at this stage)

view this post on Zulip Richard Feldman (Feb 12 2025 at 16:47):

better to leave that for a later pass

view this post on Zulip Sam Mohr (Feb 12 2025 at 16:48):

We still put them in a big array to keep IR nodes smaller

view this post on Zulip Sam Mohr (Feb 12 2025 at 16:48):

Not sure if you think that's actively a bad idea

view this post on Zulip Richard Feldman (Feb 12 2025 at 16:48):

oh I think we can just reference original source range

view this post on Zulip Sam Mohr (Feb 12 2025 at 16:48):

Oh, that's better

view this post on Zulip Sam Mohr (Feb 12 2025 at 16:48):

I'll hook that up

view this post on Zulip Richard Feldman (Feb 12 2025 at 16:49):

that does mean the interpreter has to redo a bit of work

view this post on Zulip Richard Feldman (Feb 12 2025 at 16:49):

to do things like resolve \ escapes

view this post on Zulip Richard Feldman (Feb 12 2025 at 16:49):

which maybe isn't worth it

view this post on Zulip Richard Feldman (Feb 12 2025 at 16:50):

could always have 2 different IR nodes, one for like "literal with no escapes" that just stores a source code range and another for "escaped string literal" which points to the post-processed one

view this post on Zulip Sam Mohr (Feb 12 2025 at 16:51):

I guess that's okay complexity to eat today since we'd only need to deal with this during check

view this post on Zulip Sam Mohr (Feb 12 2025 at 16:51):

The later phases would get back a normal string

view this post on Zulip Brendan Hansknecht (Feb 12 2025 at 23:46):

Richard Feldman said:

we want to deduplicate those when compiling to a static binary, so that we don't put the same string in the binary's readonly section more than once

I think we can trivially do that when generating llvm ir. We generate a single ir module so we can literally keep a map of seen strings if we want.

view this post on Zulip Brendan Hansknecht (Feb 12 2025 at 23:48):

Richard Feldman said:

that does mean the interpreter has to redo a bit of work

I guess that would mean we have to keep all source in memory which doesn't sound good. Or the interpreter has to re-open every file to pull all the strings, which sounds kinda strange, but ok.


Last updated: Jul 06 2025 at 12:14 UTC