Stream: ideas

Topic: customizing dbg rendering in the host

Richard Feldman (Dec 04 2023 at 18:04):

Richard Feldman said:

giving platforms a way to customize what Inspect output gets sent to the host

an idea for how to do this:

1. Give platform modules a new header entry where they platform author specifies an InspectFormatter that will be used whenever dbg gets run. They can make this generate utf8 strings, with or without ansi escapes in a terminal, or raw bytes encoded in some way that a GUI can decode to get syntax highlighting info, etc
2. roc_dbg expects to receive the output of this

Richard Feldman (Dec 04 2023 at 18:04):

so then basically all the dbg formatting is specified in Roc code, which I think will be a lot easier

Richard Feldman (Dec 04 2023 at 18:05):

than trying to do it in the host

Brendan Hansknecht (Dec 04 2023 at 18:05):

Can it also just produce an ast? I think we could make it produce anything and just pass a pointer to that to roc_dbg

Brendan Hansknecht (Dec 04 2023 at 18:06):

Then glue can generate for it so the platform can use it.

Richard Feldman (Dec 04 2023 at 18:56):

we could, although that seems more costly at runtime and probably more complex for host authors

Richard Feldman (Dec 04 2023 at 18:57):

like an ast has to have a bunch of pointers and allocations, whereas the Roc implementation can just traverse what we already have in memory and turn it into a string or bytes with no extra allocations etc

Brendan Hansknecht (Dec 04 2023 at 18:58):

Well, it is up to the host author if they want to support that at all. I assume most will just accumulate to a Str.

Brendan Hansknecht (Dec 04 2023 at 18:58):

The main gain will be any sort of dbg guis or dbg with richer output

Brendan Hansknecht (Dec 04 2023 at 18:59):

If I want a gui ast anyway, might as well just generate the ast instead of idk, inspecting into a json like format and then parsing that on the host side.

Richard Feldman (Dec 04 2023 at 19:41):

I figure if a GUI author wants to do that, traversing (parsing) a compact binary representation will be about the same perf as traversing ast nodes though - maybe even better because of locality

Richard Feldman (Dec 04 2023 at 19:42):

like either way they look at a discriminant in memory, load some other memory based on what it was, do something with that memory, and repeat

Brendan Hansknecht (Dec 04 2023 at 19:45):

I mean I guess that depends if a platform author wants to deal with high performance binary encoding and decoding just to enable pretty debug statements. I know if I were that platform author, I would prefer just to get a full data structure. It is a debug message after all. I don't care much about perf. I mostly care about convenience of implementation and correctness.

Sky Rose (Dec 05 2023 at 10:32):

I imagine there would be a lot of basic platforms that want the easiest solution of printing an unformatted string, and a few high production value platforms that will want the power to do something fancy.

In that sense, as long as there's a reasonable default if you don't specify InspectFormatter at all, then this discussion of ast vs binary is just for the few platforms opting in to fancier formatting, and minimum effort is less of a consideration.

Sky Rose (Dec 05 2023 at 10:33):

But also, if a binary representation and an ast are close enough in performance, then you should just go for the one that's easier to work with. Which I think would be the structured data, so that you write one traversal in the host, instead of a serialized in Roc and a deserializer in the host.

But that makes a couple assumptions:

1. It's only fancy high-effort platforms that would do this, not simple platforms that are thin wrappers to get into Roc.
2. There's good glue code for the host language.
3. The platform author is fine writing in the host language instead of Roc (which is probably a safe assumption if they're writing a fancy debug viewer).

Richard Feldman (Dec 05 2023 at 12:00):

I kinda assume it's equally convenient for host authors though - either way they'll need to reach for a third-party thing to decode it (if it's an AST, they'll need glue to generate the bindings for them, and if it's a compact binary serialization they'll need a parser)

Brendan Hansknecht (Dec 05 2023 at 16:01):

You also need to write a compact binary serializer that supports all roc types and generates the same format as is supported in the host. That is too much effort for most people is my guess (just wait, people will start sending json cause that already has a serializer on the roc side)

Brendan Hansknecht (Dec 05 2023 at 16:02):

Also, 100% agree that most platforms probably will never touch this and will just take a string and print it.

Brendan Hansknecht (Dec 05 2023 at 16:03):

So we want to either default to what we currently have or expose the DbgFormatter so that platforms can specify it as the formatter they want.

Richard Feldman (Dec 05 2023 at 16:50):

Brendan Hansknecht said:

You also need to write a compact binary serializer that supports all roc types and generates the same format as is supported in the host. That is too much effort for most people is my guess (just wait, people will start sending json cause that already has a serializer on the roc side)

oh I actually think we should make this be a thing :grinning_face_with_smiling_eyes:

Richard Feldman (Dec 05 2023 at 16:50):

like a nice obvious fast default for anyone who wants to save some Roc values to disk, or talk between two Roc processes (including over the network) etc

Richard Feldman (Dec 05 2023 at 16:53):

I think there's a decent argument for having that binary format as a builtin actually

Richard Feldman (Dec 05 2023 at 16:53):

basically an obvious roc-specific default choice for serialization

Brendan Hansknecht (Dec 05 2023 at 17:12):

oh I actually think we should make this be a thing

Sure, but how is that different from just sending the roc type over via dbg? A formatter can generate literally any roc type. I guess the main argument is that we go from roc in memory to roc as bytes to avoid the cost of wrapping each roc type with a tag to specify what it is? Like the roc as bytes would have this tagged information, but it would be generated without extra pointers and allocatoins?

Also, I guess the most performant dbg format if you truly want an ast and more control would be pass a record of (typedescriptor, originalvalue). type descriptor would be a list of tags where they use offsets instead of pointers for recursive types. That way, no data has to move or be serialized at all. The host can read the tags and do dynamic casting and such.

Richard Feldman (Dec 05 2023 at 17:15):

true! That could work

Last updated: Jun 16 2026 at 16:19 UTC