Luke Boswell (Jan 09 2025 at 22:32): Luke Boswell (Jan 09 2025 at 22:32):I'd really like to have type checking in doc comment code blocks right now... I've manually gone through and updated all the docs for basic-cli and basic-websever by hand and probably have broken things along the way. Sam has helped pick up some snake_case and PNC issues, but without copying the snippet into an editor it's hard to know if it's ok.
Not sure if it's a big job to add this. Even nicer, would be if the language server could provide the orange and red squiggles in the editor :smile:
Anthony Bullard (Jan 09 2025 at 22:33):If we have a markdown library this is trivial to automate
Sam Mohr (Jan 09 2025 at 22:34):It is a big job to automate this
Sam Mohr (Jan 09 2025 at 22:35):We currently just parse doc comments as text, and highlight them totally separately for the docs
Anthony Bullard (Jan 09 2025 at 22:36):I know but if we can convert it to markdown we could do it trivially
Sam Mohr (Jan 09 2025 at 22:36):We'd now need to parse them in addition to the rest of the code
Sam Mohr (Jan 09 2025 at 22:36):Let me think about that
Anthony Bullard (Jan 09 2025 at 22:37):I mean the type checking and validating the blocks
Sam Mohr (Jan 09 2025 at 22:37):We'd probably want to do our own "close-to-markdown" parsing anyway, since the long-term goal is for us to have basically no dependencies (LLVM and Zig)
Sam Mohr (Jan 09 2025 at 22:38):That could work
Anthony Bullard (Jan 09 2025 at 22:39):All you need to parse out is the code blocks which we already do
Sam Mohr (Jan 09 2025 at 22:39):Even if it's parsed as markdown for us, there's still a lot of work waiting because code blocks for docs need to be treated differently in a few different ways
Anthony Bullard (Jan 09 2025 at 22:39):And then just throw them in a module or something and execute them
Anthony Bullard (Jan 09 2025 at 22:40):Of course there’s some massaging to do
Sam Mohr (Jan 09 2025 at 22:40):The massaging is the part I'm expecting is a lot of effort
Sam Mohr (Jan 09 2025 at 22:40):But yes in theory it's simple
Sam Mohr (Jan 09 2025 at 22:40):Very doable if someone is committed
Luke Boswell (Jan 09 2025 at 22:47):We don't want to support full markdown... at least I know Richard has mentioned we don't want to support html or anything like that in Doc comments.
Luke Boswell (Jan 09 2025 at 22:48):One option would be just to provide this check with roc docs where it's being parsed anyway
Anthony Bullard (Jan 09 2025 at 22:49):We can also do it during roc check as a separate check
Luke Boswell (Jan 09 2025 at 22:49):So the typical path ignores comments, keeping things fast... and then for those who want to check the CI run that checks roc docs etc will pick it up
Anthony Bullard (Jan 09 2025 at 22:49):Like Rust does
Sam Mohr (Jan 09 2025 at 22:49):Sounds like we want to parse our own partial markdown. That means basically:
Anthony Bullard (Jan 09 2025 at 22:49):And probably bold/italics
Sam Mohr (Jan 09 2025 at 22:49):We are gonna be doing some nice caching from the results of parsing and canonicalization
Sam Mohr (Jan 09 2025 at 22:50):We should take advantage of the parsing work we already have in the parse crate and do it there
Notification Bot (Jan 09 2025 at 22:52):26 messages were moved here from #compiler development > casual conversation by Luke Boswell.
Luke Boswell (Jan 09 2025 at 22:55):Is there a minimal scope here where we just parse out the ## ```roc and ## ``` part and do something with it... we could leave the other parts for a future expansion right?
Sam Mohr (Jan 09 2025 at 22:56):I think it's not that much effort to add the other stuff, the code blocks are the harder part IMO.
Sam Mohr (Jan 09 2025 at 22:57):But we can definitely start with just code blocks
Sam Mohr (Jan 09 2025 at 22:57):It'd help break this up
Sam Mohr (Jan 09 2025 at 22:57):I'm just thinking that another way to break this up is to first parse as markdown and leave code blocks as text blobs
Sam Mohr (Jan 09 2025 at 22:57):And after that, parse the code blocks themselves
Sam Mohr (Jan 09 2025 at 22:59):But if we're optimizing for getting code blocks type-checked and everything else is okay not worrying about, then we can run in a straight line, sure
Sam Mohr (Jan 09 2025 at 23:00):First, it'd be nice to get Richard's agreement on this, since it's a big change.
Sam Mohr (Jan 09 2025 at 23:01):It's one thing if he thinks this is just too much work compared to other important stuff.
Sam Mohr (Jan 09 2025 at 23:02):But it felt like the pushback last time in #contributing > Ironing out doc code blocks was around docs always needing to type check and that getting in the way of them being useful teaching tools
Sam Mohr (Jan 09 2025 at 23:03):But if we implement the ... keyword first, then we can probably take that pain away
Richard Feldman (Jan 09 2025 at 23:15):Luke Boswell said:
We don't want to support full markdown... at least I know Richard has mentioned we don't want to support html or anything like that in Doc comments.
specifically I don't want to support escaping to arbitrary HTML
Richard Feldman (Jan 09 2025 at 23:15):so that we aren't coupled to only being able to output docs to things that can render html
Richard Feldman (Jan 09 2025 at 23:15):e.g. if we want to be able to render in an editor that doesn't include a whole browser
Richard Feldman (Jan 09 2025 at 23:16):also yeah, I think if we have ... I'd be open to trying this
Sam Mohr (Jan 09 2025 at 23:21):As nice as this is, we should probably put it below everything else we're working on for 0.1.0
Sam Mohr (Jan 09 2025 at 23:22):But if someone really wanted to implement this, I'd be happy to help them do it! It'd be very valuable
Luke Boswell (Jan 09 2025 at 23:43):Sam Mohr said:
But if we're optimizing for getting code blocks type-checked and everything else is okay not worrying about, then we can run in a straight line, sure
My thinking here for why we might want to prioritise this part of it sooner than later is that it helps ensure documentation quality (particularly with all the breaking and syntax changes) across the ecosystem.
It's a bit of chicken and egg, but quality documentation for all the platforms & packages helps reinforce the culture around high quality roc packages, and for new people experiencing roc (who may not be experienced with the magic of Elm ecosystem)
I've been debating with myself how I would prioritise this, but I'm thinking it would be nice to have (just this code block part) before Richard's blog post and we expect a few new people to be visiting and checking things out.
So somewhere after |args| and "string ${interpolation}, but before we make the static-dispatch/abilities/module params etc changes.
Jasper Woudenberg (Jan 10 2025 at 07:33):Quick note in case it's useful: I know from integrating libcmark into my static site builder platform that it will by default escape HTML, only parse it out if you pass a special flag.
Sam Mohr (Jan 10 2025 at 09:42):We could maybe start with using a library, which would get us to the end result faster
Sam Mohr (Jan 10 2025 at 09:43):Just remember that Roc doesn't want to have any deps besides LLVM and Zig in the longterm
Richard Feldman (Jan 10 2025 at 14:04):and probably Blake3. I could see an argument for a markdown parser being a dependency though
Anthony Bullard (Jan 10 2025 at 14:31):You could find a minimal one with a liberal license and just vendor it
Last updated: Jul 06 2025 at 12:14 UTC