We don't currently have an issue on GitHub owning doc comment code blocks, e.g.
## Repeat a string n times.
##
## ```
## echoed_words =
## Str.repeat("hello", 3)
##
## expect echoed_words == "hellohellohello"
## ```
There are a few things we should iron out before we make this available for implementing by contributors.
First, we want to be able to validate that they compile at all. I think we should parse the contents of doc comments as top-level code that can't be exposed or used by the rest of the module, and report warnings/errors just like the rest of the module. The only difference is that docs sometimes have top-level expressions, which aren't currently syntactically legal outside of docs. Should we allow this just in docs, or should we force users to put those values in defs? I vote the latter because it probably only increases readability.
We also want to make sure that doc code blocks with top-level expect
s have those tests run. I think these tests should always be run with roc test
. We could optionally have a --ignore-docs
flag for roc test
if we want to allow only running non-doc tests, but I think we shouldn't do this unless there is a lot of demand.
Rust has the ability to hide lines of docs by putting a second set of doc comment prefixes, e.g.
## ```
## ## import Hidden
## Hidden.run_func(123, "abc")
## ```
We should allow hiding doc lines in the generated code, but they should be otherwise treated exactly the same as normal doc code block lines.
Lastly, if a doc should not be treated normally, we should allow using annotations like in Rust to explicitly handle these cases in a trustworthy way. Syntactically, we would put a word right after the starting triple-backtick of the code block as one would notate the language in the block. All of these should be accompanied by a hint in the generated docs. I think the following annotations would be good:
has_warnings
asserts that there is at least one parse/type/etc. warning in the code. If not, roc check
fails.has_errors
asserts that there is at least one parse/type/etc. error in the code, warnings are allowed but there can be zero or a hundred and it doesn't matter. If not, roc check
fails.should_crash
still ensures that there are no parse/type/etc. errors in the code block, but ensures that when run, the block must crash
. If it doesn't, roc test
fails. (Maybe roc check
could catch this with constant folding, meaning we would just need has_errors
.)There are some other annotations that Rust allows, but I think we shouldn't allow them:
compile_fail
is a more general version of has_warnings
/has_errors
, I think we can be more specific on this.ignore
is a very general "don't worry about it" annotation that allows bucket ignoring, we should force more specific annotations.no_run
makes sense for effectful code, but we can't run effectful code unless it's using Store.get!
and Store.set!
in test environments.edition20XX
sets the Rust version that runs, but we can just use the proposed version of Roc at the top of the package.If the above plan is accepted, I'll some GitHub issues. First a parent issue for organizing these features into a single place, and then the following children:
expect
s in doc code blocks with roc test
"## ##
"I like the idea of making sure documentation examples compile and run as they should!
I'm not sure how I feel about the annotations, I've not used those before. Do you have an example of some Rust documentation you like that uses these annotations to good effect?
GitHub search didn't help that much. I'm trying to remember the examples I've seen. The main one is should_panic
but I'm not remembering where I saw it. I'll work on it
https://doc.rust-lang.org/rustdoc/write-documentation/documentation-tests.html#attributes
This is the docs on how to use attributes, but not an example that shows they should exist
here’s one example: https://github.com/ayazhafiz/xorf/blob/108d67903f2df6db6dcd1c05c571594c529413d6/src/hash_proxy.rs#L65. it’s kind of useful for checking you’re actually in the bounds of the language. not sure it’s super useful.
I have mixed feelings about this in general. I would definitely stay away from attributes for now until there is a clear need. With regard to checking that programs are well-formed, I am biased towards it being opt-in. There are two reasons for this.
yeah I think this is a topic with nonobvious tradeoffs
the number one goal of docs is teaching
and sometimes the best way to teach is to give examples that don't fully work - e.g. you have a chunk missing and put in a comment like # this is where you put the thing
that can be optimal for teaching even though it doesn't compile
separately, a thing I don't love about docs having tests get run in them automatically is that it means they now have a split focus. You're no longer trying to optimize only for teaching, but also for your test suite
Though we generally can do well with putting a valid but obviously placeholding value, e.g.
Str.repeat("YOUR VAL HERE", 123)
sometimes those happen to be 1-to-1 but sometimes making the test better makes it a worse example
which creates an unhealthy tension when tests and docs have been coupled
Richard Feldman said:
separately, a thing I don't love about docs having tests get run in them automatically is that it means they now have a split focus. You're no longer trying to optimize only for teaching, but also for your test suite
I think we can still set the standard of "doc code blocks are valid Roc, even if some lines are hidden". You don't have to put any expect
s. If we don't even ensure that doc blocks are syntactically correct, then readers have much lower confidence that they can apply what they see.
I hear you and I agree, though I wonder if this is a non problem. I’ve very very rarely run into issues like this (only two in my head) and in both cases you can ask the author what’s wrong. I think if someone is already taking the time to write documentation, and moreover include code, they’re going to do so correctly
sorry by correct i mean syntactically valid
I don't agree
I just got a compiler panic when updating Weaver because I forgot to update a function name buried in a 100-line module header that didn't get changed from camelCase to snake_case
I have come to strongly appreciate tools that take away mental load
And having doc code blocks tell me "If it compiles, it's valid Roc" means I just don't have to worry about it
I think every developer has felt the pain of documentation always going out of date, which is why "self-documenting code" is so touted in our industry
I think you're right that the first time you write the docs, they'll probably be correct
But lots of PRs in our Roc compiler repo just move old docs around
I also put a lot of value in the guaranteed correctness of doc code blocks.
And I don't push on people to update them every time because a) it happens so often, and b) I'm jaded because I tend to devalue docs since I know they're somewhat likely to be wrong
I also put a lot of value in the guaranteed correctness of doc code blocks.
It feels like a boost for the entire Roc ecosystem
Also
I’ve very very rarely run into issues like this (only two in my head) and in both cases you can ask the author what’s wrong.
Most code I work on and don't own is libraries owned by people I've never talked to before. I don't think it's feasible to expect users to reach out to code owners
Sam Mohr said:
I just got a compiler panic when updating Weaver because I forgot to update a function name buried in a 100-line module header that didn't get changed from camelCase to snake_case
Can you elaborate on this? I don't follow the connection to docs
When I say "module header", I mean:
## Welcome to Weaver!
##
## Use [oldFuncName] to do a thing.
module [weave, combine, ...]
got it
okay definitely +1 to the case of you have hyperlinks and making sure they actually link
I also have a lot of docs that I almost broke. Weaver has 39 (13 num types times 3) basically copy-pasted functions that parse numbers as args. All of them need to be manually updated
can those be unit tests?
## Add an option that takes an `I128` and can be given multiple times
## to your CLI builder.
##
## Parsing arguments will fail if any calls of the option don't provide
## a value, or the values are not all numbers.
##
## ```
## expect
## { parser } =
## Opt.i128List { long: "answer" },
## |> Cli.finish { name: "example" }
## |> Cli.assertValid
##
## parser ["example", "-a", "1", "--answer=2", "--answer", "3"]
## == SuccessfullyParsed [1, 2, 3]
## ```
It's a type of denormalization
I think that localizing all code examples to a common set of tests makes my code cleaner, but it adds layers of indirection to someone using Weaver
So it's better for me to copy the same example to every function in the module so that it's immediately obvious when you hover the function how to use it
Meaning I have dozens of code examples that can easily go out-of-date
I think this is something that normalization could solve very well and probably be better than requiring you to copy the examples everywhere
Having docs generation lift up expect tests to the docs
Maybe some annotation to highlight the test in the docs or something
Not saying that's the answer but it feels like a better direction than copy/pasting the same docs and checking they're up-to-date - you only have one site to update
I agree that it feels like there's a better solution, but I can't think of one that isn't really fancy or makes the API worse to use
But I'd love to do that
Let's ignore this being a test for a minute
## Add an option that takes an `I128` and can be given multiple times
## to your CLI builder.
##
## Parsing arguments will fail if any calls of the option don't provide
## a value, or the values are not all numbers.
##
## ```roc
## { parser } =
## Opt.i128_list { long: "answer" },
## |> Cli.finish { name: "example" }
## |> Cli.assert_valid
##
## parser ["example", "-a", "1", "--answer=2", "--answer", "3"]
## == SuccessfullyParsed [1, 2, 3]
## ```
Let's say this is the code example.
I'd really like if code examples get type checked and all.
Just a quick note, not interrupting you:
I still think long-form docs with a bunch of examples are more of an education tool as Richard mentioned and I think some opt-in checking is useful but I don't think it should be the default.
I would also suggest maybe punting this until there are many more packages in the ecosystem and seeing what the common patterns across the ecosystem look like
Would you be opposed?
Okay, we can punt
I think you could do it with just two block tags “hide_header” and “hidden_module <modulename>”
I think it's fine but (1) I think that example should be an expect and (2) it should be opt-in if you have an example you want to type-check
I'll have to go look for some doc-based code examples that I feel would be better off not being type-checked
So using triple backticks, roc as the language, and then one of those. The first would hide the header of whatever file it it’s, and the second would not show the block at all but add the module as one that could be used by other blocks in the same document
Hard to show here
Because Richard's example of xyz = # your answer
can usually be done with either of these
xyz = "your answer"
-- or
# your answer below
xyz = null
And then we get that mental burden lowering of "code examples all type check unless they say they don't"
Anthony Bullard said:
So using triple backticks, roc as the language, and then one of those. The first would hide the header of whatever file it it’s, and the second would not show the block at all but add the module as one that could be used by other blocks in the same document
This has the disadvantage of making the docs less markdown-like, since you can no longer read them contiguously, but I'd personally be okay with this.
@Ayaz Hafiz I'm much less experienced in the guiding of a language's development, so I'm curious about a metaprocess you seem to have running here. You suggest we should punt this until we have more ecosystem code to base our decision on, which mirrors what Richard has said for some of the other stuff I've suggested recently. I generally am trying to get ahead of problems I see in other languages with all aspects for Roc, but between:
I've gotten pushback on that approach. Do you think you know of a conscious heuristic (even a rough one) I can apply to figure out when we need to wait to solve a problem in Roc's design? I don't like the idea of making everyone update their code until after a problem is confirmed to exist, instead of just preempting it.
First off, I apologize if I made it seem that this is not worth discussing. That's not my intention. I think everything is worth thought and I don't think anyone has say what's worth discussing vs not
Not interpreted that way. You've been extremely courteous the whole time I've been working on Roc!
I see this now mostly as an optimization problem to get the language to a position of stability from where it can grow. Rewinding a few years, we added a lot of features in a short time. For a while I thought (esp. with Richard and Folkert) we could continue developing and fixing bugs at a consistent rate. Unfortunately, I think what has happened is that we sprayed too wide, too fast, and our time availability went down significantly. The outcome of this has been a lot of bugs and few solid foundations to stand on. Things have gotten much better since e.g. 2021/2022, but as you know, there is still a lot to do to get the foundational core of Roc to work correctly. I'm happy to share particular instances of this in my mind if it's helpful. Regardless, my opinion is that the optimization problem should be to develop a good, stable, working foundational core first, and then branch off from that.
I understand the problem you're talking about with stability, no need for examples, I see it when I work on the compiler
And when I did AoC last year
There are a few advantages of doing one thing right before branching off. The first is that you build a foundation people can build really sophisticated things on top of (like you have with Weaver) and then figure out how to support the emergent use cases better. In my experience trying to find the emergent use cases ahead of time is really hard, and you often have a few false starts. The second is that you give people a foundation that they can work with productively. People use the software and have a good time, and the discussion switches from "when will this stop crashing" to "how do I make this use case better". Roc is more in the latter direction now which is good, but the former still occurs way too much for what should be simple tasks, IMO.
Okay, fair. Between the monomorphization rewrite, lambda set rewrite, canonicalization rewrite, and ROAR, we're doing a lot of work to get a very robust core!
Okay, that's a great explanation!
We're trying to avoid buying a new car while we're still behind on our credit card payments
We've paid off most of our bills, but we still have a credit score of 500
Thanks for taking the time to explain
The third, and what I think is most important, is that keeping the surface area small and getting it right the first time gives a better basis for where to branch off. If you need to backtrack you only need to throw off a leaf rather than rebuild everything (or rearrange the leaves when you realize the core is bad). Also, when you have a core that isn't going to change much and can see what sophisticated things people are building on top of it, it gives you better ideas for what actually matters. A good example of this the new effects/for loop/imperative stuff. I think it's very hard to have seen that if you didn't have a lot of Roc code to look at and reflect on what was difficult about it for people
Yeah the car analogy is good
Fair.
Also just as a personal reflection/quirk/whatever, for me it's very difficult to understand the consequences of features without people using them. For me it's much easier to talk to someone and figure out what their pain is and solve that than to try to predict the pain. I'm probably just not smart enough to see the pain ahead of time, but sometimes the pain is invisible. And to predict an invisible pain I think is really difficult
Maybe I've been using too much Arrakian spice
I try to apply YAGNI (You Ain't Gonna Need It) when doing application dev
But I guess that instinct doesn't kick in as much with language dev because I assume that we're trying to handle future problems
But I guess Roc has found success in reacting more than proacting
yeah, i mean it's also fun right
So I'll try to lean more towards it
for me it's more fun to think of what could be than to reflect on what's not going well
eh, definitely not saying you should lean more towards it
but hopefully explains my perspective
It does explain your perspective
Ayaz Hafiz said:
eh, definitely not saying you should lean more towards it
I find success in life applying virtue ethics: the philosophy is to align yourself and your decisions on good principles until they become second nature. I find it leads to less need to resolve conflict between what I do and what I should do.
As a result, I tend to naturally make decisions I'm happy with, but have less practice holding competing ideas in my head and letting them "weigh on the scale".
So I prefer to align on principles, but it's good practice to hear your opinion, disagree, and figure out how to manage that.
What's our conclusion here? I definitely want at least a minimal version of this that we can use to test the snippets in the tutorial in CI. It looks terrible for Roc if we have code that doesn't work in the tutorial.
To be fair, that's a different part of Roc, right? That means we should add a GH issue that's called "Allow testing code blocks in roc markdown snippets"
"Allow testing code blocks in roc markdown snippets"
That's already implemented :) I need the ability to hide imports etc.
that's a different part of Roc, right?
Not really, there is a lot of shared functionality
For the features outlined in this thread, I would vote that we at least add some opt-in behavior to start the implementation with. I think that 90% of the work is parsing and typechecking doc comments, I presume we only highlight syntactically based on my memory.
@Anton Roc code blocks can't hide lines with the same syntax, since ## ##
is just for doc comments
Not sure what the right syntax would be
what if we just try out whatever seems best for the tutorial, but don't enable it in the compiler in general?
Oh, I know
like focus on the use case of the tutorial, defer the question of what should be in the language proper
I'm okay with that goal
@Anton do we need to hide imports for the tutorial?
I'd like them to be viewable with a toggle
We could just allow something like
Some tutorial stuff
```roc,hide_imports
import Hidden
Hidden.call_func(123)
Should be simple enough to implement
Then it just affects the generated HTML but not the Roc code, so we can still typecheck and all normally
I do think we'd want specific control over what lines to hide, like we want to able to hide the definition of something
for this snippet that's currently in the tutorial:
stoplightColor =
if something > 0 then
Red
else if something == 0 then
Yellow
else
Green
We could do a Jupyter Notebook thing: all markdown blocks in a single file are part of the same module
And then we have a toggleable
(name is flexible) annotation that makes entire blocks toggleable
That gives us control per block without needing a special new Roc directive just for markdown
something I like to do decently often is to have like a ...
in examples to omit things - could support that with a find/replace of …
(the actual Unicode character) with (crash "")
before compiling, so those always type-check
although that could result in compiler warnings of unreachable code :thinking:
Hmm, that's an interesting idea!
Do we have a precedent for any compiler warnings/errors that are there for dev but not production?
Wait, what am I saying
I think having ...
be the Python equivalent where it means "this is a stub of some body" would be a great feature to have
That would let people do
xyz = ... # your val here
-- Or just this by convention
abc = if True then ... else 123
And then we have a syntactically valid way to have your incomplete code blocks that still typecheck
Making it easier for us to typecheck and test doc code blocks if we go that road in the future
https://python.land/python-ellipsis
In maybe 45 minutes, I'll make a thread for this. It could work with inline annotations for host functions
Just for my curiosity, I'm wondering if people have preferences on formatting hidden code lines. Rust doesn't have real problems besides looking silly with hidden code lines because whitespace isn't significant there.
/// Split a string.
///
/// ```
/// /// fn test_splitting() {
/// let x = "a b c";
/// /// let delimiter = " ";
/// let split = x.split(delimiter);
///
/// assert_eq!(&split[..], &["a", "b", "c"]);
/// /// }
In Roc, whitespace is important, and parsing fails with incorrect indentation. The current suggestion is one of these two options:
## ```
## my_func = |str|
## ## import Helpers { str } as H
## x = H.build(123)
##
## str.repeat(x)
## ```
## ```
## my_func = |str|
## ## import Helpers { str } as H
## x = H.build(123)
##
## str.repeat(x)
## ```
I'm not super happy with either, but the first is more readable IMO for the source markdown, though both would generate the same HTML docs. Is there a third option, or do we have to pick from the lesser of two evils. If so, which evil?
We don't need to decide until this discussion is picked back up
right-hand hide?
## ```
## my_func = |str|
## import Helpers { str } as H # @@hide (or whatever name)
## x = H.build(123)
##
## str.repeat(x)
## ```
`
```
That's a solution if we are okay opening the door to add directives to comments.
Actually, I think putting doc comments after the line could work
## ```
## my_func = |str|
## import Helpers { str } as H ##
## x = H.build(123)
##
## str.repeat(x)
## ```
this is docs specific right? ##
on the left is as much a directive
It's pretty subtle
Yeah, I agree
Just because something is visually indistinct from other Roc tools, it doesn't mean it's not totally different under the hood
But it would be nice to not invent a totally new feature for this
another hot take but i wonder if supporting hiding is a good idea. I think it makes sense if you have multiple code blocks that flow naturally from one to the other but are distinct for some reason. But for that maybe it's better to squash them altogether and run check over that. The downside of hiding is you can't copy-paste and have the code run correctly. If the code block is so complicated that it needs hidden pieces, maybe it's a bad API.
Probably true. The only case I care about is hiding imports, but we could just always have imports collapsed by default and toggleable, and imports would be included if you copied the block
I think we should start without code hiding for the reasons you point out
im curious of an example of a code block that has a lot of imports, if you have one in mind
I'm looking across GitHub and I'm only seeing a few imports per example, so you probably have a point. They just don't have teaching value IMO, they only make copy-paste-ability better
So hiding them cleans up the code, but it's not necessary
Or should I say, it makes examples more concise
We can't squash all the code blocks together in the tutorial, we'd have duplicate name errors.
Last updated: Jul 05 2025 at 12:14 UTC