though as I said they look great, there are some mild improvements that maybe people would have an opinion on
take a function like this for instance (see attached if below), it's really not clear on first glance that the state, elem -> ... are the step function, rather looks like just repeated arguments (at least state).
suggestion - maybe parenthesize is or at least find a way to inline those at the docs gen?
the name of the function is clickable, which is convenient for creating a quick header link, but annoying when the function name is long and i just want to copy paste it with ctrl c.
suggestion- perhaps make a # before the function name for header linking? may not look as clean but much more ergonomic for sure
this im 90% sure saw an issue on it but just in case- some function docs are just missing, even if docs gen should generate them
this one is more involved and i for sure sure will be happy to help if you agree- it'd be nice if there was one simple example usage on every function
perhaps mildly unrelated- but maybe we should make something like this https://docs.python.org/3/library/itertools.html#itertools-recipes
not even in docs, but maybe in a separate page, for sure people will find it useful to both see some style from the roc devs and gain intuition on how to make such helper functions themselves. that would also make a clear space for these functions that are just on the line of being useful but not enough to be on stdlib
maybe parenthesize is or at least find a way to inline those at the docs gen?
I think parenthesis and parameter names would be amazing in the docs.
I started working on some of the docs stuff a little while ago but have paused for now. Some things to note; there are plans to change or remove some of the modules, like Json and maybe parts of Str like scalars. Also the syntax for Abilities needs further work to settle on a better syntax. There isn't much documentation for Abilities as a result. They are useable, but takes a while to figure out. I feel like we need someone to read the previous discussions and synthesize into a proposal which considers the various tradeoffs to progress Abilities. For syntax highlighting we are now at the point where we can probably make that work, the static-site platform uses the roc parser to tokenise code snippets which is what we need for docs examples. Another thing is I would love to know if there is a good way to package docs in with the URL tarball? maybe we could roc docs be able to boot into a local browser window the docs for a URL package?
*move modules out of builtins to separate packages, not necessarily remove
oh yeah the formatter for the types in docs is old and incomplete; it could definitely use some love!
if anyone's interested in working on that lmk, I'd be happy to get you started on it! :smiley:
To clarify, the parenthesis are there in the original definition, it's always been planned to support them to be "rendered"
walkFromUntil : List elem, Nat, state, (state, elem -> [Continue state, Break state]) -> state
perhaps make a # before the function name for header linking? may not look as clean but much more ergonomic for sure
How about a :link: symbol on hover like github does for markdown headers?
it'd be nice if there was one simple example usage on every function
:100: I love examples
I've made issue #5096 for the examples.
Our current example format with expect does add a lot of noise:
expect Str.split "1,2,3" "," == ["1","2","3"]
Perhaps we should just render that as follows:
> Str.split "1,2,3" ","
["1","2","3"]
So I got a bit carried away and started on the docs syntax highlighting. It was quicker than I thought, so here is the PR #5097 which should be ready for review.
but maybe we should make something like this https://docs.python.org/3/library/itertools.html#itertools-recipes
I've been wanting to start on a database/repo for roc examples/snippets for a long time.
Anybody any objections to creating roc-lang/examples?
It would be cool to have equivalent python, javascript... code below the roc example as well so that large language models trained on github learn how to translate python to roc :)
Luke Boswell said:
So I got a bit carried away and started on the docs syntax highlighting. It was quicker than I thought, so here is the PR #5097 which should be ready for review.
yooooo that's amazing! This has been an often-requested feature for years! :heart_eyes:
Anton said:
Our current example format with
expectdoes add a lot of noise:expect Str.split "1,2,3" "," == ["1","2","3"]Perhaps we should just render that as follows:
> Str.split "1,2,3" "," ["1","2","3"]
yeah I don't love mixing docs examples and test examples
l prefer "repl-like" version
The reason I started adding expect was to have working code examples that people could copy paste. Assumption was that expect would be eventually supported in REPL. I haven't managed to make highlighting work for the REPL examples, so in the tutorial those first few are still html and not a markdown code block.
Related idea... what if we could add expects underneath a function, and they were somehow associated with that function, so that we can render the docs and have a collapsed view which you can expand to see tests. Maybe these could even be "runnable" using wasm? Then we can add lots of tests which also form part of the documentation and can be checked in CI with roc test. Then we could keep code snippets for purely semantic explanation and dont need to necessarily be fully complete code.
This could resolve the discussion re # >>> xyz verse # xyz too. Anything that is a comment is assumed to maybe not be valid code, while anything that is actual code in a module should be complete and valid.
Or it would be cool if there was a way to see any test that uses a function. Then you could group them up and see all the related tests from the docs. It wouldn't matter where the test was defined etc.
I think that's potentially interesting, but I definitely worry about the incentive to just write tests and not give actual examples geared toward teaching
like "just read the tests, what's the problem?"
whereas good examples in docs have been chosen specifically to help you understand, and they're presented in a particular order, and there aren't an overwhelming number of them
none of which are necessarily true of good tests
That makes a lot of sense, we should be mindful of the intended purpose or outcome we want to achieve. Tests are for correctness, examples are for communicating.
| Module | Status | Comment |
|---|---|---|
| Bool | Good | All functions include description and example |
| Box | Good | All functions include description and example |
| Decode | Not Started | No documentation |
| Dict | OK | All functions have a description, most have an example |
| Encode | Not Started | No documentation |
| Hash | OK | All functions have a description, no examples |
| Json | Not Started | No documentation |
| List | OK | Most functions have a description and/or example, verbose comments |
| Num | OK | Some functions have a description and/or example, verbose comments |
| Result | Good | All functions include description and example |
| Set | Good | All functions include description and example |
| Str | Best | All functions include description and example |
expect (see example below)## Headings can be collapsed by default to have sub-sections in doc comments? Or should this kind of longer detail eventually migrate into a Roc book or something and be linked to?Encode and Decode if abilities syntax might change?Hasher? I'm not really sure what this is useful for.numbers : Set U64
numbers = Set.fromList [1,2,3,4,5]
values = [1,2,3,4,5]
expect Set.toList numbers == values
Module introductions are a good way to summarise key concepts
:plus:
I think it is nice when using top level variables followed by an expect
I'm not sure if I interpreted this correctly but I still believe in this comment I made earlier. It's very easy for beginners to get overwhelmed so I think there is a lot of value in simplicity in this case.
Maybe ## Headings can be collapsed by default to have sub-sections in doc comments?
I don't know of all the kinds of ## Headings but I think performance details should be collapsed by default.
Or should this kind of longer detail eventually migrate into a Roc book or something and be linked to?
Something like Performance details for a specific function seems suited best for the docs.
I would like to add a copy to clipboard for examples, possibly adding an app/interface header, so users can paste into a file and run the example.
Copy to clipboard button sounds great! I like the header so that it is self-contained and works out-of-the-box when copying but am also somewhat concerned about it adding noise if it's everywhere. Perhaps we could show them on hover or something.
What if we have a "REPL" code fence, and when you click a button it runs the code in WASM and prints the output?
:+1:
one quick request for copy-on-clipboard button: I've tried to make things gracefully degrade on the tutorial and docs pages for people who have JavaScript disabled, so I'd like to have JS actually create these elements rather than having them always there and just not do anything if you have JS disabled :big_smile:
Richard Feldman said:
I think that's potentially interesting, but I definitely worry about the incentive to just write tests and not give actual examples geared toward teaching
As a possibly interesting connection, this reminds of Pyret, a language made for teaching programming, which distinguishes between examples (where: blocks) and tests (check: blocks): https://dcic-world.org/2023-02-21/testing.html#%28part._from-examples-to-tests%29
Thats super interesting! Let's explore a little further...
One way we could do something similar is using comments on a top level expect. We already have # comments for internal focussed communication like "maintainers need to know this, but not users" etc, and ## documentation comments for external focussed communication like "users need to know these things". Only the ## get rendered into the generated documentation.
So what if a module could support the following documentation options;
## documentation is written in markdown and can have inline snippets and fenced code examples.unchecked, repl, with default to roc check code examples in documentation comments. ## comments before the module header act like an introduction or summary for the whole module.## comments before an individual statement document that statement.## comments before an expect can be tailored for the purpose of documentating a test for users. Worked Example.
The below example uses both Set.fromList and Set.difference functions so in theory could be linked to both of those functions.
## This test shows users how to use the `Set.difference` function.
expect
first = Set.fromList [Left, Right, Up, Down]
second = Set.fromList [Left, Right]
difference = Set.fromList [Up, Down]
Set.difference first second == difference
Sorry, cant figure out how to escape ticks `
Actually... somewhere along the way I think I completely diverged from examples. Sorry, I think I got caught up re-visiting previous ideas. Just trying to write things down to consolidate them in one place. The examples in doc comments work really well currently. I think I grabbed onto the idea of using expect for examples and ran with that. It started as an exploration of making examples, but I think the comments work well enough and I haven't found a compelling reason to do them like Pyret. The key idea I think above is the delineation between internal and external audience for a # verse ## comment.
I might stop here and let someone else get a word in... but one more thing to add. Apologies if I'm off on a random tangent here, but I think this is neat.
I would love the ability to hash the individual tests somehow and get a unique identifier that can be referenced externally (i.e. from the generated docs). I can imagine a future where a suite of tests can be used in an automated way for verification.
Suppose I have a model with the requirements for my software. I can now demonstrate each of them are resolved by referencing the relevant tests. If the code changes, that reference will be invalidated, indicating the requirement needs to be reviewed. This could be an efficient way to make a formal argument that a particular module meets some set of requirements or specficiations.
This would be most useful when selling of engineering work for a contract; I think this kind of automation supports a more agile approach instead of manually reviewing code or manually testing with each update.
That's almost like using the doc comment above the expect to "name the test", in the same way as you'd do with function names in other languages - except you can also use full english sentences/etc without it being awkward.
There's value in brainstorming but it seems a little early to me to focus on a verification-oriented feature.
Similar to the where block in Pyret I do like the convention of putting a few illustrative expect tests above a function.
I just started playing with the static files in crates/docs/src/static/ with hopes of moving the search bar to the header as requested in #5220.
(I figured I would wait to commit to implementing these changed until I was confident I was able to do so. Didn't want to discourage anyone else from working on it and possibly doing it better or faster than I.)
Unfortunately, iterating on this is slow because any change to index.html means I have to wait 11+ seconds to see the results:
$ hyperfine "cargo run -- docs crates/compiler/builtins/roc/main.roc" --prepare "echo "\n" >> ./crates/docs/src/static/index.html"
Benchmark 1: cargo run -- docs crates/compiler/builtins/roc/main.roc
Time (mean ± σ): 11.234 s ± 0.049 s [User: 0.858 s, System: 0.240 s]
Range (min … max): 11.191 s … 11.333 s 10 runs
What would happen if we generated the docs using roc glue instead? Would it be faster since we don't have to wait for rustc every time we make a change? If it would be faster, are there reasons not to do it?
This idea is inspired by the conversation in https://roc.zulipchat.com/#narrow/stream/304641-ideas/topic/multi-purpose.20glue
The workflow I'm imagining is: roc glue docs-gen.roc ./generated-docs ./crates/compiler/builtins/roc/main.roc
Considering that eventually the doc-generation code will be pretty stable, part of me is hesitant to suggest rewriting the code just for a use case that will eventually all but disappear. But perhaps there are other benefits that could make it worth it? Do we like the idea of being able to say "Roc is so good for writing static site generators that we generate our package documentation using Roc!" ?
:thinking:
Maybe instead of using an HTML template I could work on a module with an API that resembles Elm’s Html module but just spits out strings. Might be more work than it’s worth? How would performance compare to inserting values in a template?
Glue doesn't expose the information you would need to generate docs currently
So would take a lot more work on the compiler first.
a risk of doing that is that it means if we want to make a future improvement to docs generation, we have to expose the information necessary to do that to roc glue, which we many not want to :sweat_smile:
Maybe we could make it so that debug builds for this the docs module don't ingest the html files? Instead they load them from disk. That way instead of compilation time and pipeline, it is just copying some html files to the output directory or something (or even referencing them directly in the repo, no rebuild at all from cargos point of view)?
Otherwise, maybe we could look at the exact dependency tree here and fix that, though you will still have to linke the final roc binary which is slow. This is already pretty optimal. Basically all just time to link final binary.
Just ideas to speed this flow up
that sounds like a great solution! :100:
As a side note, this is where lld could theoretically help a ton. Sadly, there is no clean way to specify it anymore. It gets overwritten my our .cargo/config.toml. We really should work to get rid of all of these lines. They block all rust flags set by the user in their ~/.cargo/config.toml.
Yeah if you aren't sure you want to make roc glue a more general-purpose tool, then my idea doesn't make as much sense.
But if the generation time drops to a second or two, that should help a lot!
Is that a relatively low-effort change?
I'd offer to make the change myself, but I'm not sure I'm good enough with Rust.
Is that a relatively low-effort change?
Yeah, should be pretty easy
find all the include_str! and instead have a config that replaces them with file loading on dev builds. We will just load the files relative to target dir.
I'll open a draft PR. Would you like me to open an issue as well or nah?
Just go for the PR. If you don't think you can figure it out, file an issue.
https://github.com/roc-lang/roc/pull/5375
So much for the draft lol. It went quick.
I think that we'd benefit a lot from using Docusaurus, I can set it up if that's okay
What are the key benefits of Docusaurus that would make it valuable for our use case?
Setup structure, easy docs with Markdown (+ react, as it renders MDX), easy search integration with Algolia (which should be free for FOSS), the possibility of rendering the API docs with direct links to them from other pages, and also the possibility of compiling code blocks so examples are not out of sync with language versions
I'm gonna give it a go, I think that having a centralized place with docs of all kinds will benefit the community a lot.
The docs stuff sounds like something @Georges Boris was talking about recently.
iirc Richard had the intention of not using something for docs that would add lots of dependencies to the build step - but we talked about that a while ago, don't know if something changed!
that being said, my personal opinion is that it would be great to experiment with community projects around docs - having the content available (maybe through roc glue) so people could play around with rendering them using docusaurus, etc, or even building something new for roc - maybe they come up with something great that doesn't necessarily fit within roc's main repo.
Nikita Tchayka said:
Setup structure, easy docs with Markdown (+ react, as it renders MDX), easy search integration with Algolia (which should be free for FOSS), the possibility of rendering the API docs with direct links to them from other pages, and also the possibility of compiling code blocks so examples are not out of sync with language versions
we actually already have docs rendering from markdown!
e.g. https://www.roc-lang.org/builtins/Str is rendered from https://github.com/roc-lang/roc/blob/main/crates/compiler/builtins/roc/Str.roc
two things that are very important to me about docs:
roc edit to have to ship an entire HTML renderer :sweat_smile: Algolia (which should be free for FOSS)
this is very interesting! I didn't know Algolia had a free plan - we should look into that for search across packages
Richard Feldman said:
Algolia (which should be free for FOSS)
this is very interesting! I didn't know Algolia had a free plan - we should look into that for search across packages
Yeah. I'm using Docusaurus at work in production. The decision was taken after many weeks of research. It is a solution that gives a lot out of the box, while maintaining flexibility.
I'll give this a go :)
from https://www.algolia.com/for-open-source/
Screen-Shot-2023-08-09-at-7.47.22-AM.png
What’s included in the Algolia for Open Source (AOS) plan?
Standard PAYG plan and to include 200 units for free which translates to 200,000 records and 200,000 search requests (and up to 2,000,000 indexing operations) per month (a $180/month value).
so 200K search requests per month
I wonder how many monthly requests something like crates.io gets
200K sounds like a lot but I don't really know when you factor in things like bots and scrapers
a thing I want to be careful about is getting committed to something that has a limited free option, and then when we cross that threshold we suddenly need to either:
Maybe we can use algolia for docs search and figure out something else for packages?
Having something like Hoogle setup shouldn't be hard. Also, indexing content in a Vector Store and searching through embeddings is something feasible. We did something like this for the documentation of the open source framework we made at work: https://docs.boosterframework.com/ (Let me know if I should remove the link)
Nikita Tchayka said:
Maybe we can use algolia for docs search
as in searching within a particular package's docs?
we have an in-page search option currently https://www.roc-lang.org/builtins/Str
As in searching the site:
roc-things markdown fileand so on
oh
wouldn't a general web search work for that? :thinking:
personally I never use site-specific search boxes like that
Me neither (although I must admit that algolia works really well), that's why I pushed at work having a chat thing that teaches you how to do the things that you ask it to do :laughing:
yeah I think in that case I'd rather not take on the dependency (and possible future emergency rewrite)
but I don't have a good solution in mind for package search
Algolia might be a good fit for that unless we run into bots/scraping issues with the limit
I'm not sure what could be done about that
elm-packages has a good solution for that which works at a sufficiently small size, which is that it does everything in the browser
because the entire list of packages is in the thousands, so they can just be sent to the browser in a JSON blob, and then the searching is done in JS
Makes sense
but that stops working at some size of package repo haha
Yeah, we can think in the search part later on then. Imma make the spike that I wanted to do without anything related to search so you can give me your opinion :)
the spike on which part?
Making a centralized hub with Docusaurus
with everything we got in the website but a bit more accessible, I had trouble navigating things yesterday when I started learning about Roc
Once you know where's everything, it is easier, but it takes a while :sweat_smile:
ah gotcha
so full disclosure, I don't think Docusaurus is something I'm open to having on roc-lang.org
I do think making things on the website more accessible is a good goal though!
have you seen https://www.roc-lang.org/wip btw?
it's the WIP version of the next version of the website, which has a goal of making it look more like a normal programming language website in terms of navigation etc :big_smile:
the "docs" link at the top right corner being an example of that
there are more details about that project in https://roc.zulipchat.com/#narrow/stream/316715-contributing/topic/2023.20website.20update
Richard Feldman said:
so full disclosure, I don't think Docusaurus is something I'm open to having on roc-lang.org
Got it! I guess that you mean so because of the dependencies that it brings in, which I agree. I was thinking in using Docusaurus as a first stage that would enable us to reach the goal of having a site faster without having to think in stuff like responsive design, etc...
E.g. this is how it looks on mobile (absolutely not bashing anyone, these things are hard, and we do them in our free time)
I could try working out a layout, if that's okay for everyone involved in the new website
oh yeah totally! Luke posted about the status of it here:
Luke Boswell said:
I want to give an update on progress with the 2023 website for Georges Boris Éber Freitas Dias and anyone else who is interested in contributing. PRs most welcome :smile:
We have made a start on the basic structure for the new site which for now is sitting at
www/wip_new_websiteon main, and includes a local build script usingroc run build.roc. It is generated using the static-site-gen platform which gives us Roc code syntax highlighting and enables most of the content to be written in markdown.Ideally we will update the site prior to Roctoberfest, as this supports that activity and in preparation for 2023 Advent of Code.
The new site has the following structure;
- index home page, minimal content with links to other pages
- design goals describe the design and user requirements for the language and ecosystem design
- tutorial to guide people new to Roc through the basics
- install quick start for getting and using Roc
- examples a collection of code which show Roc in action
- community information about the community and contributing
- sponsor information about how to sponsor Roc
- docs links to the documentation and guides
The initial focus has been to try and scope out what should be included. For now this basically just pages and headings which loosly define structure, and are supported by dot points or comments to capture ideas. Richard provided input here so we have a reasonable starting point.
There is a lot of things that need work. For example there has been almost no work on styling or content. It is still very much a work in progress and all contributions welcome.
Some ideas for things that would be hlepful;
- styles and layout for the pages to look good
- integrate the web REPL for the front page
- write a short language reference (move some things out of tutorial e.g. de-sugaring table, keywords)
- write a brief explanation of platform/application abstraction versus libraries as common in most other languages
- write a brief summary of the current state of using Roc for Tools & Scripts, Web, Networking & Servers, Graphical, Scientific, Embedded etc
it explicitly doesn't have real styling yet; if you wanted to work on that, that would be amazing! :heart_eyes:
Richard Feldman said:
oh yeah totally! Luke posted about the status of it here:
Luke Boswell said:
I want to give an update on progress with the 2023 website for Georges Boris Éber Freitas Dias and anyone else who is interested in contributing. PRs most welcome :smile:
We have made a start on the basic structure for the new site which for now is sitting at
www/wip_new_websiteon main, and includes a local build script usingroc run build.roc. It is generated using the static-site-gen platform which gives us Roc code syntax highlighting and enables most of the content to be written in markdown.Ideally we will update the site prior to Roctoberfest, as this supports that activity and in preparation for 2023 Advent of Code.
The new site has the following structure;
- index home page, minimal content with links to other pages
- design goals describe the design and user requirements for the language and ecosystem design
- tutorial to guide people new to Roc through the basics
- install quick start for getting and using Roc
- examples a collection of code which show Roc in action
- community information about the community and contributing
- sponsor information about how to sponsor Roc
- docs links to the documentation and guides
The initial focus has been to try and scope out what should be included. For now this basically just pages and headings which loosly define structure, and are supported by dot points or comments to capture ideas. Richard provided input here so we have a reasonable starting point.
There is a lot of things that need work. For example there has been almost no work on styling or content. It is still very much a work in progress and all contributions welcome.
Some ideas for things that would be hlepful;
- styles and layout for the pages to look good
- integrate the web REPL for the front page
- write a short language reference (move some things out of tutorial e.g. de-sugaring table, keywords)
- write a brief explanation of platform/application abstraction versus libraries as common in most other languages
- write a brief summary of the current state of using Roc for Tools & Scripts, Web, Networking & Servers, Graphical, Scientific, Embedded etc
Awesome!
Richard Feldman said:
it explicitly doesn't have real styling yet; if you wanted to work on that, that would be amazing! :heart_eyes:
Absolutely
Do we have GH Issues labels for tracking website tasks?
No, I don't think so
I'll add a README to the wip_new_website folder to make it easier to get started.
Anton said:
No, I don't think so
Would be lovely to have it so organization is super easy :blush:
I'll get on that after I'm done with the README
I was thinking about including this chart (not the verbatim screenshot, but something that includes that) would that be okay?
It's a convincing chart, but I'm not sure it sets expectations appropriately. We don't have performance like that on every use case. What do you think @Richard Feldman?
Maybe add an "our goal is to make Roc fast"
also that chart was made a long time ago. To make it public now we should re-run the benchmarks, maybe make them public/reproducable too?
I have to head out for 30 minutes
yeah I think let's leave that chart out - it was the first significant benchmark we had, but we have a lot more benchmarks now and we've found several cases where we're not that good, so I'd rather not give people the wrong impression that we're always that fast (yet!)
about algolia + site specific search. I'm a huge fan of tailwindcss's website+docs. They use algolia and it makes a world of difference with what we would get by searching through google - I keep their site whenever I'm working on something new.
even though I mostly use it for docs, not guides, maybe Roc could benefit from something like that - we have a lot of ground to cover that are not really "packages": language features, using platforms, creating platforms, techniques(?)
(I just thought about techniques but when working with Elm we have a lot of common techniques that are tied to the language - would be amazing to have that in a more structured way alongside the main guides)
(maybe Roc's techniques would overlap with Elm's in some areas but the language has quite a few different possibilities so there will definitely be Roc-unique stuff worth collecting)
algolia likely relies on a vector database of embeddings. I wonder if we could create a vector db file that is updated nightly and hosted on github. The roc CLI could be equipped with the ability to search through this file and download it on demand. If the vector db file is small enough we could even include it with the regular nightly.
I've started a github project for the new website to group the issues.
Would you like the responsive design issue @Nikita Tchayka?
To be honest, I have spent the day tinkering with CSS and webdev and the task goes far beyond my estimation. Initially I wanted to use Docusaurus because everything was set up and it was a matter of tweaking fonts and colors, but I'm not gonna continue with the task as I'm not that experienced in frontend dev to do so, specially without a framework
fair enough!
I might make a reference page for myself tho
Last updated: Jun 16 2026 at 16:19 UTC