Richard Feldman (Jun 29 2023 at 00:11): Richard Feldman (Jun 29 2023 at 00:11):if anyone's interested in setting up a VS Code Extension for Roc that does type-checking and such, here are two ingredients that could be used to do it!
I, for one, would personally use this if someone wanted to make it :smiley:
Richard Feldman (Jun 29 2023 at 00:49):or even better, contribute it to @Ivan Demchenko's https://github.com/raqystyle/roc-vscode-unofficial
Brendan Hansknecht (Jun 29 2023 at 01:06):Is an LSP used for syntax highlighting in vscode or does it use treesitter or something else for that?
Ayaz Hafiz (Jun 29 2023 at 01:15):LSP can provide syntax highlighting but mine doesn’t (currently) (though it would be easy to add via the parser)
Ayaz Hafiz (Jun 29 2023 at 01:16):you can also have an extension that highlights via tree sitter
Ayaz Hafiz (Jun 29 2023 at 01:16):fwiw you do not need an extension to use the LSP, you can add it to VScodes lsp settings directly
Richard Feldman (Jun 29 2023 at 01:27):intriguing! I couldn't figure out how to do that :sweat_smile:
any tips?
Ayaz Hafiz (Jun 29 2023 at 18:11):maybe I was wrong about custom binaries without an extension
Ivan Demchenko (Oct 24 2023 at 06:57):Dear all,
I'm happy to announce a new version of 0.1.0 of the VSCode extension for Roc! :tada: Kudos to @Ayaz Hafiz for his PR to integrate the language server he's been working on :exploding_head: into the extension.
Currently, the extension doesn't come with the LSP binaries. So, there's a bit of a setup required.
@Richard Feldman I guess, we could follow the GTP 4's suggestion if the LSP binaries were available for distribution as part of the extension. I say so because I think it would be beneficial to separate the build steps (IDEs vs LSP). Alternatively, the LSP bins could be included as part of the Roc package.
Luke Boswell (Oct 24 2023 at 07:22):Screenshot-2023-10-24-at-18.19.05.png
I'm a total noob with vscode extensions. I'm not sure If I'm using it right. I built the binaries (both release and debug) set those in the extension settings. I can see Roc code is highlighted. But I'm not sure if I need to start it?
Ivan Demchenko (Oct 24 2023 at 09:51):Luke Boswell said:
Thank you Luke for your bug report. This should not be happening. I'll fix it ASAP.
Luke Boswell (Oct 24 2023 at 10:23):Cool, glad to help. I thought that was me. :upside_down:
Ivan Demchenko (Oct 24 2023 at 11:13):Okay everyone,
Good news! The bug was fixed, and the new version 0.1.1 is out in the wild.
Thank you @Luke Boswell for reporting this.
Ivan Demchenko (Oct 24 2023 at 13:41):I just implemented the automatic release process, but it triggered a version bump. So, the version of the extension is now 1.0.0. Well, technically, there's no difference. But I decided to let you know if case you wondered :)
Erik (Oct 24 2023 at 19:43):if anyone got overexcited like me and built a roc_ls binary yesterday and got an error on startup about module_id or something like that... just pull and build the roc_ls binary again and it should work straight away :)
Richard Feldman (Oct 25 2023 at 00:15):coooooool :heart_eyes:
Screenshot-2023-10-24-at-8.15.01-PM.png
Richard Feldman (Oct 25 2023 at 00:16):amazing work @Ayaz Hafiz!!!
Richard Feldman (Oct 25 2023 at 00:22):@Ivan Demchenko what are your thoughts on bundling this into the extension, so people can install the extension and have it Just Work without needing to set up anything else or configure anything?
I have some experience distributing the Roc compiler binaries in npm from roc-npm - is there overlap between how npm packages things and how vs code packages extensions? :thinking:
Luke Boswell (Oct 26 2023 at 20:32):Super minor nit, but I thought I would share the below. Where does the extension manage syntax highlighting? Is it using the tokens from the roc parser or just matching on strings? Happy to help if you can point me in the right direction.
Screenshot-2023-10-27-at-07.31.00.png
Hannes (Oct 26 2023 at 23:56):It uses a textmate grammar which is essentially a bunch of regexes
Richard Feldman (Oct 27 2023 at 00:00):I already made one PR to adjust those regexes - it looked like this: https://github.com/raqystyle/roc-vscode-unofficial/pull/9
Hannes (Oct 27 2023 at 00:13):We're still parsing the single quotes wrong though, I'll make a quick PR now to fix that
Richard Feldman (Oct 27 2023 at 00:59):I also noticed we don't color backlash escapes differently (in either single quotes or double quotes) for string interpolation - I'd love to see it color the \( as a different color from the rest of the string literal (and the ) at the end too), and then have everything inside the parens highlighted like a normal expression (since that's what it is!) but I have no idea how hard that is :big_smile:
Hannes (Oct 27 2023 at 01:02):PR submitted: https://github.com/raqystyle/roc-vscode-unofficial/pull/16
Hannes (Oct 27 2023 at 01:06):I believe adding support for string interpolation syntax shouldn't be too hard. Textmate grammars have a way to reuse patterns, so you'd define some patterns for normal syntax and then include them in a "string interpolation" pattern, and then include that pattern in the "string" pattern
Ivan Demchenko (Oct 27 2023 at 07:08):Thank you all for your amazing contribution! The fix is live :tada:
Ivan Demchenko (Oct 27 2023 at 07:12):@Richard Feldman I'd love the extension to come with the LS binaries. The LS code is on the branch that @Ayaz Hafiz started. The question is where to get them from. We have a few options.
IMHO, Option #2 has more benefits than #1 as it would help with the Roc's adoption rate. People download Roc and get the LS too. They wanna try the language, not install an editor.
Hannes (Oct 27 2023 at 07:52):I suspect the language server would've been part of the Roc binary if it wasn't for the plans to make a Roc editor. I don't know if those plans have changed significantly enough to reconsider adding official LSP support?
Anton (Oct 27 2023 at 10:18):I believe we indeed previously decided to go with an official LSP. Bundling the binaries seems to make sense, especially at this time, otherwise it would be difficult to keep both in sync.
Richard Feldman (Oct 27 2023 at 12:43):I actually think #1 would be the better option. Thinking ahead, in the future I want to have a libroc C library that can provide a superset of the LSP functionality (including being able to run Roc plugins written in userspace) with debugging support, hot loading, no JSON RPC overhead, etc.
I've looked into how this can work with all the popular editors, and it seems doable!
In that world, we'd need the extension to talk directly to the libroc library rather than the roc binary, so I think it makes sense to build toward that world by bundling the lsp library with the extension now, so that in the future we can more easily swap out what's in the bundle with something else!
Richard Feldman (Oct 27 2023 at 12:43):thoughts?
Ayaz Hafiz (Oct 27 2023 at 13:02):I think option 2 is strictly better from a practical perspective right now. You can integrate LSP with effectively any editor; the same is not true with a C library-that requires a lot of effort on a per editor basis, and we know that it will not be available any time soon. Having a separate binary/library also means that it is far too easy to let the ls binary drift from the compiler binary. There are other ways to solve this, like a package manager, but that will also not be available any time soon.
Ayaz Hafiz (Oct 27 2023 at 13:03):Even with a C library, integrating into the binary seems useful. You can query the binary directly to tell you the location of the dylib, rather than having to manage the version issues separately
Richard Feldman (Oct 27 2023 at 13:39):these are good points. I worry about this ending up being something the binary has to support forever though, because if we ever want to remove it it'll break too many things :sweat_smile:
Ayaz Hafiz (Oct 27 2023 at 13:46):Yeah, that's fair. I just worry that we might be letting perfect be the enemy of good here if we try to do something like multiple binary release channels right now, as it just increases the surface area of what we (as an ecosystem) need to support - while the ecosystem is pretty small and our ability to, for example, deal with per-editor integrations, or reconcile version mismatches, is not very high. I think if we want to remove it from the binary at some point, there are a few migration paths you can imagine that are not cumbersome when the ecosystem is larger than it is today, but still fairly small. For example, the LSP command can automatically install the LSP shim that exists elsewhere and refuse to connect until you change all referenced commands. Or, you can migrate editor integrations automatically, without it being a user-facing issue.
Richard Feldman (Oct 27 2023 at 13:51):zooming out for a sec, I'm trying to avoid coupling the public roc API to other protocols, so that we don't get in a situation where people are saying "hey please update the language and do a release ASAP because we're blocked and there's no way for us to unblock ourselves short of a language release."
examples of this include:
roc to llvm updatesroc to Unicode updatesroc to lsp for the same reason.with that in mind, I wonder if there's a way we could keep the relevant logic in the roc binary but expose it in a way that lsp (and others) can access functionality with it in a way that can be upgraded independently from roc itself - like for example what we do with roc glue and giving it a .roc script that describes what to do
Richard Feldman (Oct 27 2023 at 13:52):so I wonder if there's some way to, for example (in the future, as a smooth migration path, not right now) expose from the binary a command where you can run it and it runs a .roc file that provides access to all the editor commands we support, as roc functions
Richard Feldman (Oct 27 2023 at 13:53):then one thing you could do with that is implement the json-rpc logic that lsp needs
Richard Feldman (Oct 27 2023 at 13:54):but also you could do other things, like shared memory ipc, or downloading a dylib for your system and roc version, and telling you where it was downloaded, etc
Ayaz Hafiz (Oct 27 2023 at 14:03):Yeah I agree
Ayaz Hafiz (Oct 27 2023 at 14:05):It's just that right now the primary distribution mechanism is nightly releases that you can pull down at any time, right - and that is a very convenient piece. I think we have a lot of things to change in general if we decide to a move to a slower release cycle (i.e. versioned releases). I just don't want us to add more complexity until we need to, given the usage patterns today.
Richard Feldman (Oct 27 2023 at 14:08):yeah for sure! I'm game for that as long as we have an idea of how to avoid long-term coupling to the protocol, and I'm satisfied that if nothing else the above idea would give us a path to do that :thumbs_up:
Richard Feldman (Oct 27 2023 at 14:08):so I'm on board with strategy #2!
Brendan Hansknecht (Oct 27 2023 at 14:10):Dont people already download a tar of file. What's the issue with throwing an extra binary in it?
Luke Boswell (Nov 02 2023 at 21:49):The API changed recently for List.dropFirst etc.
If you gets this error, like I did, then you need to re-build the language server. :smile:
Screenshot-2023-11-03-at-08.38.55.png
Luke Boswell (Nov 03 2023 at 05:44):Putting this here as I'm not sure if this is RLS or the extension. I suspect this is the langsrv @Ayaz Hafiz ??
Screenshot-2023-11-03-at-16.42.57.png
Ayaz Hafiz (Nov 03 2023 at 06:05):yeah looks like a Roc bug. do you get the same isue when checking the code?
Luke Boswell (Nov 03 2023 at 07:28):Yes, we do running roc check crates/cli_testing_examples/benchmarks/RBTreeDel.roc gives us the UNSAFE PATTERN error.
Luke Boswell (Nov 03 2023 at 07:34):I think it is related to https://github.com/roc-lang/roc/issues/4152, I note that Bool is an opaque type though this behaviour seems strange at first. I assumed boolean would be exhaustive with true and false cases.
Ayaz Hafiz (Nov 03 2023 at 13:03):Ohh, yeah, that’s why. So this is not a bug.
Ayaz Hafiz (Nov 03 2023 at 13:05):yeah i agree it feels strange. But i don’t think we should change the behavior
Richard Feldman (Nov 03 2023 at 13:12):yeah I think we should improve the error message
Richard Feldman (Nov 03 2023 at 13:12):like special case bools to explain what's going on
jan kili (Feb 01 2025 at 17:21):Now that it's time to update editor extensions for the many syntax changes in 0.0.0-alpha2, does it make sense to shift our VSCode support strategy?
Timeline of VSCode support, as far as I can tell:
Should we refactor roc-vscode-unofficial to depend on tree-sitter-roc? That would make Kakoune the only community-supported editor that doesn't use the tree sitter (yet). Alternatively, we could create a new tree-sitter-based extension, to see which works better.
I'm happy to try/help, though I've never edited an editor extension or tree sitter before.
If we do that refactor, would that have any implications for Roc's first-party language server by eliminating a/the primary dependency on it?
Sam Mohr (Feb 01 2025 at 17:35):If everything depends on tree-sitter-roc, we don't have to worry about stuff going out of date. I'm definitely a proponent of that plan
Sam Mohr (Feb 01 2025 at 17:36):The Kakoune config for Roc only has 2 stars and hasn't been updated in 9 months: https://github.com/evanrelf/roc.kak
Sam Mohr (Feb 01 2025 at 17:36):I'm not that worried about it
Eli Dowling (Feb 01 2025 at 17:43):Unless something drastic has changed since I last checked. Highlighting via tree sitter isn't supported in vscode.
Does the vscode extension not support semantic tokenization?
(For those who aren't quite as deep into the editor tooling rabbit hole:
Semantic tokenization is syntax highlighting emitted by the language server, that should be much more accurate than the textmate highlights that vscode uses by default and should be equivalent or better than tree-sitter.
)
I thought the language server supported semantic tokenization? If not we could get that happening without much trouble, it would free us from updating the textmate grammar/make it much less relevant.
Eli Dowling (Feb 01 2025 at 17:50):Also, the tree sitter updates have been discussed in another thread.
I'll be able to work on them when I've finished my holiday (mid Feb ), with help from @Sam Mohr and @Anthony Bullard we should be able to get well ahead of the current roc syntax hopefully in a way that future proofs it for quite a while.
I already got started and things were going very smoothly.
Sam Mohr (Feb 01 2025 at 17:51):Awesome! I'm prioritizing the compiler rewrite skeleton, but can get back to this when that's done
Eli Dowling (Feb 01 2025 at 17:52):No rush. Just thought it was worth summarising the other thread here for clarity :)
Ivan Demchenko (Feb 06 2025 at 17:36):Hello, lovely people!
I am sorry for disappearing and neglecting the extension. Unfortunately, I've been looking for a job.
Anyway, I think I can dedicate some time (between interviews) to support the wonderful community here.
Sam Mohr (Feb 06 2025 at 17:41):We should be putting together a corpus for tree-sitter, which can be used to update this as well
jan kili (Mar 16 2025 at 03:25):Eli Dowling said:
Unless something drastic has changed since I last checked. Highlighting via tree sitter isn't supported in vscode.
It looks like that is drastically changing soon! https://github.com/microsoft/vscode/issues/210475
jan kili (Mar 16 2025 at 03:27):TL;DR: Microsoft is currently implementing tree-sitter support, so that (a) via settings, users can color more than just TextMate scopes, and (b) via extensions, language ecosystem contributors can add entire highlights.scm files. Maybe there's a (c) & (d) too, related to more powerful features getting unlocked by having semantic context for the cursor, but idk. They're currently testing on insiders with TypeScript and INI file syntaxes.
Eli Dowling (Mar 16 2025 at 06:48):Wow. That's exciting
Last updated: Jul 06 2025 at 12:14 UTC