Plans for autocomplete in language server? · contributing

I'm wondering if there is any plan for adding autocomplete to the language server? I believe I've seen it mentioned that autocomplete isn't possible right now because the compiler doesn't support it. If that's likely to be a long way off how interested would folks here be in using tree-sitter to provide autocomplete?
It would be basic, no docs (at least from imports, you could provide docs from the local file) and limited ability to tell the type's of imports and fields of records(may be possible), but tree-sitter has enough info to at least provide autocomplete that only includes functions/variables actually available within the scope you are in. This wouldn't be too hard to implement in a very basic way to tide people over.
Just gauging interest :)

Ayaz Hafiz (Nov 29 2023 at 15:09):

In principle the language server should be able to support auto-complete without any considerable effort, someone just needs to implement it

Ayaz Hafiz (Nov 29 2023 at 15:10):

But if the tree-sitter autocomplete is also trivial to set up, I think it would definitely be useful

Eli Dowling (Nov 30 2023 at 01:34):

Oh okay, well maybe I'll look into setting up a proper implementation. While we are on the topic, why is the language server not written in roc?
Seems like it would be a good project to show off some quality roc code, language Servers tend to include most common coding patterns, error handling, io, IPC, event driven architecture, testing, etc.

Ayaz Hafiz (Nov 30 2023 at 01:38):

Just because the compiler is also written in Rust, so it enables a lot of code reuse. Writing a language server in Roc would be great. But it would require a lot of effort (parsing, type checking) that I didn't want to put in. If someone wants to try to write a Roc-based LSP, it could be a great project. You could even use the Roc compiler as a platform and have the LSP-specific logic in Roc to begin with

Eli Dowling (Nov 30 2023 at 01:52):

Oh yeah, that makes sense, I assumed that it might be for tighter integration with the compiler. I was thinking you'd have to expose the compiler services to roc via some json-rpc type interface and call into it, but making it a platform does seem like a solution. Shows that I'm clearly not thinking about things the roc way yet :sweat_smile:

Ayaz Hafiz (Nov 30 2023 at 01:54):

Richard Feldman (Nov 30 2023 at 01:55):

long-term I actually would like to eliminate the JSON-RPC overhead of LSP in most editor extensions, by having the compiler expose a libroc C library that extensions can call directly in the same way! :rocket:

Richard Feldman (Nov 30 2023 at 01:56):

but LSP is definitely a huge step forward for Roc tooling, and is also worth having indefinitely regardless because there will always be new editors that will support LSP but won't have a dedicated Roc extension etc.

Eli Dowling (Nov 30 2023 at 01:58):

Well that would be interesting... Though, how much overhead do you think jsonrpc actually adds? I couldn't say accurately, but I've worked on 3 language Servers and I would guess that the time spent in the compiler doing stuff like type checking completely dwarfs it.

Eli Dowling (Dec 01 2023 at 04:34):

@Richard Feldman I felt like my statement there wasn't very well substantiated, so i did some testing to back it up.
I made a jsonrpc connection between two ocaml programs( the language I've most recently been doing some lsp stuff in) and then sequentially sent 1000 requests that had 1000 lines of json as the response.
No work was done on either end except deserializing the request and serializing the response.
The time per request on my laptop was 0.2ms.
I would consider this completely meaningless overhead.

Eli Dowling (Dec 01 2023 at 15:45):

https://github.com/roc-lang/roc/pull/6134
Okay that's an initial draft for the completion, it's all a bit of a mess but I thought I'd better at least let people know I'm working on it :)

Eli Dowling (Dec 02 2023 at 04:11):

If anyone wants to use this for AOC or the like it's currently working well, supporting local variables,function args, pattern matching, destructing etc. no types, tags, or imports though. And I couldn't figure out how to get the data from a backpassing expression so that one might be a little while. Should add more in the coming days

Artur Domurad (Dec 04 2023 at 17:53):

Are there any plans to add roc_ls to the nightly builds?
I'm coding roc mainly on a laptop on which I can't use tools like nix to build it from source :(

Anton (Dec 04 2023 at 18:06):

We've talked about that before but we did not want it bundled with the nightly forever and thus did not want people to rely on it being there. We failed to come up with a good compromise. I can't seem to find that dicussion...

Brendan Hansknecht (Dec 04 2023 at 18:09):

I thought the most recent dicussion on this was that we should probably directly add it to the roc executable as a sub command.

Brendan Hansknecht (Dec 04 2023 at 18:10):

Pearce Keesling (Dec 04 2023 at 19:01):

Fwiw I was able to build roc_ls without needing nix. I do think I installed llvm on brew though.

Artur Domurad (Dec 04 2023 at 19:24):

It worked :O
git clone ... and cargo build -p roc_lang_srv --release - without nix.

Pearce Keesling (Dec 05 2023 at 04:53):

I think it all ends up playing nicer since language server doesn't need to do any codegen

Eli Dowling (Dec 05 2023 at 07:10):

Unfortunately I think that's the last one I'm going to be able to easily implement, it seems like the current way the the document is loaded doesn't expose imports and type definitions. I have been digging around the the compiler a bit but I think someone with some more experience would be better off taking a look at how to surface that info.
I can clean this up and see about closing the PR with the current parts working.

I am also taking a look at fixing the nasty delay in completion and hover. Currently we tie the completion and hover to the document having been typechecked. My proposed solution is that we store:
The latest typechecked document,
The last typecheck that didn't completely fail, (this I've already done)
Ideally you could get symbols almost immediately for any changed versions of the document(I assume the parsing step is very quick), and use the typecheck results from the previous version. If splitting that out of the loading process is difficult we can just immediately provide the lineinfo for the new document which is enough for completions, and we could change how hover works to use that too.

I am in the process of implementing the second solution, but rust is hard and it's been ages, so I'm lost in a hell of lifetimes and borrow checking right now. :sweat_smile:

Johan Lindskogen (Dec 06 2023 at 06:39):

@Eli Dowling Not sure if this is the right place to ask, but I tried running roc_ls with the vscode extension from your fork, and it keeps crashing with this message:

got document
starting to get completion items for prefix:
decl:"simple"
check if prefix "p" matches "simple"
decl:"main"
check if prefix "p" matches "main"
decl:"run"
check if prefix "p" matches "run"
check if prefix "p" matches "part2"
check if prefix "p" matches "part1"
check if prefix "p" matches "countWays"
check if prefix "p" matches "parse2"
check if prefix "p" matches "parse"
check if prefix "p" matches "input"
completion begin
check if prefix "p" matches "parseNums"
got completions:
[Error - 07:37:46] Request completionItem/resolve failed.
  Message: Method not found
  Code: -32601

Eli Dowling (Dec 06 2023 at 06:49):

Johan Lindskogen (Dec 06 2023 at 06:53):

app "hello"
    packages { pf: "https://github.com/roc-lang/basic-cli/releases/download/0.7.0/bkGby8jb0tmZYsy2hg1E_B2QrCgcSTxdUlHtETwm5m4.tar.br" }
    imports [pf.Stdout]
    provides [main] to pf

simple1 = 100

main =

    Stdout.line
        "done"

Johan Lindskogen (Dec 06 2023 at 06:55):

I just realised the error might be that I am running your branch against the nightly build of roc. Is it really important that I keep roc_ls in sync with the roc binary?

Eli Dowling (Dec 06 2023 at 15:14):

It shouldn't be. I'll check out your example soonish, I'm currently still rewriting a bunch of stuff to remove all the delays in completion, so it might take me a little while.

Eli Dowling (Dec 10 2023 at 00:55):

@Johan Lindskogen Sorry for the wait. I just tested this with the latest version and it all seems good :)

Johan Lindskogen (Dec 11 2023 at 10:55):

thread 'main' panicked at 'Cannot block the current thread from within a runtime. This happens because a function attempted to block the current thread while the thread is being used to drive asynchronous tasks.', crates/lang_srv/src/registry.rs:140:14
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Stream: contributing

Topic: Plans for autocomplete in language server?

Eli Dowling (Nov 29 2023 at 15:04):