LSP: add tree-sitter as main parsing option #176
Draft
+916
−262
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add the tree-sitter grammar parser and use it for the main LSP functions. Functions changed: - completion: completion now also accepts visible variables for completion, which we obtain with tree-sitter. - hover: now it searches inside the tree-sitter AST instead of the AST generated by the SimplicityHL compiler. Also added more hover options, such as types and variables. - goto-definition and references: also updated to search using the tree-sitter AST and to use tree-sitter’s query features for finding definitions and references. As we now use the AST provided by tree-sitter, the functions working with the compiler AST are removed, as is the `miniscript` crate. Added the `documentation` module, because all SimplicityHL function documentation was previously inside the `completion` module, which was confusing since this documentation is also used by other functions, such as `hover`.
Collaborator
|
This change introduces a bigger maintenance scope than I had expected I would like to reiterate with the team to understand what we can do to improve the language instead of trying to introduce more parsers With this addition, we will have 4 different grammars (pest, nom, vscode, tree-sitter) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This integrates tree-sitter grammar parser in order to implement more features for LSP. The following features have now been added:
also fixes issue with reference and hover for functions inside generics type (for_while, fold and array_fold).
Previously, navigation was based on constructs built from the original compiler parser, but it removes a lot of important information needed for this, such as the location of some tokens.
Tree-sitter allows us to track token positions, make queries to find individual elements, perform quick searches by position, and perform incremental parsing.
We could also use the original grammar for pest parser, using the existing grammar file inside the compiler, but that would require rewriting the parsing logic from scratch. Also, tree-sitter is more convenient to use for LSP features.