Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to tree-sitter 0.22.6 #485

Merged
merged 6 commits into from
Jun 21, 2024
Merged

Upgrade to tree-sitter 0.22.6 #485

merged 6 commits into from
Jun 21, 2024

Conversation

mjambon
Copy link
Member

@mjambon mjambon commented Jun 12, 2024

Uses semgrep/ocaml-tree-sitter-core#77

This is the long list of parsers that can't build with tree-sitter 0.22.6. Most of them only need an upgrade of the tree-sitter-xxx submodule:

*** Failed to build or test the following languages: bash c-sharp dart elixir fsharp hack hcl html java javascript lua php python r ruby rust sfapex sml solidity typescript vue

Only c-sharp and hack are known to fail permanently due to memory exhaustion during tree-sitter generate.

Security

  • Change has no security implications (otherwise, ping the security team)

@mjambon mjambon requested review from aryx and brandonspark June 12, 2024 22:20
@mjambon mjambon requested a review from a team as a code owner June 12, 2024 22:20
@mjambon mjambon requested a review from nmote June 12, 2024 22:22
@aryx
Copy link
Collaborator

aryx commented Jun 14, 2024

fails in CI

@mjambon
Copy link
Member Author

mjambon commented Jun 19, 2024

@aryx yes, it's expected to fail for most languages. Their tree-sitter-xxx submodule needs to be upgraded and I don't want to do it without doing the OCaml integration work.

Copy link
Collaborator

@aryx aryx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what is the plan? Who is gonna update the failing languages?
Should we merge this before we fix all the failing languages?

@mjambon
Copy link
Member Author

mjambon commented Jun 20, 2024

So what is the plan?

See internal note: https://semgrepinc.slack.com/archives/C048LGPK46L/p1718771440113229

Who is gonna update the failing languages?

Whoever needs to generate a parser for a language.

The problem is that upgrading a parser requires OCaml integration work done in semgrep and I'm not willing to do that (unless the parser is unused). I'll look into upgrading the unused parsers.

Normally, the semgrep developer working on a parser would upgrade to the latest tree-sitter parser but if they want to only extend something like how semgrep's metavariables are parsed without upgrading, they can do so by continuing to use the older tree-sitter (0.20.6).

Should we merge this before we fix all the failing languages?

Yes, because this doesn't affect the semgrep repo.

@mjambon mjambon merged commit dc31834 into main Jun 21, 2024
2 of 3 checks passed
@mjambon mjambon deleted the martin/upgrade-tree-sitter branch June 21, 2024 00:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants