WIP c3 function indexing and completion

[sanoprime]

Its work in progress.

Added:

• SDK sources/classes;
• function indexes (C3FuncDef)
• completion of functions (very basic, it knows all declared functions from project and SDK but to make it really work, PSI have to be stable)
• few error recoveries in C3.bnf

What have to be done:

• error recovery in C3.bnf for stable PSI
• after stable PSI I can work on better completion.

Completion now works only when function is not broken from parser perspective:

fn void test() {
    int a = pri; // this is ok
    int b = pri// this breaks parser and completion
    int c = io:: // this breaks too
    int d = io::pri ; // this works
}

@lerno
I would also add few things to this PR:

• I want to make struct, const and module indexes/code completion
• Goto declaration for structs
• error recovery and tokens=[] into C3.bnf (from C3.flex) so I can iterate faster on error recovery.

[lerno]

I got this error:

com.intellij.diagnostic.PluginException: Failed to build stub tree for test.c3 [Plugin: org.c3lang.c3intellij]
	at com.intellij.diagnostic.PluginProblemReporterImpl.createPluginExceptionByClass(PluginProblemReporterImpl.java:23)
	at com.intellij.diagnostic.PluginException.createByClass(PluginException.java:89)
	at com.intellij.psi.stubs.StubTreeBuilder.handleStubBuilderException(StubTreeBuilder.java:105)
	at com.intellij.psi.stubs.StubTreeBuilder.lambda$buildStubTree$3(StubTreeBuilder.java:142)
	at com.intellij.psi.impl.PsiManagerImpl.runInBatchFilesMode(PsiManagerImpl.java:455)
	at com.intellij.psi.stubs.StubTreeBuilder.buildStubTree(StubTreeBuilder.java:132)
	at com.intellij.psi.stubs.StubTreeBuilder.buildStubTree(StubTreeBuilder.java:91)
	at com.intellij.psi.stubs.PerFileElementTypeStubModificationTracker.preciseCheck(PerFileElementTypeStubModificationTracker.java:163)
	at com.intellij.psi.stubs.PerFileElementTypeStubModificationTracker.lambda$endUpdatesBatch$2(PerFileElementTypeStubModificationTracker.java:97)
	at com.intellij.openapi.application.ReadAction.lambda$run$1(ReadAction.java:64)
	at com.intellij.openapi.application.impl.ApplicationImpl.runReadAction(ApplicationImpl.java:923)
	at com.intellij.openapi.application.ReadAction.compute(ReadAction.java:76)
	at com.intellij.openapi.application.ReadAction.run(ReadAction.java:63)
	at com.intellij.psi.stubs.PerFileElementTypeStubModificationTracker.endUpdatesBatch(PerFileElementTypeStubModificationTracker.java:91)
	at com.intellij.util.indexing.events.ChangedFilesCollector$4.endBatch(ChangedFilesCollector.java:252)
	at com.intellij.util.indexing.events.ChangedFilesCollector$5.lambda$endBatch$2(ChangedFilesCollector.java:289)
	at com.intellij.util.ConcurrencyUtil.withLock(ConcurrencyUtil.java:264)
	at com.intellij.util.indexing.events.ChangedFilesCollector$5.endBatch(ChangedFilesCollector.java:288)
	at com.intellij.util.indexing.events.VfsEventsMerger.processChanges(VfsEventsMerger.java:109)
	at com.intellij.util.indexing.events.ChangedFilesCollector.processFilesInReadAction(ChangedFilesCollector.java:271)
	at com.intellij.util.indexing.events.ChangedFilesCollector.processFilesToUpdateInReadAction(ChangedFilesCollector.java:230)
	at com.intellij.util.indexing.events.ChangedFilesCollector.ensureUpToDate(ChangedFilesCollector.java:191)
	at com.intellij.util.indexing.FileBasedIndexImpl.ensureUpToDate(FileBasedIndexImpl.java:785)
	at com.intellij.psi.stubs.StubIndexEx.processAllKeys(StubIndexEx.java:314)
	at com.intellij.psi.stubs.StubIndex.processAllKeys(StubIndex.java:70)
	at com.intellij.psi.stubs.StubIndexEx.getAllKeys(StubIndexEx.java:302)
	at org.c3lang.intellij.completion.FunctionCompletionContributor$1.addCompletions(FunctionCompletionContributor.java:45)
	at com.intellij.codeInsight.completion.CompletionProvider.addCompletionVariants(CompletionProvider.java:23)
	at com.intellij.codeInsight.completion.CompletionContributor.fillCompletionVariants(CompletionContributor.java:156)
	at com.intellij.codeInsight.completion.CompletionService.getVariantsFromContributor(CompletionService.java:81)
	at com.intellij.codeInsight.completion.impl.CompletionServiceImpl.lambda$getVariantsFromContributor$0(CompletionServiceImpl.java:333)
	at com.intellij.platform.diagnostic.telemetry.helpers.TraceKt.runWithSpan(trace.kt:81)
	at com.intellij.codeInsight.completion.impl.CompletionServiceImpl.getVariantsFromContributor(CompletionServiceImpl.java:332)
	at com.intellij.codeInsight.completion.CompletionService.getVariantsFromContributors(CompletionService.java:73)
	at com.intellij.codeInsight.completion.CompletionResultSet.runRemainingContributors(CompletionResultSet.java:150)
	at com.intellij.codeInsight.completion.CompletionResultSet.runRemainingContributors(CompletionResultSet.java:142)
	at com.intellij.codeInsight.completion.CompletionResultSet.runRemainingContributors(CompletionResultSet.java:138)
	at com.intellij.codeInsight.template.impl.LiveTemplateCompletionContributor$1.addCompletions(LiveTemplateCompletionContributor.java:87)
	at com.intellij.codeInsight.completion.CompletionProvider.addCompletionVariants(CompletionProvider.java:23)
	at com.intellij.codeInsight.completion.CompletionContributor.fillCompletionVariants(CompletionContributor.java:156)
	at com.intellij.codeInsight.completion.CompletionService.getVariantsFromContributor(CompletionService.java:81)
	at com.intellij.codeInsight.completion.impl.CompletionServiceImpl.lambda$getVariantsFromContributor$0(CompletionServiceImpl.java:333)
	at com.intellij.platform.diagnostic.telemetry.helpers.TraceKt.runWithSpan(trace.kt:81)
	at com.intellij.codeInsight.completion.impl.CompletionServiceImpl.getVariantsFromContributor(CompletionServiceImpl.java:332)
	at com.intellij.codeInsight.completion.CompletionService.getVariantsFromContributors(CompletionService.java:73)
	at com.intellij.codeInsight.completion.CompletionService.getVariantsFromContributors(CompletionService.java:56)
	at com.intellij.codeInsight.completion.CompletionService.performCompletion(CompletionService.java:130)
	at com.intellij.codeInsight.completion.BaseCompletionService.performCompletion(BaseCompletionService.java:48)
	at com.intellij.codeInsight.completion.impl.CompletionServiceImpl.lambda$performCompletion$1(CompletionServiceImpl.java:351)
	at com.intellij.platform.diagnostic.telemetry.helpers.TraceKt.runWithSpan(trace.kt:81)
	at com.intellij.codeInsight.completion.impl.CompletionServiceImpl.performCompletion(CompletionServiceImpl.java:340)
	at com.intellij.codeInsight.completion.CompletionProgressIndicator.lambda$calculateItems$11(CompletionProgressIndicator.java:871)
	at com.intellij.util.indexing.FileBasedIndex.lambda$ignoreDumbMode$0(FileBasedIndex.java:213)
	at com.intellij.openapi.util.RecursionManager$1.computePreventingRecursion(RecursionManager.java:110)
	at com.intellij.util.indexing.FileBasedIndexEx.ignoreDumbMode(FileBasedIndexEx.java:721)
	at com.intellij.util.indexing.FileBasedIndex.ignoreDumbMode(FileBasedIndex.java:212)
	at com.intellij.util.indexing.DumbModeAccessType.ignoreDumbMode(DumbModeAccessType.java:43)
	at com.intellij.codeInsight.completion.CompletionProgressIndicator.calculateItems(CompletionProgressIndicator.java:867)
	at com.intellij.codeInsight.completion.CompletionProgressIndicator.runContributors(CompletionProgressIndicator.java:855)
	at com.intellij.codeInsight.completion.CodeCompletionHandlerBase.lambda$startContributorThread$7(CodeCompletionHandlerBase.java:373)
	at io.opentelemetry.context.Context.lambda$wrap$1(Context.java:212)
	at com.intellij.codeInsight.completion.AsyncCompletion.lambda$tryReadOrCancel$5(CompletionThreading.java:162)
	at com.intellij.openapi.application.impl.ApplicationImpl.tryRunReadAction(ApplicationImpl.java:1133)
	at com.intellij.codeInsight.completion.AsyncCompletion.tryReadOrCancel(CompletionThreading.java:160)
	at com.intellij.codeInsight.completion.CodeCompletionHandlerBase.lambda$startContributorThread$8(CodeCompletionHandlerBase.java:364)
	at io.opentelemetry.context.Context.lambda$wrap$1(Context.java:212)
	at com.intellij.codeInsight.completion.AsyncCompletion.lambda$startThread$0(CompletionThreading.java:85)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:186)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$executeProcessUnderProgress$13(CoreProgressManager.java:604)
	at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:679)
	at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:635)
	at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:603)
	at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:61)
	at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:173)
	at com.intellij.codeInsight.completion.AsyncCompletion.lambda$startThread$1(CompletionThreading.java:81)
	at com.intellij.openapi.application.impl.ApplicationImpl$2.run(ApplicationImpl.java:272)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.util.concurrent.Executors$PrivilegedThr

[lerno]

Something also happened with function attributes. It looks like it's always expecting them...? But lookup for functions work! HORRAY!

[lerno]

The above happened as I was typing "module test"

[sanoprime]

Yeah, I have to fix PSI tree (recoverWhile), otherwise weird errors will happen. Ill keep you posted.

[lerno]

Very nice already just to have that working.

[lerno]

Is it ready to merge now?

[sanoprime]

Not yet. Sorry for confusion.