[BUG] Test is marked as successful even though it was not executed

[mardukbp]

I have a JUnit test for which the Maven goal testCompile (belonging to the maven-compiler-plugin) fails. Nevertheless, when the test is executed using the green play button in the gutter, the green triangle becomes a green tick. Even though the Debugging Console shows that the class was not found.

In addition to marking the test as failed it would be very useful to provide the output of the failed Maven execution in the Debugging Console. That the class was not found is a consequence of the compilation failure, and this stacktrace does not help to fix the problem. The output of Maven does.

[jdneo]

Is it possible to provide a sample project for this issue?

[mardukbp]

I learned that when a test is not executed the last test result is shown. After deleting %APPDATA%\Code\User\workspaceStorage\workspace-UUID\testResults I got the green triangle again. It would be good to have a command to clear this cache (Java: Clean Java Language Server Workspace does not do it). But in any case, if a test cannot be executed because the class does not exist I would expect that the test is marked as failed (though it would be better to have a Blocked status).

I added testCompile as prelaunchTask in order for the compilation error to be listed under Problems, thus solving my second point. But now I get a permanent toaster that reads: "Run Tests: Resolving launch configuration...". And every time I run the test another toaster appears above the previous one.

Here is the project: https://github.com/mardukbp/error-prone-demo

[jdneo]

It would be good to have a command to clear this cache

You can use Test: Clear All Results

My understanding is that you hope the tests should be marked as failed/error when the prelaunch task fails? Is that correct?

[mardukbp]

In my view a prelaunchTask is an explicit precondition and the existence of SomeTest.class is an implicit precondition. Whenever a precondition is not satisfied the test cannot be executed (in my project both preconditions fail). So strictly speaking the test itself doesn't fail, but a past execution result doesn't represent the current state of affairs.

The failure of the prelaunchTask is reported as an error in Problems and the runtime failure due to the missing class is reported in the Debugging Console. This information should be associated with the test execution and AFAIK marking the test as failed is a way to accomplish that.

[jdneo]

Ok, I got your point.

We need some kind of UI hint to let the users know tests are not executed due to some reasons (compile error, etc...). Currently displaying a pass status in the previous session is misleading.

[mardukbp]

And what about the toasters that don't go away when the execution fails? Am 28. Oktober 2022 08:57:11 MESZ schrieb Sheng Chen ***@***.***>:

…

Ok, I got your point. We need some kind of UI hint to let the users know tests are not executed due to some reasons (compile error, etc...). Currently displaying a pass status in the previous session is misleading. -- Reply to this email directly or view it on GitHub: #1499 (comment) You are receiving this because you authored the thread. Message ID: ***@***.***>