Skip to content

Conversation

@jonahgraham
Copy link
Member

#1375 identified that some tests are not running in JUnit5 runner because discovery is different than JUnit3 runner.

This PR changes the tests to use JUnit5 (in core tests only for now). However much of the purpose of this PR is to modernize the tests. This PR also works around eclipse-jdt/eclipse.jdt.ui#2591

Fixes #1375

@jonahgraham
Copy link
Member Author

There are more commits coming on this PR

@github-actions
Copy link

github-actions bot commented Oct 31, 2025

Test Results

  584 files   -   1    584 suites   - 1   13m 47s ⏱️ -1s
9 920 tests +  3  9 878 ✅  -  15  24 💤 ±0  14 ❌ +14  4 🔥 +4 
9 839 runs   - 116  9 797 ✅  - 134  24 💤 ±0  14 ❌ +14  4 🔥 +4 

For more details on these failures and errors, see this check.

Results for commit 863c40b. ± Comparison against base commit 6574af3.

This pull request removes 1331 and adds 1334 tests. Note that renamed tests count towards both.
NonIndexingTests ‑ test16bitUnicodeEscapeSequence
NonIndexingTests ‑ testAccessGlobalVariableFromConstexprFunction
NonIndexingTests ‑ testAccessGlobalVariableFromGlobalConstexpr
NonIndexingTests ‑ testAccessMemberOfCompositeParameter
NonIndexingTests ‑ testAccessOfStaticField
NonIndexingTests ‑ testAliasDeclaration
NonIndexingTests ‑ testAliasTemplate1
NonIndexingTests ‑ testArgumentEvaluation
NonIndexingTests ‑ testAssignmentOfArrays
NonIndexingTests ‑ testAssignmentOfCompositeValues
…
org.eclipse.cdt.core.parser.tests.ast2.cxx14.constexpr.ArrayTests$NonIndexingTests ‑ testAssignmentOfArrays
org.eclipse.cdt.core.parser.tests.ast2.cxx14.constexpr.ArrayTests$NonIndexingTests ‑ testAssignmentOfMultiDimensionalArrays
org.eclipse.cdt.core.parser.tests.ast2.cxx14.constexpr.ArrayTests$NonIndexingTests ‑ testGlobalArrayAccessValue
org.eclipse.cdt.core.parser.tests.ast2.cxx14.constexpr.ArrayTests$NonIndexingTests ‑ testInitializationOfArrays
org.eclipse.cdt.core.parser.tests.ast2.cxx14.constexpr.ArrayTests$NonIndexingTests ‑ testInitializationOfMultiDimensionalArrays
org.eclipse.cdt.core.parser.tests.ast2.cxx14.constexpr.ArrayTests$NonIndexingTests ‑ testPassArrayToFunctionAsPointerAndModifyCell
org.eclipse.cdt.core.parser.tests.ast2.cxx14.constexpr.ArrayTests$NonIndexingTests ‑ testPassMultiDimensionalArrayToFunctionAsPointerAndModifyCell
org.eclipse.cdt.core.parser.tests.ast2.cxx14.constexpr.ArrayTests$NonIndexingTests ‑ testPointerArithmeticsOnMultidimensionalArray
org.eclipse.cdt.core.parser.tests.ast2.cxx14.constexpr.ArrayTests$NonIndexingTests ‑ testPointerDereferencingOnArrayName
org.eclipse.cdt.core.parser.tests.ast2.cxx14.constexpr.ArrayTests$NonIndexingTests ‑ testPointerToArrayReturnedFromMemberFunction1
…

♻️ This comment has been updated with latest results.

}

private static void cxx20SetUp() {
@BeforeEach
Copy link
Contributor

@i-garrison i-garrison Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at least for this test this change did not worked well:

  • as far as I can see in my local run IndexBindingResolutionTestBase.setUpStrategy() fires and runs parser before IndexConceptTests.cxx20SetUp is fired so it's already too late to tell parser to recognize concepts
  • could be that a few other tests are affected by similar issue

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for having a sneak peak - I think IndexConceptTest should do:

	@Override
	@BeforeEach
	protected void setUpStrategy() throws Exception {
		TestScannerProvider.sDefinedSymbols.put("__cpp_concepts", "201907L");
		super.setUpStrategy();
	}

instead of its own BeforeEach.

I misunderstood something in my first pass in refactoring and I will correct it before marking ready for review.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW The most recent push does not resolve this yet.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem I introduced is that some setUp methods were like this:

setUp() {
  super.setUp()
  localsetup codehere
}

And some were like this:

setUp() {
  localsetup codehere
  super.setUp()
}

And I didn't adequately pay attention to the difference in these two.

This isn't needed as JDT automatically injects what it needs when
running tests.
…nit5

SemanticTestBase is the base class for a huge hierarchy of tests and
all those test updates are included in this commit.
@jonahgraham jonahgraham force-pushed the convert-tests-to-junit5 branch from b8ad90f to 863c40b Compare October 31, 2025 20:51
@jonahgraham
Copy link
Member Author

The sizes of the diff can make it hard to see the important stuff. I think there should be some combination of -I command line options for git diff to ignore some stuff. What I am using is meld with these Text Filters:

image
  • ^\t+@Test
  • ^\t+public static TestSuite.*
  • ^\t+return suite\(.*
  • ^\t+}
  • ^import.*

This makes the number of diff manageable and easy enough to review.

@jonahgraham
Copy link
Member Author

I have decided to split this up into separate PRs as reviewing multiple commits in a single PR made it hard to see what was happening reliably.

I will label them all https://github.com/eclipse-cdt/cdt/issues?q=state%3Aopen%20label%3Ajunit5 to make the easier to follow.

@jonahgraham jonahgraham deleted the convert-tests-to-junit5 branch October 31, 2025 22:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Potential issue with tests derived from IndexBindingResolutionTestBase and JUnit5

2 participants