Generate tests from a JUnit5 Extension

[vmassol]

Hello everyone,

Since #371 is closed, is there a solution to be able to create a JUnit5 Extension that will generate tests dynamically?

Right now I'm forced to use a @TestFactory and a class to be extended, as in (in the example the CompatibilityTest class has a method annotated with @TestFactory):

class HTML5CompatibilityTest extends CompatibilityTest
{
}

But it would be nicer if I could write:

@CompatiblityTest
class HTML5CompatibilityTest
{
}

The @CompatiblityTest would trigger an Extension that generate the tests dynamically (in my case, the tests are expressed in some files and the Extension would generate the tests based on that).

Note: It's better than extending a class since this wouldn't touch the test hierarchy.

Any idea?

Thanks a lot

[sormuras]

Is writing a custom test engine an option?

A TestEngine facilitates discovery and execution of tests for a particular programming model.

[sormuras]

Here is also some code for a very simplistic test engine implementation with at least some test descriptors being generated: https://github.com/junit-team/junit5-samples/tree/main/junit5-modular-world/src/main/ice.cream - taking in some of your configuration files and creating test descriptors based on them should do-able.

Sure, Surefire's Platform support should work with any new test engine you through into the mix.

[sormuras]

Find a good article about it here: https://blogs.oracle.com/javamagazine/post/junit-build-custom-test-engines-java

[sormuras]

And here's a video about the topic: https://www.youtube.com/watch?v=fQN5f5Ou3UI

[sormuras]

I have the feeling that the gains I'll win are small for such an endeavor though

One benefit could be that an IDE can present you text(?) configuration files as first-class testable things and provide a Run test... menu entry.

[sormuras]

Sure, Surefire's Platform support should work with any new test engine you through into the mix.

Mind https://stackoverflow.com/questions/62487551/running-file-based-tests-on-junit-5-platform-using-apache-maven-surefire though.

[vmassol]

@sormuras thanks a lot for all your pointers, this is great and much appreciated! :) I'll need some time to research/digest this and then decide if go in this direction or not. Thx again