Migrate test suite from JUnit 4 to JUnit Jupiter

[sbrannen]

Overview

Although we have to use JUnit 4 for various tests in spring-test that validate our JUnit 4 support, we should be able to migrate all other tests from JUnit 4 to JUnit Jupiter.

The primary challenges include parameterized tests, where the JUnit 4 style (i.e., class-level parameterization) does not always map easily to Jupiter's @ParameterizedTest support (i.e., method-level parameterization).

Related Issues

• Migrate remaining JUnit 3 based tests to JUnit 4 [SPR-13514] #18091
• Introduce Checkstyle rules to prevent improper use of JUnit APIs in production code #22962

[sbrannen]

Current work on this issue can be viewed in the following feature branch.

https://github.com/sbrannen/spring-framework/commits/issues/gh-23451-migrate-test-suite-to-junit-jupiter

[sbrannen]

Team Discussion

We need to decide how we want to migrate parameterized tests.

In sbrannen@956230a, I have introduced two options for migrating from JUnit 4's Parameterized runner to JUnit Jupiter's parameterized tests and dynamics tests.

The challenge is that JUnit Jupiter's @ParameterizedTest support is for a single "test" method; whereas, JUnit 4's Parameterized runner parameterizes the test class, allowing multiple parameterized test methods within a single test class. JUnit Jupiter will eventually provide support for parameterized containers (i.e., test classes), but that is not yet available (see junit-team/junit5#871 and junit-team/junit5#878).

Choices

In the interim, we have three choices.

1. Do not migrate existing JUnit 4 based test classes that use the Parameterized runner, but rather wait on JUnit Jupiter's official support for parameterized containers before migrating such test classes.
2. Migrate to dynamic tests (see PooledDataBufferDynamicTests).
3. Migrate to @ParameterizedTest (see PooledDataBufferParameterizedTests).

Pros and Cons

The advantage of dynamic tests is that they allow for arbitrary nesting of containers and tests. That's why we have 8 reported tests for PooledDataBufferDynamicTests, analogous to the 8 reported tests for the JUnit 4 based PooledDataBufferTests. The downside of dynamic tests is that they require more work to implement, and each individual dynamic test invocation does not have test lifecycle support (e.g., @BeforeEach, @AfterEach, callbacks for extensions, etc.).

The advantage of @ParameterizedTest is that it provides an API closer to JUnit 4's Parameterized runner. Plus, individual parameterized test invocations do have full lifecycle support (e.g., @BeforeEach, @AfterEach, callbacks for extensions, etc.). The downside is that we lose granularity in terms of test reporting. That's why we only have 4 reported tests for PooledDataBufferParameterizedTests.

Test Reporting

JUnit 4 Parameterized runner

JUnit Jupiter @TestFactory

JUnit Jupiter @ParameterizedTest

Complexity

Please keep in mind that the PooledDataBufferTests test class that was migrated in the above examples is rather simple: it only had two @Test methods.

On the other hand, we often have much more complex scenarios with our JUnit 4 parameterized tests. For example, some subclasses of AbstractDataBufferAllocatingTestCase have a lot of individual @Test methods. Specifically, DataBufferTests has 38 individual test methods. So we need to take that into consideration in how/whether we migrate such test classes.

[sbrannen]

In sbrannen@f0a9c30 I have introduced a third migration option for parameterized tests.

Specifically, one can emulate parameterized tests by introducing a testing trait (i.e., test interface with default methods) that is implemented by multiple @Nested test classes.

Pros and Cons

The advantage of this approach is that individual test invocations do have full lifecycle support (e.g., @BeforeEach, @AfterEach, callbacks for extensions, etc.), and we retain granularity in terms of test reporting. That's why we have 8 reported tests for PooledDataBufferNestedTests. The downside is that we have to create concrete @Nested test classes for each "parameterization" and override the abstract methods from the testing trait accordingly.

Test Reporting

JUnit Jupiter @Nested test classes with testing trait