Skip to content

rerunFailingTestsCount incorrectly marks failed ParameterizedClass test as pass #3303

@eze210

Description

@eze210

Affected version

3.5.4

Bug description

We noticed the same symptom as in #2768, but for ParameterizedClass annotation of JUnit6

To reproduce it, I've just created a maven project with the surefire plugin and JUnit6, and added this test:

package org.example;

import static org.junit.jupiter.api.Assertions.assertTrue;

import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedClass;
import org.junit.jupiter.params.provider.ValueSource;

@ParameterizedClass(name = "Parameterization with index: [{index}]")
@ValueSource(ints = {1, 2, 3})
class ParameterizedClassTest {

    private final int number;

    ParameterizedClassTest(int number) {
        this.number = number;
    }

    @Test
    void someParameterizationFails() {
        assertTrue(number < 3);
    }
}

And ran:

mvn test -Dsurefire.rerunFailingTestsCount=2

As a result, I have this log:

[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.001 s <<< FAILURE! -- in org.example.ParameterizedClassTest
[ERROR] org.example.ParameterizedClassTest.someParameterizationFails -- Time elapsed: 0.001 s <<< FAILURE!
org.opentest4j.AssertionFailedError: expected: <true> but was: <false>
        at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:158)
        at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:139)
        at org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:69)
        at org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:41)
        at org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:35)
        at org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:195)
        at org.example.ParameterizedClassTest.someParameterizationFails(ParameterizedClassTest.java:21)

[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 s -- in org.example.ParameterizedClassTest
[INFO] 
[INFO] Results:
[INFO] 
[WARNING] Flakes: 
[WARNING] org.example.ParameterizedClassTest.someParameterizationFails
[INFO]   Run 1: PASS
[INFO]   Run 2: PASS
[ERROR]   Run 3: ParameterizedClassTest.someParameterizationFails:21 expected: <true> but was: <false>
[ERROR]   Run 4: ParameterizedClassTest.someParameterizationFails:21 expected: <true> but was: <false>
[ERROR]   Run 5: ParameterizedClassTest.someParameterizationFails:21 expected: <true> but was: <false>
[INFO] 
[INFO] 
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Flakes: 1
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.814 s
[INFO] Finished at: 2026-02-19T17:17:02-03:00
[INFO] ------------------------------------------------------------------------

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions