Feature Request: Add @MatrixSource Annotation for Matrix-Style Parameterized Tests in JUnit 5 #4900
Description
Summary
I propose adding a new annotation, @MatrixSource, to the JUnit Jupiter parameterized testing framework to simplify writing tests that must run with all combinations (Cartesian product) of multiple parameter values, a pattern I call matrix tests. The name "matrix" comes from an Azure DevOps feature called "Matrix job strategy"
Currently, JUnit 5 @ParameterizedTest requires explicit listing or generation of arguments for such tests via @MethodSource or other sources, which can be verbose and cumbersome. @MatrixSource would provide a shorter and clearer syntax for common finite-domain parameter types and enable customization for other argument types that are not finite-domain.
This will improve the testing experience in case you have to write tests for a cartesian products of arguments.
Motivation
A common case involves test methods with boolean flags, enums, and other finite-domain types where the goal is to run the test for every possible combination of those parameters:
@ParameterizedTest
void doTest(boolean flag1, boolean flag2, MyEnum enumerated) { ... }
Currently, one must manually write a @MethodSource that returns the Cartesian product of these values or explicitly list them with @CsvSource, adding boilerplate and reduced readability.
Proposed Design
Basic Usage
Applying @MatrixSource without parameters on a test method will:
- Automatically detect if all parameters are finite-domain types (booleans, enums, etc.).
- Generate the Cartesian product of the parameter value domains.
- Run the test on each combination.
Example:
@MatrixSource
void testWithBooleansAndEnums(boolean flag1, boolean flag2, MyEnum myEnum) {
// test logic
}This runs the test for all combinations of true/false for each boolean and all constants of MyEnum automatically.
Customization for Non-Finite Types
For parameters that are not strictly finite domain (e.g., strings, numbers), the user must explicitly provide value sources using current source annotations (@ValueSource, @CsvSource, @MethodSource, etc.).
This may be done using a nested annotation @MatrixArgument inside @MatrixSource to specify argument names, index, and its source.
Example:
@MatrixSource({
@MatrixArgument(name = "payload", index = 1, values = @ValueSource(strings = {"foo", "bar"}))
})
void testWithBooleansAndStrings(boolean flag, String payload) {
// test logic
}
Here, flag is boolean and automatically handled; payload is a string and requires an explicit domain.
Annotation Sketch
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@ParameterizedTest
public @interface MatrixSource {
MatrixArgument[] value() default {};
}
@Target({})
public @interface MatrixArgument {
String name() default ""; //mutually exclusive with index
int index() default -1; //mutually exclusive with name
ValueSource values() default @ValueSource; //Only one source must be indicated
CsvSource csv() default @CsvSource;
MethodSource method() default @MethodSource;
// Add other supported sources as needed
}This is a draft and is based on the assumption that Java doesn't allow polymorphism in annotation. I have tried to leverage on the existing annotation in the Junit landscape.
Benefits
- Simplifies writing matrix-style parameterized tests.
@EnumSourceworks great only if it's the only parameter. - Reduces boilerplate and improves test readability.
- Automatically leverages finite-domain types for full Cartesian product generation.
- Maintains flexibility via per-parameter explicit domain specification.
Request for Discussion
- Does this align with JUnit 5 design principles and goals?
- Should support be limited strictly to finite-domain parameters by default?
- Is the nested annotation (
@MatrixArgument) the best approach for customization? - Suggestions on naming ("@matrixsource") or alternative ideas?
Looking forward to community feedback and discussion!
Thank you for considering this proposal!