[Fix][Transform-V2][JsonPath] Fix date/time conversion error when mul…

[yht0827]

Fixes: #10333

Purpose of this pull request

When using JsonPath transform with multiple date/timestamp columns, the converter received null as the field name, causing incorrect date format resolution.
Fixed by passing columnConfig.getDestField() to the converter, ensuring each column resolves its own date format correctly.

Does this PR introduce any user-facing change?

Yes. Previously, multiple date/timestamp columns in JsonPath transform could fail or produce incorrect results due to missing field name context. Now each column correctly applies its configured date/time format.

How was this patch tested?

• JsonPathTransformTest.java
  • testMultipleDateColumnsWithDifferentFormats() - Verifies multiple date columns with distinct formats are correctly converted using their respective dest_field configurations.
  • testMultipleDateColumnsWithNullValues() - Ensures null values in date columns are preserved without causing conversion errors.

Check list

• If any new Jar binary package adding in your PR, please add License Notice according
  New License Guide
• If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
• If necessary, please update incompatible-changes.md to describe the incompatibility caused by this PR.
• If you are contributing the connector code, please check that the following files are updated:
  1. Update plugin-mapping.properties and add new connector information in it
  2. Update the pom file of seatunnel-dist
  3. Add ci label in label-scope-conf
  4. Add e2e testcase in seatunnel-e2e
  5. Update connector plugin_config

[dybyte]

Please enable CI following the instructions.

[dybyte]

I think we should add null checks in JsonRowConverters#convertToLocalDate and JsonRowConverters#convertToLocalDateTime.
Also, could you add test cases for these two methods in JsonRowDataSerDeSchemaTest?

[zhangshenghang]

There is currently a problem with CI, and you need to enable it.

[yht0827]

@dybyte

Thanks for the review!

I’ll add null checks to ‎JsonRowConverters#convertToLocalDate and ‎JsonRowConverters#convertToLocalDateTime, and also add test cases for these methods in ‎JsonRowDataSerDeSchemaTest.

[DanielCarter-stack]

Issue 1: Test coverage is not comprehensive enough

Location: seatunnel-transforms-v2/src/test/java/org/apache/seatunnel/transform/JsonPathTransformTest.java:549-662

Related Context:

• Test class: JsonPathTransformTest.java
• Production class: JsonPathTransform.java:130-165
• Dependency class: JsonToRowConverters.java:259-293

Problem Description:

The newly added test cases cover the scenarios of "multiple date columns with different formats" and "containing null values", but are missing the following scenarios:

1. Regression test for a single date column (to verify the modification does not break original behavior)
2. Test for multiple date columns with the same format (to verify caching mechanism)
3. Mixed type field test (to verify it does not affect string, integer, and other types)

Potential Risks:

• Risk 1: Unable to ensure compatibility of the modification with single date column scenarios
• Risk 2: Unable to verify whether performance optimization for multiple fields with the same format has regressed
• Risk 3: Unable to verify the modification does not affect non-date type fields

Impact Scope:

• Direct impact: Test coverage of JsonPathTransform
• Indirect impact: Safety net for future refactoring
• Impact surface: Single Transform (not framework-level)

Severity: MINOR

Improvement Suggestions:

@Test
public void testSingleDateColumn() {
    // Verify that the behavior of a single date column remains consistent with the modifications
    Map<String, Object> configMap = new HashMap<>();
    configMap.put(
        JsonPathTransformConfig.COLUMNS.key(),
        Arrays.asList(
            ImmutableMap.of(
                JsonPathTransformConfig.SRC_FIELD.key(), "data",
                JsonPathTransformConfig.PATH.key(), "$.birth",
                JsonPathTransformConfig.DEST_FIELD.key(), "birth_date",
                JsonPathTransformConfig.DEST_TYPE.key(), "date")));
    // ... test logic
}

@Test
public void testMultipleDateColumnsWithSameFormat() {
    // Verify that multiple date columns with the same format can share cache
    Map<String, Object> configMap = new HashMap<>();
    configMap.put(
        JsonPathTransformConfig.COLUMNS.key(),
        Arrays.asList(
            ImmutableMap.of(..., "birth", "birth_date", "date"),
            ImmutableMap.of(..., "hired", "hire_date", "date")));
    String jsonData = "{\"birth\": \"2024-01-15\", \"hired\": \"2024-02-20\"}";
    // ... verify both fields are correctly parsed
}

@Test
public void testMixedTypeColumns() {
    // Verify mixed configuration of date columns with string and integer columns
    Map<String, Object> configMap = new HashMap<>();
    configMap.put(
        JsonPathTransformConfig.COLUMNS.key(),
        Arrays.asList(
            ImmutableMap.of(..., "date_field", "date"),
            ImmutableMap.of(..., "string_field", "string"),
            ImmutableMap.of(..., "int_field", "int")));
    // ... verify all types are correctly converted
}

Rationale:

• These tests can enhance confidence in the correctness of the modification
• Single date column test ensures backward compatibility
• Same format test verifies cache sharing works properly
• Mixed type test verifies isolation of the modification (does not affect other types)

---

Issue 4: Insufficient encapsulability of fieldFormatterMap

Location: seatunnel-formats/seatunnel-format-json/src/main/java/org/apache/seatunnel/format/json/JsonToRowConverters.java:80

Related Context:

• Current definition: public Map<String, DateTimeFormatter> fieldFormatterMap = new HashMap<>();
• Usage location: JsonToRowConverters.java:261, 281
• Caller: JsonPathTransform.java (indirectly through converter)

Problem Description:

fieldFormatterMap is declared as public, which is an unnecessary encapsulation violation. Although it does need to be accessed externally (e.g., by tests), a better approach is to provide a controlled access method.

Potential Risks:

• Risk 1: External code may directly modify the Map, causing cache pollution
• Risk 2: Difficult to change internal implementation during future refactoring (e.g., replacing with ConcurrentMap)

Impact Scope:

• Direct impact: API design of JsonToRowConverters
• Indirect impact: All code using JsonToRowConverters
• Impact surface: Format layer (multiple Formats use similar patterns)

Severity: MINOR

Improvement Suggestions:

// Current (line 80)
public Map<String, DateTimeFormatter> fieldFormatterMap = new HashMap<>();

// Suggested change to
private Map<String, DateTimeFormatter> fieldFormatterMap = new HashMap<>();

// If tests need access, add package-level visible access methods
Map<String, DateTimeFormatter> getFieldFormatterMap() {
    return fieldFormatterMap;
}

// Or provide controlled access methods
public DateTimeFormatter getCachedFormatter(String fieldName) {
    return fieldFormatterMap.get(fieldName);
}

Rationale:

• Follows encapsulation principle: internal implementation details should not be exposed as public
• Prevents accidental modification: external code should not directly manipulate cache
• Preserves refactoring freedom: underlying implementation can be replaced in the future without affecting API

Note: This issue is outside the scope of this PR and is an existing problem in the codebase. However, since it was discovered during review, it should be recorded for future improvement.

---

[yht0827]

@dybyte @DanielCarter-stack Thanks for the detailed reviews!

I've added the following test cases to cover the requested scenarios:

JsonPathTransformTest

Test Method	Purpose
testMultipleDateColumnsWithDifferentFormats	Multiple date/timestamp columns with different formats
testMultipleDateColumnsWithNullValues	Null values don't cause conversion errors
testSingleDateColumn	Regression test - single date column still works
testMultipleDateColumnsWithSameFormat	Same format columns can share cache
testMixedTypeColumns	Mixed types (date + string + int) work correctly
testInvalidDateFormat	Invalid date format throws exception
testLeapYearDate	Leap year (2024-02-29) parses correctly

JsonRowDataSerDeSchemaTest

Test Method	Purpose
testMultipleDateColumnsWithDifferentFormats	Ensures multiple LocalDate columns using different patterns (yyyy-MM-dd, yyyy.MM.dd) are parsed independently and correctly

---

@DanielCarter-stack I'm also interested in working on issue 4. Will create a follow-up PR after this one is merged.

Please review when you have time.

[dybyte]

LGTM if CI passes. Thanks @yht0827 !

[dybyte]

@yht0827 Could you please retry the CI without adding any additional commits?

[yht0827]

@yht0827 Could you please retry the CI without adding any additional commits?

I’ve retriggered the CI without adding any new commits, but it failed again.

[yht0827]

@zhangshenghang @DanielCarter-stack

Could you please take another look at this PR?

The JsonPath multi date/datetime column issue has been fixed, and the requested null checks and tests have all been applied. If everything looks good, I’d really appreciate your approval.