WritableJson.toByteArray() may corrupt supplementary characters

[yyuneu]

What happened?

AppendableByteArray encodes each append call independently with endOfInput set to false. If the input ends with a high surrogate, CharsetEncoder returns UNDERFLOW without consuming it. Because that input buffer is then discarded, the high surrogate is lost. The low surrogate in the next append call is malformed on its own and is replaced with ?.

JsonValueWriter writes strings one UTF-16 code unit at a time, so this affects supplementary characters, including many emoji. For example:

String emoji = new String(Character.toChars(0x1F600));
JsonWriter<Map<String, Object>> writer = JsonWriter.standard();
WritableJson json = writer.write(Map.of("msg", "hello " + emoji));

json.toJsonString();                                     // {"msg":"hello 😀"}
new String(json.toByteArray(), StandardCharsets.UTF_8);  // {"msg":"hello ?"}

Structured logging is also affected. StructuredLogEncoder for Logback and StructuredLogLayout for Log4j2 use formatAsBytes, which reaches WritableJson.toByteArray(Charset).

This regression was introduced by #49428 in 4.1.0. The previous implementation used an OutputStreamWriter, which retains a pending high surrogate across writes, so 4.0.x is not affected.

What does this PR change?

AppendableByteArray now retains an unconsumed high surrogate and prepends it to the input from the next append call.

If no further input arrives, toByteArray() encodes the pending surrogate with endOfInput set to true, so an unpaired high surrogate is replaced instead of being dropped. reset() also clears the pending state so it cannot carry over when the cached instance is reused.

There are no public API changes.

Tests

Added coverage for:

• A surrogate pair appended as individual characters, using UTF-8 and UTF-16
• A surrogate pair split across two appended strings
• An unpaired high surrogate
• Supplementary characters written character by character through WritableJson

The AppendableByteArray tests compare the result with the output produced by OutputStreamWriter. All of these tests fail without this change.

:core:spring-boot:check passes on Linux with JDK 25.

Related: #51156 fixed a separate regression introduced by #49428.