Sanitize control/format characters in console logger output across all formatters

[Copilot]

Fixes #128727

Console logging currently writes untrusted control characters verbatim, allowing terminal escape/control effects and ambiguous output. This change sanitizes control/format characters across Simple and Systemd formatter paths.

• Behavioral change
  • Escapes Unicode Cc/Cf characters as \uXXXX before writing log output.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-extensions-logging
See info in area-owners.md if you want to be subscribed.

[tarekgh]

(superseded by the comment below)

[tarekgh]

Review of the control character sanitization changes

The security motivation here is solid. Preventing terminal escape injection (ANSI sequences, bidi overrides, etc.) is worth doing. But the current implementation has several problems that need to be addressed before merging.

---

\n, \r, and \t should not be escaped

The sanitizer uses UnicodeCategory.Control which catches every character in U+0000-U+001F, including \n, \r, and \t. These are not security threats. They are structural formatting characters that the formatters depend on.

Both SimpleConsoleFormatter and SystemdConsoleFormatter have explicit downstream logic that operates on real newlines:

• SimpleConsoleFormatter.WriteMessage calls message.Replace(Environment.NewLine, _newLineWithMessagePadding) to add indentation padding after each newline in exception text.
• SystemdConsoleFormatter.WriteReplacingNewLine calls message.Replace(Environment.NewLine, " ") to flatten multi-line messages into a single line (required by systemd/journald).

Because the sanitizer runs before these calls, it converts \n to the literal text \u000A. The downstream Replace calls then find no real newlines and become no-ops. This breaks multi-line exception formatting in Simple mode (no padding) and breaks the single-line guarantee in Systemd mode.

The fix should target only the actually dangerous characters: ESC (\x1B), BEL (\x07), backspace (\x08), bidi overrides (\u202E, \u202D), and similar. Not \n/\r/\t.

---

Double-escaping on the JSON path

Utf8JsonWriter with JavaScriptEncoder.Default already escapes all control characters (U+0000-U+001F) and all non-BasicLatin characters (including \u202E). Pre-sanitizing the strings is redundant and produces double-escaped output.

For example, ESC (\x1B) would normally appear as \u001B in the JSON output. With the sanitizer, it becomes \\u001B, which is a literal backslash followed by u001B. JSON consumers parsing these logs would see the text \u001B instead of the actual ESC character. The test changes in JsonConsoleFormatterTests.cs confirm this: they switched to expecting \\\\u000D\\\\u000A.

The sanitizer should be skipped entirely for the JsonConsoleFormatter path, or at minimum should not run when Utf8JsonWriter is handling the escaping.

---

Breaking change with default true

Setting SanitizeControlCharacters = true by default changes the output format for every existing application without any opt-in. Exception stack traces go from properly formatted multi-line output to a single blob containing \u000A literals. This will break log parsing tools and dashboards that expect the current format.

Consider either defaulting to false or narrowing the escape set so that \n/\r/\t pass through unchanged (which would make the default safe).

---

Minor issues

• API review: adding a public property to ConsoleFormatterOptions requires going through the dotnet/runtime API review process.
• Allocations: every exception log triggers a StringBuilder allocation since exception strings always contain \n. Consider string.Create or ValueStringBuilder for the hot path.
• Test coverage: Log_ControlCharacters_SanitizationCanBeDisabled only tests Simple and Systemd formatters. JSON opt-out is not covered.
• Existing test expectations modified: the changes to ConsoleLoggerTest.cs normalize broken formatting as the new expected output rather than preserving the original behavior.

[rosebyte]

A note on JSON output and custom encoders:

JsonConsoleFormatter delegates character escaping to  Utf8JsonWriter, which by default escapes all control characters. This means dangerous characters like ESC or bidi overrides will appear as  \u001B  etc. in the JSON output without us needing to do anything extra.

However, dotnet's JSON writer supports pluggable encoders, and one of them,  JavaScriptEncoder.UnsafeRelaxedJsonEscaping, deliberately relaxes what gets escaped. If someone explicitly configures their JsonConsoleFormatterOptions.JsonWriterOptions to use this encoder, certain invisible/directional Unicode characters (the kind that can mislead someone reading raw text) will pass through unescaped.

Why we should leave it as it for now:

• The encoder's name literally contains "Unsafe", so using it is a deliberate opt-in to relaxed behaviour.
• JSON logs are typically consumed by log aggregators (Seq, ELK, Datadog, etc.) which parse the JSON and display values in their own UI, not by someone  displaying a file in a terminal.
• Adding pre-sanitisation for this edge case would complicate the default path and risk double-escaping for the majority of users who haven't changed the encoder.

If this proves to be a real-world concern, we can add targeted sanitisation to the JSON path later without any API change.

[Copilot]

Pull request overview

This PR introduces a shared sanitizer (ConsoleControlCharacterSanitizer) and applies it to the Simple and Systemd console formatter pipelines to reduce the risk of untrusted control / formatting characters influencing terminal output. It also adds new unit tests intended to validate sanitization behavior for non-JSON formatters.

Changes:

• Added ConsoleControlCharacterSanitizer and used it to sanitize message / exception / category / scope text in SimpleConsoleFormatter and SystemdConsoleFormatter.
• Updated JsonConsoleFormatter’s char state-property handling (but Json formatter still does not apply the new sanitizer).
• Added tests for sanitization behavior (currently for non-JSON formatters only) and updated minor test code comments.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
src/libraries/Microsoft.Extensions.Logging.Console/tests/Microsoft.Extensions.Logging.Console.Tests/ConsoleLoggerTest.cs	Removes Arrange/Act/Assert comments in an existing test.
src/libraries/Microsoft.Extensions.Logging.Console/tests/Microsoft.Extensions.Logging.Console.Tests/ConsoleFormatterTests.cs	Adds new sanitization tests (currently limited to non-JSON formatters).
src/libraries/Microsoft.Extensions.Logging.Console/src/SystemdConsoleFormatter.cs	Sanitizes message/exception/category and scope string output prior to writing.
src/libraries/Microsoft.Extensions.Logging.Console/src/SimpleConsoleFormatter.cs	Sanitizes message/exception/category and scope string output prior to writing.
src/libraries/Microsoft.Extensions.Logging.Console/src/Microsoft.Extensions.Logging.Console.csproj	Adds ValueStringBuilder to support the sanitizer implementation.
src/libraries/Microsoft.Extensions.Logging.Console/src/JsonConsoleFormatter.cs	Adjusts JSON state-property writing for char values (allocates) but does not add sanitizer usage.
src/libraries/Microsoft.Extensions.Logging.Console/src/ConsoleControlCharacterSanitizer.cs	New sanitizer that escapes a hardcoded set of characters to \\uXXXX.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

[rosebyte]

@tarekgh, thank you for the idea, implemented, the perf results I posted above already use it.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.