Scanner: extend Trojan Source checks to isolates and NatSpec comments

[0xWeakSheep]

Summary

This PR addresses #16500 by extending Trojan Source mitigation in the lexer.

Specifically:

• Adds support for BiDi isolate markers (U+2066..U+2069, i.e. LRI/RLI/FSI/PDI) in validateBiDiMarkup().
• Applies BiDi validation to NatSpec doc comments (/// and /** ... */) in addition to regular comments and string literals.

Problem

The existing lexer mitigation only tracked override/embedding markers (U+202A..U+202E + U+202C) and did not track isolate markers.
Also, NatSpec comment scanning paths were not validated by the BiDi checker.

As a result, unbalanced directional markers could be accepted in some cases.

Changes

Lexer

• Updated liblangutil/Scanner.cpp:
  • Extended validateBiDiMarkup() to track:
    • override depth (LRO/RLO/LRE/RLE vs PDF)
    • isolate depth (LRI/RLI/FSI vs PDI)
  • Added a range-based variant so validation can be run on explicit comment spans.
  • Hooked validation into NatSpec scanning paths in scanSlash():
    • single-line doc comments (///)
    • multi-line doc comments (/** ... */)

Tests

Added syntax tests:

• test/libsolidity/syntaxTests/comments/natspec_singleline_unicode_direction_override_1.sol
• test/libsolidity/syntaxTests/comments/singleline_unicode_direction_isolate_1.sol

Validation

Built solc and verified parsing behavior with:

• --stop-after parsing --error-codes on the two new tests
• existing directional-underflow test still reporting expected directional error

Notes

Error text remains the existing directional-override diagnostics for isolate imbalance as well, consistent with current scanner error categories.

[github-actions]

Thank you for your contribution to the Solidity compiler! A team member will follow up shortly.

If you haven't read our contributing guidelines and our review checklist before, please do it now, this makes the reviewing process and accepting your contribution smoother.

If you have any questions or need our help, feel free to post them in the PR or talk to us directly on the #solidity-dev channel on Matrix.