bug: Some multi-byte Unicode characters are not highlighted

[itojum]

Did you check existing issues?

• I have read all the tree-sitter docs if it relates to using the parser
• I have searched the existing issues of tree-sitter-c

Tree-Sitter CLI Version, if relevant (output of tree-sitter --version)

tree-sitter 0.26.10

Describe the bug

Some multi-byte Unicode characters are not being highlighted. Upon checking in the Playground, I discovered that an error occurs during tree generation.

Sample code that fails

:言 # Kanji
:你 # Simplified Chinese characters
:일 # Hangul characters

Successful sample code

:日
:广
:통

Use of AI: Launch Playground

I intend to contribute to the resolution to the fullest extent possible, such as by providing more samples and implementing the actual fixes.

Steps To Reproduce/Bad Parse Tree

All cases of failure are shown in the following tree.

program [0, 0] - [1, 0]
  ERROR [0, 0] - [0, 1]
  identifier [0, 1] - [0, 2]

Expected Behavior/Parse Tree

program [0, 0] - [1, 0]
  simple_symbol [0, 0] - [0, 2]

Repro