fix utf8 handling in C code#3
Conversation
| s.require_paths = ["lib"] | ||
|
|
||
| s.add_development_dependency "nokogiri", "~> 1.6.0" | ||
| s.add_development_dependency "rake", "~> 10.0" |
There was a problem hiding this comment.
NOTE: I can't build it with the latest rake, so fixed the version to v10.
|
CI fails because of activesupport dependency. How can I fix it? |
|
@yujinakayama ping? |
|
Sorry for the late response. Right now I'm busy but will review in this week 🙏 |
The CI issues are fixed in the current master. Could you please rebase against the master? 🙏 |
|
|
||
| for (link_end = 0; link_end < size; ++link_end) { | ||
| uint8_t c = data[link_end]; | ||
| char c = data[link_end]; |
There was a problem hiding this comment.
I understand this change forces the c to be virtually 7-bit ASCII but it's a bit obscure. Maybe the way in vmg#463 and defining a macro something like is_ascii() (c < 0x7f) would be more explicit.
There was a problem hiding this comment.
I understand this change forces the c to be virtually 7-bit ASCII
No, it might be a part of UTF-8's middle bytes, and the change of the type doesn't change the bit pattern of c.
Because c is dealt as char, it should be char, not uint8_t (i.e. unsigned char). I believe this code is the best.
gfx
force-pushed
the
fix_utf8_handling
branch
from
2718458 to
a316a54
Compare
|
Rebased. |
|
Thanks! |
|
Released as v3.2.2.2. |
|
Thanks! |
3 participants
Fix #2, which comes in redcarpet.
Some tests are borrowed from vmg#464