pcre2_callout_enumerate fails for patterns with Unicode character classes

[NWilson]

This is a bug in PCRE2 10.45 and 10.46, not present in 10.44 and earlier.

Applications which call pcre2_callout_enumerate should potentially downgrade to PCRE2 10.44 until we release an update of PCRE2 with the fix.

Since this function is (clearly!!) not used often by applications, I am not currently treating this as very high severity. There is no way for an attacker to make any application call this function, if it is not currently using it.

The root cause seems to be commit 24f9d8d (#540).

(I apologise for pointing to the specific commit - I am not blaming anyone. I just think we need to be sure we understand when the bug was introduced.)

The layout of the pcre2_code object was altered from struct, nametable, code to struct, nametable, classlists, code. Nearly all the relevant code was updated, but clearly, pcre2_callout_enumerate was missed out.

We also are guilty of having inadequate code coverage of this function. We should be confident to make changes like this and know that our test suite will catch any errors. Sadly, the test suite only covers a few happy paths through pcre2_callout_enumerate, on a handful of simple patterns, and none of them have complicated character classes.

---

Please comment if you want to express an opinion on how we handle this bug (whether to do a rapid release, or to hurry up our scheduled 10.47, or to just fix it whenever 10.47 comes around, with no extra hurry).

[ltrzesniewski]

Please comment if you want to express an opinion on how we handle this bug (whether to do a rapid release, or to hurry up our scheduled 10.47, or to just fix it whenever 10.47 comes around, with no extra hurry).

It's unclear what type of bug this is. Does the function return incorrect results, or is there a buffer overflow somewhere? You seem to mean the latter, but I didn't find any issue by adding character classes to an existing unit test.

I ran the tests on the #801 branch with and without the fix: pcre2test sometimes loops infinitely on testinput4 line 574 when the fix is not applied.

I reproduced the issue on PCRE.NET with that pattern. Applying your fix solves the problem, so I will do a patch release (I'll just wait for you to validate #801 since it's not merged yet). But as you say, no one uses pcre2_callout_enumerate, so I don't have a strong opinion on what you should do there. I suppose it depends on what other features you want to include in 10.47.

[NWilson]

I am sorry I didn't make the class of bug clear.

It's an or of bounds read of arbitrary size, arbitrarily far from the allocated region (although it's hard to read memory very far away).

There's a loop that reads in bytes of bytecode. A few of the instructions are followed by a 16-bit offset to the next instruction, otherwise the loop just reads forwards byte by byte.

The problem is that it starts on the wrong location if the pattern has a character class containing Unicode characters >u+ff. The character class data is misinterpreted as bytecode, so the loop can jump pretty much anywhere.

You basically get a crash. Anyone using this function with Unicode patterns in 10.45/46 is bound to have discovered this, leading me to conclude there's little usage.

I will see what others think before deciding what to do. The patch can be considered official, if you want to apply it.

[ltrzesniewski]

Ok, thanks for the details! I suppose ASLR makes the crashes happen randomly.