`ContentSearchService` crashes with "String index is out of bounds" on CJK content

[qqfly]

Describe the bug

Searching a Chinese-language EPUB via publication.search(query:) reliably crashes with a Swift precondition failure inside ContentSearchService:

libswiftCore.dylib: _assertionFailure — String index is out of bounds
  String.index(_:offsetBy:)
  Iterator.makeLocator(matchStart:matchEnd:)   (ContentSearchService.swift:510)
  Iterator.search(dangerZoneCapacity:)
  Iterator.next()

How to reproduce?

1. Open any reflowable Chinese EPUB (multi-byte CJK text; ours also mixes Latin fragments).
2. let iterator = try await publication.search(query: "测试").get()
3. _ = await iterator.next() → crash on books/queries where matches land after mixed-width text.

Readium version

3.11.0

OS version

iOS 26.5

Testing device

iPhone 15 Pro (device) + iPhone 17 (simulator)

Environment

macOS: 26.5.2
platform: arm64
Xcode 26.6
Build version 17F113

Additional context

Analysis

In ContentSearchService.search(dangerZoneCapacity:), match ranges returned by the search algorithm are converted to window offsets via searchSlice.distance(from:to:) (Character distances), and later makeLocator(matchStart:matchEnd:) re-derives indices with windowText.index(windowText.startIndex, offsetBy: matchStart). When the window has been trimmed (trimAmount bookkeeping) and/or when findRanges operates on NSString/UTF-16 semantics (e.g. .diacriticInsensitive via ICU), the character-offset arithmetic drifts on multi-byte CJK content, producing offsets past windowText.endIndex — String.index(_:offsetBy:) then traps.

A defensive fix would be index(_:offsetBy:limitedBy:) with a nil-guard in makeLocator/search; a root fix is keeping all offset bookkeeping in a single unit (either String.Index end-to-end, or UTF-16 offsets converted at the boundary).

Note that ContentSearchService has been the default EPUB search backend since 3.9.0 (PR #787), and the 3.9.0–3.11.0 changelogs show no CJK/offset-related fixes in Shared — so the same window/offset arithmetic likely affects all three releases, not just 3.11.0.

Workaround

We replaced the service with a custom SearchService (per-segment String.range(of:options:) matching, no window arithmetic) via setSearchServiceFactory, which is stable on CJK.

[mickael-menu]

Thank you for reporting this issue. Would you be able to share an EPUB and a search query we could use to reproduce the crash? Even if it's a partial or fake EPUB, that's fine. Thanks.

[qqfly]

Attached is a self-contained reproducer (24 KB): a Swift package pinned to
swift-toolkit 3.11.0, with the EPUBs and an XCTest.

./run.sh          # picks whatever iOS simulator you have installed

The EPUBs are already in the zip — make_epub.py is only there if you want to
regenerate them. All the prose is text I wrote for this report, so there is
nothing copyrighted in it.

Two tests crash, five pass:

test	fixture	result
testCombiningMark	paragraph starts with U+0301	crash
testVariationSelector	paragraph starts with U+FE0F	crash
testCleanControl	plain CJK prose	144 hits
testCRLF	CRLF line endings	144 hits
testEmojiZWJ	ZWJ emoji inside paragraphs	144 hits
testFullWidth	full-width latin/digits in CJK	144 hits
testDiacritic	café / Ångström, precomposed + decomposed	144 hits

I kept the passing ones in because they narrow the trigger down to grapheme
clusters that merge at a concatenation boundary.

What seems to happen: appendToWindow tracks window offsets by adding
Character counts, but Swift recomputes grapheme breaking across the
concatenation:

" ".count + "\u{0301}猫".count   // 3
(" " + "\u{0301}猫").count       // 2

So when an element's text starts with a combining character, it merges with the
separator space and windowTextCount ends up larger than the real
windowText.count. Once a match lands near the end of the window,
makeLocator's index(_:offsetBy: matchEnd) walks past the end and traps.

Recalibrating windowTextCount = windowText.count after each append would fix
the drift; index(_:offsetBy:limitedBy:) in makeLocator and
extractSnippetBefore/After would at least stop the trap. Carrying
String.Index instead of Int offsets would remove the whole class of bug.

One caveat: our original crash was on a commercial Chinese EPUB I can't share,
and that book has no combining marks or variation selectors (it does have CRLF,
but testCRLF passes). So there may be a second path to the same drift that
these fixtures don't cover. Happy to test a patch against the real book.

readium-search-crash-repro.zip

[mickael-menu]

@qqfly Would you mind checking if this branch addresses the issue for you? fix-issue-876. Thank you! Unfortunately it's too late for 3.x and this we'll be available in 4.0.0-alpha.1.

#878

[qqfly]

Confirmed — fix-issue-876 resolves it. I re-ran the reproducer against the
branch (afa729d); the two crashing cases now pass and still return the correct
144 hits each, so matches aren't being dropped:

testCombiningMark      ✅ 144 hits   (was: crash on 3.11.0)
testVariationSelector  ✅ 144 hits   (was: crash on 3.11.0)
testCleanControl       ✅ 144 hits
testCRLF               ✅ 144 hits
testEmojiZWJ           ✅ 144 hits
testFullWidth          ✅ 144 hits
testDiacritic          ✅ 144 hits
** TEST SUCCEEDED **

Thanks for the quick fix.