Implement Glushkov's NFA into cuDF

[lingyany-nv]

Description

Add bit-parallel Glushkov NFA regex engine with shared memory optimization

Implement Glushkov's NFA for regex string matching in cuDF to be more GPU friendly, references (1) hyperscan paper (2) HybridSA paper (3) vectorscan repo. Basically, this is Glushkov's NFA compared with the other popular Thompson's NFA (also used in current cuDF regex). The Glushkov engine represents NFA state as a single uint64_t bitmask (max 64 positions), requiring zero GPU working memory per thread compared to Thompson NFA's per-thread state arrays. A shared memory cache further accelerates execution by cooperatively loading read-only program data (reach masks, shift masks, exception successors) into SMEM at kernel entry.

Key changes

• Two-phase O(n) unanchored search algorithm (glushkov.inl): Phase 1 scans forward, injecting start states each character and recording provisional match ends. Phase 2 rescans only the match region to find the true leftmost start. Each character is processed at most twice.
• Leftmost-first correctness via priority-kill (glushkov.inl, glushkov_regcomp.cpp): A runtime glushkov_priority_kill clears lower-priority alternative paths at accept time. A compile-time conflict detector (frontier_has_priority_conflict) conservatively falls back to Thompson when bit-index ordering cannot guarantee Thompson-compatible leftmost-first semantics.
• Automatic fallback: Patterns with anchors (^, $, \b, \B), >64 positions, nullable top-level expressions, capture group requirements (extract, backref_re), or priority conflicts transparently fall back to Thompson NFA — no user intervention needed.
• Shared memory DataSource abstraction (glushkov.cuh, utilities.cuh): Templates compute_follow and compute_reach over glushkov_global_source vs glushkov_shmem_source, with cooperative SMEM loading in kernel wrappers.

Limitations

• do not support capturing groups (e.g. extract, replace_with_backrefs)
• do not support zero-width assertions like BOL/EOL/BOW/NBOW
• max 64 character-consuming positions since we are using uint64_t
• do not support lazy quantifiers
• empty/degenerate patterns rejected
• do not support nullable patterns as well as some ambiguous alternation patterns

When above condition is detected, it falls back to use the current Thompson's NFA.

Unit tests + benchmark

• Priority-kill parity tests: Verify Glushkov matches Thompson for overlapping-prefix alternations (foo|foobar, cat|catch, a|aa) across all 5 operations (contains, count, findall, replace, split)
• Nullable fallback parity: Confirm nullable patterns (a*, \d*, (ab)?) transparently fall back to Thompson and produce identical results
• Mixed-engine regression test (MixedEngineReplace): Exercises multi-pattern replace where one pattern is Glushkov-backed and another falls back to Thompson
• Spark-rapids compatibility: ~60 regex patterns from spark-rapids integration tests validated under both engines via parametrized Python tests
• Benchmarks: 6–9 patterns per benchmark covering char classes, alternation, bounded repetition, dot wildcards, and late-failure stress patterns; state.skip() guards for Glushkov-unsupported combinations (anchors, backrefs)
• Extended more complex regexes in the current split_re/contains/replace_re/count, it showed 1.01-6.62x speedup.

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.