Skip to content

perf(tk-encode): SIMD-first encode pipeline — fused split cache + de-virtualized BPE - #2234

Draft
ArthurZucker wants to merge 5 commits into
perf/tk-encode-pretok-basefrom
perf/fused-split-cache
Draft

perf(tk-encode): SIMD-first encode pipeline — fused split cache + de-virtualized BPE#2234
ArthurZucker wants to merge 5 commits into
perf/tk-encode-pretok-basefrom
perf/fused-split-cache

Commits