Skip to content

Add SYCL Kernels for XPU backend #1679

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 38 commits into
base: main
Choose a base branch
from
Open

Add SYCL Kernels for XPU backend #1679

wants to merge 38 commits into from

Conversation

xiaolil1
Copy link

@xiaolil1 xiaolil1 commented Jun 15, 2025 •
edited
Loading

This is the pull request for the SYCL Kernels targeting the XPU backend.

  • It features the implementation of the "dequantize_blockwise," "dequantize_4bit," and "dequant & gemv_4bit fusion" kernels.
  • The target low-precision quantization datatypes encompass NF4, FP4 and General8bits.
  • This PR aims to eliminate the dependency on IPEX and improve the performance.

xiaolil1 and others added 19 commits June 15, 2025 16:08
@xiaolil1
Add SYCL Kernels for XPU backend
dd7b173
@xiaolil1
Merge pull request #1 from xiaolil1/jiqing
df93cdd 
fix transpose
@jiqing-feng
fix transpose
872aa02 
Signed-off-by: jiqing-feng <[email protected]>
@jiqing-feng
fix log and format
04437a3 
Signed-off-by: jiqing-feng <[email protected]>
@jiqing-feng
revert cpu changes
d585bea 
Signed-off-by: jiqing-feng <[email protected]>
@jiqing-feng
clean ipex_xpu
1781611 
Signed-off-by: jiqing-feng <[email protected]>
@jiqing-feng
clean ipex import
c982781 
Signed-off-by: jiqing-feng <[email protected]>
@jiqing-feng
fix ipex cpu import
a4c5f8c 
Signed-off-by: jiqing-feng <[email protected]>
@jiqing-feng
fix typo
4f076bb 
Signed-off-by: jiqing-feng <[email protected]>
@jiqing-feng
fix comments
76d7178 
Signed-off-by: jiqing-feng <[email protected]>
@xiaolil1
Merge pull request #2 from xiaolil1/jiqing
b31ea62 
revert cpu changes
@xiaolil1
refine gemv_4bit kernel
452aa84
@jiqing-feng
Merge branch 'main' into main
e8ac8b5
@xiaolil1 @leizhenyuan
enable FP4 for dequant_4bit and gemv_4bit
8620a95
@xiaolil1
refine FP4 dequantization performance
00f064b
@jiqing-feng
remove check for better performance
d60750f 
Signed-off-by: jiqing-feng <[email protected]>
@xiaolil1
Merge pull request #3 from xiaolil1/jiqing
59f2aa8 
remove check for better performance
@jiqing-feng
fix doc
aad358f 
Signed-off-by: jiqing-feng <[email protected]>
@xiaolil1
Merge pull request #4 from xiaolil1/jiqing
45e4451 
fix doc
@matthewdouglas matthewdouglas added Low Priority (will be worked on after all priority issues) Intel labels Jun 17, 2025
@matthewdouglas matthewdouglas self-assigned this Jun 17, 2025
@matthewdouglas matthewdouglas self-requested a review June 17, 2025 16:19
@matthewdouglas matthewdouglas added this to the v0.48.0 milestone Jun 17, 2025
@fengyuan14
Copy link

Can we use a more accurate title for the commit? or reviewers would get confused if all SYCL kernels are included in the PR.

fengyuan14
fengyuan14 reviewed Jun 18, 2025
View reviewed changes
fengyuan14
fengyuan14 reviewed Jun 18, 2025
View reviewed changes
fengyuan14
fengyuan14 reviewed Jun 18, 2025
View reviewed changes
fengyuan14
fengyuan14 reviewed Jun 18, 2025
View reviewed changes
fengyuan14
fengyuan14 reviewed Jun 18, 2025
View reviewed changes
fengyuan14
fengyuan14 reviewed Jun 18, 2025
View reviewed changes
fengyuan14
fengyuan14 reviewed Jun 18, 2025
View reviewed changes
fengyuan14
fengyuan14 reviewed Jun 18, 2025
View reviewed changes
fengyuan14
fengyuan14 reviewed Jun 18, 2025
View reviewed changes
xiaolil1
xiaolil1 commented Jun 18, 2025
View reviewed changes
xiaolil1 and others added 17 commits June 18, 2025 03:53
@xiaolil1
clean code
1e21ee9
@xiaolil1
Merge branch 'main' into main
4e7f5c1
@jiqing-feng
fix tests
1601652 
Signed-off-by: jiqing-feng <[email protected]>
@jiqing-feng
rm comments
1cc25ff 
Signed-off-by: jiqing-feng <[email protected]>
@xiaolil1
Merge pull request #5 from xiaolil1/jiqing
c44f38e 
fix tests
@xiaolil1
fix memory issue
9f283bd
@xiaolil1
fix ut failure
9897eae
@jiqing-feng
adjust threshold
411a276 
Signed-off-by: jiqing-feng <[email protected]>
@jiqing-feng
fix xpu check
b6a3524 
Signed-off-by: jiqing-feng <[email protected]>
@jiqing-feng
change test_functional check
1c4f478 
Signed-off-by: jiqing-feng <[email protected]>
@jiqing-feng
fix test_module
e5cf821 
Signed-off-by: jiqing-feng <[email protected]>
@xiaolil1
Merge pull request #6 from xiaolil1/jiqing
502fe83 
Fix xpu check
@jiqing-feng
fix device check
8b54381 
Signed-off-by: jiqing-feng <[email protected]>
@jiqing-feng
Merge pull request #7 from xiaolil1/jiqing_test
1e0f661 
fix device check
@jiqing-feng
fix tests
99698d2 
Signed-off-by: jiqing-feng <[email protected]>
@jiqing-feng
Merge pull request #8 from xiaolil1/jiqing
b88236a 
fix tests
@jiqing-feng
Merge branch 'main' into main
56c48bc
@jiqing-feng
Copy link
Contributor

jiqing-feng commented Jun 24, 2025 •
edited
Loading

Hi @matthewdouglas . The PR is ready to be reviewed. The sycl kernel could get 0-150% speed-up compared to triton on 4bit models. Could you take the 1st round review? Please let me know if you have any concerns. Thanks!

@xiaolil1 xiaolil1 marked this pull request as ready for review June 25, 2025 01:51
@jiqing-feng jiqing-feng mentioned this pull request Jun 27, 2025
Closed
@fengyuan14 fengyuan14 mentioned this pull request Jun 27, 2025
Closed
@xiaolil1 xiaolil1 changed the title Add SYCL Kernels for XPU backend Add SYCL Kernels for QLoRA XPU backend Jun 27, 2025
@xiaolil1 xiaolil1 changed the title Add SYCL Kernels for QLoRA XPU backend Add SYCL Kernels for XPU backend Jun 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fengyuan14 fengyuan14 fengyuan14 left review comments

@matthewdouglas matthewdouglas Awaiting requested review from matthewdouglas

Assignees

@matthewdouglas matthewdouglas

Labels
Intel Low Priority (will be worked on after all priority issues)
Projects
None yet
Milestone
v0.48.0
Development

Successfully merging this pull request may close these issues.
4 participants
@xiaolil1 @fengyuan14 @jiqing-feng @matthewdouglas