Skip to content

Commit 633d1c7

Browse files
Add PyTorch 2.6 QuickStart for Intel GPU (#13024)
* Add quickstart for install IPEX-LLM with PyTorch 2.6 on Intel GPUs * Add jump links * Rename * Small fix * Small fix * Update based on comments * Small fix
1 parent 34b1b14 commit 633d1c7

File tree

6 files changed

+241
-0
lines changed

6 files changed

+241
-0
lines changed

docs/mddocs/Overview/install_gpu.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,9 @@
22

33
## Windows
44

5+
> [!NOTE]
6+
> For installation with PyTorch 2.6, please refer to this [guide](../Quickstart/install_pytorch26_gpu.md#windows-quickstart) for more information.
7+
58
### Prerequisites
69

710
IPEX-LLM on Windows supports Intel iGPU and dGPU.
@@ -171,6 +174,9 @@ If you met error when importing `intel_extension_for_pytorch`, please ensure tha
171174

172175
## Linux
173176

177+
> [!NOTE]
178+
> For installation with PyTorch 2.6, please refer to this [guide](../Quickstart/install_pytorch26_gpu.md#linux-quickstart) for more information.
179+
174180
### Prerequisites
175181

176182
IPEX-LLM GPU support on Linux has been verified on:

docs/mddocs/Quickstart/install_linux_gpu.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,9 @@
55

66
This guide demonstrates how to install IPEX-LLM on Linux with Intel GPUs. It applies to Intel Data Center GPU Flex Series and Max Series, as well as Intel Arc Series GPU and Intel iGPU.
77

8+
> [!NOTE]
9+
> For installation with PyTorch 2.6, please refer to this [guide](./install_pytorch26_gpu.md#linux-quickstart) for more information.
10+
811
> [!NOTE]
912
> For installation on Intel Arc B-Series GPU (such as **B580**), please refer to this [guide](./bmg_quickstart.md).
1013

docs/mddocs/Quickstart/install_linux_gpu.zh-CN.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,9 @@
55

66
本指南将引导你如何在带有 Intel GPU 的 Linux 系统上安装 IPEX-LLM。适用于 Intel 数据中心的 GPU Flex 和 Max 系列,以及 Intel Arc 系列 GPU 和 Intel iGPU。
77

8+
> [!NOTE]
9+
> 如果需要安装 IPEX-LLM PyTorch 2.6 版本,请参阅本[指南](./install_pytorch26_gpu.md#linux-quickstart)获取详细信息。
10+
811
> [!NOTE]
912
> 如果是在 Intel Arc B 系列 GPU 上安装(例,**B580**),请参阅本[指南](./bmg_quickstart.md)
1013
Lines changed: 223 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,223 @@
1+
# Install IPEX-LLM on Intel GPU with PyTorch 2.6
2+
3+
This guide demonstrates how to install IPEX-LLM on Intel GPUs with PyTorch 2.6 support.
4+
5+
IPEX-LLM with PyTorch 2.6 provides a simpler prerequisites setup process, without requiring manual installation of oneAPI. Besides, it offers broader platform support with AOT (Ahead of Time) Compilation.
6+
7+
> [!TIP]
8+
> For details on which device IPEX-LLM PyTorch 2.6 supports with AOT compilation, you could refer to here ([Windows](https://pytorch-extension.intel.com/installation?platform=gpu&version=v2.6.10%2Bxpu&os=windows&package=pip#:~:text=following%20system%20requirements%3A-,1.1.%20Hardware,-Supported%20by%20prebuilt) or [Linux](https://pytorch-extension.intel.com/installation?platform=gpu&version=v2.6.10%2Bxpu&os=linux%2Fwsl2&package=pip#:~:text=following%20system%20requirements%3A-,1.1.%20Hardware,-Supported%20by%20prebuilt)) for more information.
9+
10+
## Table of Contents
11+
- [Windows Quickstart](#windows-quickstart)
12+
- [Install Prerequisites](#install-prerequisites)
13+
- [Install `ipex-llm`](#install-ipex-llm)
14+
- [Runtime Configurations](#runtime-configurations)
15+
- [Verify Installation](#verify-installation)
16+
- [Linux Quickstart](#linux-quickstart)
17+
- [Install Prerequisites](#install-prerequisites-1)
18+
- [Install `ipex-llm`](#install-ipex-llm-1)
19+
- [Runtime Configurations](#runtime-configurations-1)
20+
- [Verify Installation](#verify-installation-1)
21+
22+
## Windows Quickstart
23+
24+
### Install Prerequisites
25+
26+
#### Update GPU Driver
27+
28+
We recommend updating your GPU driver to the [latest](https://www.intel.com/content/www/us/en/download/785597/intel-arc-iris-xe-graphics-windows.html). A system reboot is necessary to apply the changes after the installation is complete.
29+
30+
#### Setup Python Environment
31+
32+
Visit [Miniforge installation page](https://conda-forge.org/download/), download the **Miniforge installer for Windows**, and follow the instructions to complete the installation.
33+
34+
<div align="center">
35+
<img src="https://llm-assets.readthedocs.io/en/latest/_images/quickstart_windows_gpu_miniforge_download.png" width=80%/>
36+
</div>
37+
38+
After installation, open the **Miniforge Prompt**, create a new python environment `llm-pt26`:
39+
```cmd
40+
conda create -n llm-pt26 python=3.11
41+
```
42+
Activate the newly created environment `llm-pt26`:
43+
```cmd
44+
conda activate llm-pt26
45+
```
46+
47+
### Install `ipex-llm`
48+
49+
With the `llm-pt26` environment active, use `pip` to install `ipex-llm` for GPU:
50+
51+
- For **Intel Core™ Ultra Processors (Series 2) with processor number 2xxH (code name Arrow Lake)**:
52+
53+
Choose either US or CN website for `extra-index-url`:
54+
55+
- For **US**:
56+
57+
```cmd
58+
pip install --pre --upgrade ipex-llm[xpu_2.6_arl] --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/arl/us/
59+
```
60+
61+
- For **CN**:
62+
63+
```cmd
64+
pip install --pre --upgrade ipex-llm[xpu_2.6_arl] --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/arl/cn/
65+
```
66+
67+
> [!TIP]
68+
> For other Intel Core™ Ultra Processors, such as 2xxHX, please refer to the installation instruction below (i.e. for **other Intel iGPU and dGPU**).
69+
70+
- For **other Intel iGPU and dGPU**:
71+
72+
```cmd
73+
pip install --pre --upgrade ipex-llm[xpu_2.6] --extra-index-url https://download.pytorch.org/whl/xpu
74+
```
75+
76+
### Runtime Configurations
77+
78+
For optimal performance, it is recommended to set several environment variables. Please check out the suggestions based on your device.
79+
80+
With the `llm-pt26` environment active:
81+
82+
- For **Intel Arc™ A-Series GPU (code name Alchemist)**
83+
84+
```cmd
85+
set SYCL_CACHE_PERSISTENT=1
86+
set UR_L0_USE_IMMEDIATE_COMMANDLISTS=0
87+
```
88+
89+
> [!TIP]
90+
> It is recommanded to experiment with `UR_L0_USE_IMMEDIATE_COMMANDLISTS=0` or `1` for best performance on Intel Arc™ A-Series GPU.
91+
92+
- For **other Intel iGPU and dGPU**:
93+
94+
```cmd
95+
set SYCL_CACHE_PERSISTENT=1
96+
:: [optional] The following environment variable may improve performance, but in some cases, it may also lead to performance degradation
97+
set SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
98+
```
99+
100+
> [!NOTE]
101+
> The environment variable `SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS` determines the usage of immediate command lists for task submission to the GPU. It is highly recommanded to experiment with `SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1` or `0` on your device for best performance.
102+
>
103+
> You could refer to [here](https://www.intel.com/content/www/us/en/developer/articles/guide/level-zero-immediate-command-lists.html) regarding more information about Level Zero Immediate Command Lists.
104+
105+
### Verify Installation
106+
107+
You can verify if `ipex-llm` is successfully installed following below steps:
108+
109+
- Open the **Miniforge Prompt** and activate the Python environment `llm-pt26` you previously created:
110+
111+
```cmd
112+
conda activate llm-pt26
113+
```
114+
115+
- Set environment variables according to the [Runtime Configurations section](#runtime-configurations).
116+
117+
- Launch the Python interactive shell by typing `python` in the Miniforge Prompt window and then press Enter.
118+
119+
- Copy following code to Miniforge Prompt **line by line** and press Enter **after copying each line**.
120+
121+
```python
122+
import torch
123+
from ipex_llm.transformers import AutoModel, AutoModelForCausalLM
124+
tensor_1 = torch.randn(1, 1, 40, 128).to('xpu')
125+
tensor_2 = torch.randn(1, 1, 128, 40).to('xpu')
126+
print(torch.matmul(tensor_1, tensor_2).size())
127+
```
128+
129+
It should output following content at the end:
130+
131+
```
132+
torch.Size([1, 1, 40, 40])
133+
```
134+
135+
- To exit the Python interactive shell, simply press Ctrl+Z then press Enter (or input `exit()` then press Enter).
136+
137+
138+
## Linux Quickstart
139+
140+
### Install Prerequisites
141+
142+
#### Install GPU Driver
143+
144+
We recommend following [Intel client GPU driver installation guide](https://dgpu-docs.intel.com/driver/client/overview.html) to install your GPU driver.
145+
146+
#### Setup Python Environment
147+
148+
Download and install the Miniforge as follows if you don't have conda installed on your machine:
149+
150+
```bash
151+
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
152+
bash Miniforge3-Linux-x86_64.sh
153+
source ~/.bashrc
154+
```
155+
156+
You can use `conda --version` to verify you conda installation.
157+
158+
After installation, create a new python environment `llm-pt26`:
159+
```bash
160+
conda create -n llm-pt26 python=3.11
161+
```
162+
Activate the newly created environment `llm-pt26`:
163+
```bash
164+
conda activate llm-pt26
165+
```
166+
167+
### Install `ipex-llm`
168+
169+
With the `llm-pt26` environment active, use `pip` to install `ipex-llm` for GPU:
170+
171+
```bash
172+
pip install --pre --upgrade ipex-llm[xpu_2.6] --extra-index-url https://download.pytorch.org/whl/xpu
173+
```
174+
175+
### Runtime Configurations
176+
177+
For optimal performance, it is recommended to set several environment variables. Please check out the suggestions based on your device.
178+
179+
With the `llm-pt26` environment active:
180+
181+
```bash
182+
unset OCL_ICD_VENDORS
183+
export SYCL_CACHE_PERSISTENT=1
184+
# [optional] The following environment variable may improve performance, but in some cases, it may also lead to performance degradation
185+
export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
186+
```
187+
188+
> [!NOTE]
189+
> The environment variable `SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS` determines the usage of immediate command lists for task submission to the GPU. It is highly recommanded to experiment with `SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1` or `0` on your device for best performance.
190+
>
191+
> You could refer to [here](https://www.intel.com/content/www/us/en/developer/articles/guide/level-zero-immediate-command-lists.html) regarding more information about Level Zero Immediate Command Lists.
192+
193+
### Verify Installation
194+
195+
You can verify if `ipex-llm` is successfully installed following below steps:
196+
197+
- Activate the Python environment `llm-pt26` you previously created:
198+
199+
```cmd
200+
conda activate llm-pt26
201+
```
202+
203+
- Set environment variables according to the [Runtime Configurations section](#runtime-configurations-1).
204+
205+
- Launch the Python interactive shell by typing `python` in the terminal and then press Enter.
206+
207+
- Copy following code to Miniforge Prompt **line by line** and press Enter **after copying each line**.
208+
209+
```python
210+
import torch
211+
from ipex_llm.transformers import AutoModel, AutoModelForCausalLM
212+
tensor_1 = torch.randn(1, 1, 40, 128).to('xpu')
213+
tensor_2 = torch.randn(1, 1, 128, 40).to('xpu')
214+
print(torch.matmul(tensor_1, tensor_2).size())
215+
```
216+
217+
It should output following content at the end:
218+
219+
```
220+
torch.Size([1, 1, 40, 40])
221+
```
222+
223+
- To exit the Python interactive shell, simply press Ctrl+C then press Enter (or input `exit()` then press Enter).

docs/mddocs/Quickstart/install_windows_gpu.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,9 @@
55

66
This guide demonstrates how to install IPEX-LLM on Windows with Intel GPUs.
77

8+
> [!NOTE]
9+
> For installation with PyTorch 2.6, please refer to this [guide](./install_pytorch26_gpu.md#windows-quickstart) for more information.
10+
811
> [!NOTE]
912
> For installation on Intel Arc B-Series GPU (such as **B580**), please refer to this [guide](./bmg_quickstart.md).
1013

docs/mddocs/Quickstart/install_windows_gpu.zh-CN.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,9 @@
55

66
本指南将引导你如何在具有 Intel GPUs 的 Windows 系统上安装 IPEX-LLM。
77

8+
> [!NOTE]
9+
> 如果需要安装 IPEX-LLM PyTorch 2.6 版本,请参阅本[指南](./install_pytorch26_gpu.md#windows-quickstart)获取详细信息。
10+
811
> [!NOTE]
912
> 如果是在 Intel Arc B 系列 GPU 上安装(例,**B580**),请参阅本[指南](./bmg_quickstart.md)
1013

0 commit comments

Comments
 (0)