Merge pull request #112 from MooreThreads/fmo-update

update(Version): Update to v2.7.0

Author: fmo-mt

CMakeLists.txt

@@ -380,6 +380,12 @@ target_compile_definitions(${MUSA_KERNELS_LIB}
 target_compile_definitions(${PLUGIN_NAME}
                            PUBLIC -DREAL_MUSA_VERSION=${REAL_MUSA_VERSION})
 
+set(MUSA_COMP_VERSION "10400")
+target_compile_definitions(${MUSA_KERNELS_LIB}
+                           PUBLIC -DMUSA_COMP_VERSION=${MUSA_COMP_VERSION})
+target_compile_definitions(${PLUGIN_NAME}
+                           PUBLIC -DMUSA_COMP_VERSION=${MUSA_COMP_VERSION})
+
 add_dependencies(${MUSA_KERNELS_LIB} ${CODEGEN_TARGET})
 
 string(REPLACE ";" " " PYTORCH_MUSA_ARCH_readable "${TORCH_MUSA_ARCH_LIST}")

README.md

@@ -4,7 +4,7 @@
 <!-- toc -->
 
 - [Overview](#overview)
-- [Installation](#installation)  
+- [Installation](#installation)
   - [Prerequisites](#prerequisites)
   - [Docker Image](#docker-image)
   - [From Python wheels](#from-python-wheels)
@@ -21,12 +21,14 @@
 
 ## Overview
 
-**torch_musa** is an extended Python package based on PyTorch. Combined with PyTorch, users can take advantage of the strong power of Moore Threads graphics cards through **torch_musa**. 
+**torch_musa** is an extended Python package based on PyTorch. Combined with PyTorch, users can take advantage of the strong power of Moore Threads graphics cards through **torch_musa**.
 
 **torch_musa**'s APIs are consistent with PyTorch in format, which allows users accustomed to PyTorch to migrate smoothly to **torch_musa**, so for the usage users can refer to [PyTorch Official Doc](https://docs.pytorch.org/docs/stable/index.html), all you need is just switch the backend string from "cpu" or "cuda" to "musa".
 
 **torch_musa** also provides a bundle of tools for users to conduct cuda-porting, building musa extension and debugging. Please refer to [README.md](torch_musa/utils/README.md).
 
+For some customize optimizations, like **Dynamic Double Casting** and **Unified Memory Management**, please refer to [README.md](torch_musa/README.md).
+
 If you want to write your layers in C/C++, we provide a convenient extension API that is efficient and with minimal boilerplate. No wrapper code needs to be written. You can see [a ResNet50 example here](torch_musa/examples/cpp/README.md).
 
 --------------------------------------------------------------------------------
@@ -94,21 +96,32 @@ USE_MCCL=0 bash build.sh -c
 
 ## MUSA Supported Repositories
 We provide some widely used PyTorch environment repositories, which have all adapted with our MUSA platform. Besides, many repositories have supported MUSA backend upstream,
-like [Transformers](https://github.com/huggingface/transformers.git), [Accelerate](https://github.com/huggingface/accelerate.git), you can install them with `pip install [repo-name]`.
+like [Transformers](https://github.com/huggingface/transformers.git), [Accelerate](https://github.com/huggingface/accelerate.git),
+you can install them from PyPi with `pip install [repo-name]`.
 
-### torchvision and torchaudio
-PyTorch v2.5.0 needs `torchvision==0.20.0` and `torchaudio==2.5.0`, and for torch_musa users we
-shouldn't have them installed like `pip install torchvision==0.20.0`, instead, we should build
-them from source:
+### torchvision
+For torch_musa v2.7.0 and later, install torchvision from our musified one:
+```shell
+git clone https://github.com/MooreThreads/vision -b v0.22.1-musa --depth 1
+cd vision && python setup.py install
+```
+
+
+For torch_musa v2.5.0 and earlier, install torchvision from source with:
 ```shell
-# build & install torchvision
-git clone https://github.com/pytorch/vision.git -b v0.20.0 --depth 1
-cd visoin && python setup.py install
+git clone https://github.com/pytorch/vision -b ${version} --depth 1
+cd vision && python setup.py install
+```
+the `version` is depend on torch version, for example you have torch_musa v2.5.0 with torch v2.5.0, install `torchvision==0.20.0`.
 
-# build & install torchaudio
-git clone https://github.com/pytorch/audio.git -b v2.5.0 --depth 1
+### torchaudio
+Install torchaudio from source:
+them from source:
+```shell
+git clone https://github.com/pytorch/audio.git -b ${version} --depth 1
 cd audio && python setup.py install
 ```
+the `version` is same as the torch version.
 
 ### Other Repositories
 There are many widely used pytorch-related repositories, and we musified some of them and put them into our [GitHub](https://github.com/MooreThreads), here's the list:
@@ -117,6 +130,7 @@ There are many widely used pytorch-related repositories, and we musified some of
 | pytorch3d | musa-dev | https://github.com/MooreThreads/pytorch3d | python setup.py install |
 | pytorch_sparse | master | https://github.com/MooreThreads/pytorch_sparse | python setup.py install |
 | pytorch_scatter | master | https://github.com/MooreThreads/pytorch_scatter | python setup.py install |
+| torchvision | v0.22.1-musa | https://github.com/MooreThreads/vision | python setup.py install |
 | More to come... | | | |
 
 If users find any question about these repos, please file issues in torch_musa, and if anyone  musify a repository, you can
@@ -145,7 +159,7 @@ The following two key changes are required when using **torch_musa**:
   ```
   # ensure torchvision is not installed
   pip uninstall torchvision
-  
+
   git clone https://github.com/pytorch/vision.git
   cd vision
   python setup.py install

build.sh

@@ -16,7 +16,7 @@ TORCH_MUSA_HOME=$CUR_DIR
 PYTORCH_PATH=${PYTORCH_REPO_PATH:-$(realpath ${TORCH_MUSA_HOME}/../pytorch)}
 TORCH_PATCHES_DIR=${TORCH_MUSA_HOME}/torch_patches/
 KINETO_URL=${KINETO_URL:-https://github.com/MooreThreads/kineto.git}
-KINETO_TAG=v2.0.1
+KINETO_TAG=v2.7.0
 
 BUILD_WHEEL=0
 DEBUG_MODE=0
@@ -27,7 +27,7 @@ USE_KINETO=${USE_KINETO:-1}
 ONLY_PATCH=0
 CLEAN=0
 COMPILE_FP64=1
-PYTORCH_TAG=v2.5.0
+PYTORCH_TAG=v2.7.1
 PYTORCH_BUILD_VERSION="${PYTORCH_TAG:1}"
 PYTORCH_BUILD_NUMBER=0 # This is used for official torch distribution.
 USE_MCCL=${USE_MCCL:-1}
@@ -119,7 +119,7 @@ done
 
 cmd_check(){
   cmd="$1"
-  if command -v ${cmd} >/dev/null 2>&1; then 
+  if command -v ${cmd} >/dev/null 2>&1; then
     echo "- cmd exist  : ${cmd}"
   else
     echo -e "\033[34m- cmd does not exist, automatically install \"${cmd}\"\033[0m"
@@ -132,7 +132,7 @@ precommit_install(){
   root_dir="$(dirname "$(realpath "${BASH_SOURCE:-$0}" )")"
   if [ ! -f ${root_dir}/.git/hooks/pre-commit ]; then
     pushd $root_dir
-    pre-commit install 
+    pre-commit install
     popd
   fi
 }
@@ -174,7 +174,7 @@ apply_torch_patches() {
     popd
   fi
 
-  for file in $(find ${TORCH_PATCHES_DIR} -type f -print); do
+  for file in $(find ${TORCH_PATCHES_DIR} -type f -not -path "*/kineto/*" -print); do
     if [ "${file##*.}"x = "patch"x ]; then
       echo -e "\033[34mapplying patch: $file \033[0m"
       pushd $PYTORCH_PATH
@@ -183,13 +183,27 @@ apply_torch_patches() {
       popd
     fi
   done
+
+  if [ ${USE_KINETO} -eq 1 ]; then
+    for file in $(find ${TORCH_PATCHES_DIR}/kineto -type f -print); do
+      if [ "${file##*.}"x = "patch"x ]; then
+        echo -e "\033[34mapplying patch: $file \033[0m"
+        pushd $PYTORCH_PATH
+        git apply --check $file
+        git apply $file
+        popd
+      fi
+    done
+  fi
 }
 
 update_kineto_source() {
   echo -e "\033[34mUpdating Kineto...\033[0m"
   pushd ${PYTORCH_PATH}
+  # remove the current kineto
   rm -rf ${PYTORCH_PATH}/third_party/kineto
   git submodule update --init --recursive --depth 1
+  # remove the official kineto
   rm -rf ${PYTORCH_PATH}/third_party/kineto
   popd
   echo -e "\033[34mUpdating KINETO_URL, might take a while...\033[0m"
@@ -230,7 +244,7 @@ update_submodule() {
       pushd ${PYTORCH_PATH}
       git submodule update --init --recursive --depth 1
       popd
-    elif [ "${remote_url}" = "${KINETO_URL}" ] && [ "${current_tag}" = "${KINETO_TAG}" ]; then
+    elif [ "${remote_url}" = "${KINETO_URL}" ]; then
       pushd ${PYTORCH_PATH}/third_party/kineto
       echo  -e "\033[34mUpdating KINETO submodule, might take a while...\033[0m"
       git submodule update --init --recursive
@@ -244,6 +258,13 @@ update_submodule() {
       popd
       rm -rf ${PYTORCH_PATH}/third_party/kineto
       mv /tmp/kineto ${PYTORCH_PATH}/third_party
+      if [ "${current_tag}" != "${KINETO_TAG}" ]; then
+        echo  -e "\033[34mUpdate the kineto to the [${KINETO_TAG}]\033[0m"
+        pushd ${PYTORCH_PATH}/third_party/kineto
+        git fetch origin tag ${KINETO_TAG}
+        git checkout ${KINETO_TAG}
+        popd
+      fi
     else
       update_kineto_source
     fi

cmake/utils.cmake

@@ -34,28 +34,28 @@ endfunction()
 
 function(parse_real_musa_version outputvar)
   find_program(
-    MUSA_TOOLKIT_VERSION_EXECUTABLE
-    NAMES musa_toolkits_version
+    MUSA_RUNTIME_VERSION_EXECUTABLE
+    NAMES musa_runtime_version
     PATHS "$ENV{MUSA_HOME}"
     PATH_SUFFIXES bin bin64
     NO_DEFAULT_PATH)
-  mark_as_advanced(MUSA_TOOLKIT_VERSION_EXECUTABLE)
-  if(MUSA_TOOLKIT_VERSION_EXECUTABLE)
-    execute_process(COMMAND ${MUSA_TOOLKIT_VERSION_EXECUTABLE}
-                    OUTPUT_VARIABLE MUSA_TOOLKITS_VERSION)
+  mark_as_advanced(MUSA_RUNTIME_VERSION_EXECUTABLE)
+  if(MUSA_RUNTIME_VERSION_EXECUTABLE)
+    execute_process(COMMAND ${MUSA_RUNTIME_VERSION_EXECUTABLE}
+                    OUTPUT_VARIABLE MUSA_RUNTIME_VERSION)
     string(REGEX REPLACE ".*\"([0-9]+)\\.([0-9]+)\\.([0-9]+)\".*" "\\1"
-                         MUSA_TOOLKITS_VERSION_MAJOR ${MUSA_TOOLKITS_VERSION})
+                         MUSA_RUNTIME_VERSION_MAJOR ${MUSA_RUNTIME_VERSION})
     string(REGEX REPLACE ".*\"([0-9]+)\\.([0-9]+)\\.([0-9]+)\".*" "\\2"
-                         MUSA_TOOLKITS_VERSION_MINOR ${MUSA_TOOLKITS_VERSION})
+                         MUSA_RUNTIME_VERSION_MINOR ${MUSA_RUNTIME_VERSION})
     string(REGEX REPLACE ".*\"([0-9]+)\\.([0-9]+)\\.([0-9]+)\".*" "\\3"
-                         MUSA_TOOLKITS_VERSION_PATCH ${MUSA_TOOLKITS_VERSION})
+                         MUSA_RUNTIME_VERSION_PATCH ${MUSA_RUNTIME_VERSION})
 
     math(
       EXPR
-      MUSA_TOOLKITS_VERSION_INT
-      "${MUSA_TOOLKITS_VERSION_MAJOR} * 1000 + ${MUSA_TOOLKITS_VERSION_MINOR} * 10 + ${MUSA_TOOLKITS_VERSION_PATCH}"
+      MUSA_RUNTIME_VERSION_INT
+      "${MUSA_RUNTIME_VERSION_MAJOR} * 1000 + ${MUSA_RUNTIME_VERSION_MINOR} * 10 + ${MUSA_RUNTIME_VERSION_PATCH}"
     )
-    set(${outputvar} "${MUSA_TOOLKITS_VERSION_INT}" PARENT_SCOPE)
+    set(${outputvar} "${MUSA_RUNTIME_VERSION_INT}" PARENT_SCOPE)
     mark_as_advanced(${outputvar})
   endif()
 endfunction()

docs/markdown/README.md

@@ -0,0 +1,24 @@
+# torch_musa开发者文档
+
+中文版的开发者文档，请参考：
+
+[前言](source/01_foreword.md)
+
+[简介](source/02_introduction.md)
+
+[torch_musa安装和编译](source/03_compile_and_install.md)
+
+[torch_musa快速入门](source/04_quick_start.md)
+
+[torch_musa算法开发](source/05_ops_dev.md)
+
+[torch_musa第三方扩展支持](source/06_third_party_lib_extension.md)
+
+[torch_musa性能优化](source/07_optimization.md)
+
+[torch_musa调试工具](source/08_debug.md)
+
+[YOLOv5迁移案例](source/09_YOLO_transfer.md)
+
+[FAQ](source/10_FAQ.md)
+

docs/markdown/source/01_foreword.md

@@ -0,0 +1,61 @@
+---
+title: 前言
+description: torch_musa 前言
+hide_table_of_contents: False
+---
+
+# 前言
+
+## 版本记录
+
+| 文档名称 | 摩尔线程 Torch_MUSA 开发者手册 |
+| ---- | --------------------- |
+| 版本号  | V 1.3.0               |
+| 作者   | Moore Threads         |
+| 修改日期 | 2024年11月18日           |
+
+## 更新历史
+
+- **V0.1.0**
+  
+  **更新时间**：2023.05.29
+  
+  **更新内容**：
+  
+  - 完成初版 Torch_MUSA 开发者手册。
+
+- **V1.1.0**
+  
+  **更新时间**：2024.03.31
+  
+  **更新内容**：
+  
+  - 增加调试工具章节。
+  
+  - 增加 MUSAExtension 章节。
+  
+  - 增加性能分析章节。
+  
+  - 快速入门章节增加更多示例代码。
+  
+  - 完善 FAQ 章节。
+  
+  - 增加经典模型 YOLOv5 迁移示例章节。
+
+- **V1.2.0**
+  
+  **更新时间**：2024.07.28
+  
+  **更新内容**：
+  
+  - 完善 MUSAExtension 章节。
+  
+  - 增加算子适配说明，支持更多算子适配的方法。
+
+- **V1.3.0**
+  
+  **更新时间**：2024.11.18
+  
+  **更新内容**：
+  
+  - 版本升级

docs/markdown/source/02_introduction.md

@@ -0,0 +1,35 @@
+---
+title: 简介
+description: torch_musa 简介
+hide_table_of_contents: False
+---
+
+# 简介
+
+## MUSA概述
+
+MUSA (Metaverse Unified System Architecture) 是摩尔线程公司为摩尔线程GPU推出的一种通用并行计算平台和编程模型。它提供了GPU编程的简易接口，用MUSA编程可以构建基于GPU计算的应用程序，利用GPUs的并行计算引擎来更加高效地解决比较复杂的计算难题。同时摩尔线程还推出了MUSA工具箱（MUSAToolkits），工具箱中包括GPU加速库，运行时库，编译器，调试和优化工具等。MUSAToolkits为开发人员在摩尔线程GPU上开发和部署高性能异构计算程序提供软件环境。
+
+关于MUSA软件栈的更多内容，请参见MUSA官方文档。
+
+## PyTorch概述
+
+PyTorch是一款开源的深度学习编程框架，可以用于计算机视觉，自然语言处理，语音处理等领域。PyTorch使用动态计算，这在构建复杂架构时提供了更大的灵活性。PyTorch使用核心Python概念，如类、结构和条件循环，因此理解起来更直观，编程更容易。此外，PyTorch还具有可以轻松扩展、快速实现、生产部署稳定性强等优点。
+
+关于PyTorch的更多内容，请参见PyTorch官方文档。
+
+## torch_musa概述
+
+为了摩尔线程GPU能支持开源框架PyTorch，摩尔线程公司开发了torch_musa。在PyTorch v2.0.0基础上，torch_musa以插件的形式来支持摩尔线程GPU，最大程度与PyTorch代码解耦，便于代码维护与升级。torch_musa利用PyTorch提供的第三方后端扩展接口，将摩尔线程高性能计算库动态注册到PyTorch上，从而使得PyTorch框架能够利用摩尔线程显卡的高性能计算单元。利用摩尔线程显卡CUDA兼容的特性，torch_musa内部引入了cuda兼容模块，使得PyTorch社区的CUDA kernels经过porting后就可以运行在摩尔线程显卡上，而且CUDA Porting的工作是在编译torch_musa的过程中自动进行，这大幅降低了torch_musa算子适配的成本，提高模型开发效率。同时，torch_musa在Python前端接口与PyTorch社区CUDA接口形式上基本保持一致，这极大地降低了用户的学习成本和模型的迁移成本。
+
+本手册主要介绍了基于MUSA软件栈的torch_musa开发指南。
+
+## torch_musa核心代码目录概述
+
+- torch_musa/tests 测试文件。
+- torch_musa/core 主要包含Python module，提供amp/device/memory/stream/event等模块的Python前端接口。
+- torch_musa/csrc c++侧实现代码；
+  - csrc/amp 提供混合精度模块的C++实现。
+  - csrc/aten 提供C++ Tensor库，包括MUDNN算子适配，CUDA-Porting算子适配等等。
+  - csrc/core 提供核心功能库，包括设备管理，内存分配管理，Stream管理，Events管理等。
+  - csrc/distributed 提供分布式模块的C++实现。