CANN (Compute Architecture of Neural Networks), developed by Huawei, is a heterogeneous computing architecture for AI scenarios.
It provides multi-layer programming interfaces to help users quickly build AI applications and services based on the Ascend platform.

CANN backend in CTranslate2, enables running AI models on the Ascend NPU extending the existing CPU & CUDA workflows.
One can find more on Ascend NPU and CANN library here.

Examples of projects that support CANN include ONNX Runtime & OpenCV.

Implementation as well as detailed description of tests, environment setup & benchmarks can be found in pull request #1606 .

Notes

• In the context of the development of this feature we also submitted issue GEMM operator calculates the c output shape incorrectly when input a is transpose? #1583 .
• In case CANN Backend support has increased demand, a new Pull request/subproject will be introduced contributing the respective CI involving dedicated Ascend hardware.