Consider using cibuildwheel

[sevagh]

Hello,

Has CuPy ever considered using cibuildwheel to help build and publish CuPy wheels?
https://github.com/pypa/cibuildwheel

I set up a cibuildwheel GitHub Action and had some good luck, for example here:
https://github.com/sevagh/NVTX/blob/dev/.github/workflows/cibw.yml#L27

However, not sure how the interaction is with the CUDA Toolkit.

[kmaehashi]

Thanks for the suggestion! I once saw it used by NumPy. I agree using GitHub Acitons is the right direction to build a release pipeline and also make the binary build process transparent to end-users.

The first thing in my mind is to move everything into GitHub Actions and set up self-hosted runners. We can then try replacing the current tool with cibuildwheel. Currently, our build infra is a bit complicated mainly because we use real hardware to run smoke tests:

• Internal CI/CD (FlexCI, on GCP) with NVIDIA GPU - CUDA/{Linux,Windows}/x86_64
• Internal Jenkins (on-prem) with AMD GPU - ROCm/Linux/x86_64
• 🆕 GitHub Actions (self-hosted runner on Cloud) (here) - CUDA/Linux/aarch64

The tooling in this repository has allowed us to run wheel builds everywhere, but I'd like to use the standard tools whenever possible.

[sevagh]

FYI I discovered that using the manylinux_2_28 environment for cibuildwheel (uses AlmaLinux 8, RHEL 8 compatible) allows one to install nvcc + CTK easily with the NVIDIA RHEL 8 repo: https://github.com/sevagh/rmm/blob/feat/wheel-ci/.github/workflows/cibuildwheel.yml#L42

Works for both aarch64 and x86_64.

So it's a potential idea for moving towards cibuildwheel and eliminating custom tooling.