[PRI2-585] Enable broader Linux suport from PyPi wheels

[mikasenghaas]

Local setup

❯ python --version && maturin --version
Python 3.11.12
maturin 1.8.3

Activate virtual environment using source .venv/bin/activate. Then, run the builds.

Build wheels for manylinux_2_34 (default)

❯ maturin build --compatibility manylinux_2_34
📦 Including license file "/home/ubuntu/prime-iroh/LICENSE"
🍹 Building a mixed python/rust project
🔗 Found pyo3 bindings
🐍 Found CPython 3.12 at /home/ubuntu/prime-iroh/.venv/bin/python
📡 Using build options features from pyproject.toml
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.24s
📦 Built wheel for CPython 3.12 to /home/ubuntu/prime-iroh/target/wheels/prime_iroh-0.2.1-cp312-cp312-manylinux_2_34_x86_64.whl

Build wheels for manylinux_2_31 locally fails

❯ maturin build --compatibility manylinux_2_31
📦 Including license file "/home/ubuntu/prime-iroh/LICENSE"
🍹 Building a mixed python/rust project
🔗 Found pyo3 bindings
🐍 Found CPython 3.12 at /home/ubuntu/prime-iroh/.venv/bin/python
📡 Using build options features from pyproject.toml
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.24s
💥 maturin failed
  Caused by: Error ensuring manylinux_2_31 compliance
  Caused by: Your library is not manylinux_2_31 compliant because of the presence of too-recent versioned symbols: ["libc.so.6 offending symbols: pthread_join@GLIBC_2.34, pthread_setspecific@GLIBC_2.34, pthread_attr_getguardsize@GLIBC_2.34, pthread_key_create@GLIBC_2.34, pthread_create@GLIBC_2.34, dlsym@GLIBC_2.34, pthread_detach@GLIBC_2.34, pthread_attr_setstacksize@GLIBC_2.34, pthread_attr_getstack@GLIBC_2.34, fstat64@GLIBC_2.33, pthread_setname_np@GLIBC_2.34, pthread_key_delete@GLIBC_2.34, stat64@GLIBC_2.33, pthread_getattr_np@GLIBC_2.32"]. Consider building in a manylinux docker container

Build wheels in docker for manylinux_2014

❯ docker run --rm -v $(pwd):/io ghcr.io/pyo3/maturin:latest build --compatibility manylinux2014
📦 Including license file "/io/LICENSE"
🍹 Building a mixed python/rust project
🔗 Found pyo3 bindings
🐍 Found CPython 3.9 at /opt/python/cp39-cp39/bin/python3
📡 Using build options features from pyproject.toml
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.27s
📦 Built wheel for CPython 3.9 to /io/target/wheels/prime_iroh-0.2.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

The key is to build for broad support using Docker. In CI, we can use the PyO3/maturin-action to build wheels from source. Also, we now include a source distribution so that users not covered by the wheels can install prime-iroh from source.