Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .github/workflows/build-wheels-platforms.yml
Original file line number Diff line number Diff line change
Expand Up @@ -130,9 +130,15 @@ jobs:
supported_python_versions: ${{ needs.get-supported-versions.outputs.supported_python }}
oldest_supported_python: ${{ needs.get-supported-versions.outputs.oldest_supported_python }}

# Repair Linux wheels with auditwheel for manylinux compatibility
repair-manylinux-wheels:
needs: [build-wheels, build-python-version-dependent-wheels]
name: Repair manylinux wheels
uses: ./.github/workflows/manylinux-repair.yml

# TODO Uncomment this when we are ready to upload the wheels
#upload-python-wheels:
# needs: [build-wheels, build-python-version-dependent-wheels]
# needs: [repair-manylinux-wheels] # Changed to depend on repaired wheels
# name: Upload Python wheels
# uses: ./.github/workflows/upload-python-wheels.yml
# secrets: inherit
50 changes: 50 additions & 0 deletions .github/workflows/manylinux-repair.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name: manylinux-repair

# Repairs Linux wheels using auditwheel for broad Linux compatibility
# https://github.com/pypa/auditwheel

on:
workflow_call:

jobs:
repair-wheels:
name: Repair ${{ matrix.arch }} wheels
runs-on: ubuntu-latest
strategy:
matrix:
arch: [x86_64 , aarch64]
include:
- arch: x86_64
platform: manylinux_2_28_x86_64
- arch: aarch64
platform: manylinux_2_28_aarch64

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Download wheel artifacts
uses: actions/download-artifact@v4
with:
path: ./downloaded_wheels

- name: Set up QEMU for cross-platform builds
if: matrix.arch == 'aarch64'
uses: docker/setup-qemu-action@v3
with:
platforms: arm64

- name: Repair wheels in manylinux container
run: |
docker run --rm \
-v $(pwd):/work \
-w /work \
quay.io/pypa/${{ matrix.platform }} \
python3 repair_wheels.py

- name: Re-upload artifacts with repaired wheels
uses: actions/upload-artifact@v4
with:
name: wheels-repaired-${{ matrix.arch }}
path: ./downloaded_wheels
overwrite: true
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,8 +118,13 @@ File for the requirements needed for the build process and the build script.
### os_dependencies
When there is a need for additional OS dependencies to successfully build the wheels on a specific platform and architecture, the `.sh` script in the `os_dependencies` directory can be adjusted.

## Manylinux wheels
We are using [`auditwheel`](https://github.com/pypa/auditwheel) package to repair already built wheels which need static link for the libraries to achieve manylinux tag. This logic is done by the [manylinux workflow](./.github/workflows/manylinux-repair.yml) and the [`repair_wheels.py` script](./repair_wheels.py)

## Activity Diagram
The main file is `build-wheels-platforms.yml` which is scheduled to run periodically to build Python wheels for any requirement of all [ESP-IDF]-supported versions.


![IDF Python wheels - Activity diagram](./resources/idf-python-wheels_diagram.svg "IDF Python wheels - Activity diagram")

*The diagram was generated with the open-source tool [PlantUML](https://plantuml.com) (and edited)*
Expand Down
79 changes: 79 additions & 0 deletions repair_wheels.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
#
# SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO LTD
#
# SPDX-License-Identifier: Apache-2.0
#
"""
Repairs Linux wheels using auditwheel to ensure manylinux compatibility.
"""

import subprocess

from pathlib import Path


def main():
wheels_dir = Path("./downloaded_wheels")
temp_dir = Path("./temp_repair")
temp_dir.mkdir(exist_ok=True)

# Find all wheel files
wheels = list(wheels_dir.rglob("*.whl"))

if not wheels:
print(f"No wheels found in {wheels_dir}")
raise SystemExit("No wheels found in downloaded_wheels directory")

print(f"Found {len(wheels)} wheels\n")

repaired_count = 0
skipped_count = 0

# Repair each wheel
for wheel in wheels:
print(f"Processing: {wheel.name}")

if "-py3-none-any" in wheel.name or "-py2.py3-none-any" in wheel.name:
print(" → Skipping pure Python wheel")
skipped_count += 1
continue

# Clean temp directory before each
for old_wheel in temp_dir.glob("*.whl"):
old_wheel.unlink()

# auditwheel repair automatically:
# - Detects if wheel needs repair
# - Bundles required libraries
# - Sets appropriate manylinux tag
result = subprocess.run(
["auditwheel", "repair", str(wheel), "-w", str(temp_dir)], capture_output=True, text=True
)

# auditwheel output
if result.stdout:
print(f" {result.stdout.strip()}")
if result.stderr:
print(f" {result.stderr.strip()}")

# Check if a repaired wheel was created
repaired = next(temp_dir.glob("*.whl"), None)

if repaired and repaired.name != wheel.name:
# A new wheel was created with a different name (repaired)
repaired.rename(wheel.parent / repaired.name)
wheel.unlink() # Remove original
print(f" → Replaced with repaired wheel: {repaired.name}\n")
repaired_count += 1
else:
print(" → Keeping original wheel")
skipped_count += 1

print("---------- STATISTICS ----------")
print(f"Total wheels: {len(wheels)}")
print(f"Kept wheels: {skipped_count}")
print(f"Repaired wheels: {repaired_count}")


if __name__ == "__main__":
main()
Loading
Loading