Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
136 changes: 136 additions & 0 deletions .github/workflows/build-uamqp.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
# SPDX-FileCopyrightText: 2026 The RISE Project
# SPDX-License-Identifier: MIT
---
# This workflow is based on the `ManyLinux` job of
# https://github.com/Azure/azure-uamqp-python/blob/v1.6.11/.azure-pipelines/client.yml
name: Build uamqp wheels (riscv64)

on:
workflow_dispatch:
inputs:
version:
description: 'uamqp version to build (git tag, e.g. 1.6.11)'
required: true
default: '1.6.11'
pull_request:
paths:
- '.github/workflows/build-uamqp.yml'

concurrency:
group: ${{ github.workflow }}-${{ inputs.version || '1.6.11' }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

permissions:
contents: read # to fetch code (actions/checkout)

env:
# `inputs.version` is empty on pull_request events; default to 1.6.11 there.
UAMQP_VERSION: ${{ inputs.version || '1.6.11' }}
MANYLINUX_RISCV64_IMAGE: quay.io/pypa/manylinux_2_39_riscv64

jobs:
setup:
uses: $/.github/workflows/_setup.yml

build_wheels:
needs: [setup]
name: Build uamqp ${{ inputs.version || '1.6.11' }} ${{ matrix.python }}-manylinux_riscv64
runs-on: ubuntu-24.04-riscv
timeout-minutes: 90
strategy:
fail-fast: false
matrix:
# No cp314t: several .pyx callbacks read context_pyobj.ob_refcnt
# directly to detect in-flight GC, which doesn't exist on
# free-threaded builds' PyObject layout -- fails identically on
# every arch, not a riscv64 issue (gotcha 227). Upstream itself
# ships no cp314 or cp314t wheel at all; cp314 happens to build
# clean here.
python: ["cp312", "cp313", "cp314"]

steps:
- name: Checkout azure-uamqp-python v${{ env.UAMQP_VERSION }}
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: Azure/azure-uamqp-python
ref: v${{ env.UAMQP_VERSION }}
persist-credentials: false

- name: Build wheels
uses: pypa/cibuildwheel@1828c10ab37f080699c7b81cea34097c684a7074 # v4.2.0
with:
output-dir: wheelhouse/
only: ${{ matrix.python }}-manylinux_riscv64
env:
CIBW_MANYLINUX_RISCV64_IMAGE: ${{ env.MANYLINUX_RISCV64_IMAGE }}
# setup.py's build_ext runs `cmake` against the vendored
# src/vendor/azure-uamqp-c tree itself (not scikit-build-core), and
# links the result against system OpenSSL; upstream's own
# pyproject.toml sets [tool.cibuildwheel.linux] before-build to
# build OpenSSL 3.0.15 from source for manylinux2014's ancient one
# (and that build fails here anyway: the image's perl has no
# FindBin, gotcha 46). CIBW_BEFORE_ALL_LINUX would not have
# replaced it -- before-all and before-build are separate hooks
# (gotcha 225); CIBW_BEFORE_BUILD_LINUX replaces that list wholesale;
# the riscv64 image is Rocky 10 with a current openssl-devel, so
# use that instead (mirrors build-aioquic.yml). The vendored C
# tree's own LICENSE (Azure IoT SDKs, MIT) differs textually from
# uamqp's own and lives under src/vendor/, so setuptools'
# root-only glob won't pick it up on its own (gotcha 44); stage a
# copy at the root.
CIBW_BEFORE_BUILD_LINUX: >-
dnf -y install openssl-devel &&
cp /usr/share/licenses/openssl-libs/LICENSE.txt {project}/LICENSE.openssl &&
cp src/vendor/azure-uamqp-c/LICENSE {project}/LICENSE.azure-uamqp-c
# Cython has a riscv64 wheel only on our registry.
# deps/azure-macro-utils-c declares cmake_minimum_required(2.8.11),
# which CMake 4 (the pip-resolved cmake build dependency) refuses
# outright; CMAKE_POLICY_VERSION_MINIMUM is a no-op for the
# already-3.5+ top-level project and unblocks that vendored one
# (gotcha 207). CIBW_ENVIRONMENT replaces upstream's own
# [tool.cibuildwheel.linux] environment table wholesale (gotcha
# 107), so restore its CFLAGS: GCC 14 turns
# -Wincompatible-pointer-types into a hard error by default, and
# the generated c_uamqp.c hits it (upstream already carries this
# same fix for its own CI, gotcha 226).
CIBW_ENVIRONMENT: >-
PIP_EXTRA_INDEX_URL=https://pypi.riseproject.dev/simple/
CMAKE_POLICY_VERSION_MINIMUM=3.5
CFLAGS="-Wno-error=incompatible-pointer-types -Wunused-function"
# Mirror the non-live half of upstream's own suite (tests/, not
# samples/): the ManyLinux job's `samples` doctest run needs live
# Event Hub/Service Bus credentials and only executes on Microsoft's
# internal pipeline (`condition: ne(...TeamProject..., 'public')`).
# tests/ exercises the built c_uamqp extension directly.
CIBW_TEST_REQUIRES: pytest
CIBW_TEST_SOURCES: tests
CIBW_TEST_COMMAND: python -m pytest {project}/tests

- name: Check the extension and licences made it into the wheel
run: |
python3 - wheelhouse/*.whl <<'EOF'
import sys, zipfile
for whl in sys.argv[1:]:
names = zipfile.ZipFile(whl).namelist()
assert any(n.startswith("uamqp/c_uamqp") and n.endswith(".so") for n in names), names
licences = {n.rsplit("/", 1)[1]
for n in names if ".dist-info/licenses/" in n} - {""}
assert licences == {"LICENSE", "LICENSE.openssl", "LICENSE.azure-uamqp-c"}, (whl, licences)
print(whl, "ok")
EOF

- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: uamqp-${{ env.UAMQP_VERSION }}-${{ matrix.python }}-manylinux_riscv64
path: wheelhouse/*.whl
if-no-files-found: error

publish:
name: Publish uamqp ${{ inputs.version || '1.6.11' }}
needs: [setup, build_wheels]
permissions:
contents: write
pull-requests: write
uses: $/.github/workflows/_publish-wheel.yml
with:
artifact-pattern: uamqp-${{ inputs.version || '1.6.11' }}-*-manylinux_riscv64
2 changes: 1 addition & 1 deletion skills/python-project-porting/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ Load these on demand — they are one level deep from here.

### Finding the right gotcha

The 190 gotchas are split into themed files under `references/gotchas/`. **Read the one file
The 221 gotchas are split into themed files under `references/gotchas/`. **Read the one file
that matches your current step** rather than loading them all — each file opens with an
`## In this file` list of its entries. Three ways in:

Expand Down
5 changes: 4 additions & 1 deletion skills/python-project-porting/references/gotchas-index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Gotchas index — router for the themed gotcha files

The porting gotchas (200 of them) live in [`references/gotchas/`](gotchas/), split by theme so only the relevant slice loads. Every gotcha keeps a **permanent number** cited elsewhere as "gotcha N" (and in workflow comments as "CLAUDE.md gotcha N"). Numbers are stable IDs — **not sequential**, and four are **reused** with different content (two each of 33, 55, 56, 57), disambiguated by theme below.
The porting gotchas (221 of them) live in [`references/gotchas/`](gotchas/), split by theme so only the relevant slice loads. Every gotcha keeps a **permanent number** cited elsewhere as "gotcha N" (and in workflow comments as "CLAUDE.md gotcha N"). Numbers are stable IDs — **not sequential**, and four are **reused** with different content (two each of 33, 55, 56, 57), disambiguated by theme below.

## How to find the gotcha you need

Expand Down Expand Up @@ -89,6 +89,8 @@ The porting gotchas (200 of them) live in [`references/gotchas/`](gotchas/), spl
- **216** — An abi3 build's own mandatory floor interpreter (gotcha 96) can itself be the one
- **217** — Upstream's own `repair-wheel-command` commonly re-runs abi3audit itself via
- **221** — `quay.io/pypa/musllinux_1_2_riscv64` is a real, working image — every prior port
- **225** — `CIBW_BEFORE_ALL_LINUX` and `CIBW_BEFORE_BUILD_LINUX` are two different hooks —
- **227** — A build that touches `PyObject` internals directly (`ob_refcnt`, `ob_type`,

### Rust, maturin & PyO3 — [`gotchas/rust-maturin-and-pyo3.md`](gotchas/rust-maturin-and-pyo3.md)

Expand Down Expand Up @@ -132,6 +134,7 @@ The porting gotchas (200 of them) live in [`references/gotchas/`](gotchas/), spl
- **138** — Two more manylinux-image facts, in the vein of gotchas 46 and 51.
- **139** — RISC-V SIMD in an upstream that already supports riscv64: two traps, both invisible
- **207** — A vendored dependency three submodules deep can declare a `cmake_minimum_required`
- **226** — GCC 14 turns `-Wincompatible-pointer-types` (and `-Wimplicit-function-declaration`,

### Native dependencies & linking — [`gotchas/native-deps-and-linking.md`](gotchas/native-deps-and-linking.md)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ To pull up one entry: `grep -n '^N\. ' references/gotchas/cibuildwheel-matrix-an
- **134** — cibuildwheel's default abi3 audit rejects a wheel for exporting its *own*
- **204** — cibuildwheel 4.2.0 doesn't offer cp313t as a build target on *any* platform —
- **221** — `quay.io/pypa/musllinux_1_2_riscv64` is a real, working image — every prior port
- **225** — `CIBW_BEFORE_ALL_LINUX` and `CIBW_BEFORE_BUILD_LINUX` are two different hooks —
- **227** — A build that touches `PyObject` internals directly (`ob_refcnt`, `ob_type`,
- **209** — A multi-grammar tree-sitter-`<lang>` repo does not necessarily need a
- **56** — `py-build-cmake` projects: the free-threaded job dies at *configure* unless
- **201** — When `package-dir` is a monorepo subdirectory and the package's own build script
Expand Down Expand Up @@ -401,3 +403,45 @@ To pull up one entry: `grep -n '^N\. ' references/gotchas/cibuildwheel-matrix-an
own dependency footprint (`requires_dist`, `CIBW_TEST_REQUIRES`) against the
registry before assuming musllinux riscv64 needs to be dropped by default; try it
when nothing pulls in a numpy-shaped blocker.

225. **`CIBW_BEFORE_ALL_LINUX` and `CIBW_BEFORE_BUILD_LINUX` are two different hooks —
overriding the one upstream *doesn't* use leaves theirs running too (the uamqp
case).** Both env vars replace their matching `[tool.cibuildwheel.linux]` key
independently (gotcha 107's cascade, one level more specific); they don't share a
slot, so setting `CIBW_BEFORE_ALL_LINUX` while upstream's own pyproject sets
`before-build` doesn't touch it at all — cibuildwheel runs **both**, yours first,
then theirs. uamqp's `before-build` builds OpenSSL from source for manylinux2014;
the riscv64 override installed `openssl-devel` via `before-all` to skip that, but
upstream's still-present `before-build` then `yum remove`d it again and tried the
source build anyway (failing separately on missing `perl-FindBin`, gotcha 46).
Both jobs looked like they ran (the openssl-devel install step's log was right
there) which made the real cause — the wrong hook name — easy to miss.
- **Read which key upstream's `[tool.cibuildwheel.linux]` table actually sets**
before choosing which `CIBW_*_LINUX` var to override; `before-all` runs once per
container, `before-build` once per interpreter, and only the matching one
replaces upstream's list.
- When in doubt, grep the job log for upstream's own command line (here,
`install_openssl.sh`) — if it's still present after your override step, you
overrode the wrong hook, not the right one with a bug in it.

227. **A build that touches `PyObject` internals directly (`ob_refcnt`, `ob_type`,
etc.) instead of through the stable/limited API fails to *compile* under
free-threaded Python, and the failure is identical on every architecture — trim
the matrix, don't patch around it (the uamqp case).** Several of uamqp's `.pyx`
files read `context_pyobj.ob_refcnt == 0` inside a C callback to guess whether the
Python-side context object is mid-garbage-collection before touching it. Free-threaded
builds replace `PyObject`'s single `ob_refcnt` field with a different
layout (per-thread local/shared reference counts), so the field plain doesn't exist
and the cp314t build fails at `gcc: error: 'PyObject' {aka 'struct _object'} has no
member named 'ob_refcnt'` — a compile error, not a runtime one, so it can't be
waved off with `CIBW_TEST_SKIP` the way gotcha 216's coverage gaps can.
- **Check whether upstream ships a cp314/cp314t wheel at all before assuming this
needs a patch.** uamqp's own PyPI releases stop at cp313 — nobody has hit this
upstream because nobody has built it free-threaded yet. Dropping `cp314t` from
the matrix (keeping `cp312`/`cp313`/`cp314`, all of which built and passed tests
clean) mirrors upstream's own supported set rather than inventing support they
don't have.
- **This is a real correctness gap, not a riscv64 build quirk** — the same source
would fail identically compiling cp314t on x86_64/aarch64. Reaching for the
pattern of gotcha 26 (bump the toolchain) or 107/226 (fix `CFLAGS`) doesn't apply;
there's no flag that makes a nonexistent struct member exist.
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ To pull up one entry: `grep -n '^N\. ' references/gotchas/manylinux-image-and-to
- **138** — Two more manylinux-image facts, in the vein of gotchas 46 and 51.
- **139** — RISC-V SIMD in an upstream that already supports riscv64: two traps, both invisible
- **207** — A vendored dependency three submodules deep can declare a `cmake_minimum_required`
- **226** — GCC 14 turns `-Wincompatible-pointer-types` (and `-Wimplicit-function-declaration`,

---

Expand Down Expand Up @@ -266,3 +267,25 @@ To pull up one entry: `grep -n '^N\. ' references/gotchas/manylinux-image-and-to
- **Confirm exactly which vendored `CMakeLists.txt` needs it by reading the
"CMake Error at ..." path in the configure log**, not by guessing from the
top-level project's own declared minimum — it is rarely the direct dependency.

226. **GCC 14 turns `-Wincompatible-pointer-types` (and `-Wimplicit-function-declaration`,
`-Wimplicit-int`) from a warning into a hard error by default for C code — a
toolchain-version fact, not a riscv64 one, that bites old-style C sources compiled
against a newer manylinux image than upstream targets (the uamqp case).** uamqp's
Cython-generated `c_uamqp.c` calls into the vendored `azure-uamqp-c` C API with
loosely-typed pointers that were always technically wrong but only warned under the
GCC upstream's own manylinux2014 image ships. `manylinux_2_39_riscv64` (Rocky 10)
carries GCC 14.3.1, so the same code hard-fails there — refines gotcha 26 from "too
old to build at all" to "new enough to enforce what an old one let slide". uamqp's
own `pyproject.toml` already carries the fix — `[tool.cibuildwheel.linux]
environment = {..., CFLAGS="-Wno-error=incompatible-pointer-types
-Wunused-function"}` — for exactly this reason, presumably hit on a newer x86_64 CI
image at some point.
- **A `CIBW_ENVIRONMENT` override that adds `PIP_EXTRA_INDEX_URL` (or anything
else) silently drops that CFLAGS too, since `CIBW_ENVIRONMENT` replaces the whole
table (gotcha 107)** — read upstream's `environment` entry before overriding and
carry forward anything build-relevant, not just the keys your port needed to add.
- Confirm it's this exact class before reaching for the flag: the compiler error
text names the diagnostic (`error: ... incompatible-pointer-types` in
`[-Wincompatible-pointer-types]`), and it appears identically on any sufficiently
new GCC/Clang regardless of architecture — nothing riscv64-specific to chase.