Synchronize CI/CD, Flatbuffers vendoring and wamp-ai and wamp-cicd Submodules between autobahn-python and zlmdb #1822
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
oberstet
changed the title
Add shared flatc infrastructure from wamp-cicd: - Bump .cicd submodule to include new scripts/flatc/ files - Add src/autobahn/_flatc/ module with get_flatc_path() and run_flatc() - Add scripts/smoke_test.py and scripts/smoke_test_flatc.py - Add justfile recipes: test-smoke, test-wheel-install, test-sdist-install This syncs autobahn-python with zlmdb's FlatBuffers bundling approach, using shared code from wamp-cicd/scripts/flatc/ for DRY. Note: This work was completed with AI assistance (Claude Code).
- hatch_build.py: Add _build_flatc(), _generate_reflection_bfbs(), and _update_flatbuffers_git_version() methods for wheel builds - pyproject.toml: Add flatc console script entry point - flatbuffers/__init__.py: Add version() function and _git_version import - flatbuffers/_git_version.py: Git version tracking file - flatbuffers/reflection.fbs: Copy reflection schema for runtime access - scripts/update_flatbuffers.sh: External script for updating vendored flatbuffers - justfile: Simplify update-flatbuffers recipe, add test-bundled-flatc - .gitignore: Exclude flatc binary and reflection.bfbs build artifacts The wheel now includes: - Bundled flatc compiler built from deps/flatbuffers - reflection.bfbs generated during build - Git version tracking for vendored runtime Tested: wheel install, sdist install, smoke tests all pass Note: This work was completed with AI assistance (Claude Code).
…nux_2_34 The manylinux_2_34 container uses a toolchain that defaults to x86_64_v2 ISA level (SSE4.2, POPCNT), which causes auditwheel to fail with: ValueError: Cannot repair the wheel, because required library "flatc" could not be located. x86_64_v2 (ISA level) is not compatible with manylinux platform tag's (x86_64) requirement. The manylinux_2_28 container uses baseline x86_64 ISA which is compatible with all manylinux platform tags. This matches zlmdb which already uses manylinux_2_28 successfully. Note: This work was completed with AI assistance (Claude Code).
CFFI compilation under QEMU ARM64 emulation is very slow. CPython builds can take 20+ minutes due to emulation overhead. This matches zlmdb which already uses 30 minute timeouts. Note: This work was completed with AI assistance (Claude Code).
CFFI and flatc compilation under QEMU ARM64 emulation is very slow. Builds can take 45+ minutes due to emulation overhead. Note: This work was completed with AI assistance (Claude Code).
The wheels-docker workflow was changed to use manylinux_2_28 instead of manylinux_2_34 for ISA compatibility. The release workflow must use the matching artifact names. Note: This work was completed with AI assistance (Claude Code).
1. Add .github/workflows/README.md documenting the CI/CD architecture, artifact flow, and platform coverage (adapted from zlmdb) 2. Add _generate_wamp_bfbs() to hatch_build.py to compile wamp.bfbs from the WAMP FlatBuffers schemas during wheel build. This binary schema file enables runtime schema introspection for WAMP messages. 3. Add check_zlmdb_flatbuffers_version_in_sync() to autobahn/__init__.py This is the reciprocal of zlmdb.check_autobahn_flatbuffers_version_in_sync() and verifies both libraries use the same vendored FlatBuffers runtime for reliable data-in-transit/data-at-rest interoperability. 4. Add wamp.bfbs to .gitignore as it's a build artifact Note: This work was completed with AI assistance (Claude Code).
…n check 1. Rename `install-flatc` to `install-flatc-system` with prominent warning explaining that users normally DON'T need this since autobahn bundles flatc in wheels. Adds interactive confirmation prompt. 2. Enhance smoke_test_flatc.py to verify flatc binary version matches the vendored FlatBuffers runtime version. This ensures consistency between the compiler and runtime library. Note: This work was completed with AI assistance (Claude Code).
This was referenced Dec 15, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
fixes #1821 [WIP]
Related Issue(s)
Closes or relates to #1821
Checklist
the style guidelines of this project
is effective or that my feature works
updated the changelog
in this PR