DimOS Interactive Viewer — full diff vs upstream rerun-io/rerun#8
Open
spomichter wants to merge 25 commits intoupstream-mainfrom
Open
DimOS Interactive Viewer — full diff vs upstream rerun-io/rerun#8spomichter wants to merge 25 commits intoupstream-mainfrom
spomichter wants to merge 25 commits intoupstream-mainfrom
Conversation
…ration Adds click-to-coordinate support to the custom_callback viewer example. When a user clicks on an entity in a 2D/3D spatial view, world-space coordinates are sent via TCP to a Python bridge for downstream processing. Changes: - interaction/protocol.rs: ViewerEvent + AppCommand enums (bincode serde) - interaction/handle.rs: InteractionHandle mpsc wrapper with unit tests - interaction/sender.rs: TCP client with auto-reconnect to Python bridge - viewer.rs: on_event callback intercepting SelectionChange events, 100ms debounce, rapid-click detection - build-viewer.yml: CI workflow for Linux x64 + macOS arm64 Uses the official StartupOptions::on_event callback API — zero changes to rerun core crates. All modifications are in examples/rust/custom_callback/. DIM-643
Python TCP server + bincode codec that receives click events from the custom viewer. Provides @on_click decorator API for handling world-space coordinates. Files: - bincode_codec.py: pure Python bincode encoder/decoder - bridge_bincode.py: production TCP server (bincode protocol) - bridge.py: development TCP server (JSON protocol) - test_bridge.py: 4 JSON bridge tests - test_bridge_bincode.py: 6 bincode bridge tests DIM-643
Rerun's picking only fires on entity clicks. This adds a helper that logs a large, nearly-invisible subdivided mesh at z=0 so clicks on empty floor space still produce world-space coordinates. Usage: from ground_plane import log_ground_plane log_ground_plane() # 200m x 200m at z=0, alpha=1 DIM-643
Replace the TCP-based ViewerEventSender with an LCM UDP multicast publisher for sending click events from the Rerun viewer to Python. Changes: - Add interaction/lcm.rs: LcmPublisher that encodes and sends geometry_msgs/PointStamped messages via UDP multicast (239.255.76.67:7667) - Update viewer.rs: use LcmPublisher instead of ViewerEventSender, publish click events on '/clicked_point#geometry_msgs.PointStamped' - Update interaction/mod.rs: export LCM types (ClickEvent, LcmPublisher, click_event_from_ms, click_event_now) The LCM encoding is verified byte-for-byte compatible with Python's dimos_lcm.geometry_msgs.PointStamped via unit tests (10 Rust tests) and cross-language integration tests (15 Python tests). Wire format: [8B fingerprint hash][Header: seq+stamp+frame_id][Point: x+y+z] Fingerprint: 0x9cd764738ea629af (matches Python LCM recursive hash)
- remove python_bridge/ directory (TCP bridge replaced by LCM) - remove interaction/sender.rs (TCP event sender, unused) - remove ControlViewer connection (port 8889) - remove Control panel wrapper (old demo UI) - simplify protocol.rs: keep only Click variant - simplify handle.rs: keep only send_click - clean up build-viewer.yml: fix paths, update action versions (v4) - rename app env to 'DimOS Interactive Viewer' - final Box::new(rerun_app) instead of Control::new wrapper
refactor: remove dead TCP bridge code, clean up for LCM-only transport
feat(viewer): interactive click-to-coordinate with Python bridge
* feat(packaging): add PyPI distribution with maturin for dimos-viewer - Rename binary from custom_callback_viewer to dimos-viewer - Remove unused custom_callback_app binary - Add pyproject.toml with maturin build backend (bindings=bin) - Add dimos_viewer Python package with main(), launch(), _find_viewer_binary() - Add __main__.py for 'python -m dimos_viewer' support - Add comprehensive Python tests (wrapper, binary, install) - Replace build-viewer.yml with build-and-publish.yml CI workflow - Build matrix: linux-x64, linux-arm64, macos-arm64 - Python versions: 3.10, 3.11, 3.12, 3.13 - Rust tests + Python tests - PyPI publish on tag push (v*) via maturin upload - GitHub Release with wheel artifacts - Version 0.1.0 (independent semver, based on Rerun 0.30.0-alpha.1) - Add README with usage, platform support, and versioning docs Closes DIM-646 (dimos-viewer side) * chore: version 0.30.0a1, remove dead comms/panel/app code - Version tracks Rerun base: 0.30.0a1 (based on Rerun 0.30.0-alpha.1) - Remove comms/ (689 lines TCP protocol, unused) - Remove panel.rs (demo UI wrapper, unused) - Remove app.rs (demo app binary, already removed from Cargo.toml) - lib.rs now only exports interaction module * ci: replace custom build-and-publish.yml with build-viewer.yml Rework CI workflow to follow Rerun's patterns: - Renamed build-and-publish.yml → build-viewer.yml - Added push-to-main trigger alongside PR and tag triggers - Simplified wheel build: removed redundant --interpreter flag (bindings='bin' produces py3-none universal wheels) - Added concurrency group to cancel stale runs - Added workflow_dispatch for manual triggers - Cleaner job structure: check → build-wheel → test-wheel → publish → release - Uses GitHub Artifacts for wheel storage (not GCS) - Publish to PyPI via maturin upload on v* tags only Rerun's reusable_build_and_upload_wheels.yml cannot be called directly because it is tightly coupled to building rerun_py (uses pixi, GCS sccache, rerun-cli download, and rerun_py-specific feature flags). Instead, this workflow follows the same structural patterns: platform matrix, manylinux_2_28 compat, artifact upload per platform. * fix: use PyPI trusted publisher (OIDC) instead of API token
- Linux x64: build inside quay.io/pypa/manylinux_2_28_x86_64 (matches Rerun) - System deps via dnf inside container - Rust installed inside container - maturin -i python3 for interpreter discovery - Shell defaults: bash --noprofile --norc -euo pipefail (matches Rerun) - 60 min timeout per job (matches Rerun) - Removed linux-arm64 (no arm runners yet) - PyPI publish via trusted publisher OIDC (no API token)
- Linux: python3 -m pip install --upgrade pip before maturin (old pip fails) - macOS: actions/setup-python@v5 to avoid PEP 668 system Python restriction
Switch from raw manylinux_2_28 base to ghcr.io/rerun-io/ci_docker:0.17.0 which already has all system deps, Rust, and working pip pre-installed. Matches Rerun's exact build environment.
ghcr.io/rerun-io/ci_docker is private (requires GHCR auth). Use quay.io/pypa/manylinux_2_28_x86_64 (public) and install the same deps from Rerun's ci_docker/Dockerfile inline.
The manylinux_2_28 container ships Python 3.6 as system python3. Using -i python3 made maturin tag the wheel as cp36-cp36m, which is incompatible with Python 3.10+. Switch to /opt/python/cp310-cp310/bin/python3 (pre-installed in manylinux images) so the wheel is tagged cp310 and installable on modern Python.
bindings = 'bin' produces a standalone binary, not a Python extension. The -i python3.10 flag incorrectly tagged the wheel as cp310-only, making it uninstallable on Python 3.12. Without -i, maturin produces a py3-none wheel that works on any Python 3.x.
- Put /opt/python/cp310-cp310/bin on PATH so maturin finds Python 3.10 instead of system Python 3.6 - Use plain 'python3' and 'maturin' commands (no hardcoded paths) - Simplify test: single Python version, just verify wheel installs and binary is findable (no pytest — this is a Rust binary, not a Python lib) - Rerun runs Python tests because rerun-sdk has Python bindings; dimos-viewer is just a binary in a wheel, no Python API to test
- Put /opt/python/cp310-cp310/bin on PATH so maturin finds Python 3.10 instead of system Python 3.6 in manylinux container - bindings=bin should still produce py3-none universal wheel - Simplified test job: single Python 3.12 (matches DimOS), no pytest (viewer is a Rust binary, not a Python library) - Rerun also tests with single Python version (3.10)
python-packages in pyproject.toml causes maturin to tag wheels with the build interpreter version (cp310). Test must use same version. Matches Rerun's approach (PYTHON_VERSION=3.10).
Add clap CLI argument parsing to dimos-viewer so it accepts the same flags that rerun_bindings.spawn() passes to the stock rerun binary: --port (default 9877) --memory-limit (default 75%) --server-memory-limit (default 1GiB) --hide-welcome-screen --expect-data-soon This enables seamless integration with the Rerun SDK: import rerun_bindings rerun_bindings.spawn(executable_name='dimos-viewer', port=9877) No need to fork rerun-sdk or pre-launch the viewer manually. The DimOS bridge can call rerun_bindings.spawn() directly with our binary name as a drop-in replacement.
Move all DimOS-specific code out of examples/rust/custom_callback/ into
a top-level dimos/ directory. Restore custom_callback to its upstream
Rerun state for easier future merges.
Directory structure:
dimos/
Cargo.toml (dimos-viewer crate)
pyproject.toml (maturin packaging)
dimos_viewer/ (Python wrapper)
tests/ (Python tests)
src/
viewer.rs (main binary with CLI args)
lib.rs
interaction/ (LCM transport module)
Changes:
- Created dimos/ crate with all viewer + LCM code
- Restored examples/rust/custom_callback/ to upstream Rerun state
- Updated CI workflow to reference dimos/ instead of custom_callback
- Added dimos/ to workspace members
- Both crates compile and test independently
Verified:
- cargo check -p dimos-viewer ✓
- cargo test -p dimos-viewer ✓ (13 tests pass)
- cargo check -p custom_callback ✓ (upstream intact)
- dimos-viewer --help shows correct CLI args
'Topic :: Scientific/Engineering :: Robotics' doesn't exist on PyPI. Changed to 'Topic :: Scientific/Engineering'.
Remove python-packages from pyproject.toml so maturin produces py3-none universal wheels instead of cpXXX version-specific ones. The Python wrapper (dimos_viewer/) was unused — the binary is the product and rerun_bindings.spawn(executable_name='dimos-viewer') finds it on PATH directly. Bumped to v0.30.0a2 since PyPI won't accept re-uploads of v0.30.0a1. Test job now uses Python 3.12 to verify universal compatibility.
maturin bindings=bin always tags wheels with the discovered Python interpreter. Removed dimos_viewer/ directory (unused since we call rerun_bindings.spawn directly) and switched manylinux build to use Python 3.12 to match DimOS requirements. Wheel will be tagged cp312-cp312 instead of cp310-cp310.
Tests referenced dimos_viewer package which was removed. Verify step now just checks binary is on PATH and --help works.
maturin bindings=bin always tags wheels with the discovered Python version. Instead of fighting it, build one wheel per Python version per platform (6 wheels total: 3 versions x 2 platforms). This matches how most binary Python packages handle multi-version support — one wheel per (platform, cpython-version) combo. Test job also runs per Python version to verify each wheel installs correctly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This PR shows all changes made to the upstream rerun-io/rerun fork to create
dimos-viewer— an interactive Rerun viewer with click-to-navigate support for DimOS robots.Key Changes
New:
dimos/directory (DimOS viewer)dimos/Cargo.toml— Rust crate for thedimos-viewerbinarydimos/src/viewer.rs— Main viewer entry point with Rerun-compatible CLI argsdimos/src/interaction/— Click event handling → LCM PointStamped publishingdimos/pyproject.toml— Maturin packaging config (bindings = "bin")dimos/README.md— Package documentationNew: CI/CD
.github/workflows/build-viewer.yml— Build matrix (Python 3.10/3.11/3.12 × linux-x64/macos-arm64), test, publish to PyPI via trusted publisher (OIDC)Modified from upstream
Cargo.toml(workspace) — Addeddimos-viewerto workspace membersCargo.lock— Updated with dimos-viewer depsUpstream files preserved
examples/rust/custom_callback/— Restored to exact upstream state (zero diff)Architecture
PyPI Package
pip install dimos-viewer— pypi.org/project/dimos-viewerdimos-vieweron PATHrerun_bindings.spawn(executable_name='dimos-viewer')