Skip to content

Performance improvements to the gripper and head cameras - #41

Merged
hello-robot-shehab merged 15 commits into
mainfrom
feature/gripper_camera_performance_enhancements
Aug 19, 2026
Merged

Performance improvements to the gripper and head cameras#41
hello-robot-shehab merged 15 commits into
mainfrom
feature/gripper_camera_performance_enhancements

Conversation

@hello-robot-shehab

@hello-robot-shehab hello-robot-shehab commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Changes in this PR:

  • Updated gripper camera's pipeline to match stretch4_gripper_modeling_and_control's settings for adding a sync node, lowering buffer size to 1, and a sync threshold to 15ms.
  • Changed head cameras to also use a sync node
  • Improved rerun logging performance by moving logging to a background thread.
  • Changed stretch_camera_show --gripper to not do point cloud anymore -> stretch_rgbd_show --gripper does that instead.

To test

  • Run stretch_camera_show -lrc --rerun --show_fps and observe that the FPS is near 30fps (it's slightly less because center camera is at 10fps

  • Run stretch_camera_show -lr --rerun --show_fps and observe that the FPS is 30fps.

  • Run stretch_camera_show -c --rerun --show_fps and observe that the FPS is 10fps.

  • Run stretch_camera_show -g --rerun --show_fps and observe that the FPS is 30fps.

  • Run stretch_rgbd_show -g --show_fps and make sure a point cloud from the gripper is displayed.

  • Make sure to have ~/repos/stretch4_rgbd and ~/repos/stretch4_gripper_modeling_and_control cloned, and checkout cd ~/repos/stretch4_rgbd && git checkout feature/combined_left_and_right_rgbd in stretch4_rgbd, and then run python3 test/test_camera_benchmark.py, you should get an output like:

================================================================================
                      PERFORMANCE COMPARISON REPORT
================================================================================
Implementation                                     | Init Latency | Realized FPS | Sensor-to-Host Latency | Jitter (Arrival) | Drop Rate  | Frames Recv
-------------------------------------------------------------------------------------------------------------------------------------------------
Head (stretch4_rgbd)                               | 5.66 s      | 30.0         | 46.3 ± 6.7 ms          | 11.16 ms     | 0.0 %      | 301
Head (stretch4_body)                               | 4.82 s      | 30.1         | 50.2 ± 1.8 ms          | 1.99 ms     | 0.0 %      | 302
Gripper (stretch4_gripper_modeling_and_control)    | 2.99 s      | 30.1         | 74.0 ± 3.8 ms          | 1.64 ms     | 0.0 %      | 301
Gripper (stretch4_body)                            | 3.41 s      | 29.9         | 75.1 ± 3.8 ms          | 3.43 ms     | 0.3 %      | 299

================================================================================
                           REGRESSION CHECK
================================================================================

Evaluating Head Camera:
  * Latency: Old=46.3ms, New=50.2ms (Limit=53.2ms)
  * FPS:     Old=30.0, New=30.1 (Limit=25.5)
  [+] PASSED: Performance is equivalent within threshold bounds!

Evaluating Gripper Camera:
  * Latency: Old=74.0ms, New=75.1ms (Limit=85.1ms)
  * FPS:     Old=30.1, New=29.9 (Limit=25.6)
  [+] PASSED: Performance is equivalent within threshold bounds!

@hello-robot-shehab
hello-robot-shehab marked this pull request as ready for review June 11, 2026 18:53
@hello-robot-shehab hello-robot-shehab changed the title Performance improvements to the gripper camera Performance improvements to the gripper and head cameras Jun 11, 2026
@hello-robot-shehab
hello-robot-shehab force-pushed the feature/gripper_camera_performance_enhancements branch 2 times, most recently from c605809 to 8bc2381 Compare June 11, 2026 19:06
@hello-robot-shehab
hello-robot-shehab marked this pull request as draft June 22, 2026 22:51
@hello-robot-shehab
hello-robot-shehab force-pushed the feature/gripper_camera_performance_enhancements branch from 955a881 to 61a22f0 Compare June 25, 2026 19:25
@hello-robot-shehab
hello-robot-shehab force-pushed the feature/gripper_camera_performance_enhancements branch 2 times, most recently from 7356998 to 06a0585 Compare August 3, 2026 16:55
@hello-robot-shehab
hello-robot-shehab force-pushed the feature/gripper_camera_performance_enhancements branch from eb8ec59 to b4e613d Compare August 4, 2026 20:33
@hello-robot-shehab
hello-robot-shehab marked this pull request as ready for review August 4, 2026 20:34
@hello-robot-shehab
hello-robot-shehab force-pushed the feature/gripper_camera_performance_enhancements branch from bbfc2b6 to abe4774 Compare August 5, 2026 16:20
@hello-ola

Copy link
Copy Markdown
Contributor

All tests passed

image
First frame received! Initialization Latency: 3.400 s

================================================================================
                      PERFORMANCE COMPARISON REPORT
================================================================================
Implementation                                     | Init Latency | Realized FPS | Sensor-to-Host Latency | Jitter (Arrival) | Drop Rate  | Frames Recv
-------------------------------------------------------------------------------------------------------------------------------------------------
Head (stretch4_rgbd)                               | 5.26 s      | 30.0         | 49.0 ± 6.9 ms          | 11.16 ms     | 0.0 %      | 300
Head (stretch4_body)                               | 5.36 s      | 30.0         | 50.6 ± 1.3 ms          | 1.67 ms     | 0.0 %      | 301
Gripper (stretch4_gripper_modeling_and_control)    | 3.05 s      | 30.1         | 71.3 ± 3.6 ms          | 1.66 ms     | 0.0 %      | 302
Gripper (stretch4_body)                            | 3.40 s      | 29.9         | 72.7 ± 2.0 ms          | 2.82 ms     | 0.3 %      | 298

================================================================================
                           REGRESSION CHECK
================================================================================

Evaluating Head Camera:
  * Latency: Old=49.0ms, New=50.6ms (Limit=56.3ms)
  * FPS:     Old=30.0, New=30.0 (Limit=25.5)
  [+] PASSED: Performance is equivalent within threshold bounds!

Evaluating Gripper Camera:
  * Latency: Old=71.3ms, New=72.7ms (Limit=82.0ms)
  * FPS:     Old=30.1, New=29.9 (Limit=25.6)
  [+] PASSED: Performance is equivalent within threshold bounds!

Saved performance report to: camera_regression_results.md

@hello-ola
hello-ola self-requested a review August 10, 2026 21:24

@hello-ola hello-ola left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! note that test requires branch feature/combined_left_and_right_rgbd on stretch4_rgbd

@hello-robot-shehab
hello-robot-shehab force-pushed the feature/gripper_camera_performance_enhancements branch 2 times, most recently from 8e98866 to 18a60df Compare August 19, 2026 22:08
@hello-robot-shehab
hello-robot-shehab force-pushed the feature/gripper_camera_performance_enhancements branch from 18a60df to 86301c5 Compare August 19, 2026 23:18
@hello-robot-shehab
hello-robot-shehab merged commit 0e2fcb3 into main Aug 19, 2026
@hello-robot-shehab
hello-robot-shehab deleted the feature/gripper_camera_performance_enhancements branch August 19, 2026 23:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants