[Open Call] Robotic Control via Embodied Chain-of-Thought Reasoning, leverage vision-language-action models (VLAs) for robotics using Jetson Orin

[elainedanwu]

Objective:

Develop a robotic control system using embodied chain-of-thought reasoning (ECoT) to enable robots to think, perceive, and act more effectively. By integrating Vision-Language-Action (VLA) models, the task will leverage the NVIDIA Jetson Orin to improve decision-making, task planning, and action execution in robotic systems. This setup will enable robots to reason iteratively about tasks before acting, allowing for greater flexibility and generalization in unstructured environments.

Simulation with MimicGen

MimicGen creates randomized episodes from as few as 10 teleoperated examples by utilizing scene graph information and task/subtask metadata about which objects in the environment are targets of the current subtask, in order to interpolate the original teloperated trajectories into their new random locations and poses. This generates large amounts of unique training data to improve robustness, without needing large amounts of human effort for the robot learning new skills and behaviors.

Deliverables:

1. Docker Images and Files:

   • Include all dependencies, configurations, and environment variables required for the system.

2. Tutorial Documentation:

   • Provide detailed deployment steps, configuration methods, button usage guides, and solutions for common issues. Please refer to Jetson AI Lab: https://www.jetson-ai-lab.com/openvla.html

3. Source Code and Development Documentation:

   • Include all source code, comments, and detailed development documentation to facilitate maintenance and feature expansion.

4. Test Report:

   • Include results from functional testing, performance testing, and user experience testing.

Reference Links:

• Robotic Control via Embodied Chain-of-Thought Reasoning
• OpenVLA - Vision/Language Action Models for Embodied Robotics

[kouroshkarimi]

@elainedanwu
it is my pleasure to do this...