Pico teleoperation demo written in python for both mujoco simulation and robot hardware.

This project provides a framework for controlling robots in robot hardware and MuJoCo simulation through XR (VR/AR) input devices. It allows users to manipulate robot arms using natural hand movements captured through XR controllers.

1. Download and install XRoboToolkit PC Service. Run the installed program before running the following demo.

2. Clone the repository:

   git clone https://github.com/XR-Robotics/XRoboToolkit-Teleop-Sample-Python.git
   cd XRoboToolkit-Teleop-Sample-Python

3. Installation Note: The setup scripts are currently only tested on Ubuntu 22.04. It is recommended to setup a Conda environment and install the project using the included script.

   bash setup_conda.sh --conda <optional_env_name>
   conda activate <env_name>
   bash setup_conda.sh --install

   If installing on system python:

   bash setup.sh

Use the following instructions to run example scripts. For a more detailed description, please refer to teleop_details.md.

To run the teleoperation demo with a UR5e robot in MuJoCo simulation:

python scripts/simulation/teleop_dual_ur5e_mujoco.py

This script initializes the MujocoTeleopController with the UR5e model and starts the teleoperation loop.

To run the teleoperation demo with a UR5e robot in Placo visualization:

python scripts/simulation/teleop_x7s_placo.py

This script initializes the PlacoTeleopController with the X7S robot and starts the teleoperation loop.

• Shadow hand simulation in Mujoco

  python scripts/simulation/teleop_shadow_hand_mujoco.py

• Inspire hand in Placo Visualization

  scripts/simulation/teleop_inspire_hand_placo.py

To run the teleoperation demo with the physical dual UR arms and Dynamixel-based head:

1. Normal Operation:

   python scripts/hardware/teleop_dual_ur5e_hardware.py

   This script initializes the DynamixelHeadController and DualArmURController and starts the teleoperation loops for both head tracking and arm control.

2. Resetting Arm Positions: If you need to reset the UR arms to their initial/home positions and initialize the robotiq grippers, you can run the script with the --reset flag:

   python scripts/hardware/teleop_dual_ur5e_hardware.py --reset

   This will execute the reset procedure defined in the DualArmURController and then exit.

3. Visualizing IK results: To visualize the inverse kinematics solution with placo during teleoperation, run the script with the --visualize_placo flag.

   python scripts/hardware/teleop_dual_ur5e_hardware.py --visualize_placo

To run the teleoperation demo with a Galaxea A1X arm, follow the instruction to setup the ROS environment. In two command line windows, run the following scripts.

roslaunch HDAS A1XY.launch

python scripts/hardware/teleop_a1x_hardware.py

XR Robotics dependencies:

• xrobotookit_sdk: Python binding for XRoboToolkit PC Service SDK, MIT License

Robotics Simulation and Solver

• mujoco: robotics simulation, Apache 2.0 License
• placo: inverse kinematics, MIT License

Hardware Control

• dynamixel_sdk: Dynamixel control functions, Apache-2.0 License
• ur_rtde: interface for controlling and receiving data from a UR robot, MIT License
• Galaxea A1XY: Ros Noetic package for controlling A1XY manipulators

This project is licensed under the MIT License - see the LICENSE file for details.