In industrial applications, multiple robots with different specialized capabilities must work in concert to carry out complex tasks. This project showcases how coordination between multiple robots can be achieved via the Unity Editor and robotics simulation packages, along with ROS 2, to carry out a find-and-ferry task in a warehouse. This demonstration also highlights the advantage of using Unity over other robotics simulation tools where multi-agent simulations like this are challenging to accomplish. Our simulation consists of two types of robots, which we call Findbot and Ferrybot. Multiple Findbots are responsible for finding target cubes in a warehouse environment using machine learning, and a single Ferrybot navigates to, picks up, and drops off these cubes at a designated location. To accomplish this, each Findbot is equipped with a camera for detecting the cube, while the Ferrybot has a robotic arm for picking it up. This example project will be useful for robotics developers and researchers looking to use Unity’s robotics tools in their own simulations.
Overall, this was a great experience because we were able to use and integrate a wide array of Unity packages into our project. For instance, we used the Computer Vision Perception Package for data collection to train our pose estimation model. We also used an inverse kinematics package (mentioned in Jacob’s project above) on Ferrybot for picking up the cubes. Taking a dependency on a project being developed in parallel with ours was also a major challenge, but it was a great opportunity for learning collaboration and communication. It is also very rewarding to know that our project will be used to prepare a ROSCon 2021 workshop.
Source: Unity Technologies Blog