Nvidia cosies up to Open Robotics for hardware-accelerated ROS

Nvidia has linked up with Open Robotics to drive new artificial intelligence capabilities in the Robot Operating System (ROS).

The non-exclusive agreement will see Open Robotics extending ROS 2, the latest version of the open-source robotics framework, to better support Nvidia hardware – and in particular its Jetson range, low-power parts which combine Arm cores with the company's own GPU and deep-learning accelerator cores to drive edge and embedded artificial intelligence applications.

"Our users have been building and simulating robots with Nvidia hardware for years, and we want to make sure that ROS 2 and Ignition work well on those platforms," Brian Gerkey, Open Robotics' chief exec, told The Register.

"We get most excited by two things: robots and open source. This partnership has both. We're working together with Nvidia to improve the developer experience for the global robotics community by extending the open source software on which roboticists rely. We're excited to work directly with Nvidia and have their support as we extend our software to take maximum advantage of their hardware."

The team-up will see Open Robotics working on ROS to improve the data flow between the various processors – CPU, GPU, NVDLA, and Tensor Cores – on Nvidia's Jetson hardware as a means to boost processing of high-bandwidth data.

As part of that, Open Robotics' Ignition and Nvidia's Isaac Sim simulation environments are to gain interoperability – meaning robot and environment models can be moved from one to the other, at least when the software is finished some time early next year.

As for why Nvidia's accelerated computing portfolio, and in particular its embedded Jetson family of products, should appeal to robot-makers, Gerkey said: "Nvidia has invested heavily in compute hardware that's relevant for modern robotics and AI workloads. Robots ingest and process large data volumes from sensors such as cameras and lasers. Nvidia's architecture allows that data flow to happen incredibly efficiently."

• Japan, Singapore, perhaps the whole world.... Get ready for robot waiters from Softbank and Keenon
• Royal Navy will be getting autonomous machines – for donkey work humans can't be bothered with
• Businesses put robots to work when human workers are hard to find, argue econo-boffins
• Spot the dog? No, we couldn't either because Spot is a robot employed by United Kingdom Atomic Energy Authority

Murali Gopalakrishna, head of product management, Intelligent Machines, at Nvidia said of the hookup: "Nvidia's GPU-accelerated computing platform is at the core of many AI robot applications and many of those are developed using ROS, so it is logical that we work closely with open robotics to advance the field of robotics.

The work also brings with it some new Isaac GEMs, hardware-accelerated packages for ROS designed to replace code which would otherwise run on the CPU. The latest GEMs include packages for handling stereo imaging and point cloud data, colour space conversion, lens distortion correction, and the detection and processing of AprilTags – QR Code-style 2D fiducial tags developed at the University of Michigan.

The partnership doesn't mean the two are going steady, though. "We are eager to extend ROS 2 in similar ways on other accelerated hardware," Gerkey told us of planned support for other devices like Intel's Myriad X and Google's TPU – to say nothing of GPU hardware from Nvidia rival AMD.

"In fact, we plan for the work we do together with Nvidia to lay the foundation for additional extensions for additional architectures. To other hardware manufacturers: please contact us to talk about extensions for your platform!"

The latest Isaac GEMs are available on Nvidia's GitHub repository now; the interoperable simulation environments, meanwhile, aren't expected to release until the (northern hemisphere) spring of 2022.

Nvidia's Gopalakrishna said it was possible for ROS developers to begin experimenting before the release date. "The simulator already has a ROS 1 and ROS 2 bridge and has examples of using many of the popular ROS packages for navigation (nav2) and manipulation (MoveIT). Many of these developers are also leveraging Isaac Sim to generate synthetic data to train the perception stack in their robots. Our spring release will bring additional functionality like interoperability between Gazebo Ignition and Isaac Sim."

When we asked what performance uplift could users expect from the new Isaac GEMs compared to CPU-only packages, we were told: "The amount of performance gain will vary depending on how much inherent parallelism exists in a given workload. But we can say that we are seeing an order of magnitude increase in performance for perception and AI related workloads. By using the appropriate processor to accelerate the different tasks, we see increased performance and better power efficiency."

As for additional features in the pipeline, Gopalakrishna said: "Nvidia is working with Open Robotics to make the ROS framework more streamlined for hardware acceleration and we will also continue to release multiple new Isaac GEMs, our hardware accelerated software packages for ROS.

"Some of these will be DNNs which are commonly used in robotics perception stacks. On the simulator side, we are working to add support for more sensors and robots and release more samples that are relevant to the ROS community." ®