Software

AI + ML

Nvidia cosies up to Open Robotics for hardware-accelerated ROS

Hopes to tempt roboticists over to its Jetson platform with new simulation features, drop-in acceleration code


Nvidia has linked up with Open Robotics to drive new artificial intelligence capabilities in the Robot Operating System (ROS).

The non-exclusive agreement will see Open Robotics extending ROS 2, the latest version of the open-source robotics framework, to better support Nvidia hardware – and in particular its Jetson range, low-power parts which combine Arm cores with the company's own GPU and deep-learning accelerator cores to drive edge and embedded artificial intelligence applications.

"Our users have been building and simulating robots with Nvidia hardware for years, and we want to make sure that ROS 2 and Ignition work well on those platforms," Brian Gerkey, Open Robotics' chief exec, told The Register.

"We get most excited by two things: robots and open source. This partnership has both. We're working together with Nvidia to improve the developer experience for the global robotics community by extending the open source software on which roboticists rely. We're excited to work directly with Nvidia and have their support as we extend our software to take maximum advantage of their hardware."

The team-up will see Open Robotics working on ROS to improve the data flow between the various processors – CPU, GPU, NVDLA, and Tensor Cores – on Nvidia's Jetson hardware as a means to boost processing of high-bandwidth data.

As part of that, Open Robotics' Ignition and Nvidia's Isaac Sim simulation environments are to gain interoperability – meaning robot and environment models can be moved from one to the other, at least when the software is finished some time early next year.

As for why Nvidia's accelerated computing portfolio, and in particular its embedded Jetson family of products, should appeal to robot-makers, Gerkey said: "Nvidia has invested heavily in compute hardware that's relevant for modern robotics and AI workloads. Robots ingest and process large data volumes from sensors such as cameras and lasers. Nvidia's architecture allows that data flow to happen incredibly efficiently."

Murali Gopalakrishna, head of product management, Intelligent Machines, at Nvidia said of the hookup: "Nvidia's GPU-accelerated computing platform is at the core of many AI robot applications and many of those are developed using ROS, so it is logical that we work closely with open robotics to advance the field of robotics.

The work also brings with it some new Isaac GEMs, hardware-accelerated packages for ROS designed to replace code which would otherwise run on the CPU. The latest GEMs include packages for handling stereo imaging and point cloud data, colour space conversion, lens distortion correction, and the detection and processing of AprilTags – QR Code-style 2D fiducial tags developed at the University of Michigan.

The partnership doesn't mean the two are going steady, though. "We are eager to extend ROS 2 in similar ways on other accelerated hardware," Gerkey told us of planned support for other devices like Intel's Myriad X and Google's TPU – to say nothing of GPU hardware from Nvidia rival AMD.

"In fact, we plan for the work we do together with Nvidia to lay the foundation for additional extensions for additional architectures. To other hardware manufacturers: please contact us to talk about extensions for your platform!"

The latest Isaac GEMs are available on Nvidia's GitHub repository now; the interoperable simulation environments, meanwhile, aren't expected to release until the (northern hemisphere) spring of 2022.

Nvidia's Gopalakrishna said it was possible for ROS developers to begin experimenting before the release date. "The simulator already has a ROS 1 and ROS 2 bridge and has examples of using many of the popular ROS packages for navigation (nav2) and manipulation (MoveIT). Many of these developers are also leveraging Isaac Sim to generate synthetic data to train the perception stack in their robots. Our spring release will bring additional functionality like interoperability between Gazebo Ignition and Isaac Sim."

When we asked what performance uplift could users expect from the new Isaac GEMs compared to CPU-only packages, we were told: "The amount of performance gain will vary depending on how much inherent parallelism exists in a given workload. But we can say that we are seeing an order of magnitude increase in performance for perception and AI related workloads. By using the appropriate processor to accelerate the different tasks, we see increased performance and better power efficiency."

As for additional features in the pipeline, Gopalakrishna said: "Nvidia is working with Open Robotics to make the ROS framework more streamlined for hardware acceleration and we will also continue to release multiple new Isaac GEMs, our hardware accelerated software packages for ROS.

"Some of these will be DNNs which are commonly used in robotics perception stacks. On the simulator side, we are working to add support for more sensors and robots and release more samples that are relevant to the ROS community." ®

Send us news
2 Comments

Publisher halts AI article assembly line after probe

Plus: Academics debate citing ChatGPT as research co-author; Getty Images sues Stability AI

Software glitch revokes copyright protection for AI-generated comic book

So is AI artwork copyrightable? Not quite as case remains up in the air

Midjourney, DeviantArt face lawsuit over AI-made art

Lawyer known for GitHub Copilot case to argue artists' legal struggle

Universities offered software to sniff out ChatGPT-written essays

Teachers hopefully get their hands on the tool within months

AI lawyer to fight first legal case in court, startup claims

Plus: How much would you pay for ChatGPT? And British AI drug biz gets snapped up for half a billion

What keeps this FBI director up at night? China’s AI work, for one

Hacking the world with ML is Uncle Sam's job, buddy

OpenAI's ChatGPT is a morally corrupting influence

No principles or virtues, people accepting everything it says – this bot is perfect for political life

Publisher breaks news by using bots to write inaccurate stories

Worse still, human editors appear not to have caught the mistakes

Writing tool from AI21 Labs won't do all the hard work for you

Wordtune Spices is more like an AI-powered Grammarly

Adobe: Take user data to train generative AI models? We'd never do that

Controversial policy change forces chief product officer to speak out

Microsoft to offer ChatGPT-as-a-service from Azure real soon now

Addition of the chatbot everyone's talking about, and to, comes as Azure OpenAI goes GA

Microsoft: You're not out of love with cloud, you're just 'optimizing' it for a bit

Q2 results include modest revenue increase, steep consumer and net income decrease