Simplifying hardware acceleration for robots with ROS 2 – TechTalks


By Victor Mayoral-Vilches

When developing a robot, system integration arguably consumes the largest proportion of project resources; much more than developing the final application. With the advent of low-end industrial collaborative robots, companies have emerged that focus solely on developing software to run on existing hardware. However, there is a critical relationship between the hardware and software capabilities of a robot. Retaining design control over hardware is necessary to create more specialized, energy-efficient, secure, and high-performance robots.

Material challenge. Software skills

But there is a hurdle that must be overcome if roboticists are to deliver the better and faster robots that will be needed in the future. In today’s post-Moore computing world, upgrading hardware to adopt the latest generation microprocessor cannot provide the desired upgrade in application performance. The way forward is no longer to wait for the last chip lithograph. Hardware acceleration is often the only way to get the necessary gains.

This hardware challenge complicates the lives of developers in disciplines like robotics, whose skills tend to be software-oriented. This means they must face the prospect of designing adaptive computing hardware if they are to meet market demands for new industrial robots. End users – companies that use robots to increase productivity in areas such as production lines and warehouses – are looking for equipment that offers additional dexterity, more precise position control, superior vision-based capabilities , improved data capture and reduced power consumption.

The main principle underlying hardware acceleration in robotics is that a mixed control and data-based approach to software development, unlike the traditional control-based approach, allows teams to design custom calculations that allocate the optimal amount of hardware resources for an application.

Regarding the implementation, a heterogeneous calculation model is required. This takes advantage of the strengths of CPUs and GPUs, which excel in control flow calculations, while leveraging the strengths of FPGAs to handle data flow calculations. This approach simultaneously provides the flexibility and full control of CPU/GPUs to implement complex computations, along with the low power, high performance, low latency, and deterministic nature of hardware acceleration. Various vendors now offer adaptive System-on-Chips (SoC) and System-on-Module (SOM) devices – such as the AMD-Xilinx Kria™ SOM and its associated Kria robotics stack – that offer the benefits of such a mixed calculation. The table compares these different models.

Comparison of hardware computer models in robotics

Adaptive SoCs and SOMs allow roboticists to create machine behaviors by programming an architecture that creates the right data paths and control mechanisms. However, complex engineering skills are required to program such architectures using established tools and techniques.

Appropriate hardware and embedded design expertise is rare among roboticists, who are used to creating behaviors in the form of computer graphs that solve the robotic task at hand. They often use C++ to create complex real-time deterministic systems through advanced software engineering practices.

Building on ROS

A different approach is now needed to help roboticists take advantage of available hardware acceleration technologies. Ideally, such an approach should allow them to create their custom hardware by working in a familiar development environment – such as ROS – and using familiar tools – such as Gazebo for simulation.

ROS is the de facto industry standard for robotics application development and more since the arrival of ROS 2 in 2020. It has become the default software development kit (SDK) for robotics applications in all the sectors. Many groups now use ROS and Gazebo.

Previous initiatives to integrate adaptive computing into ROS have addressed the challenge from a hardware engineer’s perspective. They assumed that the user had previous experience with embedded and hardware streams and was therefore familiar with concepts such as RTL, HDL, and HLS, and the design tools used to manipulate them. Likewise, the deployment in embedded targets requires a certain familiarity with Yocto, OpenEmbedded and the associated tools.

Understanding that most roboticists do not come from this background, the ROS 2 Hardware Acceleration Working Group (HAWG) takes a ROS-centric approach to integrating embedded flows directly into the ROS ecosystem. Its goal is to provide an experience similar to what roboticists get when building ROS workspaces on their desktop workstations.

The HAWG’s work builds on published research on optimizing the ROS computational graph to take advantage of adaptive computing, as well as proposals for tools and methodologies for accelerating portions of the graph in programmable logic. On top of that, HAWG now offers an architecture (figure below) that focuses on familiar languages ​​like C++ and OpenCL.

ROS Stack 2 and Hawg
The ROS 2 and HAWG stacks together make hardware acceleration easy.

The proposed architecture aims to be not only platform independent, and therefore suitable for edge, workstation, data center and/or cloud targets, but also technology independent to enable targeting of FPGAs, CPUs and GPUs as well as easily portable to various modules and boards.

Ultimately, this work should enable the majority of roboticists to take advantage of hardware acceleration opportunities to produce future generations of advanced and sophisticated robots.

About the Author

Victor Mayoral-Vilches

Víctor Mayoral-Vilches is a Robotic System Architect at Adaptive & Embedded Computing Group, AMD. With a solid technical background in robotics, embedded systems and cybersecurity, he has spent the last 10 years building robots. Prior to joining AMD-Xilinx, he founded and led three robotics startups by building teams of over 30 engineers and leading them in initiatives and research projects in the fields of robotics, cybersecurity and technology. ‘artificial intelligence. Víctor was selected as one of the ten most innovative people under 35 in Spain by the MIT Technology Review in 2017 and holds several positions as national experts representing Spain in ISO and IEC committees for standards in robotics, safety and cybersecurity.


Comments are closed.