Working with ROS 2 and Perception Stack

Robots interact with the environment thanks to their sensors. Without sensors, robots are blind machines unable to react to the world’s stimuli and can only perform cyclic motions. However, adding sensors to a robot also means the necessity to implement proper algorithms to process the sensor data. One of the most common sensor types used in robotics is visual sensors, like cameras. In this chapter, we will discuss how to interface standard cameras with our laptops and elaborate images using Open Computer Vision (OpenCV), a popular open-source library to process video streams. In addition, we will see the main limitations of cameras, that is, the missing information about the depth. For this reason, we will discuss another type of sensor: depth sensors and the processing of their data (point cloud data). Apart from these libraries that offer functionalities to elaborate raw image and spatial data, we will discuss artificial intelligence...