Teaching the Robot to Navigate and Avoid Stairs

Let’s have a quick review of our quest to create a robot that picks up toys. We’ve created a toy detector and trained the robot arm. What’s next on our to-do list? We need to drive the robot to the location of the toy in order to pick it up. That sounds important.

This chapter covers navigation and path planning for our toy-grabbing robot helper. You have to admit that this is one of the most difficult problems in robotics. There are two parts to the task – figuring out where you are (localization), and then figuring out where you want to go (path planning). Most robots at this point would be using some sort of simultaneous localization and mapping (SLAM) algorithm that would first map the room, and then figure out where the robot is within it. But is this really necessary? First of all, SLAM generally requires some sort of 3D sensor, which we don’t have, and a lot of processing, which we don’...

Technical requirements

We require the Robot Operating System Version 2 (ROS 2) for this chapter. This book uses the Foxy Fitzroy release: http://wiki.ros.org/foxy/Installation. This chapter assumes that you have completed Chapter 6, where we gave the robot a voice and the ability to receive voice commands. We will be using the Mycroft interface and voice text-to-speech system, which is called Mimic: https://github.com/MycroftAI/mimic3. You’ll find the code for this chapter in the GitHub repository for this book at https://github.com/PacktPublishing/Artificial-Intelligence-for-Robotics-2e.

We will also be using the Keras library for Python (https://keras.io), which is a powerful library for machine learning applications and lets us build custom neural networks. You can install it using the following command:

pip install keras

You will also need PyTorch, which is installed with this command:

pip3 install torch torchvision torchaudio --index-url https://download.pytorch...

Task analysis

As we do for each chapter, let’s review what we are aiming to accomplish. We will be driving the robot around the house, looking for toys. Once we have a toy, we will take that toy to the toy box and put it away by dropping it into the toy box. Then, the robot will go look for more toys. Along the way, we need to avoid obstacles and hazards, which include a set of stairs going downward that would definitely damage the robot.

Note

I used a baby gate to cover the stairs for the first part of testing and put pillows on the stairs for the second part. There is no need to bounce the robot down the stairs while it is still learning.

We are going to start with the assumption that nothing in this task list requires the robot to know where it is. Is that true? We need to find the toy box – that is important. Can we find the toy box without knowing where it is? The answer is, of course, that the robot can just search for the toy box using its camera until...

Understanding the SLAM methodology

SLAM is a common methodology for navigating indoor robots. Before we get into the specifics, let’s look at two key issues:

• The first problem we have in indoor robot driving is that we don’t have a map
• The second problem we have is that we have no frame of reference to locate ourselves – GPS does not work indoors

That is two problems – we need a map, and then we need a way to locate ourselves on that map. While SLAM starts with the letter S for “simultaneous,” in truth, most robots make a map, store it away, and then drive on it later. Of course, while maps are being made, the robot must make the map and then locate itself on the map – usually in the center.

How does SLAM work? The sensor usually associated with SLAM is the spinning LIDAR. You can think of LIDAR as laser radar – it uses a laser to measure the distance to objects and spins in a circle to collect data all around...

Exploring alternative navigation techniques

In this section, we’ll look at some potential alternative methods for navigation that we could use for our robot now that we’ve ruled out the SLAM methodology:

• We could just drive around randomly, looking for toys. When we find a toy, the robot picks it up and then drives around randomly looking for the toy box. When it sees the toy box, it drives up to it and deposits the toy. But we still need a method to avoid running over obstacles. We could follow a process called structure from motion (SfM) to get depth information out of our single camera and use that to make a map. Structure from motion requires a lot of textures and edges, which houses may not have. It also leaves lots of voids (holes) that must be filled in the map. Structure from motion uses parallax in the video images to estimate the distance to the object in the camera’s field of view. There has been a lot of interesting work in this area, and I...

Introducing the Floor Finder technique

What I will be presenting in this chapter is my version of a Floor Finder technique that is different from RoboRealm, or other floor-finder algorithms, but that accomplishes the same results. Let’s break this simple concept down for ease of understanding.

We know that the floor directly in front of the robot is free from obstacles. We use the video image pixels of the area just in front of the robot as an example and look for the same texture to be repeated farther away. We are matching the texture of the part of the image we know is the floor with pixels farther away. If the textures match, we mark that area green to show that it is drivable and free of obstacles. We will be using bits of this technique in this chapter. By the way, did you notice that I said texture and not color? We are not matching the color of the floor, because the floor is not all one color. I have a brown carpet in my upstairs game room, which still has considerable...

Implementing neural networks

So, what does a neural network do? We use a neural network to predict some association of an input with an output. When we use a CNN, we can associate a picture with some desired output. What we did in our previous chapter was to associate a class name (toys) with certain images. But what if we tried to associate something else with images?

How about this? We use a neural network to classify the images from our camera. We drive the robot around manually, using a joystick, and take a picture about four times a second. We record what the robot is doing in each picture – going forward, turning right, turning left, or backing up. We use that information to predict the robot’s motion command given the image. We make a CNN, with the camera image as the input and four outputs – commands for go forward, go left, or go right. This has the advantage of avoiding fixed obstacles and hazards automatically. When we get to the stairs (remember...

Summary

This chapter introduced some concepts for robot navigation in an unstructured environment, which is to say, in the real world, where the designers of the robot don’t have control over the content of the space. We started by introducing SLAM, along with some of the strengths and weaknesses of map-based navigation. We talked about how Roomba navigates, by random interaction and statistical models. The method selected for our toy-gathering robot project, Albert, combined two algorithms that both relied mostly on vision sensors.

The first was the Floor Finder, a technique I learned when it was used by the winning entry in the DARPA Grand Challenge. The Floor Finder algorithm uses the near vision (next to the robot) to teach the far vision (away from the robot) what the texture of the floor is. We can then divide the room into things that are safe to drive on, and things that are not safe. This deals with our obstacle avoidance. Our navigation technique used a trained...

Questions

1. Regarding SLAM, what sensor is most commonly used to create the data that SLAM needs to make a map?
2. Why does SLAM work better with wheel odometer data available?
3. In the Floor Finder algorithm, what does the Gaussian blur function do to improve the results?
4. The final step in the Floor Finder is to trace upward from the robot position to the first red pixel. In what other way can this step be accomplished (referring to Figure 7.3)?
5. Why did we cut the image in half horizontally before doing our neural network processing?
6. What advantages does using the neural network approach provide that a technique such as SLAM does not?
7. If we used just a random driving function instead of the neural network, what new program or function would we have to add to the robot to achieve the same results?
8. How did we end up avoiding the stairs in the approach presented in the chapter? Do you feel this is adequate? Would you suggest any other means for accomplishing...

Further reading

• Deep Obstacle Avoidance by Sullivan and Lawson, published by Naval Research Labs, Rosebrock, Adrian.
• Artificial Intelligence with Python Cookbook by Ben Auffarth, Packt Publishing, 2020
• Artificial Intelligence with Python – Second Edition, by Prateek Joshi, Packt Publishing, 2020
• Python Image Processing Cookbook by Sandipan Dey, Packt Publishing, 2020