Summary

Our task for this chapter was to use machine learning to teach the robot how to use its robot arm. We used two techniques with some variations. We used a variety of reinforcement learning techniques, or Q-learning, to develop a movement path by selecting individual actions based on the robot’s arm state. Each motion was scored individually as a reward, and as part of the overall path as a value. The process stored the results of the learning in a Q-matrix that could be used to generate a path. We improved our first cut of the reinforcement learning program by indexing, or encoding, the motions from a 27-element array of possible combinations of motors as numbers from 0 to 26, and likewise indexing the robot state to a state lookup table. This resulted in a 40x speedup of the learning process. Our Q-learning approach struggled with the large number of states that the robot arm could be in.

Our second technique was a GA. We created individual random paths to make a...