In the previous chapter, we learned how Recurrent Neural Networks (RNNs) are essential for music generation because they make it possible to operate on a sequence of vectors and remember past events. This second part is really important in music generation since past events play an important role in defining the global musical structure. Let's consider the example of a broken minor ninth chord of "A," "C," "E," "G," "B." To predict the last note, "B," the network has to remember four events back to know that this is probably a minor ninth chord being played.
Unfortunately, as the gap between the relevant information and the point where it is needed grows, RNNs become unable to learn the dependency. In theory, the network could be able to do it, but in practice, it is really difficult...
