10.4 Summary
Until this chapter, our mathematical study was quite close to machine learning. Vectors, matrices, functions: they are all there at the ground zero of theory and practice.
This time, however, we’ve gone far below the surface. We rarely work directly with sequences in practice, but despite appearances, they are all over the place, providing a solid theoretical foundation for everything that is quantitative.
So, what did we learn? What numbers are, for one. Going from natural numbers to real numbers is nothing short of a revelation, allowing us to see the evolution of the concept of a number. But deep down, sequences hold the concept of numbers together. And whenever we talk about sequences, limits and convergence enter the picture.
To summarize, gradient descent is about the limit of the sequence
converging to a local minima of f if the stars are aligned. In the following chapters, our main goal is to understand xn+1 = xn −hf′(xn). What is f′...
