Chapter 6. Linear Regression and Gradient Descent

In the previous chapters, we've implemented non-parametric models including kNN and k-means and their applications to supervised classification and unsupervised clustering. In this chapter, we will proceed with the supervised learning by discussing algorithms for regression, this time focusing on the parametric models. Linear regression is the simple yet powerful tool for this kind of task. Linear regression was historically the first machine learning algorithm, so the math behind it is well developed, and you can find many books dedicated to this one topic exclusively. We will see when to use linear regression and when not to, how to analyze its errors, and how to interpret its results. As for the Swift part, we will get our feet wet with Apple's numerical libraries—the Accelerate framework.

Linear regression will serve as an example to explain an important mathematical optimization technique, gradient descent. This iterative algorithm will...