In the preceding list, step 4 implies an iterative process where we try models, parameters, and features until we get the best result that we can. Let's now think about a classification problem, where we want to separate squares from circles, as shown in the following diagram. At the beginning of the process, we will probably be in a situation that is similar to the first chart (on the left-hand side). The model fails to efficiently separate the two shapes and both sides are a mixture of both squares and circles. This is called underfitting and refers to a model that fails to represent the characteristics of the dataset:
As we continue tuning parameters and adjusting the model to the training dataset, we might find ourselves in a situation that is similar to the third chart (on the right-hand side). The model accurately splits the dataset...
