Join our book community on Discord

https://packt.link/EarlyAccessCommunity

A well-known British statistician, George Box, is famous for saying that “All [statistical] models are wrong, but some are useful.” As machine learning (ML) practitioners, this is valuable to keep in mind because it reminds us that no matter how much we train a model or fine-tune its hyperparameters, no matter how good it seems to perform when making predictions in production, there will always be instances where it makes a wrong prediction, or the underlying data distribution changes, or any number of potential events that can make our model work less effectively. This is why, during training, we implement techniques to roughly simulate how our model will perform in a variety of situations.

This chapter explores advanced cross-validation methods, model evaluation metrics, and techniques for selecting the best models. Exercises focus on applying cross-validation and evaluation techniques to improve...