Chapter 4: Exploring Bayesian Optimization
Bayesian optimization (BO) is the second out of four groups of hyperparameter tuning methods. Unlike grid search and random search, which are categorized as uninformed search methods, all of the methods that belong to the BO group are categorized as informed search methods, meaning they are learning from previous iterations to (hopefully) provide a better search space in the future.
In this chapter, we will discuss several methods that belong to the BO group, including Gaussian process (GP), sequential model-based algorithm configuration (SMAC), Tree-structured Parzen Estimators (TPE), and Metis. Similar to Chapter 3, Exploring Exhaustive Search, we will discuss the definition of each method, the differences between them, how they work, and the pros and cons of each method.
By the end of this chapter, you will be able to explain BO and its variations when someone asks you. You will not only be able to explain what they are, but also...
