Summary

In this chapter, we introduced hyperparameter tuning (through HyperDrive) and Azure Automated Machine Learning. We observe that both techniques can help you to efficiently retrieve the best model for your ML task.

Grid sampling works great with classical ML models, and also when the number of tunable parameters is fixed. All values on a discrete parameter grid are evaluated. In random sampling, we can apply a continuous distribution for the parameter space and select as many parameter choices as we can fit into the configured training duration. Random sampling performs better on a large number of parameters. Both sampling techniques can/should be tuned using an early stopping criterion.

Unlike random and grid sampling, Bayesian optimization probes the model performance in order to optimize the following parameter choices. This means that each set of parameter choices and the resulting model performance are used to compute the next best parameter choices. Therefore, Bayesian...