Summary

In this chapter, we discussed AutoML, which aims to provide methods for model selection and hyperparameter optimization. AutoML is useful for beginners who have little expertise on making decisions such as how many layers to put in a model, which optimizer to use, and so on. AutoML is also useful for experts to both speed up the model training process and discover superior model architectures for a given task that would be nearly impossible to figure manually.

We looked at two different AutoML tools that can be used with PyTorch. First, we discussed Auto-PyTorch, which does the task of both finding an optimal neural architecture and finding the perfect hyperparameter setting. We used the MNIST handwritten digit classification task from Chapter 1, Overview of Deep Learning Using PyTorch, to find the best model for this task, using Auto-PyTorch. We obtained a best accuracy of 96.4%.

Next, we explored Optuna which is another AutoML tool that automates hyperparameter search...