9.4 Alternatives to Bayesian deep learning
While the focus of the book is on Bayesian inference with DNNs, these aren’t always the best choice for the job. Generally speaking, they’re a great choice when you have large amounts of high dimensional data. As we discussed in Chapter 3, Fundamentals of Deep Learning (and as you probably know), deep networks excel in these scenarios, and thus adapting them for Bayesian inference is a sensible choice. On the other hand, if you have small amounts of low-dimensional data (with tens of features, fewer than 10,000 data points), then you may be better off with more traditional, well-principled Bayesian inference, such as via sampling or GPs.
That said, there has been interest in scaling GPs, and the research community has developed GP-based methods that both scale to large amounts of data and are capable of complex non-linear transformations. In this section, we’ll introduce these alternatives in case you wish to pursue...
