Probabilistic TensorFlow

Uncertainty is a fact of life; whether you are doing a classification task or a regression task, it is important to know how confident your model is in its prediction. Till now, we have covered the traditional deep learning models, and while they are great at many tasks, they are not able to handle uncertainty. Instead, they are deterministic in nature. In this chapter, you will learn how to leverage TensorFlow Probability to build models that can handle uncertainty, specifically probabilistic deep learning models and Bayesian networks. The chapter will include:

• TensorFlow Probability
• Distributions, events, and shapes in TensorFlow Probability
• Bayesian networks using TensorFlow Probability
• Understand uncertainty in machine learning models
• Model aleatory and epistemic uncertainty using TensorFlow Probability

All the code files for this chapter can be found at https://packt.link/dltfchp12

Let’...

TensorFlow Probability

TensorFlow Probability (TFP), a part of the TensorFlow ecosystem, is a library that provides tools for developing probabilistic models. It can be used to perform probabilistic reasoning and statistical analysis. It is built over TensorFlow and provides the same computational advantage.

Figure 12.1 shows the major components constituting TensorFlow Probability:

Figure 12.1: Different components of TensorFlow Probability

At the root, we have all numerical operations supported by TensorFlow, specifically the LinearOperator class (part of tf.linalg) – it contains all the methods that can be performed on a matrix, without the need to actually materialize the matrix. This provides computationally efficient matrix-free computations. TFP includes a large collection of probability distributions and their related statistical computations. It also has tfp.bijectors, which offers a wide range of transformed distributions.

Bijectors encapsulate...

TensorFlow Probability distributions

Every distribution in TFP has a shape, batch, and event size associated with it. The shape is the sample size; it represents independent and identically distributed draws or observations. Consider the normal distribution that we defined in the previous section:

normal = tfd.Normal(loc=0., scale=1.)

This defines a single normal distribution, with mean zero and standard deviation one. When we use the sample function, we do a random draw from this distribution.

Notice the details regarding batch_shape and event_shape if you print the object normal:

print(normal)

>>> tfp.distributions.Normal("Normal", batch_shape=[], event_shape=[], dtype=float32)

Let us try and define a second normal object, but this time, loc and scale are lists:

normal_2 = tfd.Normal(loc=[0., 0.], scale=[1., 3.])
print(normal_2)

>>> tfp.distributions.Normal("Normal", batch_shape=[2], event_shape=[], dtype...

Summary

This chapter introduced TensorFlow Probability, the library built over TensorFlow to perform probabilistic reasoning and statistical analysis. The chapter started with the need for probabilistic reasoning – the uncertainties both due to the inherent nature of data and due to a lack of knowledge. We demonstrated how to use TensorFlow Probability distributions to generate different data distributions. We learned how to build a Bayesian network and perform inference. Then, we built Bayesian neural networks using TFP layers to take into account aleatory uncertainty. Finally, we learned how to account for epistemic uncertainty with the help of the DenseVariational TFP layer.

In the next chapter, we will learn about TensorFlow AutoML frameworks.

References

1. Dillon, J. V., Langmore, I., Tran, D., Brevdo, E., Vasudevan, S., Moore, D., Patton, B., Alemi, A., Hoffman, M., and Saurous, R. A. (2017). TensorFlow distributions. arXiv preprint arXiv:1711.10604.
2. Piponi, D., Moore, D., and Dillon, J. V. (2020). Joint distributions for TensorFlow probability. arXiv preprint arXiv:2001.11819.
3. Fox, C. R. and Ülkümen, G. (2011). Distinguishing Two Dimensions of Uncertainty, in Essays in Judgment and Decision Making, Brun, W., Kirkebøen, G. and Montgomery, H., eds. Oslo: Universitetsforlaget.
4. Hüllermeier, E. and Waegeman, W. (2021). Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine Learning 110, no. 3: 457–506.

Join our book’s Discord space

Join our Discord community to meet like-minded people and learn alongside more than 2000 members at: https://packt.link/keras