Probability and Statistics

In this chapter, we will cover two of the most important areas of mathematics—probability and statistics. These are two terms that you've likely come across a number of times in your everyday life. People use it to justify just about everything that occurs or when they're trying to prove a point. Once you are done with this chapter, you will have a firm grasp of both of them and will understand how they both are related and how they differ.

This chapter will cover the following topics:

• Understanding the concepts in probability
• Essential concepts in statistics

Understanding the concepts in probability

Probability theory is one of the most important fields of mathematics and is essential to the understanding and creation of deep neural networks. We will explore the specifics of this statement in the coming chapters. For now, however, we will focus our effort toward gaining an intricate understanding of this field.

We use probability theory to create an understanding of how likely it is that a certain event will occur. Generally speaking, probability theory is about understanding and dealing with uncertainty.

Classical probability

Let's suppose we have a random variable that maps the results of random experiments to the properties that interest us. The aforementioned random...

Essential concepts in statistics

While probability allows us to measure and calculate the odds of events or outcomes occurring, statistics allows us to make judgments and decisions given data generated by some unknown probability model. We use the data to learn the properties of the underlying probabilistic model. We call this process parametric inference.

Estimation

In estimation, our objective is given n iid samples with the same distribution as X (the probability model). If the PDF and probability mass function (PMF) is , we need to find θ.

Formally, we define a statistic as an estimate of θ.

A statistic is a function, T, of the data, , so that our estimate is . Therefore, T(x) is the sampling distribution...

Summary

In this chapter, we learned a lot of concepts. I recommend going through the chapter again if needed because the topics in this chapter are very important to gaining a deep understanding of deep learning. Many of you may be wondering what the chapters you have learned so far have to do with neural networks; we will tie it all together in a couple more chapters.

The next chapter focuses on both convex and non-convex optimization methods and builds the foundation for understanding the optimization algorithms used in training neural networks.