Chapter 8. Probability Distributions, Covariance, and Correlation

In Chapter 6, Dimensionality Reduction with Principal Component Analysis, we discussed principal component analysis. In the previous chapter, we discovered association rules using apriori in R. In this chapter, we will examine the following:

• Probability distributions

• A short introduction to descriptive statistics (mean and standard deviation)

• Covariance and correlation, notably what they mean and how they are computed

• How to perform correlation analysis in R

Probability distributions

In this section, we very briefly examine important distributions for common statistical problems with data consisting of quantities: the normal distribution and Student's t-distributions. We first introduce the idea of distributions with a discrete uniform distribution. We conclude with binomial distribution. We will try to be as non-technical as possible in this introduction to allow readers without statistical knowledge to follow easily; however, don't worry, we will be highly technical when explaining how to build functions that estimate correlations and regression coefficients.

Introducing probability distributions

Here, we introduce the idea of distributions using discrete uniform and binomial distributions.

Discrete uniform distribution

You might remember that, in Chapter 2, Visualizing and manipulating data using R, we examined outcomes of the roulette game. We showed that each of the 37 numbers (0 to 36) in European roulette has an equal probability of occurring...

Covariance and correlation

Before going in depth into the topic of this section, let me remind the reader of three mathematical notions that will be used in this chapter: arithmetic mean, variance, and standard deviation. Some have been already discussed in other chapters, but a more formal definition is interesting for the purposes of the chapter.

The arithmetic mean is a measure of central tendency. Considering a sample of observations of an attribute—for instance, the height of individuals—the arithmetic mean is simply the sum of the values of the observations divided by the number of observations. We are interested in computing the mean height of three individuals measuring 160 cm, 170 cm, and 180 cm.

The formula for the mean is:

Type the following in the R console to compute the arithmetic mean of this sample:

(160 + 170 + 180) / 3

R outputs the following:

[1] 170

Check the solution by typing this:

mean(c(160,170,180))

Our computation of the mean was correct—R outputs:

[1] 170

Variance is a...

Summary

In this chapter, we have briefly examined what statistical distributions are and why they are important when doing statistical inference. After a short introduction to descriptive statistics (mean and standard deviation), we have discovered how to obtain covariance and correlation coefficients programmatically. In the next chapter, we will discuss regression analysis in R.