Since the standard error of our sample measures how closely we expect our sample mean to match the population mean, we could also consider the inverse—the standard error measures how closely we expect the population mean to match our measured sample mean. In other words, based on our standard error, we can infer that the population mean lies within some expected range of the sample mean with a certain degree of confidence.
Taken together, the "degree of confidence" and the "expected range" define a confidence interval. While stating confidence intervals, it is fairly standard to state the 95 percent interval—we are 95 percent sure that the population parameter lies within the interval. Of course, there remains a 5 percent possibility that it does not.
Whatever the standard error, 95 percent of the population mean will lie between -1.96 and 1.96 standard deviations of the sample mean. 1.96 is therefore the critical z-value for a 95 percent confidence interval.
