6.4 z-Scores and the Standard Normal Distribution

We now understand how to describe and present our data visually and numerically. These simple tools, and the principles behind them, will help you interpret information presented to you and understand the basics of a variable. Moving forward, we now turn our attention to how scores within a distribution are related to one another, how to precisely describe a score’s location within the distribution, and how to compare scores from different distributions.

Normal Distributions

The normal distribution is the most important and most widely used distribution in statistics. It is sometimes called the “bell curve,” although the tonal qualities of such a bell would be less than pleasing. It is also called the “Gaussian curve” of Gaussian distribution after the mathematician Karl Friedrich Gauss.

The Gaussian distribution is commonly used partly because of its convenient characteristics in that it is symmetric, bell-shaped and unimodal. Each distribution is given by two parameters, the mean, [latex]\mu[/latex], and the standard deviation, [latex]\sigma[/latex]. For a normal distribution, we can denote its characteristics using [latex]N(\mu,\sigma)[/latex]. Mathematically, there is a very complex function that represents the Gaussian distribution, and it is written as the following:

[latex]f(x)=\frac{1}{\sigma \sqrt{2\pi}}e^{-\frac{1}{2}\left(\frac{x-\mu}{\sigma}\right)^2}[/latex]

where: [latex]-\infty[latex]-\infty<\mu<\infty[/latex], and [latex]\sigma>0[/latex].

In order to calculate the probability of an event occurring by hand, we would require a high level of mathematics knowledge and calculus skills. Luckily, there are methods that have been developed so that we can find the probability of a certain event occurring without these techniques and advanced mathematics. In this section and the next, we will learn how to use a [latex]z[/latex]-table to calculate certain probabilities.

Strictly speaking, it is not correct to talk about “the normal distribution” since there are many normal distributions. Normal distributions can differ in their means and in their standard deviations. Figure 6.4.1 shows three normal distributions. The blue (left-most) distribution has a mean of [latex]-3[/latex] and a standard deviation of [latex]0.5[/latex], the distribution in red (the middle distribution) has a mean of [latex]0[/latex] and a standard deviation of [latex]1[/latex], and the black (right-most) distribution has a mean of [latex]2[/latex] and a standard deviation of [latex]3[/latex]. These as well as all other normal distributions are symmetric with relatively more values at the centre of the distribution and relatively few in the tails. What is consistent about all normal distribution is the shape and the proportion of scores within a given distance along the [latex]x[/latex]-axis. We will focus on the standard normal distribution (also known as the unit normal distribution), which has a mean of [latex]0[/latex] and a standard deviation of [latex]1[/latex] (i.e., the red distribution in Figure 6.4.1).

These properties enable us to use the normal distribution to understand how scores relate to one another within and across a distribution. But first, we need to learn how to calculate the standardized score that makes up a standard normal distribution.

 If [latex]x[/latex] is a random variable and has a normal distribution with mean [latex]\mu[/latex] and standard deviation [latex]\sigma[/latex], then the Empirical Rule states the following:

• About [latex]68\%[/latex] of the values of [latex]x[/latex] lie between a [latex]-1\sigma[/latex] and [latex]+1\sigma[/latex] of the mean [latex]\mu[/latex] (within one standard deviation of the mean).
• About [latex]95\%[/latex] of the values of [latex]x[/latex] lie between [latex]-2\sigma[/latex] and  [latex]+2\sigma[/latex] of the mean [latex]\mu[/latex] (within two standard deviations of the mean).
• About [latex]99.7\%[/latex] of the values of [latex]x[/latex] lie between [latex]-3\sigma[/latex] and  [latex]+3\sigma[/latex] of the mean [latex]\mu[/latex] (within three standard deviations of the mean).
• The [latex]z[/latex]-scores for [latex]+1\sigma[/latex] and [latex]-1\sigma[/latex] are [latex]+1[/latex] and [latex]-1[/latex], respectively.
• The [latex]z[/latex]-scores for [latex]+2\sigma[/latex] and [latex]-2\sigma[/latex] are [latex]+2[/latex] and [latex]-2[/latex], respectively.
• The [latex]z[/latex]-scores for [latex]+3\sigma[/latex] and [latex]-3\sigma[/latex] are [latex]+3[/latex] and [latex]-3[/latex], respectively.

The Empirical Rule is also known as the [latex]68[/latex]–[latex]95[/latex]–[latex]99.7[/latex] rule.

Example 6.4.1

Suppose [latex]x[/latex] has a normal distribution with mean [latex]50[/latex] and standard deviation [latex]6[/latex].

Solution

Step 1: About [latex]68\%[/latex] of the [latex]x[/latex] values lie within one standard deviation of the mean.

Therefore, about [latex]68\%[/latex] of the [latex]x[/latex] values lie between [latex]-1\sigma=(-1)(6)=-6[/latex] and [latex]1\sigma=(1)(6)=6[/latex] of the mean [latex]50[/latex]. The values [latex]50-6=44[/latex] and [latex]50+6=56[/latex] are within one standard deviation from the mean [latex]50[/latex]. The [latex]z[/latex]-scores are [latex]-1[/latex] and [latex]+1[/latex] for [latex]44[/latex] and [latex]56[/latex], respectively.

Step 2: About [latex]95\%[/latex] of the [latex]x[/latex] values lie within two standard deviations of the mean.

Therefore, about [latex]95/%[/latex] of the [latex]x[/latex] values lie between [latex]-2\sigma=(-2)(6)=-12[/latex] and [latex]2\sigma=(2)(6)=12[/latex]. The values [latex]50-12=38[/latex] and [latex]50+12=62[/latex] are within two standard deviations from the mean [latex]50[/latex]. The [latex]z[/latex]-scores are [latex]-2[/latex] and [latex]+2[/latex] for [latex]38[/latex] and [latex]62[/latex], respectively.

Step 3: About [latex]99.7\%[/latex] of the [latex]x[/latex] values lie within three standard deviations of the mean.

Therefore, about [latex]95\%[/latex] of the [latex]x[/latex] values lie between [latex]-3\sigma=(-3)(6)=-18[/latex] and [latex]3\sigma=(3)(6)=18[/latex]from the mean [latex]50[/latex]. The values [latex]50-18=32[/latex] and [latex]50+18=68[/latex] are within three standard deviations of the mean [latex]50[/latex]. The [latex]z[/latex]-scores are [latex]-3[/latex] and [latex]+3[/latex] for [latex]32[/latex] and [latex]68[/latex], respectively.

Try It

2) From 1984 to 1985, the mean height of 15 to 18-year-old males from Chile was [latex]172.36cm[/latex], and the standard deviation was [latex]6.34cm[/latex]. Let [latex]Y=[/latex] the height of 15 to 18-year-old males in 1984 to 1985. Then [latex]Y\sim N(172.36, 6.34)[/latex].

a. About [latex]68\%[/latex] of the [latex]y[/latex] values lie between what two values? These values are                and               . The [latex]z[/latex]-scores are               and               , respectively.
b. About [latex]95\%[/latex] of the [latex]y[/latex] values lie between what two values? These values are                and               . The [latex]z[/latex]-scores are                and               , respectively.
c. About [latex]99.7\%[/latex] of the [latex]y[/latex] values lie between what two values? These values are                and               . The [latex]z[/latex]-scores are                and               , respectively.

Solution

a. About [latex]68\%[/latex] of the values lie between [latex]166.02cm[/latex] and [latex]178.7cm[/latex]. The [latex]z[/latex]-scores are [latex]-1[/latex] and [latex]1[/latex].
b. About [latex]95\%[/latex] of the values lie between [latex]159.68cm[/latex] and [latex]185.04cm[/latex]. The [latex]z[/latex]-scores are [latex]-2[/latex] and [latex]2[/latex].
c. About [latex]99.7\%[/latex] of the values lie between [latex]153.34cm[/latex] and [latex]191.38cm[/latex]. The [latex]z[/latex]-scores are [latex]-3[/latex] and [latex]3[/latex].

Z Scores

A z-score is a standardized version of a raw score ([latex]x[/latex]) that gives information about the relative location of that score within its distribution. The formula for converting a raw score into a [latex]z[/latex]–score is

[latex]z=\frac{x-\mu}{\sigma}[/latex] for values from a population and

[latex]z=\frac{x-M}{s}[/latex] for values from a sample.

As you can see, [latex]z[/latex]–scores combine information about where the distribution is located (the mean/centre) with how wide the distribution is (the standard deviation/spread) to interpret a raw score ([latex]x[/latex]). Specifically, [latex]z[/latex]–scores will tell us how far the score is away from the mean in units of standard deviations and in what direction.

The value of a [latex]z[/latex]–score has two parts: the sign (positive or negative) and the magnitude (the actual number). The sign of the [latex]z[/latex]–score tells you in which half of the distribution the [latex]z[/latex]–score falls: a positive sign (or no sign) indicates that the score is above the mean and on the right-hand side or upper end of the distribution, and a negative sign tells you the score is below the mean and on the left-hand side or lower end of the distribution. The magnitude of the number tells you, in units of standard deviations, how far away the score is from the centre or mean. The magnitude can take on any value between negative and positive infinity, but for reasons we will see soon, they generally fall between [latex]-3[/latex] and [latex]3[/latex].

Let’s look at some examples. A [latex]z[/latex]–score value of [latex]-1.0[/latex] tells us that this [latex]z[/latex]–score is [latex]1[/latex] standard deviation (because of the magnitude [latex]1.0[/latex]) below (because of the negative sign) the mean. Similarly, a [latex]z[/latex]–score value of [latex]1.0[/latex] tells us that this [latex]z[/latex]–score is [latex]1[/latex] standard deviation above the mean. Thus, these two scores are the same distance away from the mean but in opposite directions. A [latex]z[/latex]–score of [latex]-2.5[/latex] is two-and-a-half standard deviations below the mean and is therefore farther from the centre than both of the previous scores, and a [latex]z[/latex]–score of [latex]0.25[/latex] is closer than all of the ones before. For now, we will use a rough cut-off of [latex]1.5[/latex] standard deviations in either direction as the difference between close scores (those within [latex]1.5[/latex] standard deviations or between [latex]z=-1.5[/latex] and [latex]z=1.5[/latex]) and extreme scores (those farther than [latex]1.5[/latex] standard deviations — below [latex]z=-1.5[/latex] or above [latex]z=1.5[/latex]).

We can also convert raw scores into [latex]z[/latex]-scores to get a better idea of where in the distribution those scores fall. Let’s say we get a score of [latex]68[/latex] on an exam. We may be disappointed to have scored so low, but perhaps it was just a very hard exam. Having information about the distribution of all scores in the class would be helpful to put some perspective on ours. We find out that the class got an average score of [latex]54[/latex] with a standard deviation of [latex]8[/latex]. To find out our relative location within this distribution, we simply convert our test score into a [latex]z[/latex]-score.

[latex]z=\frac{X-\mu}{\sigma}=\frac{68-54}{8}=1.75[/latex]

We find that we are [latex]1.75[/latex] standard deviations above the average, above our rough cut-off for close and far. Suddenly our [latex]68[/latex] is looking pretty good!

Figure 6.4.3 shows both the raw score and the [latex]z[/latex]–score on their respective distributions. Notice that the red line indicating where each score lies is in the same relative spot for both. This is because transforming a raw score into a [latex]z[/latex]–score does not change its relative location, it only makes it easier to know precisely where it is.

[latex]z[/latex]–Scores are also useful for comparing scores from different distributions. Let’s say we take the SAT and score [latex]501[/latex] on both the math and critical reading sections. Does that mean we did equally well on both? Scores on the math portion are distributed normally with a mean of [latex]511[/latex] and standard deviation of [latex]120[/latex], so our [latex]z[/latex]–score on the math section is

[latex]z_\textrm{math}=\frac{501 - 511}{120}=-0.08[/latex]

which is just slightly below average (note the use of “math” as a subscript; subscripts are used when presenting multiple versions of the same statistic in order to know which one is which and have no bearing on the actual calculation). The critical reading section has a mean of [latex]495[/latex] and standard deviation of [latex]116[/latex], so

[latex]z_\textrm{CR}=\frac{501-495}{116}=0.05[/latex]

So even though we were almost exactly average on both tests, we did a little bit better on the critical reading portion relative to other people.

Finally, [latex]z[/latex]–scores are incredibly useful if we need to combine information from different measures that are on different scales. Let’s say we give a set of employees a series of tests on things like job knowledge, personality, and leadership. We may want to combine these into a single score we can use to rate employees for development or promotion, but look what happens when we take the average of raw scores from different scales, as shown in Table 6.4.1.

Because the job knowledge scores were so big and the scores were so similar, they overpowered the other scores and removed almost all variability in the average. However, if we standardize these scores into [latex]z[/latex]–scores, our averages retain more variability and it is easier to assess differences between employees, as shown in Table 6.4.2.

Setting the Scale of a Distribution

Another convenient characteristic of [latex]z[/latex]–scores is that they can be converted into any “scale” that we would like. Here, the term scale means how far apart the scores are (their spread) and where they are located (their central tendency). This can be very useful if we don’t want to work with negative numbers or if we have a specific range we would like to present. The formulas for transforming [latex]z[/latex] to [latex]x[/latex] are:

[latex]x=z\sigma+\mu[/latex] for a population and

[latex]x=zs+M[/latex] for a sample.

Notice that these are just simple rearrangements of the original formulas for calculating [latex]z[/latex] from raw scores.

Let’s say we create a new measure of intelligence, and initial calibration finds that our scores have a mean of [latex]40[/latex] and standard deviation of [latex]7[/latex]. Three people who have scores of [latex]52, 43[/latex], and [latex]34[/latex] want to know how well they did on the measure. We can convert their raw scores into [latex]z[/latex]–scores:

[latex]\begin{align*}z=\frac{52-40}{7}&=1.71\\[2ex]z=\frac{43-40}{7}&=0.43\\[2ex]z=\frac{34-40}{7}&=-0.80\end{align*}[/latex]

A problem is that these new [latex]z[/latex]–scores aren’t exactly intuitive for many people. We can give people information about their relative location in the distribution (for instance, the first person scored well above average), or we can translate these [latex]z[/latex]–scores into the more familiar metric of IQ scores, which have a mean of [latex]100[/latex] and standard deviation of [latex]16[/latex]:

[latex]\begin{array}{ccccccc}IQ&=&1.71\left(16\right)&+&100&=&127.6\\IQ&=&0.43\left(16\right)&+&100&=&106.88\\IQ&=&-0.80\left(16\right)&+&100&=&87.20\end{array}[/latex]

We would also likely round these values to [latex]127[/latex], [latex]107[/latex], and [latex]87[/latex], respectively, for convenience.

Z Scores and the Area under the Curve

[latex]z[/latex]–Scores and the standard normal distribution go hand-in-hand. A [latex]z[/latex]–score will tell you exactly where in the standard normal distribution a value is located, and any normal distribution can be converted into a standard normal distribution by converting all of the scores in the distribution into [latex]z[/latex]–scores, a process known as standardization.

We saw in the previous section, that standard deviations can be used to divide the normal distribution: [latex]68\%[/latex] of the distribution falls within [latex]1[/latex] standard deviation of the mean, [latex]95\%[/latex] within (roughly) [latex]2[/latex] standard deviations, and [latex]99.7\%[/latex] within [latex]3[/latex] standard deviations. Because [latex]z[/latex]–scores are in units of standard deviations, this means that [latex]68\%[/latex] of scores fall between [latex]z=-1.0[/latex] and [latex]z=1.0[/latex] and so on. We call this [latex]68\%[/latex] (or any percentage we have based on our [latex]z[/latex]-scores) the proportion of the area under the curve. Any area under the curve is bounded by (defined by, delineated by, etc.) by a single [latex]z[/latex]-score or pair of [latex]z[/latex]-scores.

An important property to point out here is that, by virtue of the fact that the total area under the curve of a distribution is always equal to [latex]1.0[/latex] (see section on Normal Distributions at the beginning of this section), these areas under the curve can be added together or subtracted from [latex]1[/latex] to find the proportion in other areas. For example, we know that the area between [latex]z=-1.0[/latex] and [latex]z=1.0[/latex] (i.e., within one standard deviation of the mean) contains [latex]68\%[/latex] of the area under the curve, which can be represented in decimal form as [latex]0.6800[/latex]. (To change a percentage to a decimal, simply move the decimal point [latex]2[/latex] places to the left.) Because the total area under the curve is equal to [latex]1.0[/latex], that means that the proportion of the area outside [latex]z=-1.0[/latex] and [latex]z=1.0[/latex] is equal to [latex]1.0-0.6800=0.3200[/latex] or [latex]32\%[/latex] (see Figure 6.4.4). This area is called the area in the tails of the distribution. Because this area is split between two tails and because the normal distribution is symmetrical, each tail has exactly one-half, or [latex]16\%[/latex], of the area under the curve.

We will have much more to say about this concept in the coming sections. As it turns out, this is a quite powerful idea that enables us to make statements about how likely an outcome is and what that means for research questions we would like to answer and hypotheses we would like to test. But first, we need to make a brief foray into some ideas about probability in the next section.