8.3 Outcomes and the Type I and Type II Errors
LEARNING OBJECTIVES
- Differentiate between Type I and Type II errors in a hypothesis test.
When we perform a hypothesis test, there are four possible outcomes depending on the actual truth (or falseness) of the null hypothesis [latex]H_0[/latex] and the decision to reject or not the null hypothesis. Ideally, the hypothesis test should tell us to not reject the null hypothesis when the null hypothesis is true and reject the null hypothesis when the null hypothesis is false. However, the outcome of the hypothesis test is based on sample information and probabilities, so there is a chance that the hypothesis test does not correctly identify the truth or falseness of the null hypothesis. The outcomes are summarized in the following table:
| Outcome of Test | Actual Truth State of the Null Hypothesis |
|
|---|---|---|
| [latex]H_0[/latex] is True | [latex]H_0[/latex] is False | |
| Do not reject [latex]H_0[/latex] | Correct Outcome | Type II Error |
| Reject [latex]H_0[/latex] | Type I Error | Correct Outcome |
The four possible outcomes in the table are:
- The decision is to not reject [latex]H_0[/latex] when [latex]H_0[/latex] is true (correct decision). That is, the test identifies [latex]H_0[/latex] is true, and in reality, [latex]H_0[/latex] is true, which means the test correctly identified [latex]H_0[/latex] as true.
- The decision is to reject [latex]H_0[/latex] when [latex]H_0[/latex] is true (incorrect decision known as a Type I error). That is, the test identifies [latex]H_0[/latex] as false, but in reality, [latex]H_0[/latex] is true, which means the test did not correctly identify [latex]H_0[/latex] as true.
- The decision is to not reject [latex]H_0[/latex] when [latex]H_0[/latex] is false (incorrect decision known as a Type II error). That is, the test identifies [latex]H_0[/latex] is true, but in reality, [latex]H_0[/latex] is false, which means the test did not correctly identify [latex]H_0[/latex] as false.
- The decision is to reject [latex]H_0[/latex] when [latex]H_0[/latex] is false (correct decision whose probability is called the Power of the Test). That is, the test identifies [latex]H_0[/latex] is false, and in reality [latex]H_0[/latex] is false, which means the test correctly identified [latex]H_0[/latex] as false.
There are two types of error that can occur in hypothesis testing. Each of the errors occurs with a particular probability.
- A Type I error occurs when the null hypothesis is rejected by the test (i.e. the test identifies the null hypothesis as false), but in reality, the null hypothesis is true. The probability of a Type I error is the significance level [latex]\alpha[/latex].
- A Type II error occurs when the null hypothesis is not rejected by the test (i.e. the test identifies the null hypothesis as true), but in reality, the null hypothesis is false. The probability of a Type II error is denoted by [latex]\beta[/latex].
Although the probabilities of a Type I or Type II error should be as small as possible because they are probabilities of errors, they are rarely zero.
EXAMPLE
Suppose the null hypothesis is
[latex]\displaystyle{H_0: \mbox{Frank's rock climbing equipement is safe.}}[/latex]
- Type I error: Frank thinks his rock climbing equipment is not safe when, in fact, the equipment is safe.
- Frank believes [latex]H_0[/latex] is false, but [latex]H_0[/latex] is actually true.
- Type II error: Frank thinks his rock climbing equipment is safe when, in fact, the equipment is not safe.
- Frank believes [latex]H_0[/latex] is true, but [latex]H_0[/latex] is actually false.
Note that, in this case, the error with the greater consequence is the Type II error. If Frank thinks his rock climbing equipment is safe and it actually is not safe, he will go ahead and use it.
TRY IT
Suppose the null hypothesis is
[latex]\displaystyle{H_0: \mbox{The blood cultures contain no traces of pathogen }X.}[/latex]
State the Type I and Type II errors.
Click to see Solution
- Type I error: The researcher thinks the blood cultures do contain traces of pathogen [latex]X[/latex], when, in fact, they do not.
- Type II error: The researcher thinks the blood cultures do not contain traces of pathogen [latex]X[/latex], when in fact, they do.
EXAMPLE
Suppose the null hypothesis is
[latex]\displaystyle{H_0: \mbox{The victim of a car accident is alive when they arrive at the ER.}}[/latex]
- Type I error: The ER staff thinks that the victim is dead when, in fact, the victim is alive.
- Type II error: The ER staff think the victim is alive when, in fact, the victim is dead.
Note that, in this case, the error with the greater consequence is the Type I error. If the ER staff think the victim is dead, then they will not treat them.
TRY IT
Suppose the null hypothesis is
[latex]\displaystyle{H_0: \mbox{A patient is not sick.}}[/latex]
Which type of error has the greater consequence, Type I or Type II? Why?
Click to see Solution
The error with the greater consequence is the Type II error: the patient will be thought well when, in fact, they are sick, and so they will not get treatment.
EXAMPLE
A genetics lab claims its product can increase the likelihood a pregnancy will result in a boy being born. Statisticians want to test this claim. Suppose that the null hypothesis is
[latex]\displaystyle{H_0: \mbox{The genetics lab product has no effect on gender outcome.}}[/latex]
- Type I error: We believe the genetics lab’s product can influence gender outcome when, in fact, the product has no effect.
- Type II error: We believe the genetics lab’s product cannot influence gender outcome when, in fact, the product does have an effect.
Note that, in this case, the error with the greater consequence is the Type I error because couples would use the product in hopes of increasing the chances of having a boy.
TRY IT
“Red tide” is a bloom of poison-producing algae—a few different species of a class of plankton called dinoflagellates. When the weather and water conditions cause these blooms, shellfish such as clams living in the area develop dangerous levels of a paralysis-inducing toxin. In Massachusetts, the Division of Marine Fisheries (DMF) monitors levels of the toxin in shellfish by regularly sampling shellfish along the coastline. If the mean level of toxin in clams exceeds [latex]800[/latex] μg (micrograms) of toxin per kg of clam meat in any area, clam harvesting is banned there until the bloom is over and levels of toxin in clams subside. Describe both a Type I and a Type II error in this context and state which error has the greater consequence.
Click to see Solution
In this scenario, an appropriate null hypothesis would be
[latex]\displaystyle{H_0: \mbox{The mean level of toxins is at most } 800\text{ μg.}}[/latex]
- Type I error: The DMF believes that toxin levels are still too high when, in fact, toxin levels are at most [latex]800[/latex] μg. The DMF continues the harvesting ban.
- Type II error: The DMF believes that toxin levels are within acceptable levels (are at most [latex]800[/latex] μg) when, in fact, toxin levels are still too high (more than [latex]800[/latex] μg). The DMF lifts the harvesting ban. This error could be the most serious. If the ban is lifted and clams are still toxic, consumers could possibly eat tainted food.
In summary, the more dangerous error would be to commit a Type II error because this error involves the availability of tainted clams for consumption.
EXAMPLE
A certain experimental drug claims a cure rate of at least [latex]75\%[/latex] for males with prostate cancer. Describe both the Type I and Type II errors in context. Which error is more serious?
- Type I: A cancer patient believes the cure rate for the drug is less than [latex]75\%[/latex] when it actually is at least [latex]75\%[/latex].
- Type II: A cancer patient believes the experimental drug has at least a [latex]75\%[/latex] cure rate when it has a cure rate that is less than [latex]75\%[/latex].
In this scenario, the Type II error contains the more severe consequence. If a patient believes the drug works at least [latex]75\%[/latex] of the time, this will most likely influence the patient’s (and doctor’s) choice about whether to use the drug as a treatment option.
Video: “Type 1 errors | Inferential statistics | Probability and Statistics | Khan Academy” by Khan Academy [3:24] is licensed under the Standard YouTube License.Transcript and closed captions available on YouTube.
Exercises
- The mean price of mid-sized cars in a region is [latex]\$32,000[/latex]. A test is conducted to see if the claim is true. State the Type I and Type II errors in complete sentences.
Click to see Answer
- Type I Error: The mean price of mid-sized cars is believed to not be [latex]\$32,000[/latex] when in fact, the mean price actually is [latex]\$32,000[/latex].
- Type II Error: The mean price of mid-sized cars is believed to be [latex]\$32,000[/latex] when in fact, the mean price is not [latex]\$32,000[/latex].
- A sleeping bag is tested to withstand temperatures of [latex]–30[/latex]°C. You think the bag cannot withstand temperatures that low. State the Type I and Type II errors in complete sentences.
Click to see Answer
- Type I Error: The sleeping bag is believed to not withstand temperatures of [latex]–30[/latex]°C when in fact, it does.
- Type II Error: The sleeping bag is believed to be able to withstand temperatures of [latex]–30[/latex]°C when, in fact, it cannot.
- A group of doctors is deciding whether or not to perform an operation. Suppose the null hypothesis is the surgical procedure will go well.
- State the Type I and Type II errors in complete sentences.
- Which error has the greater consequence?
Click to see Answer
-
- Type I Error: The doctors believe the surgical procedure will not go well when, in fact, it will.
- Type II Error: The doctors believe the surgical procedure will go well when, in fact, it will not.
- Type I because if the doctors believe the surgical procedure will not go well, they will not perform the operation and the patient will not get the necessary treatment.
- A group of divers is exploring an old sunken ship. Suppose the null hypothesis is the sunken ship does not contain buried treasure. State the Type I and Type II errors in complete sentences.
Click to see Answer
- Type I Error: The divers believe the ship contains buried treasure when, in fact, it does not.
- Type II Error: The divers believed the ship does not contain buried treasure when, in fact, it does.
- A microbiologist is testing a water sample for E-coli. Suppose the null hypothesis is: the sample contains E-coli. Which is the error with the greater consequence?
Click to see Answer
Type I because if the microbiologist believes the sample does not contain E-coli when in fact, it does, the public will not be advised to stop drinking the water.
- When a new drug is created, the pharmaceutical company must subject it to testing before receiving the necessary permission from the government authorities to market the drug. Suppose the null hypothesis is “the drug is unsafe.” What is the Type II error?
Click to see Answer
The drug is believed to be unsafe when, in fact, the drug is safe.
- A statistics instructor believes that fewer than [latex]20\%[/latex] of Evergreen Valley College (EVC) students attended the opening midnight showing of the latest Harry Potter movie. She surveys [latex]84[/latex] of her students and finds that [latex]11[/latex] of them attended the midnight showing. What is the Type I error?
Click to see Answer
The instructor believes more than [latex]20\%[/latex] of students attended the midnight showing when in fact, less than [latex]20\%[/latex] did.
- Previously, an organization reported that teenagers spent [latex]4.5[/latex] hours per week, on average, on the phone. The organization thinks that, currently, the mean is higher. Fifteen randomly chosen teenagers were asked how many hours per week they spend on the phone. The sample mean was [latex]4.75[/latex] hours with a sample standard deviation of [latex]2.0[/latex]. What is the Type I error?
Click to see Answer
The organization believes that the mean amount of time teenagers spend on the phone is greater than [latex]4.5[/latex] hours per week when, in fact, the average is [latex]4.5[/latex] hours per week.
“8.3 Outcomes and the Type I and Type II Errors” and “8.9 Exercises” from Introduction to Statistics by Valerie Watts is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License, except where otherwise noted.
