8.3 Outcomes and the Type I and Type II Errors
LEARNING OBJECTIVES
- Differentiate between Type I and Type II errors in a hypothesis test.
When we perform a hypothesis test, there are four possible outcomes depending on the actual truth (or falseness) of the null hypothesis [latex]H_0[/latex] and the decision to reject or not the null hypothesis. Ideally, the hypothesis test should tell us to not reject the null hypothesis when the null hypothesis is true and reject the null hypothesis when the null hypothesis is false. However, the outcome of the hypothesis test is based on sample information and probabilities, so there is a chance that the hypothesis test does not correctly identify the truth or falseness of the null hypothesis. The outcomes are summarized in the following table:
| Actual Truth State of the Null Hypothesis |
||
| Outcome of Test | [latex]H_0[/latex] is True | [latex]H_0[/latex] is False |
| Do not reject [latex]H_0[/latex] | Correct Outcome | Type II Error |
| Reject [latex]H_0[/latex] | Type I Error | Correct Outcome |
The four possible outcomes in the table are:
- The decision is not to reject [latex]H_0[/latex] when [latex]H_0[/latex] is true (correct decision). That is, the test identifies [latex]H_0[/latex] is true and in reality [latex]H_0[/latex] is true, which means the test correctly identified [latex]H_0[/latex] as true.
- The decision is to reject [latex]H_0[/latex] when [latex]H_0[/latex] is true (incorrect decision known as a Type I error). That is, the test identifies [latex]H_0[/latex] as false but in reality [latex]H_0[/latex] is true, which means the test did not correctly identify [latex]H_0[/latex] as true.
- The decision is not to reject [latex]H_0[/latex] when [latex]H_0[/latex] is false (incorrect decision known as a Type II error). That is, the test identifies [latex]H_0[/latex] is true but in reality [latex]H_0[/latex] is false, which means the test did not correctly identify [latex]H_0[/latex] as false.
- The decision is to reject [latex]H_0[/latex] when [latex]H_0[/latex] is false (correct decision whose probability is called the Power of the Test). That is, the test identifies [latex]H_0[/latex] is false and in reality [latex]H_0[/latex] is false, which means the test correctly identified [latex]H_0[/latex] as false.
There are two types of error that can occur in hypothesis testing. Each of the errors occurs with a particular probability.
- A Type I error occurs when the null hypothesis is rejected by the test (i.e. the test identifies the null hypothesis as false) but in reality the null hypothesis is true. The probability of a Type I error is denoted by [latex]\alpha[/latex].
- A Type II error occurs when the null hypothesis is not rejected by the test (i.e. the test identifies the null hypothesis as true) but in reality the null hypothesis is false. The probability of a Type II error is denoted by [latex]\beta[/latex].
Although the probabilities of a Type I or Type II error should be as small as possible, because they are probabilities of errors, they are rarely zero.
EXAMPLE
Suppose the null hypothesis is
[latex]\displaystyle{H_0: \mbox{Frank's rock climbing equipement is safe.}}[/latex]
- Type I error: Frank thinks his rock climbing equipment is not safe when in fact the equipment is safe.
- Frank believes [latex]H_0[/latex] is false but [latex]H_0[/latex] is actually true.
- Type II error: Frank thinks his rock climbing equipment is safe when in fact the equipment is not safe.
- Frank believes [latex]H_0[/latex] is true but [latex]H_0[/latex] is actually false.
Note that, in this case, the error with the greater consequence is the Type II error. If Frank thinks his rock climbing equipment is safe and it actually is not safe, he will go ahead and use it.
TRY IT
Suppose the null hypothesis is
[latex]\displaystyle{H_0: \mbox{The blood cultures contain no traces of pathogen }X.}[/latex]
State the Type I and Type II errors.
Click to see Solution
- Type I error: The researcher thinks the blood cultures do contain traces of pathogen [latex]X,[/latex] when in fact, they do not.
- Type II error: The researcher thinks the blood cultures do not contain traces of pathogen [latex]X[/latex], when in fact, they do.
EXAMPLE
Suppose the null hypothesis is
[latex]\displaystyle{H_0: \mbox{The victim of a car accident is alive when they arrive at the ER.}}[/latex]
- Type I error: The ER staff thinks that the victim is dead when in fact the victim is alive.
- Type II error: The ER staff think the victim is alive when in fact the victim is dead.
Note that, in this case, the error with the greater consequence is the Type I error. If the ER staff think the victim is dead, then they will not treat him.
TRY IT
Suppose the null hypothesis is
[latex]\displaystyle{H_0: \mbox{A patient is not sick.}}[/latex]
Which type of error has the greater consequence, Type I or Type II? Why?
Click to see Solution
The error with the greater consequence is the Type II error: the patient will be thought well when, in fact, they are sick, and so they will not get treatment.
EXAMPLE
A genetics lab claims its product can increase the likelihood a pregnancy will result in a boy being born. Statisticians want to test this claim. Suppose that the null hypothesis is
[latex]\displaystyle{H_0: \mbox{The genetics lab product has no effect on gender outcome.}}[/latex]
- Type I error: We believe the genetics lab’s product can influence gender outcome when in fact the product has no effect.
- Type II error: We believe the genetics lab’s product cannot influence gender outcome when in fact the product does have an effect.
Note that, in this case, the error with the greater consequence is the Type I error because couples would use the product in hopes of increasing the chances of having a boy.
TRY IT
“Red tide” is a bloom of poison-producing algae—a few different species of a class of plankton called dinoflagellates. When the weather and water conditions cause these blooms, shellfish such as clams living in the area develop dangerous levels of a paralysis-inducing toxin. In Massachusetts, the Division of Marine Fisheries (DMF) monitors levels of the toxin in shellfish by regularly sampling shellfish along the coastline. If the mean level of toxin in clams exceeds 800 μg (micrograms) of toxin per kg of clam meat in any area, clam harvesting is banned there until the bloom is over and levels of toxin in clams subside. Describe both a Type I and a Type II error in this context, and state which error has the greater consequence.
Click to see Solution
In this scenario, an appropriate null hypothesis would be
[latex]\displaystyle{H_0: \mbox{The mean level of toxins is at most 800 μg.}}[/latex]
- Type I error: The DMF believes that toxin levels are still too high when, in fact, toxin levels are at most 800 μg. The DMF continues the harvesting ban.
- Type II error: The DMF believes that toxin levels are within acceptable levels (are at most 800 μg) when, in fact, toxin levels are still too high (more than 800 μg). The DMF lifts the harvesting ban. This error could be the most serious. If the ban is lifted and clams are still toxic, consumers could possibly eat tainted food.
In summary, the more dangerous error would be to commit a Type II error, because this error involves the availability of tainted clams for consumption.
EXAMPLE
A certain experimental drug claims a cure rate of at least 75% for males with prostate cancer. Describe both the Type I and Type II errors in context. Which error is more serious?
- Type I: A cancer patient believes the cure rate for the drug is less than 75% when it actually is at least 75%.
- Type II: A cancer patient believes the experimental drug has at least a 75% cure rate when it has a cure rate that is less than 75%.
In this scenario, the Type II error contains the more severe consequence. If a patient believes the drug works at least 75% of the time, this will most likely influence the patient’s (and doctor’s) choice about whether to use the drug as a treatment option.
Watch this video: Type 1 errors | Inferential statistics | Probability and Statistics | Khan Academy by Khan Academy [3:23]
Concept Review
In every hypothesis test, the outcomes from the test are dependent on sample data and probabilities, which means that the conclusion of the test may not correctly identify the actual truth state of the null hypothesis. Such occurrences are expected. A Type I error occurs when a true null hypothesis is rejected. A Type II error occurs when a false null hypothesis is not rejected.
Attribution
“9.2 Outcomes and the Type I and Type II Errors“ in Introductory Statistics by OpenStax is licensed under a Creative Commons Attribution 4.0 International License.
