Module 5: Accuracy and Significant Figures

sboschman

11 Module 5: Accuracy and Significant Figures — Technical Mathematics

<!– pb_fixme –>

In the first few modules, we rarely concerned ourselves with rounding; we assumed that every number we were told was exact and we didn’t have to worry about any measurement error. However, every measurement contains some error. A standard sheet of paper is $8.5$ inches wide and $11$ inches high, but it’s possible that the actual measurements could be closer to $8.4999$ and $11.0001$ inches. Even if we measure something very carefully, with very sensitive instruments, we should assume that there could be some small measurement error.

Exact Values and Approximations

A number is an exact value if it is the result of counting or a definition.

A number is an approximation if it is the result of a measurement or of rounding.

Exercises

Identify each number as an exact value or an approximation.

1. An inch is $\frac{1}{12}$ of a foot.

2. This board is $78$ inches long.

3. There are $14$ students in class.

4. A car’s tachometer reads $3,000$ rpm.

5. A right angle measures $90$ °.

6. The angle of elevation of a ramp is $4$ °.

Suppose a co-worker texts you that they will arrive in $20$ minutes. It’s hard to tell how precise this number is, because we often round off to the nearest $5$ or $10$ minutes. You might reasonably expect them to arrive anytime within the next $15$ to $25$ minutes. If your co-worker texts that they will arrive in $17$ minutes, though, it is likely that their GPS told them that more precise number, and you could reasonably expect them to arrive within $16$ to $18$ minutes.

Accuracy and Significant Figures

African-American women were vital to NASA’s success in the 1960s, as shown in the movie Hidden Figures.

Because measurements are inexact, we need to consider how accurate they are. This requires us to think about significant figures—often abbreviated “sig figs” in conversation—which are the digits in the measurement that we trust to be correct. The accuracy of a number is equal to the number of significant figures.^[1] The following rules aren’t particularly difficult to understand but they can take time to absorb and internalize, so we’ll include lots of examples and exercises.

Significant Figures

All nonzero digits are significant.
Ex: $12,345$ has five sig figs, and $123.45$ has five sig figs.
All zeros between other nonzero digits are significant.
Ex: $10,045$ has five sig figs, and $100.45$ has five sig figs.
Any zeros to the right of a decimal number are significant.
Ex: $123$ has three sig figs, but $123.00$ has five sig figs.
Zeros on the left of a decimal number are NOT significant.
Ex: $0.123$ has three sig figs, and $0.00123$ has three sig figs.
Zeros on the right of a whole number are NOT significant unless they are marked with an overbar.
Ex: $12,300$ has three sig figs, but $12,30\overline{0}$ has five sig figs.

Another way to think about #4 and #5 above is that zeros that are merely showing the place value—where the decimal point belongs—are NOT significant.

Exercises

Determine the accuracy (i.e., the number of significant figures) of each number.

7.
$63,400$

8.
$63,040$

9.
$63,004$

10.
$0.085$

11.
$0.0805$

12.
$0.08050$

As mentioned above, we use an overbar to indicate when a zero that looks insignificant is actually significant.

For example, the precision^[2] of $7,400$ is the hundreds place; if we rounded anything from $7,350$ to $7,449$ to the nearest hundred, we would write the result as $7,400$ . An overbar shows that the number is more precise than it appears. If we rounded anything from $7,395$ to $7,404$ to the nearest ten, the result would be $7,400$ , but it isn’t clear anymore that the number was rounded to the tens place. Therefore, to show the level of precision, we write the result as $7,4\overline{0}0$ . If we rounded anything from $7,399.5$ to $7,400.4$ to the nearest one, the result would be again $7,400$ , and we again can’t see how precise the rounded number really is. Therefore, to show that the number is precise to the ones place, we write the result as $7,40\overline{0}$ .

Exercises

Determine the accuracy (i.e., the number of significant figures) of each number.

13.
$8,000$

14.
$8,\overline{0}00$

15.
$8,0\overline{0}0$

16.
$8,00\overline{0}$

Two things to remember: we don’t put an overbar over a nonzero digit, and we don’t need an overbar for any zeros on the right of a decimal number because those are already understood to be significant.

Accuracy-Based Rounding

As we saw in a previous module about decimals, it is often necessary to round a number. We often round to a certain place value, such as the nearest hundredth, but there is another way to round. Accuracy-based rounding considers the number of significant figures rather than the place value.

Accuracy-based rounding:

Locate the rounding digit to which you are rounding by counting from the left until you have the correct number of significant figures.
Look at the test digit directly to the right of the rounding digit.
If the test digit is 5 or greater, increase the rounding digit by 1 and drop all digits to its right. If the test digit is less than 5, keep the rounding digit the same and drop all digits to its right.

Exercises

Round each number so that it has the indicated number of significant figures.

17. $21,837$ (two sig figs)

18. $21,837$ (three sig figs)

19. $21,837$ (four sig figs)

20. $4.2782$ (two sig figs)

21. $4.2782$ (three sig figs)

22. $4.2782$ (four sig figs)

When the rounding digit of a whole number is a $9$ that gets rounded up to a $0$ , we must write an overbar above that $0$ .

Similarly, when the rounding digit of a decimal number is a $9$ that gets rounded up to a $0$ , we must include the $0$ in that decimal place.

Exercises

Round each number so that it has the indicated number of significant figures. Be sure to include trailing zeros or an overbar if necessary.

23. $13,997$ (two sig figs)

24. $13,997$ (three sig figs)

25. $13,997$ (four sig figs)

26. $2.5996$ (two sig figs)

27. $2.5996$ (three sig figs)

28. $2.5996$ (four sig figs)

Mt. Everest, Lohtse, and Nupse in the early morning

The mountain we know as Mt. Everest is called Sagarmatha in Nepal and Chomolungma in Tibet. On December 8, 2020, it was jointly announced by Nepal and China that the summit has an elevation of $29,031.69$ ft, replacing the previously accepted elevation of $29,029$ ft.^[3]

29. Round $29,031.69$ ft to two sig figs.

30. Round $29,031.69$ ft to three sig figs.

31. Round $29,031.69$ ft to four sig figs.

32. Round $29,031.69$ ft to five sig figs.

33. Round $29,031.69$ ft to six sig figs.

Accuracy when Multiplying and Dividing

Suppose you needed to square the number $3\frac{1}{3}$ . You could rewrite $3\frac{1}{3}$ as the improper fraction $\frac{10}{3}$ and then figure out that $(\frac{10}{3})^2 = \frac{100}{9}$ , which equals the repeating decimal $11.111...$

Because most people prefer decimals to fractions, we might instead round $3\frac{1}{3}$ to $3.3$ and find that $3.3^2=10.89$ . However, this is not accurate because $11.111...$ rounded to the nearest hundredth should be $11.11$ . The answer $10.89$ looks very accurate, but it is a false accuracy because there is round-off error involved. If we round our answer $10.89$ to the nearest tenth, we would get $10.9$ , which is still not accurate because $11.111...$ rounded to the nearest tenth should be $11.1$ . If we round our answer $10.89$ to the nearest whole number, we would get $11$ , which is accurate because $11.111...$ rounded to the nearest whole number is indeed $11$ . It turns out that we should be focusing on the number of significant figures rather than the place value; because $3.3$ has only two sig figs, our answer must be rounded to two sig figs.

Suppose instead that we round $3\frac{1}{3}$ to $3.33$ and find that $3.33^2=11.0889$ . Again, this is not accurate because $11.111...$ rounded to the nearest ten-thousandth should be $11.1111$ . If we round $11.0889$ to the nearest thousandth, we would get $11.089$ , which is still not accurate because $11.111...$ rounded to the nearest thousandth should be $11.111$ . If we round $11.0889$ to the nearest hundredth, we would get $11.09$ , which is still not accurate because $11.111...$ rounded to the nearest hundredth should be $11.11$ . Only when we round to the nearest tenth do we get an accurate result: $11.0889$ rounded to the nearest tenth is $11.1$ , which is accurate because $11.111...$ rounded to the nearest tenth is indeed $11.1$ . As above, we need to focus on the number of significant figures rather than the place value; because $3.33$ has only three sig figs, our answer must be rounded to three sig figs.

When multiplying or dividing numbers, the answer must be rounded to the same number of significant figures as the least accurate of the original numbers.

Don’t round off the original numbers; do the necessary calculations first, then round the answer as your last step.

Exercises

Use a calculator to multiply or divide as indicated. Then round to the appropriate level of accuracy.

34.
$8.75\cdot12.25$

35.
$355.12\cdot1.8$

36.
$77.3\div5.375$

37.
$53.2\div4.5$

38. Suppose you are filling a 5-gallon can of gasoline. The gasoline costs $2.579 per gallon, and you estimate that you will buy $5.0$ gallons. How much should you expect to spend?

The terms “significant digits” and “significant figures” are used interchangeably. ↵
Precision is different from accuracy, as we’ll learn in the next module, but it is mentioned here because it can be difficult to explain one without the other. ↵
https://www.washingtonpost.com/world/asia_pacific/mount-everest-height-nepal-china/2020/12/08/a7b3ad1e-389a-11eb-aad9-8959227280c4_story.html
↵

<!– pb_fixme –>

<!– pb_fixme –>
<!– pb_fixme –>

11 Module 5: Accuracy and Significant Figures — Technical Mathematics