When to use rules of significant digits vs when to avoid dilution of precision

436 views

I’m confused as to how, when and why I’m supposed to use significant digits vs when to carry all the decimals my calculator spits out. This is for surveying/civil engineering related fields in particular. My professor has covered both topics extensively but didn’t do a very good job explaining when said rules apply.

In: 0

15 Answers

Anonymous 0 Comments

Let’s say I want the sqrt(2.00)^2

I know the answer should be 2.00, but if I round too soon I get an issue

Sqrt(2.00) = 1.41

1.41^2 = 1.99

1.99 is not 2.00, so clearly something went wrong. I rounded too soon.

Always round once you’ve finished calculating, but if you need to calculate further, you need to use the exact value you calculated.

Sqrt(2.00) is indeed 1.41 and 1.41^2 is indeed 1.99, but sqrt(2.00)^2 = 2.00

You are viewing 1 out of 15 answers, click here to view all answers.