When I type e.g. **163.4 – 155** into my phone’s calculator, it returns: = **8.40000000000000*****6***. I think that after 12 years of school math, I feel entitled to say, mathematically this is wrong.
It mostly does that, (A) if the number of digits before and after the comma/point of minuend and subtrahend are equal, (B) only in subtractions and (C) only if there are decimal places. So:
|Case|Calculation|Result|Correct|
|:-|:-|:-|:-|
|-|163.4 – 155|8.40000000000000*6*|No?|
|-|0.4 – 0.3|0.10000000000000*3*|No?|
|A|1633.4 – 1555|78.4000000000000*9*|No?|
|A|1633.4 – 155|1478.4|Yes.|
|A|0.4 – 0.33|0.07|Yes.|
|A|10.4 – 0.33|10.07|Yes.|
|A?!|1163.4 – 155|1008.400000000000*1*|No?|
|B|155.4 +163|318.4|Yes.|
|C|163 – 155|8.0|Yes.|
|C|1634 – 1550|84.0|Yes.|
Additional observations: If there are decimal zeros added, they always seem to fill the decimal places, so that there are 16 digits in total. Whether the decimal places are in the minuend or in the subtrahend, does not matter. If the decimal place of minuend and/or subtrahend is .0 or if they are equal (e.g. .4 in both), this does not occur. Whether the minuend is greater than the subrahend or vice versa, does not matter. When the issue occurs, the very last digit seems random…
I know of other “software” calculators that do or have done this: **DuckDuckGo** seems to have had this “problem”, too: [See this picture](https://i.redd.it/5b2tolap5c951.png). This seems to be fixed now.
Is this just bad programming or is there something mathematical to this? Also, if it is a technical issue, why? (It’s probably still a mathematical reason?) It reoccurs over different platforms and does “too much”, but is certainly not programmed on purpose…
In: Mathematics
## ELI5:
Computers speak exclusively in binary. And converting a base 10 number to binary isn’t always clean. Its somewhat analogous to how you can cleanly represent 1/3 as a fraction, but as a decimal number, you’d need an infinite number of digits to represent it (as remember, it is 0.33333… repeating forever!). The same *kind of thing* happens when converting a decimal in base 10, to a binary number. The computer doesn’t have infinite memory though, so it can only store so many binary digits. Because of this fact, its really only approximating the number you input, So when it converts the result back to a base 10 decimal number, the result can sometimes be weirdly approximate. This won’t happen with integers though, since those transfer cleanly to binary (as you noticed).
As for why when it does the weird approximation it always fills 16 digits, that is because your computer is using a data format known as a “double precision floating point number” which is only precise out to 16 digits. Because of this fact, when it is approximating and thus slightly wrong, only that 16th decimal place will be wrong. Everything else will be correct.
***
## Slightly longer answer
Its not really “bad programming”, but rather how computers deal with numbers.
Remember, computers have to store numbers in memory, meaning they only have finite precision with which they can do arithmetic. To store numbers, there are a variety of different formats, however to maintain flexibility (useful in a calculator where you could put in any number you want), two typical standards are the floating point number, and the double precision floating point number. Often times, people just refer to these as “float” and “double” respectively.
[Floating point](https://en.wikipedia.org/wiki/Floating-point_arithmetic) numbers allow you to store *most* numbers between -3.4E+38 to +3.4E+38, to within 7 decimal places using only 32 bits of memory (the fact that its *most* is something I’ll come back to in a second). A double extends this to -1.7E+308 to +1.7E+308, to within 16 decimal places, using only 64 bits of memory! That’s quite a remarkable feat considering how large those numbers are. But now notice your observation. This weird error always occurs with 16 decimal places present, which is exactly how many digits a double precision number can store!
So what you’re running up against is the precision limit of a double precision number. Without getting into the math, you can kind of think of it as similar to how you can’t write out 1/3 in decimal, as it would never end. You’d need INFINITE precision to write out 0.333333…. forever. Similarly, when you convert some decimal numbers into binary, they cannot be represented exactly without infinite precision, and thus infinite memory.
So in your first example, 163.4 cannot be represented perfectly by a double precision floating point number, because when its converted to binary it requires MORE than 64 bits to represent. So the computer has no choice but to approximate it, and so you’ll get strange rounding errors. And those rounding errors will show up in your least significant digit (meaning, it will be in that 16th place).
There are some clever ways around this (as a quick example, you could note that when doing addition, the result will never have MORE decimal places than the maximum number of decimal places in your input. Then you can just round the output of your floating point math to whatever decimal place that is). But that takes a lot of effort for really no gain. For a cell phone calculator, noone really cares if the answer is off by such a small amount!
Anyways, I hope this helped… Let me know if I need to clarify anything!
This generally has to do with how computers represent non-integers (i.e., reals). While in mathematics 1/2 = 0.5000… with infinite precision, computer memory (which is how values are ultimately stored/represented in the computer) is finite. Correspondingly, the principal representation of reals in computers is using the common representational scheme of “floating-point numbers,” most notably the de facto standard IEEE 754. A consequence of this representation is the “errors”/artifacts you have observed in the indicated calculations, with seemingly arbitrary values in the last decimal place (a consequence of the limited-memory representation of the resulting computation).
Latest Answers