Computers (obviously) have to approximate the majority of actual mathematical numbers, as they do not have infinite storage.
If you've got two numbers, +0.0000001 and -0.0000001, but you can't represent that precision, can you see how it's less bad to round to +0.0000 and -0.0000 rather than to just 0.0000? It's encoding strictly more information.
Reading your comment gave me a realization that transformed my understanding of floats.
What just clicked for me was that in any system where we use finite precision to store exponents, we can't actually have zero as a normal value... we'll always underflow the exponent before we get to actual mathematical zero.
So +0/-0 is actually a convenient misnomer. They really are +/- epsilon.
The only way to have zero is if it's some specially handled value like NaN. Which IEEE doesn't do and that's entirely understandable.
Makes sense why you can never compare a subtraction of floats to zero (it's beyond just "rounding errors") and the existence of +0/-0 seems quite natural now.
> The only way to have zero is if it's some specially handled value like NaN. Which IEEE doesn't do and that's entirely understandable.
Wait what? Am I missing something? 0 is absolutely part of the IEEE 754 spec thanks to the existence of denormalized floating point numbers. So I would certainly call it a "specially handled value", in a sense. The existence of +/- 0 has more to do with the implementation of the leading sign bit.
Really good point. My approach to this was "if it’s not used in any mathematical algorithms, why do we need it in computers?". But in your example, you retain some information even though you can’t represent the whole truth. Thanks!
You can look these things up for yourself - they're standardised in most language's implementations in something called IEEE 754. In the cases you've asked about they're false and true. Is this what you want in all cases? No. Is this what you want in some cases? Yes. It's a tradeoff. You can still observe the difference by dividing by zero (which should be another indication that these aren't real numbers as we're conventionally understand them.)
> they're standardised in most language's implementations in something called IEEE 754
There is the IEEE 754, and there is the language's standard. One should always look at the latter because it's often the case that the language doesn't fully conform to IEEE 754.
You shouldn't really use equality on floating point numbers, except on very special circumstances (and I imagine the == behavior for 0 breaks things more often than it helps). But the wikipedia page on -0 has your case covered:
> According to the IEEE 754 standard, negative zero and positive zero should compare as equal with the usual (numerical) comparison operators, like the == operators of C and Java. In those languages, special programming tricks may be needed to distinguish the two values
> You shouldn't really use equality on floating point numbers, except on very special circumstances.
This is very common advice, so common that it gets cargo-culted into situations where it is really quite poor.
Information storage, retrieval, and transmission systems should faithfully deliver floating-point values that are good to the last bit. Round-trips through databases, transmission over network protocols, etc should all give values back that are exactly identical to what was put into them.
Oh, sure. But those applications should also not use the floating point equality operators. They deal with opaque data, and should make sure not to corrupt it.
Keep in mind that the ISO standard does require that floating point equality tests return false for values that have the exact same binary representation, and that not everything adheres to it and some environments may give you false for identical values even when the standard says it should be true. Also, != is not the negation of == for floating point. So even using those operators to test a round trip over those applications is iffy.
By "the last bit" do you mean the 32nd bit or the 64th bit? :-)
Many times I've tracked down the place in our stack where a double-precision value from user input accidentally goes through a single-precision variable in some C code somewhere and crashes some Python code later on because the values don't match "to the last bit" in the way that the programmer thought... But that's a bug in the C code - I agree completely the the system SHOULD give the value back that was put into it!
This is actually exactly what I mean. Its probably the most common bug I've come across in this class. I don't expect unit tests to capture all rounding bugs (say, due to serialization and de-serialization through text). But I do expect to capture gross errors, such as an inadvertent cast to lower-precision somewhere in the pipeline.
I've worked with highly experienced and accomplished software engineers that expected interchange through protobuf or sql to be inaccurate due to rounding. No! If you stick a finite number in, you should get the exact same finite number back out again. Direct equality is fine for most cases. The sign bit of zero and NaN should also be returned faithfully and tested using memcmp when required.
IMO, the payload bits of NaN should also be also returned faithfully, but too many systems in common practice drop them.
> Round-trips through databases, transmission over network protocols, etc should all give values back that are exactly identical to what was put into them.
Yes that's true... but what's that got to do with using an equality operator?
One should usually not rely on this, unless your language gives guarantees. I don't think even IEEE 754 gives you the guarantee you are implying.
To give you an idea, the C99 standard does not require signed zeros, and if you have them, does not dictate the behavior you are describing. I once worked on a commercial C compiler and we produced a new version that resulted in a change of sign of zero for the exact same computation (and even "worse", would give a different sign on different machines for the same compiler version). We discussed this thoroughly and decided it was OK because our docs made it clear we conform to C99 (which provides no guarantees on signed zero).
No, if x = -0.0 then x+0 = 0.0 and x*0 = -0.0. This violates fundamental tenets of arithmetic, surprises most, and makes many compiler optimizations impossible.
If you've got two numbers, +0.0000001 and -0.0000001, but you can't represent that precision, can you see how it's less bad to round to +0.0000 and -0.0000 rather than to just 0.0000? It's encoding strictly more information.