Somebody posted this anecdote a few days/weeks back, and it stuck with me:
> There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, “I don’t see the use of this; let us clear it away.” To which the more intelligent type of reformer will do well to answer: “If you don’t see the use of it, I certainly won’t let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it.”
[0] for example iirc cray had a wonky multiplier, don't remember if it was 754, that (I guess) they thought made it faster but resulted in noncommutative multiplication for many cases.
We're likely coming back into a period of divergence with ARM vs x86 parts on edge case handling that aren't strictly stated as MUST in the standard (various things related to qNaNs, sNaNs, denormal handling, +-0) but they're minor compared to the "old days".
They are probably still used in various older communication equipments.
Using floating-point had the purpose to reduce the quantization noise in comparison to 8-bit fixed-point numbers to the level corresponding to 12-bit or 13-bit fixed-point numbers.
So this was effectively a method of compressing by 50% the bit rate of voice signals.
Later, much better audio compression algorithms have been developed, allowing e.g. a 10 times compression, and such algorithms are used in the modern mobile phones.
Nevertheless, 8-bit floating-point was used for many decades in telephony.
Some complaints should be tempered, some others should be flattened into a bare acknowledgement that floating point is simply the wrong tool for the job, e.g. currency.
Maybe one "complaint" that remains is that floating point is too good and displaces progress in development and support for other number formats that are needed in their neiche like bfloat or fixed point.
IBM implemented IEEE 854 Radix-10 floating point (which later got subsumed back into IEEE 754) back in the System z9 days.
In fact, testing currency handling is the only thing that Bitcoin is actually useful for, LOL. If you take Bitcoin, you get smacked with a lot of fractional digits that break your currency system right away if you didn't do it right.
I'm not a mathematician but perhaps one could confirm, this is wrong, no? Irrationals e.g. Pi, sqrt2, cannot be represented. The floating point numbers are like a finite number of teeth on a comb pressed against the continuous real line.
I am familiar with most of the floating-point number formats that have ever been used in computers and there is no doubt that as a general-purpose numeric format all the previous floating-point alternatives have been vastly inferior to the IEEE formats.
For special-purpose niches, usually when a low precision is good enough, it is indeed possible to use some other numeric formats suitable for approximate numbers, e.g. logarithmic numbers, fixed-point numbers, low-precision FP numbers, unums/posits and a few others, which may have certain advantages over the IEEE FP numbers. e.g. a lower cost or higher speed for a given (low) precision, but even for those niches using IEEE FP numbers is usually a decent alternative, not a vastly inferior one.
A book on posit arithmetic is in the works.
A != A if A is a NaN: that's pretty sleazy.
I'm pretty sure mathematicians can come up with many different not-numbers that have to share the same label, with a rather slim chance of being equal ...
The underlying problem is that a system of calculation which propagates error symbols up the expression tree to indicate error is mixed with a system of Boolean calculation which has no such symbol. The bad calculation bubbles up a NaN up to the level of the comparison. There, the not-a-number gets eaten and becomes a Boolean true or false --- rather than becoming not-a-truth and continuing to bubble up.
There is no satisfactory way to plug NaNs into an expression that produces a clean two-valued Boolean truth with no error indication. You must separately test for the NaN.
If you perform two calculations, whose values are coming up as equal, but you didn't check the fact that both calculated the same NaN, that is your problem.
Suppose you are looking for the result of the two calculations being unequal, and they produce NaN (or at least one of them does). That's also false positive. There is no way to get around checking for NaN.
Among other things, this is super useful insofar as it gives a reliable way to test for NaN.
The fact that Windows reserves the PRN file name in every directory is probably usable and reliable to someone. That doesn't mean it's a good design for providing access to a printer device.
To test a set membership property of an object, you want a predicate function which takes the object as its only argument. For instance isnan(x).
This predicate can be efficiently implemented without relying on a bastardized equality operation.
While you might not explicitly write A == A into your code, it could occur implicitly due to some macro expansion, inline expansion or other code transformation.
I think GCC with -ffast-math gets rid of this NaN rule and does such optimizations anyway. (Your code just has to avoid generating NaNs so that the optimizations are valid.)
Since NaN values are valid representations which play a role in the system, and can be used in operations (such as comparing a NaN to a number, which is false), each of them must compare equal to itself.
If the bits on the left are the same as the bits on the right, the comparison is true. Distinct NaN bit patterns are unequal. Simple as that.
Whatever I would do, I would make sure that a comparison observes the Law of Identity.
(I'd rather not have Inf and NaN at all; operations should just generate an exception if they can't come up with a number.)
It’s the same with NULL in SQL.
Read that Chesterton quote somebody posted above. It sounds like you need to study the standard a bit more, because you don’t seem to understand why NaNs are the way they are. You’re arguing from principles that don’t apply.
That said, the ability to use double-double schemes to extend precision is wonderful and makes things much easier than they are in most of the Floating Point alternatives that have been proposed (eg Posits).
It is enough to enable exception generation for undefined operations and it is guaranteed that no NaNs can appear in any results.
It turns out that most people think that it is too much work to write suitable exception handlers, so they prefer to mask all exceptions and deal with NaNs and negative zeros.
In practice this just means that you must be careful when you write conditional expressions where floating-point numbers are compared, because the order relation becomes partial, so there are 14 possible relational operations instead of the only 6 that are possible for a total order, and you also must write the compared values in a way where the sign of a zero does not matter (in most expressions the sign of a zero does not influence the result; you must make some efforts to distinguish a negative zero from a positive zero).
> members of the committee, for the most part, were about equally altruistic. IBM's Dr. Fred Ris was extremely supportive from the outset even though he knew that no IBM equipment in existence at the time had the slightest hope of conforming to the standard we were advocating. It was remarkable that so many hardware people there, knowing how difficult p754 would be, agreed that it should benefit the community at large. If it encouraged the production of floating-point software and eased the development of reliable software, it would help create a larger market for everyone's hardware. This degree of altruism was so astonishing that MATLAB's creator Dr. Cleve Moler used to advise foreign visitors not to miss the country's two most awesome spectacles: the Grand Canyon, and meetings of IEEE p754.
> In the usual standards meetings everybody wants to grandfather in his own product. I think it is nice to have at least one example -- IEEE 754 is one -- where sleaze did not triumph. CDC, Cray and IBM could have weighed in and destroyed the whole thing had they so wished. Perhaps CDC and Cray thought `Microprocessors? Why should we worry?' In the end, all computer systems designers must try to make our things work well for the innumerable ( innumerate ?) programmers upon whom we all depend for the existence of a burgeoning market.
> Epilog: The ACM's Turing award went to Kahan in 1989.
IEEE 754 is okay as a technical standard. I've seen more denormals as a place to stash data than I ever have as a computational result, but, meh.
However, the political reason wasn't that everybody were being magically altruistic. IBM and DEC, especially, were killing it and were in no way going to allow the other to set the standard. And everybody else was keen to stop IBM or DEC from having the de facto standard which would cement their dominance further.
For example, if I remember correctly, Cray arithmetic was notorious for never actually being anything compliant (something about their multipliers).
The Intel format, which is due to William Kahan, but also to Jerome Coonen and John Palmer and a few others with lesser contributions, was a huge improvement over the IBM and DEC floating-point formats.
I have written my first programs when I was in high-school, for some IBM mainframes and DEC PDP-11 computers. Then I have used the Microsoft compilers for several languages, for Z80 / Intel 8080, which used FP formats similar to those of DEC.
The IBM and DEC formats were really ugly and writing programs for them included many pitfalls. When I began to use an IBM PC, with its much more foolproof FP format, all problems disappeared.
In general all processors introduced by Intel since their beginning and until the nineties (when their competition withered) lacked any innovative features. Every improvements in the early Intel CPUs had already been introduced earlier in CPUs from competitors.
To this lack of innovation in CPU architecture, there is a very important exception, the IEEE 754 standard, which was based on the Intel format with very minor modifications.
The Intel FP number format was one of the most important events in the history of floating-point numbers, the only other events with similar importance were when IBM made the first computers with hardware FP units (IBM 704 and NORC, in 1954) and when IBM introduced fused multiply-add (IBM POWER, in 1990).
I remember the IBM format was based on hexadecimal, and was thus criticized for wobbling precision. I thought the DEC format was pretty similar to the Intel one. What problems did you have with it, that disappeared when you switched to the IBM PC?
An Interview with the Old Man of Floating-Point (1998) - https://news.ycombinator.com/item?id=7769303 - May 2014 (17 comments)
An Interview with the Old Man of Floating-Point (1998) - https://news.ycombinator.com/item?id=6656197 - Nov 2013 (21 comments)
EDIT: I have an 8087 chip always installed to sitting on my monitor base. Because Kahan.
Kahan wanted decimal floats, not binary, and he wanted Extended Precision to be 128, not 80. I've had many hours of conversation with the man about how Intel railroaded that Standard to express the design decisions that had already been made for the i8087 coprocessor. John Palmer, who I also worked with for years, was proud of this, and told me "Whatever the i8087 is, THAT is the IEEE Standard."
Posits have a single exception value, Not a Real (NaR) for all things that fall through the protections of C and Java and all the other modern languages for things like division by zero, and the square root of a negative value. Kahan wanted the quadrillions of Not a Number (NaN) patterns to be used to encode the address of the instruction in the program to pinpoint where it happened, but the support for this in computing languages never happened. By around 2005, vendors noticed they could trap the exceptions and spend hundreds of clock cycles handling them with microcode or software, so the FLOPS claims only applied to normal floats, not subnormals or NaN or infinities, etc. This is true today for all x86 and ARM processors, and SPARC for that matter. Only the POWER series from IBM can still claim to support IEEE 754 in hardware; hardware support for IEEE 754 is all but extinct.
There are over a hundred papers published comparing posits and floats, both for accuracy on applications and difficulty of implementation. LLNL and Oxford U have definitively showed that posits are much more accurate than floats on a range of applications, so much so that a lower (power-of-two) precision can be used. Like 32-bit posits instead of 64-bit floats for shock hydrodynamics, and 16-bit posits instead of 32-bit floats for climate and weather prediction. For signal processing, 16-bit posits are about 10 dB more accurate (less noise) than 16-bit floats, which means they can perform lossless Fast Fourier Transforms (FFTs) on data from 12-bit A-to-D convertors.
For the same precision, posit hardware add/subtract units appear slightly more expensive than float add/subtract, and multiplier units are slightly cheaper for posits than for floats. This echos what was found comparing the speed of the Berkeley SoftFloat emulator with that of Cerlane Leong's SoftPosit emulator. Naive studies say posits are more expensive because they first decode the posit into float subfields, apply time-honored float algorithms, then re-encode the subfields into posit format. This does not exploit the perfect mapping of posits to 2's complement integers.
Float comparison hardware is quite complicated and expensive because there are redundant representation like –0 and +0 that have to test as equal, and redundant NaN exceptions that have to test as not equal even when their bit patterns are identical. Posit comparison hardware is unnecessary because they test exactly the same way as 2's complement integers. NaR is the 2's complement integer that has no absolute value and cannot be negated, 1000...000 in binary. It is equal to itself and less than any real-valued posit.
The name is NaR because IEEE 754 incorrectly states that imaginary numbers are not numbers, and sqrt(–1) returns NaN. The Posit Standard is more careful to say that it is not a _real_.
The Posit Standard is up to Version 4.13 and close to full approval by its Working Group. Don't use any Version 3 or earlier. The one on posithub.org may be out of date. In Version 4, the number of eS bits was fixed at 2, greatly simplifying conversions between different precisions. Unlike floats, posit precision can be changed simply by appending bits or rounding them off, without any need to decode the fraction and the scaling. It's like changing a 16-bit integer to a 32-bit integer; it costs next to nothing, which really helps people right-size the precision they're using.
