undefined | Better HN

0 pointsmark-r6y ago0 comments

My calculation shows 12 * 0.0254 = 0.30479999999999996 meters. That may be off a little due to IEEE floating-point inaccuracy.

0 comments

d-sc6y ago

2 * 4 = 8 so the last significant digit should be an ‘8’

pjc506y ago

Definitely one to cite in the "overreliance on calculators" - the comment you are replying to should have realised that multiplication can't create digits to the right like that!

(Most desktop calculators do it in BCD which would give the correct result, too)

mark-rOP6y ago

I use Python as my desktop calculator. It has a Decimal module that would have gotten it correct, but that would have been way too much trouble.

j / k navigate · click thread line to collapse