🚀 go-pugleaf

"Ron Reynolds" <ron@ronreynolds.com> writes:

> but my question is why this sort of rounding error pops up with
> such low-precision numbers and what's the accepted way of
> handling this sort of problem?

It's not a rounding problem, but a side-effect of trying to represent
decimal numbers as "a * 2^b" - not all decimal numbers fit this rather
strict limitation.

A common example which illustrates it is trying to fully represent 1/3
as a base 10 decimal. You can start writing 0.3333333333..., but as
soon as you get tired and stop, you have introduced a rounding
error. Now, if you add a condition where you _have_ to stop because
the number of digits is limited, you see that the rounding error
inevitably crops up.

If precision is important to you, do all computations with scaled
ints/longs or java.math.BigDecimal.