🚀 go-pugleaf

Alex Humphreys <ahumphreys@sema4.com> wrote:

>I'm trying to add a group of doubles (for column totals in my grid) and
>I'm getting STRANGE data with decimal points.  Seemingly randomly, it
>will throw on an extra 15 or so digits.  Now, I've read about Floating
>Point problems with this, so I'm assuming that this is an off-shoot of
>that problem.
>
>Is there a way around this problem?

Not with doubles. The problem is, doubles are stored internally using
binary, and some numbers that are exact in decimal (e.g. 8.7) are not
exact in binary. For example, try writing down 1/3 as a decimal; It
won't go, you get 0.333333333333333...... forever. But if you were
using base 3, it would go exactly, and be written 0.1. Anyway, that's
why it gets this rounding error. Normally you can program the
print-formatting to suppress it; I've never used DoubleFormatter so no
comment there.

Another way to get round the problem, if you only need a fixed number
of decimal places, is to multiply everything by a suitable scaling
factor and use integers (and clever formatting to put the decimal
points back in afterwards). E.g. instead of using dollars and cents
and having 2 decimal places of dollars to represent the cents,
multiply everything by 100 and work in cents.
>
>I'm trying to format the solution using DoubleFormatter, but can't get
>the syntax right.  I'm trying:
>
>    DoubleFormatter double_f = new DoubleFormatter(7);
>    double_f.setPattern("###.{##}");
>    Variant temp_v = new Variant();
>    double_f.setFromDouble(temp_v, 7, my_dataRow.getDouble("my_col"));
>    my_col.setText(temp_v.toString());
>
>And of course this doesn't do a darn thing.  What am I missing?
>
>Thanks.
>
>Alex