Re: Floating point bug in vitrual machine?

Article View: borland.public.jbuilder.java.language

Article #19937

Back to borland.public.jbuilder.java.language Previous Next

#19937

From: Antonio Garcia
Date: Wed, 12 Nov 1997 00:00

19 lines
1257 bytes

Eric Nagler wrote:
> 
> Yeah. It's how floating point numbers are stored internally --
> certainly not an exact representation. So it may actually be
> 511.99999999999, and then you truncate the fraction.
> 
This happens to be the case.  What threw me off initially was the the
JBuilder debugger was showing me the value in the "watch" window
as 512.00 even though it internally had the value 511.999997.

This value was calculated starting at 5.12 and then when through a
series of tests that multiplied the value * 10.0 twice.  As I stepped
through I watched the value changed to 51.20 and then to 512.00.

When the watch window showed me 512.00, silly me, I expected to get
512 by calling intValue()!

--AG

Technical Details

Message-ID: <346A1BC7.122B@eng.printronix.com>

Path: arc1-ca.pugleaf.net!unknown!not-for-mail

References: <3468E535.5AEB@eng.printronix.com> <346bf131.24828878@forums.borland.com>

Show Full Headers

Complete Article Headers

X-Google-Language: ENGLISH,ASCII-7-bit
X-Google-Thread: f74b3,70cc12fd65535774
X-Google-Attributes: gidf74b3,public
From: Antonio Garcia <garc_an@eng.printronix.com>
Subject: Re: Floating point bug in vitrual machine?
Date: 1997/11/12
Message-ID: <346A1BC7.122B@eng.printronix.com>
X-Deja-AN: 289117302
References: <3468E535.5AEB@eng.printronix.com> <346bf131.24828878@forums.borland.com>
To: Eric Nagler <76711.521@compuserve.com>
Organization: Printronix, Inc.
Newsgroups: borland.public.jbuilder.java.language

Reply to Parent Message