Re: Improper results on T800

Thread View: comp.sys.transputer

1 messages

[T] Back to Threads [V] Tree View [G] Group Articles

1 total messages Started by AT1F Fri, 26 Apr 1991 15:07

#1623

Author: AT1F
Date: Fri, 26 Apr 1991 15:07

21 lines
1055 bytes

hi all,

your floating point number problem stems from limited representation
capabilities of numbers on your computer.

the resulting error comes from calculating
(we * yin&nsitea + xdin&nsitea)

we * yin&nsitea = 1168.106778
xdin&nsitea     = 1168.106812

the two numbers differ in the 8th significant digit!
IEEE says single precision handles 7 significant digits. the results
obtained are pure random.

to overcome the problem change to 'double'.

have a nice weekend.
Stefan Pokorny, DLR (german aerospace research establishment), Cologne,
                Germany
E-Mail: at1f@dlrvmkp.bitnet

Thread Navigation

This is a paginated view of messages in the thread with full content displayed inline.

Messages are displayed in chronological order, with the original post highlighted in green.

Use pagination controls to navigate through all messages in large threads.

Back to All Threads