Re: IEEE floating point

Article View: comp.arch

Article #11866

#11866

From: John D. McCalpin
Date: Wed, 29 May 1991 20:46

26 lines
1875 bytes

> On 27 May 91 13:29:50 GMT, mccalpin@perelandra.cms.udel.edu (John D. McCalpin) said:

> On 26 May 91 14:02:57 GMT, mccalpin@perelandra.cms.udel.edu, I said:

Me>    Errors in solution of 1000x1000 system of equations
Me>            from the LINPACK benchmark suite

Me> machine			precision		RMS error
Me> ---------------------------------------------------------
Me> IBM RS/6000                  64-bit			1.3e-10
							^^^^^^^  WRONG!
Me> IBM RS/6000 (-qnomaf)	 64-bit			2.2e-12
Me> IEEE (Sun-3)		 64-bit			2.3e-13
Me> ---------------------------------------------------------

Please note that the number indicated is WRONG.  At least, I am not
able to reproduce it.  The reproducible number is 1.2e-12 for the RMS
error.  This is very close the the -qnomaf result, but still
inexplicably about 5 times larger than the IEEE number.

I apologize for the misunderstanding, and any resulting heartburn at
IBM :-).....
--
John D. McCalpin			mccalpin@perelandra.cms.udel.edu
Assistant Professor			mccalpin@brahms.udel.edu
College of Marine Studies, U. Del.	J.MCCALPIN/OMNET

Technical Details

Message-ID: <MCCALPIN.91May29164641@pereland.cms.udel.edu>

Path:

arc1-ca.pugleaf.net!utzoo!utgpu!news-server.csri.toronto.edu!rpi!usc!zaphod.mps.ohio-state.edu!uakari.primate.wisc.edu!ames!haven.umd.edu!udel!nigel.ee.udel.edu!mccalpin

References:

<9105250030.AA08036@ucbvax.Berkeley.EDU> <1991May25.222551.16365@zoo.toronto.edu> <MCCALPIN.91May26100257@pereland.cms.udel.edu> <MCCALPIN.91May27092950@pereland.cms.udel.edu>

Show Full Headers

Complete Article Headers

Path: arc1-ca.pugleaf.net!utzoo!utgpu!news-server.csri.toronto.edu!rpi!usc!zaphod.mps.ohio-state.edu!uakari.primate.wisc.edu!ames!haven.umd.edu!udel!nigel.ee.udel.edu!mccalpin
From: mccalpin@perelandra.cms.udel.edu (John D. McCalpin)
Newsgroups: comp.arch
Subject: Re: IEEE floating point
Message-ID: <MCCALPIN.91May29164641@pereland.cms.udel.edu>
Date: 29 May 91 20:46:41 GMT
References: <9105250030.AA08036@ucbvax.Berkeley.EDU>
 <1991May25.222551.16365@zoo.toronto.edu>
 <MCCALPIN.91May26100257@pereland.cms.udel.edu>
 <MCCALPIN.91May27092950@pereland.cms.udel.edu>
Sender: usenet@ee.udel.edu
Organization: College of Marine Studies, U. Del.
Lines: 26
Nntp-Posting-Host: perelandra.cms.udel.edu
In-reply-to: mccalpin@perelandra.cms.udel.edu's message of 27 May 91 13:29:50 GMT

Reply to Parent Message