Re: Bug in EXCEL v. 2.2, and a flame

Article View: comp.sys.mac

Article #2204

#2204

From: Dave Seaman
Date: Tue, 01 Aug 1989 00:58

17 lines
1238 bytes

First of all, a simple Taylor series expansion of f(x) = sqrt(x) about x=1
should show you that the square root of .999975 is .9999875 with an error on
the order of 10^(-10).  But since it was claimed that Excel 2.2 on a Mac II
did not agree, I decided to check it out.

If you display the result using "general" format (the default) in Excel 2.2,
then you get .9999875 on a Mac II.  However, if you change the numeric format
to "0.00000000000000" and then widen the cell to make room, you will see
0.99998749992187.

Since I had Mathematica handy, I tried it also and got
0.999987499921874023422240943904, which agrees to as many digits as Excel
displayed.

-- 
Dave Seaman	  					
ags@seaman.cc.purdue.edu

Technical Details

Message-ID: <3531@mentor.cc.purdue.edu>

Path: arc1-ca.pugleaf.net!utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!purdue!mentor.cc.purdue.edu!ags

References: <MYp9wBy00WB-8EBFJb@andrew.cmu.edu> <959@rex.cs.tulane.edu>

Show Full Headers

Complete Article Headers

Path: arc1-ca.pugleaf.net!utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!purdue!mentor.cc.purdue.edu!ags
From: ags@mentor.cc.purdue.edu (Dave Seaman)
Newsgroups: comp.sys.mac
Subject: Re: Bug in EXCEL v. 2.2, and a flame
Summary: Mac II results:  Excel 2.2 and Mathematica
Message-ID: <3531@mentor.cc.purdue.edu>
Date: 1 Aug 89 00:58:32 GMT
References: <MYp9wBy00WB-8EBFJb@andrew.cmu.edu> <959@rex.cs.tulane.edu>
Reply-To: ags@mentor.cc.purdue.edu (Dave Seaman)
Distribution: usa
Organization: Purdue University
Lines: 17

Reply to Parent Message