Article View: comp.lang.postscript
Article #855Re: Bug in PostScript math
From: Craig Werner
Date: Sun, 13 Aug 1989 02:23
Date: Sun, 13 Aug 1989 02:23
36 lines
1998 bytes
1998 bytes
In article <2395@aecom.yu.edu>, werner@aecom.yu.edu (Craig Werner) writes:
>
> I recently encountered a problem that seems to be do to a bug in
> the PostScript arithmetic routines built into an Apple LaserWriter IINT.
>
> Consider the following:
> 1) divide 200 by 5, save the 40 in a variable
> 2) add 200, later subtract 200
> 3) divide the variable byt the current value, the answer is 1.0
> 4) truncate the 1.0, the answer should still be 1.0.
>
> Now divide 200 by 3, saving 66.6667.
> When you truncate the 1.0 in step 4, the result is 0.0
>
Let me restate the problem. I referred to the problem as a bug
when the real problem is that it has been twelve years since I've
encountered such an egregious arithmetic precision error. (The last time
was on a TRS-80 Model I, Level I, system complete with 4K RAM.) Just
out of curiousity, what is the precision of PostScript's floating point
routines? It's obviously a lot less than I had expected.
I'm still confused as to why cvs would report the number as 1.0
instead of what it really was. However, I've used this to my advantage,
and a fix for the particular problem seems to be replacing
truncate
with 20 string cvs cvr truncate
(11 string gives me a cvs rangecheck error, and at this point
I'd rather not ask why.)
--
Craig Werner (future MD/PhD, 4.5 years down, 2.5 to go)
werner@aecom.YU.EDU -- Albert Einstein College of Medicine
(1935-14E Eastchester Rd., Bronx NY 10461, 212-931-2517)
"Sometimes you have to run as fast as you can just to stay in the same place."
Technical Details
Message-ID:
<2398@aecom.yu.edu>Path:
arc1-ca.pugleaf.net!utzoo!attcan!uunet!aecom!wernerReferences:
<2395@aecom.yu.edu>Show Full Headers
Complete Article Headers
Path: arc1-ca.pugleaf.net!utzoo!attcan!uunet!aecom!werner From: werner@aecom.yu.edu (Craig Werner) Newsgroups: comp.lang.postscript Subject: Re: Bug in PostScript math Summary: A restatement and a fix Keywords: 200/3 Message-ID: <2398@aecom.yu.edu> Date: 13 Aug 89 02:23:02 GMT References: <2395@aecom.yu.edu> Distribution: na Organization: Albert Einstein Coll. of Med., NY Lines: 36