🚀 go-pugleaf

In article <1990Oct9.192119.4453@murdoch.acc.Virginia.EDU> clc5q@shamash.cs.Virginia.EDU (Clark L. Coleman) writes:
>The September 17, 1990 issue of PC WEEK has an advertisement on pages 56-57
>from Everex. The ad claims the following Dhrystone benchmarks:
>
> Everex STEP 486/33  :  34,000 Dhrystones (19.4 MIPS)
> CRAY-X-MP/48        :  17,857 Dhrystones
> IBM 3083            :  16,666 Dhrystones
>
>Is the explanation :
	<Various possible explanations omitted>

Just thought I would quote from "Computer Architecture: A Quantitative
Approach" by Hennessy and Patterson, p.48, reproduced without permission:

	"While there are rationalizations for use early in the design,
	 there is no current valid rationale for using benchmarks and
	 kernals to evaluate working computer systems ... While kernals,
	 toy benchmarks, and synthetic benchmarks(Whetstone and 
	 Dhrystone) were an attempt to make fair comparisions among
	 different machines, use of anything less than real programs
	 after initial design studies is likely to give misleading
	 results and lead the designer astray." 

They later mention that "Dhrystone has a long history of optimizations
that skew its performance" and cite a specific string copy operation
which can be optimized by a specific compiler, resulting in a 20-30%
improvement in performance. They also mention that over 99% of real-world
string copies could not be improved by this optimization.

I don't pretend to understand the rationale behind benchmarks, but
when I read Mr. Coleman's post, I trust them even less than I did before!

>"We cannot talk of freedom unless we have private property." -- Gavriil Popov,
>Mayor of Moscow, September 11, 1990. |||  clc5q@virginia.edu

You have got that right, Gavriil old boy.
Bye
Frank E. Zabaly             UUCP:{sun,harvard,rutgers,uunet}!zabaly!convex
Convex Computer Corp                           INTERNET :zabaly@convex.com
###############################################	###########################
"Something semi-automatic                      # Caffeine, the wonder drug
 might just do the trick"                      # that works and works and
        --Timbuk3 "Standard White Jesus"       # works and works and ...
###############################################	###########################

In article <1990Oct9.192119.4453@murdoch.acc.Virginia.EDU> clc5q@shamash.cs.Virginia.EDU (Clark L. Coleman) writes:
>The September 17, 1990 issue of PC WEEK has an advertisement on pages 56-57
>from Everex. The ad claims the following Dhrystone benchmarks:
>
> Everex STEP 486/33  :  34,000 Dhrystones (19.4 MIPS)
> CRAY-X-MP/48        :  17,857 Dhrystones
> IBM 3083            :  16,666 Dhrystones
>
>Is the explanation :
>
> 1) The Everex compiler got rid of a worthless inner loop in a stupid
>     benchmark.
>
> 2) The i486-based PC is really faster on this scalar benchmark than
>     a Cray X-MP/48.
>
> 3) They figured out some way to hamper the Cray for the benchmark.
>
> 4) They're lying.
>
> 5) None of the above.
>
>I realize that Dhrystones are not meaningful for supercomputer applications,
>but I thought that the scalar integer MIPS of a Cray would still exceed a
>PC. Is there any independent source of Cray performance on this test?

I can't claim to be an independent source (I do have a bit of bias in this
particular case ;-) ), but I have a comment or two about the Cray numbers.
Looking at records I found containing benchmarks of various Cray machine/
Cray C compiler combinations over the years.  The most recent benchmark
I found is 27042 Dhrystones on a Cray X-MP/48 with C 4.0 (This compiler was
released about two years ago).  I just ran it myself with our most recent
C compiler on a Cray X-MPEA/464 (most letters and numbers, but essentially
the same machine as far as this benchmark goes), and got 28079 dhrystones.
The benchmark they are quoting is with C 2.0 (about four years old), on
an older generation of X-MP machines with a slower clock speed (9.5 ns
as opposed to 8.5 ns).  Plus the fact that the /4 part of the machine
type means that the machine has four CPU's, which is meaningless for this
benchmark.  All of the timings are for one processor.  Plus we haven't
been making X-MP's for a couple of years (our current product, the Y-MP
has a clock speed of 6 ns, a bit faster).  And finally, Cray's are designed
for floating point, not integer arithmetic, so a lot of extra conversions
from integer format to floating point format (and back) will need to be
done.

If you've stayed with me for this long, what it comes down to is this:
benchmarks, like statistics, say exactly what you want them to.  They are
not lying.  They're just not telling the whole truth.

I hope this didn't sound like a whole bunch of marketing drivel, I just
wanted to point out ways to make benchmarks look exactly how you want
them to, not just on a Cray, but any machine.

>-----------------------------------------------------------------------------
>"We cannot talk of freedom unless we have private property." -- Gavriil Popov,
>Mayor of Moscow, September 11, 1990. |||  clc5q@virginia.edu

Dave Sielaff
Cray Research, Inc.
ds@cray.com
uunet!cray!ds