🚀 go-pugleaf

>From DUFFEY@MIT-ML Mon Aug 17 22:12:25 1981

WorkS Digest               Sat, 15 Aug 1981         Volume 1 : Issue 7

Today's Topics:
                           Micro Benchmarks
----------------------------------------------------------------------

Date: 14 Aug 1981 15:28 EDT
From: Marshall.WBST at PARC-MAXC
Subject: Micro Benchmarks

A salesman from Intel was here this week and said that EDN has
retracted its benchmarks and will run another set. He said that
the new numbers show the 8086 only 10-15% slower than the 68000.
He said the retraction would be in the next issue.

Sidney Marshall - Rochester N.Y.

------------------------------

Date: 14 Aug 1981 11:36 PDT
From: Kosower at PARC-MAXC
Subject: Re: Benchmarking new Micros

   Worse yet, benchmarks can be doubly deceiving, since they
may induce you to choose a micro for all the wrong reasons.
Unless you have a very specific application in mind, or
unless you plan to design and build your own operating system
from scratch, software development facilities, system quality,
and especially user interface quality are extremely important.
A Dorado, for example, may run faster than greased lightning,
but it would be about as useful as an F-15 powered by turboprop
engines if it had IBM-quality software running on it.  After
all, most of us do not want to write reams and reams of code
in some J-random processor's assembly language (more so if it's
microcodable); so quality of the high-level language available
and quality of the compiler ARE important.  Furthermore, most
applications change, and even with changes, their lifetime is
limited, so that other development facilities (editor, debugger,
etc.) are ALSO important.  If it takes you 10 times longer to
write a program that will take 10 times longer to debug and will
eventually run 10 times slower, on processor A whose raw speed
is 10 times greater than that of processor B, which one would
you choose?  Choosing B will save you time, to say nothing of
frustration, even though it is a "slower" processor.  These are
not idle thoughts: an IBM 370/168 has tremendous raw speed, but
some of the cruftiest software ever written makes it seem slower
than the US Postal Service.  Admittedly, almost all of the
software for micros such as the 8086 and 68000 is pretty awful,
but I still think it's worth keeping the above considerations
in mind.  As Allan points out, just because something is easy
to measure does not mean it's useful or even meaningful.

                                David A. Kosower

------------------------------

Date: 14 August 1981 1832-EDT (Friday)
From: David.Lamb at CMU-10A
Subject:  EDN benchmarks

Allan Schiffman may be right that the EDN benchmarks are
"about as useless as benchmarks usually are," but the MCF
(Military Computer Family) test specifications on which
they were supposed to be based *can* be used in a sensible
fashion to evaluate a computer architecture.  The original
experiment design was set up at CMU with the co-operation
of one of the members of out Statistics department, who
designed the experiment to separate variance on the tests
into differences based on programmers, particular tests,
and the architecture itself.  The notion was to see how
good the *architecture* (instruction set, visible registers,
etc.) was, rather than any particular implementation of the
architecture.  Several different measurement scales were
set up.  One measured the number of bytes need to encode
the programs, another measured the memory accesses needed
in an idealized implementation, and the third measured the
amount of data transferred between registers in the idealized
implementation.  I'm a little disappointed that the tests are
being used now for a different purpose; it's not clear to me
that you want the same kinds of tests to do a comparison of
particular implementations of the architecture, as was the
case in the EDN tests.

------------------------------

End of WorkS Digest
*******************