🚀 go-pugleaf

In article <403@validgh.com> dgh@validgh.com (David G. Hough on validgh)
writes:
+---------------
| Why Geometric Mean is Best for SPEC
|      The progress of some end users is limited by the time it takes a fixed
| series of computational tasks to complete.  They then think about the results
| and decide what to do next.  The appropriate metric for them is the total
| elapsed time for the applications to complete, so the arithmetic mean of times
| is the appropriate summary statistic.  If rates, the inverse of times, happen
| to be available instead, the appropriate statistic is the harmonic mean of
| rates.  If application A runs ten times as long as application B, then a 2X
| improvement in application A is ten times as important as a 2X improvement in
| application B.
+---------------

In Mike Johnson's new book "Superscalar Micorprocessor Design", he argues
(pp.37-40) that the harmonic mean is the right thing to use when comparing
"speedups", because it tends to assign larger weighting to the program with
the smallest speedup. I agree. In particular, a single isolated "gigantic"
speedup is almost completely ignored, which is what you want for a "systems"
benchmark.

So it seems that the harmonic mean of the individual test ratios would be
useful as "the SPECratio" when comparing several systems to a reference
base system.


-ROb

-----
Rob Warnock, MS-1L/515		rpw3@sgi.com		rpw3@pei.com
Silicon Graphics, Inc.		(415)335-1673		Protocol Engines, Inc.
2011 N. Shoreline Blvd.
Mountain View, CA  94039-7311