🚀 go-pugleaf

In article <104996@sun.Eng.Sun.COM> khb@sun.UUCP (Keith Bierman - SPD Languages Marketing -- MTS) writes:
>>
>>  My preference is to have a third party do the numbers (no fudging allowed!).
>
>Typically no understanding of the machine either. In former
>incarnations I was on the user side. It was (and still is) that most
>third parties don't know how to run the machine right. So as a
>customer I got each vendor to run my code, and explain what they did.
>directives could increase performance fourfold .... 

  Yes, I agree that it is important to get somebody knowledgable to run
the benchmarks.  Just any third party will not do (sorry I didn't make this
clearer).

>
>It has been my experience that _most_ vendors tell the truth in these
>tests; though they may shade it a bit :>  

  Actually, some vendors shade the data quite a bit.  e.g. Intel quotes
simulated, unrolled BLAS, no-wait-state memory, and bug-free chips in
their i860 benchmarks.  Also, Sun made quite a mistake in claiming that their 
new Sparcstation 1GX could create 3D polygons twice as fast as Silicon
Graphics Personal Iris (Sun's new machine doesn't even handle 3D polygons).
(NOTE:  To be fair, Sun has claimed that their error was an honest mistake, although
it seems to me to be amazingly to their benefit to make such a mistake.)           

  And this is exactly my point.  Vendors who do stuff like this cast doubt on 
ALL the vendor numbers, making a third party solution important (and, as you
mention, the ultimate benchmark is the CUSTOMER's application).

>
> [They] chose the name of the mag w/o realizing that it was in use by one of
>the companies they would be covering .... :> *)

  Actually, as I understand it, MIPS magazine had the name before MIPS (the
company) was created.  :)

Mike Pogue
Data General Co.
Westboro, MA.

My opinions are my own....