🚀 go-pugleaf

In article <6156@npri6.npri.com>, richard@npri6.npri.com (Richard Head) writes:

> Perhaps they are irrelevant to you.  Believe it or not, some people
> (obviously not quite the established Computer Scientist you are)
> like to make comparison to a REFERENCE STANDARD (SPECmarks), so they can
> compare machines, regardless of vendor.  How can they do this with MicroVUPs?
> (That's right, Einstein, tell me how many MicroVUPs an IBM RS6000 has!
> That's exactly the information I'm looking for!)

  At the risk of actually adding some _content_ to this flamefest 8-), I am
interested in the performance of systems which run VMS because I want to buy
them (if they perform well) and run VMS on them. If VMS ever becomes avail-
able for the RS6000, then I'd be interested in knowing it's MicroVUP rating
(which I assume DR&N would gladly supply).

  If that's the market that DR&N is catering to, then I can see why they use
the ratings that they do. There are some problems with the "VUP" rating sys-
tem in general (but that doesn't stop DEC from using it) and with "MicroVUPs"
in particular. First, the 780 was generally considered to be a 1 VUP / 1 MIP
machine. That doesn't mean that a 40 VUP VAX is a 40 MIPS machine. Further,
VMS overhead has grown a great deal (meaning that a system that benchmarks
as 10 times as fast as a MicroVAX II running MicroVMS V4.1M is probably a bit
better than 10 times as fast (I'd say around 12-13 times) faster. Second, the
compilers have improved, so if the benchmarks have been recompiled they will
show better results.

  Lastly, DEC has miscalculated the VUP ratings of various processors. The
examples that come to mind are the 82x0/83x0 (much slower than they are rated)
and the 9000 series (generally faster than they are rated). Since license 
pricing is set by CPU performance, VUP numbers (and the corresponding LURT
values in the license tables) rarely go down. Some LURT numbers have changed,
sometimes accompanied by notes in the release notes, sometimes not.

  Anyway, VUP/MicroVUP/Specmark/TPS performance is so related to the OS on a
machine that a reasonable comparison across platforms isn't realistic. I find
that for some real-world apps an 11/23 outperforms a VAX 6520 (MOP service), a
12Mhz 286 with 4Mb outperforms a VAX 9440 (PC file serving), and a PDP-11/70
outperforms an 8650 (disk I/O). But I don't run my business on that PDP-11/23.
Even switching from VMS to Unix on a system with all hardware remaining exact-
ly the same yields order-of-magnitude performace differences for some tasks.

  There's an excellent discussion on DECUServe, started by Bart Lederman, cal-
led "So how fast is your VAX, anyway?" which computes instruction timings for
many instructions. That will give me the data I need to say "CPU X is faster
than CPU Y" when I'm running on the bare metal and counting cycles. It also
gives me a good general feel for overall CPU speed.

  Anyway, there's an old saying which says "the only correct benchmark is your
application". Users who are looking a purchasing a system should test their
code on the new CPU to see how it works for _them_. What works for you or for
me isn't relevant to them. The published numbers just indicate roughly if a
system is worth looking at at that level of detail or not.

	Terry Kennedy		Operations Manager, Academic Computing
	terry@spcvxa.bitnet	St. Peter's College, Jersey City, NJ USA
	terry@spcvxa.spc.edu	+1 201 915 9381