🚀 go-pugleaf

Joe Ragosta wrote:
> 
> In article <4j96jp$m7r@news.erie.net>, pce@moose.erie.net (Paul
> Christenson) wrote:
> 
> > When are people going to learn that you can't directly compare benchmarks
> > across platforms?  Especially when the different systems use different
> > processor families?
> >
> > For instance, Linux comes up with a "BogoMIPS" rating on bootup.  It's an
> > indication of how fast Linux did some timing loops.  One friend's 486DX4-100
> > has a BogoMIPS rating of 50.  Another friend's Pentium-133 also has a
> > BogoMIPS rating of 50.  Does that mean that the two machines are the same
> > speed?  Of course not.
> >
> > A friend of mine compiled up the BogoMIPS-determining code on a high end
> > SGI machine.  The machine is overall about 10x faster than the DX4-100,
> > yet its rating was LOWER than that of the DX4.
> >
> > Bottom line...  Benchmarks are useless between different platforms.
> 
> OK. Let's quit comparing computers. For that matter, the next time I need
> to buy a car, I'll just buy the first one I happen to stumble
> across--after all, you can't compare different cars. Gee, we don't need to
> spend millions of dollars on elections either. How do you compare
> candidates. We'll just assign a random number generator the task of
> allocating votes.

You completely missed the point.  He's not saying one should not compare
computers, he's saying that a benchmark is not going to give you an accurate
comparison.  If you are, you won't be comparing apples with apples (I hate
to use stupid cliches, but I couldn't think of a better one).

> You HAVE to compare platforms. How else are you going to make a decision.

Exactly.  Don't look at benchmarks.

> All you've shown is that BogoMIPS is a lousy benchmark. If you use good
> benchmarks, you get reasonably consistent performance stories. In general,
> the best benchmark is to run the application the computer user will need
> to use most often. The other benchmarks can be evaluated on how they
> approximate real life performance.

Okay, a bit of background, here.  BogoMIPS is *not* a benchmark (Bogo stands
for bogus).  Like he said, it's a timing loop.  Linus made it up so the kernel
would have a realatively consistent timing loop on any computer.  That, my
friend, is a very lousy benchmark, indeed, because it's not one.

> Most of the standard benchmarks are at least self-consistent. That is, a
> P-6 is faster than a P5 which is faster than a 486 and so on. They also
> consistently show that a 604 is in the same league as a P6 and a 601 is in
> the same league as a P5.  You could argue that finer distinction is
> impossible, but to argue that benchmarks have no use is a silly argument,
> IMO.

Of course they'll be consistent!  All that says is that all 601's are built
the same.  A good benchmark on a P6 may not account for differences in
the architecture between it and a 601.  Or how a fast processor won't get
you anywhere in a slow computer.  And, by the way, there really is no way
to compare a P6 and a 604.  It depends on what you're looking for.  It's
like deciding whether to buy a BMW or a Cadillac (assuming you can actually
afford either, or would even want to); you will inevitably compare the numbers,
but you know it won't matter.  Even if the BMW has more acceleration (higher
benchmark numbers, to aid those missing the parallel between the two), you
know the cadillac has a bigger engine, and could probably haul more weight,
if called to do so.  What really matters is whether you want American Luxury
or German engineering.

Just like what really matters with the computer is whether you want the proven
GUI of a Mac, or the flexibility of the PC (even if the GUI is somewhat less
impressive, and definitely not as stable).