🚀 go-pugleaf

In article <1224@husc6.UUCP> reiter@harvard.UUCP (Ehud Reiter) writes:
>Has anyone actually tried to evaluate the Dhrystone (and other benchmarks)
>by seeing how well it predicts performance on real applications?  It would
>seem straightforward to take ten random applications running on specific test
>data, measure their performance on some target machine/compiler combinations,
>and statistically analyze how much of the peformance differences had been
>predicted by the Dhrystone figures.
1) There's probably an interesting M.S. thesis in here somewhere.
>
>The debate on flaws of the Dhrystone is quite interesting, but it would be nice
>to have some real data on how good or bad the Dhrystone was.  I'm not even sure
>that a good benchmark is possible in principle - that is, I wonder whether
>it is possible to come up with a single number which can predict
>(with any reasonable accuracy) performance on a range of different
>applications.

2) Most people I know don't believe very much in single-number performance
metrics.

3) Althought I raised this issue in the first place, there do appear to
be a few applications that grossly correlate [and I mean grossly]
with Dhrystone, i.e., if you saw the Performance Brief I posted here a
few months ago, there was actually a reasonable correlation of it with
things like grep/diff/yacc/nroff, i.e., integer user-level programs of
moderate [but not huge] size, although it sometimes overstated the performance
of small-cached micros versus superminis.  This effect is typical of small
benchmarks: if it fits into the cache, you get something that correlates
a bit better with raw CPU/cache speed; the more it doesn't fit, the more
you're measuring cache-main-memory performance.
-- 
-john mashey	DISCLAIMER: <generic disclaimer, I speak for me only, etc>
UUCP: 	{decvax,ucbvax,ihnp4}!decwrl!mips!mash, DDD:  	408-720-1700, x253
USPS: 	MIPS Computer Systems, 930 E. Arques, Sunnyvale, CA 94086