🚀 go-pugleaf

In article <1714@tekchips.TEK.COM>, jans@tekchips.TEK.COM (Jan Steinman) writes:
> One problem with most benchmarks is that they are easily twisted to suit some 
> purpose.  The Smalltalk Benchmarks are now considered invalid because one 
> implementation caches compiled code.  

Actually, this shows my main point, which is most 30-line benchmarks
are next to worthless, anyway.  On a real-size benchmark, caching
code will show a significant performance advantage, so this
is a legitimate thing to do.

Of course, it's a lot of work to run small benchmarks; it can require
weeks to run the larger ones across a variety of machines.  So
we should not be suprised that people run around producing and
touting meaningless benchmark results, because those are the only
ones ever run.

My idea of a benchmark is to take three different 20,000 line programs,
written by three different authors from three different styles (it
would be nice if one were machine-generated, such as from C++
or Objective-C), and then see if they produce the correct result,
if they compile or run at all.  Many PC compilers would fail this
test miserably.
w
-- 
	Joel West  (c/o UCSD)
	Palomar Software, Inc., P.O. Box 2635, Vista, CA  92083
	{ucbvax,ihnp4}!sdcsvax!jww 	jww@sdcsvax.ucsd.edu
   or	ihnp4!crash!palomar!joel	joel@palomar.cts.com