🚀 go-pugleaf

|  From: inria!XV.MIT.EDU!Kevin_Crowston
|
|  ...  Also, what
|  are people's impressions of the programmes as a way to compare machines? 
|  Reasonable?  Unbiased?  Useful?  or not?  
|  
|  Kevin Crowston
|  MIT Sloan School of Management
|  
Normally, XEROX has on one of their floppies most of Gabriel's benchmarks in
Common Lisp!  I have not tried them, yet!  What are they worth?  Well, that
depends on *lots* of things.  Most (Some?) of these benchmarks test some
precise functionality of the lisp system, e.g., access to global variables,
two-dimensional arrays, procedure stacking, etc.  Unfortunately, to compare
machines on *those* precise functionalities may be valid as a comparison exactly
for the user whose application uses precisely those functionalities!!

User programs make use of a weighted sum of such functionalities plus *other*
functionalities.  Also, it is not sure what Gabriel's benchmarks as a group
measure.  There exist other benchmarks (such as "Long Loop") which measure
differently.

Furthermore, the benchmarks not only reflect on *certain* capacities of a
machine, but also on the machine's compiler and it's compiler writer group.
Next is the problem of passing the benchs.  Under what conditions? With or
without declarations?  Garbage Collection activated or not?  Benchmarks
run in what kind of virtual memory mix and run how often?  What's the mean
execution time?  Etc.

Finally, the best bench is to pass one's own application on a number of 
machines.  The problem here, is that *all* machines permit tuning of the
application in order to increase its performance -- and that requires time!!

Ralph P. Sobek		       | UUCP:  uunet!mcvax!inria!lasso!ralph,    or
			       |        ralph@lasso.uucp
LAAS-CNRS		       | Internet:  ralph@lasso.laas.fr,    or
7, avenue du Colonel-Roche     |            ralph%lasso.laas.fr@uunet.UU.NET
F-31077 Toulouse Cedex, FRANCE | ARPA:   sobek@shadow.Berkeley.EDU (forwarded\
+(33) 61-33-62-66	       | BITNET/EARN:  SOBEK@FRMOP11        \ to UUCP )
=  =  =  =  =  =  =  =  =  =  =  =  =  =  =  =  =  =  =  =  =  =  =  =  =  =  =