🚀 go-pugleaf

sterrett@nosc.mil (Anthony E. Sterrett) writes:


>Hi,
>	Would anyone like to share or disscuss techiques for
>benchmarking client-server systems?
>Thanks in Advance 
>Tony Sterrett
>sterrett@nosc.mil

I would. I'm just getting ready to start on a project like this,
and any information in this area would be a big help.
-- 
     ____  __   __       Tom@UTS.CC.Wayne.Edu
      /   /_/  /_        Thomas_Richard_Stevenson@MTS.CC.Wayne.Edu
     /.  /\ . __/.       Tom@CMS.CC.Wayne.Edu
                         UserTom@WayneMTS.Bitnet

In article <1992Oct6.152245.9931@nosc.mil> sterrett@nosc.mil (Anthony E. Sterrett) writes:
>	Would anyone like to share or disscuss techiques for
>benchmarking client-server systems?


There's an old client-server benchmark called ET-1 (transaction processing
benchmark for the banking (ATM) community).  The relevant metric is a set
of throughputs (of transactions) at various average response times.
Alternatively (and true for ET-1) there's a single metric:  throughput of
all transactions which completed in under X seconds.  [late transactions
just don't get counted]

This way both communication time and time queuing for communications get
counted in the benchmark.

Client/server performance benchmarking is:

	- great fun
	- much more complex than time shared or batch benchmarking
	- much more expensive than traditional benchmarking
	- probably won't get much easier as time goes on 

C/s performance characterization is more complex and expensive than the
more traditional computing styles (time sharing, batch).  More complex
since there are more interactions to investigate: desk top to host and
host to host (note the elegant side stepping of the client/server - 
server/client definitions:).  The more components that make up the
system, the more potential bottlenecks there are.

More expensive since the tools technology to drive c/s experiments are
more complex than ever before.  For time shared user benchmarking, a 
relatively simple remote terminal emulator could be used.  With today's 
c/s applications: mouse movements, button clicking, drag and drop, multiple
windows, multiple network stacks, and messaging formats make desk top emulation
nearly impossible.  To measure the response time at a desk top, you need
to configure and drive a desk top.  Since even the smallest servers today can 
support dozens of clients, driving a server to saturation can be a logistical 
nightmare. With high end servers, it is nigh on impossible to drive it to 
saturation with some applications, and modeling is the only hope of accurately 
sizing the server.

There are also many environments (and more coming) to deal with: X11, MS
Windows, MAC, DOS, and VTs.  Many applications span all of these environments,
and while there are many tools for each of these environments, there are no
tools that I am aware of that span all of them.  So in addition to the capital
costs of all of those desk tops, there is the cost of the software licenses for
the benchmarking tools. 

As applications spread across desk top environments, server platforms,
network operating systems, and operating systems, evaluating the 
performance of these applications across the range becomes even more 
formidable, if not impossible, task.

All together a good problem to have, and the opportunities in this space
are exciting.

Regards,


+-----------------------------------------------------------------------------+
| Marc Slater				Applications			      |
| Digital Equipment Corp (TTB1-1/E09)	 Systems			      |
| Ten Tara Blvd., Nashua, NH 03062	  Engineering			      |
| Telephone: (603) 884-6309		   Performance			      |
| FAX: (603) 884-2253                       Group			      |
| Internet: slater@mrktng.enet.dec.com	         Possibly my own opinions     |
+-----------------------------------------------------------------------------+