🚀 go-pugleaf

In article <698@mouse.UUCP>, lad@mouse.UUCP (Larry Deleski) writes:

> This is a file which I downloaded from the MouseHole BBS about a year ago. 
> At that time,  Jim Reekes wrote an article on Disktimer,  our beloved disk
> performance benchmark, and essentially exposed Disktimer for was it was 
> (and still is), a fraud.

I am the author of DiskTimer II.  I am also the author of several Macintosh
SCSI disk drivers, and of the firmware for several (non-SCSI) disk
controllers.

It is disheartening to see the republication of an article (Reekes's) which
contains so much error and which I rebutted about 18(?) months ago.  How a
simple program like DiskTimer II, whose publication included source code,
and whose user interface includes appropriate caveats about interpretation
of the results could be "a fraud" is beyond me.

> I would like to take this opportunity to point out an error in something I
> said earlier.  I stated that Disktimer did not use the Device Manager.  It
> does,  but not in the same way a typical SCSI driver does.  What Disktimer 
> fails to do is use the FileManager,  something every driver has to do.

DiskTimer II uses the disk's device driver, via _Read and _Write traps
(PBRead and PBWrite) for all disk access.  No device driver uses the File
Manager.  The File Manager uses the device driver.  Applications call
the File Manager which calls the Device Manager which calls the device driver
which either calls the SCSI Manager or accesses the SCSI hardware directly
(the latter likely to run into compatibility problems).

> Please read the following [article by Reekes] and decide for yourself.
> Reekes *does* know what he's talking about.

Reekes has no idea of what he's talking about with respect to several issues.
The following quotations indicated by ">>" are from Reekes.

>> ...the Macintosh only makes single-block requests.

False.  The File Manager makes both single- and multiple-block requests. Reekes
reiterates this error several times and bases conclusions on it.

>> ...the chain of command from application to hard disk[:]
>> Application>>FileManager>>DeviceManager>>DeviceDriver>>
>> SCSIManager>>HDController>>HardDisk

This is correct.  I quote it to call it to Deleski's attention.

>> DiskTimer uses the Macintosh's internal SystemTicks to calculate its
>> resulting times.  SystemTicks are in 1/60th of a second and DiskTimer
>> is attempting to measure times in 1/1000th of a second! (ie: A typical
>> 20MB drive's access time is 65ms and transfer rates are greater than
>> 1MB/sec.)
>> ...
>> As an analogy, using ticks tomeasure a 32k read/write, would be like using a
>> car's odometer to measure lengths of feet.

The analogy is correct, and that's why DiskTimer II does not attempt to measure
intervals such as those Reekes claims it does. DiskTimer II measures the total
elapsed time to do a sequence of operations.  The times measured are those
reported to the user, which are usually on the orders of 10-100 deciseconds
(60-600 60hz system clock interrupts). Reekes quotes Inside Macintosh to the
effect that the tick count may sometimes be inaccurate by more than one tick.
But the only cause of such inaccuracy is disabling of interrupts due to a
continuous source of interrupts.  This does not apply to DiskTimer II 
operations
if the user follows on-screen instructions.

>> DiskTimer does 200 read/write tests and averages them to the nearest
>> 60th of a second.

For each 100 read tests and write tests, DiskTimer II reports the *total*
interval in elapsed ticks, divided by 6 (it reports deciseconds, for
historical reasons).  It does not attempt to measure the timing of single
operations, and does no averaging.

>> Most importantly, DiskTimer should never be run on a disk containing 
>> user data.  ... If you were to reset, power off, or get the bomb dialog
>> during a DiskTimer test, then chances are you've lost data.

I quote this for your amusement.  By this criterion, one should never run any
software which writes to the disk.

>> [The] seek test does not consider rotational latency.  If the disk platter
>> were  required to complete an additional rotation just to bring that second
>> block under the disk heads, then it would add additional time to the seek
>> results.

Rotational latency is a component of access time.  As stated in the source
code comments, "to a avoid constant sector skew between I/Os, which might be
unfair to some drives, there is a varying delay between I/O requests.  The
delay is obtained via a pseudo-random number generator, using the same
seed on each run."

>> DiskTimer disclaims any comparisons with real world performance.

DiskTimer II tells the user that "The results to not translate directly to
perceived througput [sic] of typical Mac activity."

Moving from response to my thoughts on this matter:

DiskTimer II is a simple program that can be useful in judging the relative
performance of disk subsystems on a limited set of tasks: transfers of
24KB of data to and from the disk, and a series of single-block transfers
located 2048 blocks apart.  The latter test is invalidated, as noted by
Ephraim Vishniac, by cacheing controllers or drivers.

Implementing a "real-world" benchmark of disk performance is a time-consuming
task and requires a disk dedicated to the purpose, i.e., one that contains
no data that needs to survive the test.  Such a test requires that the exact
same files be written to each disk under test and then the exact same
operations performed in the same order.  Only a laboratory has the resources
to set up such tests.

DiskTimer II was provided as a simple, non-destructive test that any user can
run.  It should not be relied upon as a complete measurement of disk
subsystem performance.  Its measurements, where not invalidated by
special considerations, provide a few data points that may sometimes
be useful to some people.  I would have no trouble believing that a superior
tool was available (I have not looked at SCSI Evaluator).  But there's a
lot of ignorance about the technicalities of disk performance testing,
so be careful about accepting anyone's results unless you fully understand
the test techniques.