🚀 go-pugleaf

This is a file which I downloaded from the MouseHole BBS about a year ago. 
At that time,  Jim Reekes wrote an article on Disktimer,  our beloved disk
performance benchmark,  and essentially exposed Disktimer for was it was 
(and still is), a fraud.

Ephraim Vishniac would have us beleive that Reekes is a lunatic.  Ephraim
exhausted many hours at the keyboard disputing that article.  I, for one,
beleive what I read in that article.  A large portion of what Reekes wrote
is fact.

I would like to take this opportunity to point out an error in something I
said earlier.  I stated that Disktimer did not use the Device Manager.  It
does,  but not in the same way a typical SCSI driver does.  What Disktimer 
fails to do is use the FileManager,  something every driver has to do.

Please read the following and decide for yourself.  Reekes *does* know 
what he's talking about.

------------------------

			DISKTIMER II vs. THE REAL WORLD

DiskTimer II has been mistakenly used as the de facto standard for hard drive
performance. Before I attempt to discredit any results of S. Brecher's program, 
I'd like to point out that I don't blame him.  He is a good programmer, but I 
think there are some serious flaws in his benchmarkcalculations.  Normally I 
would let this utility fade off into the sunset, but a certain hard diskvendor 
has taken DiskTimer and exploited it.  Today, nearly every hard disk user is 
aware ofDiskTimer results for nearly every given drive.  Brecher disclaims any 
similarities to realworld performance, but then what do DiskTimer results 
really mean?  Well, they aremeaningless.  Let me try to explain what disk 
interleaves and DiskTimer results really mean in thereal world.



INTERLEAVES

The ONLY reason to interleave blocks on a disk is to compensate for an 
interface that cannot handlethe full data transfer rate of the disk.  A given 
disk drive has a native transfer rate that depends onthe bit density, data 
coding and spindle speed of the drive.  If the interface of the computer is 
too slow for the  drive's data rate, then the drive's controller chokes on reads
(from disk) and starves the disk on writes (to disk). When this happens, the 
transfer must be suspended until thecomputer catches up. This kind of error is 
called a data late error. 



Suppose the computer was almost as fast as the disk.  After the first block is 
read from the disk,the computer will be a bit behind in getting the data into 
memory.  But come hell or high water,here comes the second block on the hard 
disk.  Well, the computer just isn't ready yet ... DATA LATE ERROR ... and now 
we have to wait for the disk to spin completely around until the second block 
isagain under the drive's head.  By this time the computer has been waiting for 
the second block. Since there are normally 26 blocks on each track, this means 
for every block read the computerwill have to wait for 26 more blocks to 
pass!!!



Now THIS IS IMPORTANT!!!  The block interleave is NOT a measure of performance! 
 Just becausethe Dataframe XP use 1:1 interleave and the CMS S40 uses 6:1 does 
not mean the S40 is 6 timesslower.  Actually, the native transfer rate of the 
S40 is much faster than the Dataframe XP.  TheDataframe can get away with a 1:1 
interleave because its transfer rate is slower than the S40.  The Macintosh 
cannot keep up the high speed of the S40, so it is relegated to the 
substandardperformance of the Macintosh SCSI interface.



Furthermore, 1:1 interleaving is disadvantageous due to the fact that the 
Macintosh only makessingle-block requests.  In fact, an interleave of 
approximately 6:1 is optimal for handling suchrequests.  Mr. Brecher himself 
tested this by modifying his DiskTimer (which issues multi-blockrequests) to 
issue single-block (or sector) requests.  He ran this single-sector version 
ofDiskTimer, SSDiskTimer, as well as the original DiskTimer on the same drive 
at variousinterleaves.  Below are the actual results that Mr. Brecher found.  
(lower times are better)


INTERLEAVE CHART



	Interleave	SSDiskTimer	DiskTimer

	1:1		6801		507

	2:1		7206		812

	3:1		7511		1116

	4:1		7518		1523

	5:1		6513		1827

	6:1		2538		2233

	7:1		2538		2537

	8:1		2943		2943

As Steve says, This result clearly confirms that the about 6:1 estimate for 
optimal handling ofconsecutive single-block requests.  It is also interesting 
to note that DiskTimer (withmultiple-block requests) favors a drive with a 1:1 
interleave, whereas the SSDiskTimer (withsingle-block requests) demonstrates 
better performance at 6:1.  Beside the fact the Brecher hadproven to himself
that 6:1 interleave was best, he should have realized something else about
hisprogram.  Why do the DiskTimer results go from good to bad on the 
interleave scale and DiskTimer go bad to good?  I'll explain this in a 
moment.



Tom Reilly of SuperMac states; The Mac Plus is set up for a 3:1 disk 
interleave,...  This is not accurate information.  Maybe the Apple hard disk 
for the Macintosh Plus is set up with a 3:1interleave, but every drive 
manufacturer will need to determine their own interleave factor.  The best 
interleave factor for a drive is not determine by the host computer alone.  
After a developerhas completely optimized his software/hardware interface, he 
can then test all possibleinterleaves until he achieves the desired 
performance.  Different drive configurations will havedifferent optimal 
interleaves.  Furthermore Tom states; The SCSI interface on the Plus 
requiredyou to do direct ROM calls to gain performance.  Again, this is not
accurate information.  This isSuperMac's opinion, it is not my opinion and 
certainly not recommended by Apple. (The Macintosh developer guidelines clearly 
states never do direct ROM address calls.)



I am reminded of the early Macintosh programmers that considered QuickDraw too 
slow.  Instead ofconforming to Apple's guidelines, many decided to draw 
directly on the screen (ignoring windows)or jump to ROM addresses.  This 
software blew up when Apple changed the ROMs.  These programs went from high 
performance to no performance.  We can expect Apple to continueto increase 
the performance and capabilities of the Macintosh ROMs, and those that continue
to follow the developer's guidelines will reap the benefits without having to go
back to the drawingboards to write another revision (while the customer 
waits).



When I sit in front of a Macintosh Plus with a Dataframe XP20 or a CMS S20, I 
can not observeany performance differences.  Both drives appear to run my 
applications at the same rate, eventhough DiskTimer says that the XP20 is 5.48 
times faster than my CMS S20.  Gary Eisenstien,Supermac's national sales 
manager, states the Dataframe XP is three timer faster than any otherSCSI 
drive available for the Macintosh.  This is absolutly false!  How did he ever 
come to believethis?  He probably considers DiskTimer the final word for a 
drive's performance.



One should not put much faith into interleave factors alone.  One should 
understand how they affectthe user.  But I would also like to point out that 
the FileManager of the Macintosh will only requestsingle block transfers.  It 
is the FileManager that is responsible for all file I/O.  The Macintosh will
request one block, digest it, and maybe then request another block.  The time 
the Macintoshtakes to digest a block is about the same time as it takes two 
blocks to pass underneath the drive's read/write heads.  So, a drive with 3:1 
interleave is closely in tune with the Macintosh's FileManager.  Examine the 
chain of command from application to hard disk.



Application>>FileManager>>DeviceManager>>DeviceDriver>>SCSIManager>>HDController>>HardDisk



(NOTE:  In some cases, such as Dataframes, the HDController is a completely 
separate interface board. CMS uses embedded SCSI drives, where this controller 
is integrated into the hard drive.)



In a real world situation, an application will talk to the FileManager and 
request a file.  TheFileManager needs to determine which blocks that the file 
occupies by passing a message throughthe entire chain of command to get the 
hard disk and read some directory informationone-block-at-a-time.  Once the 
FileManager has found the blocks it needs, it then requests a singleblock of 
that file, again through all the managers, to the disk.  That's one block at a 
time folks. DiskTimer, on the other hand, talks directly to the DeviceManager 
with multi-block requests. Refer to the interleave chart and look at the 
difference between single and multiple block requests. Clearly the results of 
this chart demonstrate that DiskTimer favors a drive with 1:1 interleave,and 
handicaps a drive that isn't.  I wish the other version, DiskTimer, had been 
released instead. Its results are more of a real world test because of its 
single-block accessing method.



DISKTIMER RESULTS

Steve Brecher has written a benchmark program, DiskTimer, for testing the 
performance ofSCSI hard disks on the Macintosh.  Although I respect his work, I 
have to critize his methods oftesting.  Determining which hard disk is the 
better performance drive based upon DiskTimerresults can be very misleading for 
many reasons.  For one thing, DiskTimer uses device managercalls directly.  It 
never issues calls through the FileManager, which is the method of 
allapplications in the real world.  DiskTimer goes directly to the device 
manager and requestsmultiple blocks by their block number.  Applications never
know what block numbers they need. This test is unfair and does not behave as a 
real application would.  Worst of all, DiskTimer usesthe Macintosh's internal 
SystemTicks to calculate its resulting times.  SystemTicks are in1/60th of a 
second and DiskTimer is attempting to measure times in 1/1000th of a second! 
(ie: Atypical 20MB drive's access time is 65ms and transfer rates are greater 
than 1MB/sec.)



One very important issue that is not even being considered in DiskTimer results 
is whether or notthe manufacturer's SCSI drivers are performing any 
verification of the data that was transferred. If someone were to ignore 
verifying their read/writes, then they could reduce their DiskTimerresults in 
half!  Personally, I'd rather have my data verified and ignore DiskTimer 
readings.  Most importantly, DiskTimer should never be run on a disk containing 
user data.  This program firstreads a block of your data, holds it in RAM, 
performs its test on that block, and finally writes yourdata back to the drive. 
 It continues along with this sequence block-by-block.  If you were to 
reset,power off, or get the bomb dialog during a DiskTimer test, then chances 
are you've lost data.



DISKTIMER SEEK TEST

The seek test does not utilize the number of data surfaces on the hard disk, 
the number of blocksper surface, or their size.  DiskTimer can not determine if 
an actual seek has occurred. DiskTimer's seek test, according to the source 
code that I have; performs a seek by reading a block,calculates 1MB worth of 
(512 byte) blocks, and reading that last block.  This adds two block readsto 
the seek time, which has nothing to do with seeks.  And even more importantly, 
this seek testdoes not consider rotational latency.  If the disk platter were 
required to complete an additionalrotation just to bring that second block 
under the disk heads, then it would add additional time tothe seek results.  



Moreover, the DiskTimer seek test does not consider the method of bad block 
allocation.  Some drives store its allocated blocks on every track, others 
store allocation blocks at one end of the disk.  If DiskTimer were to test 
a seek to (or across) spared blocks, it would impose unfair times
for that drive.  In order to perform a valid seek test, it would require 
a SCSI Seek command, which positions the heads without attempting to read 
data.  At best, the read time needs to be subtracted along with the actual 
incurred latency.  If one examines the DiskTimer results of the
DataFame 20, you'll see seek times ranging from 38 to 80.  How can the same 
hard disk display such variances?  The Dataframe SCSI driver software could 
not possible change the true seek performance of a hard disk.  The results 
should remain constant, because that drive has an average
access time of 65ms.  The only explanation for seeing such a variance 
is because DiskTimer's testing methods are affecting the results.



FACTS ABOUT SYSTEMTICKS

Consider the Macintosh's hardware limitations.  In the Macintosh Plus world, 
the maximumtransfer rate is 320K/second.  Disregarding any software limitations 
for the moment, theMacintosh hardware can not possibly keep up the a 1:1 
interleave from the typical hard diskoperating at greater than 1MB/second.  The 
theoretical limit is about a 3:1 interleave, as Applehas recommended.  Here is 
a quote from Inside Macintosh (Apple's bible for the Macintosh);



	WARNING:  Don't rely on the tick count being exact; it will usually
	be accurate to within one tick, but may be off by more than that.



A SystemTick is 1/60th of a second.  A hard disk rotates at 3600 RPM or 60 
times per second. Therefore, a tick count is worth one rotation of a hard disk



HYPOTHESIS

What if we tested the average drive set up with the following format.



	1:1 interleave

	512 bytes per blocks

	26 blocks per track   - or -   13k per track 



ERROR FACTORS

Tick count error is +/-1 or more.  Timing a transfer of 13k, or one rotation, 
with SystemTickscould introduce a error of 100% or more.  Because the number of 
ticks could be 0S, R1 or more!  A tick is approximately 16.7ms while some hard 
disks can seek track to track in 4ms.  A drive could seek across 4 or more 
tracks before TickCount changed.  As an analogy, using ticks tomeasure a 32k 
read/write, would be like using a car's odometer to measure lengths of feet.



CONCLUSION

I understand that DiskTimer disclaims any comparisons with real world 
performance, but considerthe inherent errors built into the results it does 
present us.  A 32k file or 64 blocks wouldrequire 2.46 tracks (not cylinders) 
of storage on the average drive.  To read or write a 32k filewould require at 
least 2.46 rotations.  SystemTicks after 2.46 rotations could be 2S or R3, 
andeither of these figures may be off +/- 1 or more.  DiskTimer will introduce 
a significant errorfor each 32k read/write test.  DiskTimer does 200 read/write 
tests and averages them to thenearest 60th of a second.  Then this figure is 
converted to the nearest 10th of a second.  Thereforeany accuracy resulting 
with this program, in the real or un-real DiskTimer world, would bepurely 
coincidental.



To summarize, all DiskTimer results have been invalidated because of;



	1.  Multi-block requests

	2.  Favors a drive with 1:1 interleave 

	3.  Ignores the FileManager calls

	4.  Does not perform a true seek test

	5.  Uses SystemTicks for calculating timing data



Q.E.D.

Jim Reekes

(Acknowledgements to Bob Denny's material on disk interleave, Steve Brecher's 
source code, and other documentation that I have quoted and used as my source 
material.)


----------------------------