🚀 go-pugleaf

In <1iueqiINN6kr@fido.asd.sgi.com> knobi@knobi.munich.sgi.com (Martin Knoblauch) writes:
>In article <1993Jan12.104142.1311@wega.rz.uni-ulm.de>,
>huober_r@hal.rz.uni-ulm.de (Reiner Huober) writes:
>|> We use a 4D35 with biosym software on it. This software has a
>|> help directory with over 3900 file, one or two blocks in size each.
>|> If you copy or remove such a directory, then it takes several hours.
>|> 
>|> Is this specific to IRIX or have other systems the same behaviour
>|> if there are many files in one directory.

>  You are exceeding the disk buffer cache. It is usually not a
>good idea to have thousands of files in one directory. The directory
>gets so big, that it does not fit into the memory reserved by
>the kernel. As a result, each access to any single inode in this
>directory will directly go to disk, which is very slow. So if you
>want to copy a directory with 3900 files, each file a few inodes,
>it can take hours. All OS see this effect more or less drastic.
>The limit is about 1000 files for a "normal" IRIX kernel. When
>you exceed this limit, you will see the performance degradation
>that you describe. If you have 'sar', the system activity reporter,
>installed, you can try to monitor the system using 'sar -u' and
>'sar -b'. If the former shows high percentages of %wio and %wfs,
>this indicates a disk buffer cache problem. If 'sar -b' also
>shows low utilisation of 'rcache' and 'wcache', you have another
>indicator.

>  To improve your situation, you can try to build a new kernel with
>an increased disk buffer cache. To do so, log in as 'root' and
>cd into '/usr/sysgen/master.d'. Edit file 'kernel'. Look for a
>line containing '#define NBUF'. Under IRIX 4.0.x NBUF is set to
>0 for autoconfiguration. Replace 0 by some large nuber like 1024
>(I believe the maximum is 1500). Then rebuild the kernel using the
>'autoconfig' command and reboot.

Just as a note (because I've been burned by this one recently),
the maximum for NBUF under 4.0.X is not 1500.  There is no set
maximum.  In addition to increasing NBUF, you might also want to
consider increasing NHBUF as well.  Make sure that NHBUF, however,
is a power of two.  A recommendation to try is setting NBUF to
1024 and NHBUF to 256, reconfiguring your kernel, and testing.
Values to try beyond that are NBUF set to 2048 and NHBUF set to 512.
Give those a shot.  It should increase your directory performance.
If you have more questions, give the hotline a call for some more
recommendations.

--Matt

---------------------------------------------+-------------------------------
Matt D. Robinson (Member, Technical Staff)   |      mattr@awesome.csd.sgi.com
Silicon Graphics, Inc.                       |      mattr@leland.stanford.edu
--
---------------------------------------------+-------------------------------
Matt D. Robinson (Member, Technical Staff)   |      mattr@awesome.csd.sgi.com
Silicon Graphics, Inc.                       |      mattr@leland.stanford.edu