🚀 go-pugleaf

In article <syscrc.726293940@gsusgi1.gsu.edu> syscrc@pickle.gsu.edu (Randy Carpenter) writes:
>We have a 4d/380s with 128MB of physical memory.  It's used for
>computational chemistry.  Although there's plenty of available memory,
>long running processes never seem to stay completely resident as evident
>from the top(1) listing below.  Why does this happen?

The short answer is that only the recently referenced memory pages stay
resident.  The OS always keeps track of what is has not being referenced 
recently and makes these pages available for other uses.  Just because
you have lots of empty empty space isn't a good reason to keep around a lot
of junk you may never look at again.

>
>Ocassionally, these processes will be killed due to insufficient swap
>storage although we think there is plenty of available memory.  The
>current swap is 48MB and we will be expanding it to 128MB soon.
>

I believe that if you have a process which has grown larger than the size
of free swap space and the OS chooses to swap it out, it must swap it out in
its entirety and at that point it has no choice but to kill the process.
40Mb is a little small, 120Mb is much better but it depends on the mix
you run.  /etc/swap -l will give you an idea of how much swap space is
actually being used ...
	-db

>% top
>
>IRIX chemistry 4.0.1 11150233 IP7 Load[18.03,18.09,18.09] 23:13:15  500 procs
>    user   pid  pgrp  %cpu proc  pri  size   rss    time  command        
>  david   5641   381  56.8    5  105  3762  1656 4075:36  sybyl.40exe
>  hui     7331   278  54.5    0  100 14596  1611 2570:55  charmm21r3exe
>  david   5644   381  53.4    6  105  3762  1656 4467:02  sybyl.40exe
>  david   5643   381  52.3    2  105  3762  1656 4348:27  sybyl.40exe
>  david    469   381  52.3    *  105  3758  1652 4799:24  sybyl.40exe
>  david    470   381  52.3    *  105  3758  1652 4934:03  sybyl.40exe
>  david    468   381  52.3    *  105  3758  1652 4657:29  sybyl.40exe
>  david    467   381  51.1    *  105  3758  1652 4510:05  sybyl.40exe
>  david   5640   381  48.9    1  105  3762  1656 3890:10  sybyl.40exe
>  david   5642   381  48.9    4  105  3762  1656 4215:52  sybyl.40exe
>  tom    22880 15490  45.5    *  104  8390  2578 1060:34  sp_property
>  david    466   381  40.9    *  105  3758  1652 4321:41  sybyl.40exe
>  david   5639   381  39.8    *  105  3762  1656 3598:38  sybyl.40exe
>  david    465   381  37.5    *  105  3758  1652 4023:00  sybyl.40exe
>  david   5638   381  31.8    7  105  3762  1656 3130:09  sybyl.40exe
>  david    464   381  29.5    *  105  3758  1652 3548:53  sybyl.40exe
>  david   5637   381  28.4    *  105  3762  1656 2622:47  sybyl.40exe
>  david    411   381  23.9    *  105  3758  1652 3042:21  sybyl.40exe
>%
>-- 
>===========================================================================
>Randy Carpenter                    rcarpent@gsu.edu          % Got a light?
>Georgia State University           (404) 651-2648            No match.
>Wells Computer Center                                        %

In article <1993Jan6.072923.9471@odin.corp.sgi.com> blythe@sgi.com (David Blythe) writes:
>In article <syscrc.726293940@gsusgi1.gsu.edu> syscrc@pickle.gsu.edu (Randy Carpenter) writes:
>>We have a 4d/380s with 128MB of physical memory.  It's used for
>>computational chemistry.  Although there's plenty of available memory,
>>long running processes never seem to stay completely resident as evident
>>from the top(1) listing below.  Why does this happen?
>
>The short answer is that only the recently referenced memory pages stay
>resident.  The OS always keeps track of what is has not being referenced 
>recently and makes these pages available for other uses.  Just because
>you have lots of empty empty space isn't a good reason to keep around a lot
>of junk you may never look at again.

Really??  As long as there are free pages, I don't think an in-use page
would ever get swapped out, no matter how long it had been since it was
last touched.  Or do you just mean that transient memory usage by the OS
might occasionally fill up the really free space and cause some old stuff
to get swapped out?

>>Ocassionally, these processes will be killed due to insufficient swap
>>storage although we think there is plenty of available memory.  The
>>current swap is 48MB and we will be expanding it to 128MB soon.
>>
>I believe that if you have a process which has grown larger than the size
>of free swap space and the OS chooses to swap it out, it must swap it out in
>its entirety and at that point it has no choice but to kill the process.
>40Mb is a little small, 120Mb is much better but it depends on the mix
>you run.  /etc/swap -l will give you an idea of how much swap space is
>actually being used ...

Irix shouldn't swap out whole processes at a time -- swapping is done
page by page.

>>% top
>>
>>IRIX chemistry 4.0.1 11150233 IP7 Load[18.03,18.09,18.09] 23:13:15  500 procs
>>    user   pid  pgrp  %cpu proc  pri  size   rss    time  command        
>>  david   5641   381  56.8    5  105  3762  1656 4075:36  sybyl.40exe
>>  hui     7331   278  54.5    0  100 14596  1611 2570:55  charmm21r3exe
>>  tom    22880 15490  45.5    *  104  8390  2578 1060:34  sp_property
...and 15 more jobs with rss ~= 1600

If you add up the rss fields, it looks like there are about 30600 4K
pages of physical memory in use by these large jobs, or about 120 MB.
So in fact, if this is a typical load, there is no free memory after
all, since the OS should take up the remaining space.  The size fields
are much larger than the rss's, so most of the memory allocated by
these jobs must never be touched (otherwise they would overwhelm the
48 MB of swap space).  This is typical of Fortran programs with large
static array declarations.

        - David Hinds
          dhinds@allegro.stanford.edu

dhinds@leland.Stanford.EDU (David Hinds) writes:

> [stuff deleted...]
>
>If you add up the rss fields, it looks like there are about 30600 4K
>pages of physical memory in use by these large jobs, or about 120 MB.
>So in fact, if this is a typical load, there is no free memory after
>all, since the OS should take up the remaining space.  The size fields
>are much larger than the rss's, so most of the memory allocated by
>these jobs must never be touched (otherwise they would overwhelm the
>48 MB of swap space).  This is typical of Fortran programs with large
>static array declarations.
>
>        - David Hinds
>          dhinds@allegro.stanford.edu

Whoops.  I sampled a period when we were loaded to the gills.  I'm almost
certain this has happened in a period of low activity...perhaps I'm wrong.
Sorry.

-- 
===========================================================================
Randy Carpenter                    rcarpent@gsu.edu          % Got a light?
Georgia State University           (404) 651-2648            No match.
Wells Computer Center                                        %