🚀 go-pugleaf

John Jay Feiler writes
> I've got a large image that I need to keep all in ram, at all times.  I've  
> used get/setrlimit(RLIMIT_DATA,rtp) calls to set the data limit to the  
> maximum, and that definitely seems to help ...

More physical memory would certainly help, but basically all you have to do to  
keep it in ram is to touch the page frequently enough. If you don't touch it  
and some other application needs the physical memory then the other application  
gets it.   If you're not accessing it frequently enough to keep it in memory,  
then it seems to me that it doesn't really need to be in memory.  That's how  
demand paging systems work, and usually any attempt to fool them results in  
slightly increased performance for one particular application at the expense of  
all the others.   

> ... but is there any way to wire down  
> pages of ram from an application?  If that isn't possible, would it be  
> possible to write a LKS that wired down the pages for the image, and made  
> that chunk of ram available to another app?

I think you're missing the point.  Wiring a page means making it unavailable
for participation in the pool of pages used for demand paging.  If you should  
find a way to wire the page, you sure won't have "that chunk of ram available  
to another app."   Quite the opposite;  that chunk of RAM will be dedicated.

I've done some awful kludgy things in the past, including some code for a  
former employer that was called "belly warmer" code -- it deliberately 
forced pages to be assigned to physical pages by periodically touching at least  
one byte in the page (thus keeping them warm).  This was in the early days,  
when we used 68000 cpu's that couldn't handle their own page faults.  So, the  
solution was to freeze the clocks of the user cpu, interrupt the exec cpu (that  
always executed wired code) and have the exec cpu resolve the page fault, at  
which time the user CPU could be restarted.   So, you see, page faults were  
VERY expensive, and we went to great lengths to prevent them.   However, the  
consensus, after some time, some newer hardware (68010), and a few more  
releases of the OS, was that attempts to defeat the LRU demand paging algorithm  
were generally detrimental to overall system throughput and response time.

I hope I've convinced you to look elsewhere for a solution.

--
L. W. "Dan" Danz         (WA5SKM) VOS Mail:  Dan_Danz@vos.stratus.com
Sr Consulting Software SE         NeXT Mail: dan@az.stratus.com
Customer Assistance Center        Voice Mail/Pager: (602) 852-3107
Telecommunications Division       Customer Service: (800) 828-8513
Stratus Computer, Inc. 4455 E. Camelback #115-A, Phoenix AZ 85018

In article <1992Oct28.075558.2721@relief.com> jjfeiler@relief.com writes:
>I've got a large image that I need to keep all in ram, at all times.  I've  
>used get/setrlimit(RLIMIT_DATA,rtp) calls to set the data limit to the  
>maximum, and that definitely seems to help, but is there any way to wire down  
>pages of ram from an application?  If that isn't possible, would it be  

look at kalloc()/kfree()

-- 
Sean T. Lamont                           |   "Don't oppress me, It's
zeno@genetics.washington.edu             |    'computer of color!' "
lamont@abstractsoft.com                  |_______________________________
Abstract Software                                                         

In article <1992Oct28.075558.2721@relief.com> jjfeiler@relief.com writes:
>I've got a large image that I need to keep all in ram, at all times.  I've  
>used get/setrlimit(RLIMIT_DATA,rtp) calls to set the data limit to the  
>maximum, and that definitely seems to help, but is there any way to wire down 
>pages of ram from an application?  If that isn't possible, would it be  
>possible to write a LKS that wired down the pages for the image, and made  
>that chunk of ram available to another app?

Try creating an LKS, allocating the memory with vm_allocate, and
passing a pointer to the memory through a mach-message to the
revieving app.  That _should_ do the trick.  The memory will be mapped
copy-on-write in the receiver's address space.  It should be wired
down, but don't write to it or a non-wired down copy will be
created...


andrew
-- 
andrew@cubetech.com     | "We shall not cease from exploration
Andrew Loewenstern      |  And the end of our exploring
Cube Technologies, Inc. |  Will be to arrive where we started
                        |  And know the place for the first time." -T.S. Eliot

In article <1992Oct29.082747.825@cubetech.com> andrew@cubetech.com (Andrew Loewenstern) writes:
>In article <1992Oct28.075558.2721@relief.com> jjfeiler@relief.com writes:
>>I've got a large image that I need to keep all in ram, at all times.  I've  
>>used get/setrlimit(RLIMIT_DATA,rtp) calls to set the data limit to the  
>>maximum, and that definitely seems to help, but is there any way to wire down 
>>pages of ram from an application?  If that isn't possible, would it be  
>>possible to write a LKS that wired down the pages for the image, and made  
>>that chunk of ram available to another app?
>
>Try creating an LKS, allocating the memory with vm_allocate, and
>passing a pointer to the memory through a mach-message to the
>revieving app.  That _should_ do the trick.  The memory will be mapped
>copy-on-write in the receiver's address space.  It should be wired
>down, but don't write to it or a non-wired down copy will be
>created...
>
>
>andrew
>-- 
>andrew@cubetech.com     | "We shall not cease from exploration
>Andrew Loewenstern      |  And the end of our exploring
>Cube Technologies, Inc. |  Will be to arrive where we started
>                        |  And know the place for the first time." -T.S. Eliot

Oh yeah, it probably wouldn't be a really great idea to allocate a
whole lot of wired-down pages.  I don't think you can _ever_ reclaim
them and can really make the machine thrash if you take too much.


andrew
-- 
andrew@cubetech.com     | "We shall not cease from exploration
Andrew Loewenstern      |  And the end of our exploring
Cube Technologies, Inc. |  Will be to arrive where we started
                        |  And know the place for the first time." -T.S. Eliot

John Jay Feiler writes
>> I've got a large image that I need to keep all in ram, at all times.

dan@az.stratus.com writes:

> any attempt to fool [the OS demand paging] results in slightly increased performance for
> one particular application at the expense of  all the others.   

> the consensus [...] was that attempts to defeat the LRU demand paging algorithm  
> were generally detrimental to overall system throughput and response time.

Dan gives alot of good arguments why its usually not a good idea to
fight the demand paging system.  It does pay to understand how it works so you can
take better advantage of it.  That means it pays to learn about zones, 
copy on write optimizations and perhaps OS tuning parameters.

There is one very legitimate reason for wanting to wire a page in physical
memory irregardless of how frequently its accessed.  Thats is when it is needed
to support a task with hard real time deadlines.  Here the need is for predictability
rather than speeed. Demand paging introduces uncertain delays in individual cases even if it
increases the overall performance.  This uncertainty can make it impossible to prove that
you will meet your deadlines.  (most of the time is not good enough)

except in this special case, where you are using your computer (typically) to
control or track some physical system with hard real-time constraints,
Dan's advice is quite sound. 

-- 
---------------------------------------------------
Alex Blakemore alex@cs.umd.edu   NeXT mail accepted

In article <1992Oct29.071919.26859@u.washington.edu> zeno@phylo.genetics.washington.edu (Sean Lamont) writes:
>In article <1992Oct28.075558.2721@relief.com> jjfeiler@relief.com writes:
>>I've got a large image that I need to keep all in ram, at all times.  I've
>>used get/setrlimit(RLIMIT_DATA,rtp) calls to set the data limit to the
>>maximum, and that definitely seems to help, but is there any way to wire down
>>pages of ram from an application?  If that isn't possible, would it be
>
>look at kalloc()/kfree()

    From a user program?

    The only get/setrlimit() commands that are implemented are for the
    stack and core dump sizes.	No other parameters are implemented (bug
    report has been sent to bug_next in that regard).  Besides, the
    *rlimit() calls have nothing to do with locking pages in memory nor
    paging.

    The only way I know of to wire pages in memory is through
    kern_serv_wire_range() but it is totally undocumented.  I've never
    tried it myself and it probably wouldn't work from a user process.

    Touching pages probably will not work, the mach paging system doesn't
    have much regard for recent page usage.

    I think you will probably have to get more memory for the machine.

					    -Matt

>--
>Sean T. Lamont 			  |   "Don't oppress me, It's
>zeno@genetics.washington.edu		  |    'computer of color!' "
>lamont@abstractsoft.com		  |_______________________________
>Abstract Software

--

    Matthew Dillon		dillon@Overload.Berkeley.CA.US
    1005 Apollo Way		uunet.uu.net!overload!dillon
    Incline Village, NV. 89451	ham: KC6LVW (no mail drop)
    USA 			Sandel-Avery Engineering (702)831-8000

dillon@overload.Berkeley.CA.US (Matthew Dillon) writes:
>    The only way I know of to wire pages in memory is through
>    kern_serv_wire_range() but it is totally undocumented.  I've never
>    tried it myself and it probably wouldn't work from a user process.

I'll be the first to admit that NeXT's documentation, particularly about
loadable kernel servers is _badly_ lacking, but can we stick to facts when
we're NeXT-bashing please?

kern_serv_wire_range() is documented: check in NeXT Developer of the Digital
Librarian. As to whether it will solve the original poster's problem or not...

Graham

In article <Bx2D2p.71B@research.canon.oz.au> greyham@research.canon.oz.au (Graham Stoney) writes:
>dillon@overload.Berkeley.CA.US (Matthew Dillon) writes:
>>    The only way I know of to wire pages in memory is through
>>    kern_serv_wire_range() but it is totally undocumented.  I've never
>>    tried it myself and it probably wouldn't work from a user process.
>
>I'll be the first to admit that NeXT's documentation, particularly about
>loadable kernel servers is _badly_ lacking, but can we stick to facts when
>we're NeXT-bashing please?
>
>kern_serv_wire_range() is documented: check in NeXT Developer of the Digital
>Librarian. As to whether it will solve the original poster's problem or not...
>
>Graham

    Oops!  Your right... I had it mapped out because searching through that
    section tends to crash librarian.  I'll try indexing it again...

					    -Matt

--

    Matthew Dillon		dillon@Overload.Berkeley.CA.US
    1005 Apollo Way		uunet.uu.net!overload!dillon
    Incline Village, NV. 89451	ham: KC6LVW (no mail drop)
    USA 			Sandel-Avery Engineering (702)831-8000