🚀 go-pugleaf

"MWP" <mwp@overclockers.com.au> wrote:

>But i am accessing shared memory from external apps using OpenFileMapping and MapViewOfFile a lot (up to every 5ms).
>Would this cause these page faults?

Oh, almost definitely yes.

When you use these functions, the OS reserves a memory range, and maps
that range onto the file. As far as your used code is concerned, that
file appears to be in RAM, but it actually isn't.  Every 'page' of
memory (about 4K in size last time I looked) is noted as being 'not
there'.

(The processor uses a lookup table for this - for this purpose, it needs
1 bit per 4K of addressable memory, or 128K for 4GB.)

When you access a page for the first time, effectively the processor
checks the bit. If it says the memory isn't currently mapped, the
processor generates a page fault.

At this point, Windows catches the exception, and tries to do something
about it. In this case, it grabs a page of actual RAM, maps it to the
address required, loads the relevant data from the file into that page,
flips the bit to now say it's there, and tells the processor to restart
the instruction that caused the problem.

Oh, yes, "maps it to the address required". The address that the program
sees and that the processor actually uses for a page of memory is
different. This means another lookup table that the processor uses.

Of course, actual RAM isn't infinite, and sooner or later Windows will
need the actual RAM for something else. It'll unload the RAM, note the
address as being 'not there' again, and use the actual RAM for the
something else.

(The above ignores the fun and games of what happens when you or
something else modifies the mapped file data. If more than one process
is mapping the same file, the two different process may look at the same
page of actual RAM, but think they're at different addresses.)

And now to the conclusion:

What can cause a huge number of page faults is mapping a file at least
as large as your actual free RAM, and then repeatedly dancing from one
end to the other in steps of the page size - as each new chunk of the
file is touched, it has to be loaded, only to be discarded before it
gets needed again. For this reason, when choosing algorithms to be used
with such virtual memory, you want to try to keep them using addresses
in the same area as much as possible.

Also successfully allocating that sort of size of RAM may do the same
thing, since program memory is usually backed by 'virtual memory', which
is exactly this sort of file mapping, onto a temporary bit of file.

This is also why Windows usually grinds to a halt before actually
'running out of memory'. It spends its time continually loading and
discarding virtual memory.

Alan Bellingham
-- 
alanb@episys.com