🚀 go-pugleaf

In article <5180@crdgw1.crd.ge.com> hammondr@sunroof.crd.ge.com (Richard A Hammond) writes:
>Everybody seems to be missing the crucial fact about memory mapped files!
>
>They ONLY work for cases where the file size is < virtual address space!!!
>
>They are not a a generally useful solution which works over all possible
>implementations.  UNIX file I/O does work over all situations!

As has already been pointed out, the case of a file that is larger
than the virtual address space is relatively infrequent on all but
very small machines such as PDP-11's.  I don't think anyone considers
a PDP-11 to be a modern platform.

Furthermore, I hope that no one would implement file/memory mapping
that only mapped the entire file in.  You should be able to map n
"pages" or "segments" from "page"/"segment" i of source to
"page"/"segment" j of destination.

It may be very common for n = size of file, i = 0, and j = free chunk
of memory big enough to hold the file.  However, this is not the only
form of file/memory mapping you may want to do.

I suppose it is also known that mapping in itself does not do any I/O
unless you also specify "preloading".  The overhead of mapping 1 page
should be more or less the same as mapping 100,000.

I'll acknowledge this requires some knowledge of page or segment
sizes, but that can be done in a .h file so you can write portable
code.  A good way (which is what TOPS-20 used) is to have the
filesystem allocation unit be the same as the memory allocation unit.
As long as your code is careful to calculate the number of units based
on the page size instead of assuming that it is 512 or 1024 or
whatever, it's portable.

For historical reference, here's a description of TOPS-20's memory
mapping system call.  Note that you could use either a file or a
process for source or destination, as well as have a bi-directional
file/memory map.  No file/file mapping though.  :-) :-) Disc I/O under
TOPS-20 was simply byte copies to an operating system buffer that was
PMAPed.  Direct PMAP I/O was much faster than disc I/O!

                              PMAP     JSYS 56

   Maps one or more complete pages from a file to a process (for  input),
   from  a process to a file (for output), or from one process to another
   process.  Also unmaps pages from a process and deletes  pages  from  a
   file.  Each of the five uses of PMAP is described below.

 
   Case I:  Mapping File Pages to a Process

   This use of the PMAP call does not actually  transfer  any  data;   it
   simply  changes  the  contents of the process' page map.  When changes
   are made to the  page  in  the  process,  the  changes  will  also  be
   reflected  in the page in the file, if write access has been specified
   for the file.

   ACCEPTS IN AC1:  JFN of the file in the left half, and the page number
                    in  the file in the right half.  This AC contains the
                    source.

              AC2:  process handle in the left half, and the page  number
                    in  the  process in the right half.  This AC contains
                    the destination.

              AC3:  B0(PM%CNT)  A count is in  the  right  half  of  AC3.
                                This   count   specifies  the  number  of
                                sequential pages to be mapped.   If  this
                                bit is not set, one page is mapped.

                    B2(PM%RD)   Permit read access to the page.

                    B3(PM%WR)   Permit write access to the page.

                    B4(PM%EX)   Reserved for future use.
                                The symbol PM%RWX  can  be  used  to  set
                                B2-4.

                    B5(PM%PLD)  Preload the page being mapped  (move  the
                                page immediately instead of waiting until
                                it is referenced).

                    B9(PM%CPY)  Create a private copy of the page when it
                                is  written into (copy-on-write).  If the
                                page  is  mapped  between  two  processes
                                (Case  III  below),  both  processes will
                                receive a private copy of the page.

|                   B10(PM%EPN) The  right  half  of  AC2   contains   an
|                               extended  process  page  number.   If the
|                               section  containing  the  page  does  not
|                               exist,  an  illegal  instruction  trap is
|                               generated.

                    B11(PM%ABT) Unmap  a  page  and  throw  its   changed
                                contents  away.   This bit is significant
                                only when unmapping  process  pages  that
                                were  mapped  from  a  file  (see case IV
                                below) and OF%DUD is set in the OPENF.

                                Normally, if a page is unmapped  and  has
                                been  changed  since  the  last  time the
                                monitor updated the associated file page,
                                the monitor will remove the page from the
                                process and place it on a queue in  order
                                to  update  the file page.  PM%ABT allows
                                the page to be unmapped, but prevents the
                                monitor  from  placing  the  page  on the
                                update queue.

                                This feature is useful  in  the  case  of
                                erroneous  data  written to a mapped page
                                of a file open for  simultaneous  access.
                                In  this  case,  it is important that the
                                erroneous page be discarded  and  not  be
                                used  to  update  the file page.  Another
                                application  is  to  allow  processes  in
                                separate jobs to communicate by sharing a
                                file page (and reading/writing the  page)
                                and  avoid  the  overhead  of the monitor
                                periodically updating the page.

                    B18-35      Number of pages to be mapped if
                    (PM%RPT)    B0(PM%CNT) is set.

   RETURNS     +1:  always

   This use of PMAP changes the map of the process such that addresses in
   the  process page specified by the right half of AC2 actually refer to
   the file page specified  by  the  right  half  of  AC1.   The  present
   contents  of the process page are removed.  If the page in the file is
   currently nonexistent, it will be created when it is written (when the
   corresponding page in the process is written).  If the process page is
   in a nonexistant section, an illegal instruction trap is generated.

   This use of PMAP is legal only if the file is opened for at least read
   access.  The access bits specified in the PMAP call are ANDed with the
   access  that  was  specified  when  the  file  was  opened.   However,
   copy-on-write is always granted, regardless of the file's access.  The
   access granted is placed in the process'  map.   The  file  cannot  be
   closed  while  any  of  its  pages are mapped into any process.  Thus,
   before the file is closed, pages must be unmapped from each process by
   a PMAP call with -1 in AC1 (see below).

 
   Case II Mapping Process Pages to a File

   This use of the PMAP  call  actually  transfers  data  by  moving  the
   contents of the specified page in the process to the specified page in
   the file.  The process' map for that page becomes empty.

   ACCEPTS IN AC1:  process handle in the left half, and the page  number
                    within  the  process  in  the  right  half.   This AC
                    contains the source.

              AC2:  JFN of the file in the left half, and the page number
                    within  the file in the right half.  This AC contains
                    the destination.

              AC3:  access bits and repetition count.  (Refer to Case I.)

   RETURNS     +1:  always

   The process page and  the  file  page  must  be  private  pages.   The
   ownership  of  the  process page is transferred to the file page.  The
   present contents of the page in the file is deleted.

   The access granted to the file page is determined by ANDing the access
   specified in the PMAP call with the access specified when the file was
   opened.  This function does not update the file's  byte  size  or  the
   end-of-file  pointer in the file's FDB.  Failure to update these items
   in the FDB can prevent the reading of the file by sequential I/O calls
   such as BIN and BOUT.

   To update the file's FDB  after  using  this  PMAP  function,  do  the
   following:

        1.  Use the CLOSF call with the CO%NRJ bit set to close the  file
            but keep the JFN.

        2.  Use the CHFDB call to update the end-of-file pointer and,  if
            necessary, the byte size in the file's FDB.

        3.  Use the RLJFN call to release the JFN.

   (Refer to Section 2.2.8 for the format of the FDB fields.)

   Case III Mapping One Process' Pages to Another Process

   This use of the PMAP call normally does not  transfer  any  data;   it
   simply  changes  the contents of the page maps of the processes.  When
   changes are made to the page in one process, the changes will also  be
   reflected in the corresponding page in the other process.

   ACCEPTS IN AC1:  process handle in the left half, and the page  number
                    in  the  process in the right half.  This AC contains
                    the source.

              AC2:  a second process handle in the left  half,  and  page
                    number  in  that  process in the right half.  This AC
                    contains the destination.

              AC3:  access bits and repetition count.  (Refer to Case I.)

   RETURNS     +1:  always

   This use of PMAP changes the map of the destination process such  that
   addresses  in  the  page  specified  by the right half of AC2 actually
   refer to the page in the source process specified by the right half of
   AC1.  The present contents of the destination page are deleted.

   The access granted to the destination page is determined by the access
   specified  in  the  PMAP  call.   If  the  destination  page  is  in a
   nonexistant section, the  monitor  generates  an  illegal  instruction
   trap.

   Case IV Unmapping Pages In a Process

   As stated previously, a file cannot be closed if any of its pages  are
   mapped in any process.

   ACCEPTS IN AC1:  -1

              AC2:  process handle in the  left  half,  and  page  number
                    within the process in the right half

              AC3:  B0(PM%CNT)  Repeat  count.   Only  the  process  page
                                numbers are incremented.

                    B18-35     Number of pages to remove from process

   This format of the PMAP call removes the pages indicated in  AC2  from
   the process.

   A page that was locked with the PLOCK call may be unmapped.  Doing  so
   will  unlock  the  process'  page and return the now unlocked physical
   page to its previous state.

   Case V Deleting One or More Pages from a File

   Deletes one or more pages from a file on disk and does not affect  the
   address space of any process.

   ACCEPTS IN AC1:  -1

              AC2:  JFN of the file in the  left  half  and  page  number
                    within the file in the right half.

              AC3n: B0(PM%CNT)  Indicates that the  right  half  contains
                    the number of pages to delete.

                    B18-35      Number of pages to delete from file

   Illegal PMAP calls

   The PMAP call is illegal if:

        1.  Both AC1 and AC2 designate files.

        2.  Both AC1 and AC2 are 0.

        3.  The PMAP call designates a file with write-only access.

        4.  The PMAP call designates a file with append-only access.

        5.  The source and/or the destination designates an  execute-only
            process and the process is not self (.FHSLF).

   Can cause several software interrupts on certain file conditions.

   Generates an illegal instruction interrupt on error conditions below.

   PMAP ERROR MNEMONICS:

   ARGX06:   Invalid page number
   CFRKX3:   Insufficient system resources
   DESX1:    Invalid source/destination designator
   DESX3:    JFN is not assigned
   DESX5:    File is not open
   DESX7:    Illegal use of parse-only JFN or output wildcard-designators
   FRKHX1:   Invalid process handle
   FRKHX2:   Illegal to manipulate a superior process
   FRKHX3:   Invalid use of multiple process handle
   FRKHX7:   Process page cannot exceed 777
   FRKHX8:   Illegal to manipulate an execute-only process
   IOX11:    Quota exceeded
   IOX34:    Disk full
   IOX35:    Unable to allocate disk - structure damaged
   LNGFX1:   Page table does not exist and file not open for write
   PMAPX1:   Invalid access requested
   PMAPX2:   Invalid use of PMAP
   PMAPX3:   Illegal to move shared page into file
   PMAPX4:   Illegal to move file page into process
   PMAPX5:   Illegal to move special page into file
   PMAPX6:   Disk quota exceeded
   PMAPX7:   Illegal to map file on dismounted structure
   PMAPX8:   Indirect page map loop detected
 _____     ____ ---+---   /-\   Mark Crispin           Atheist & Proud
 _|_|_  _|_ ||  ___|__   /  /   6158 Lariat Loop NE    R90/6 pilot
|_|_|_| /|\-++- |=====| /  /    Bainbridge Island, WA  "Gaijin! Gaijin!"
 --|--   | |||| |_____|   / \   USA  98110-2098        "Gaijin ha doko ka?"
  /|\    | |/\| _______  /   \  +1 (206) 842-2385      "Niichan ha gaijin."
 / | \   | |__| /     \ /     \ mrc@CAC.Washington.EDU "Chigau. Gaijin ja nai.
kisha no kisha ga kisha de kisha-shita                  Omae ha gaijin darou."
sumomo mo momo, momo mo momo, momo ni mo iroiro aru    "Iie, boku ha nihonjin."
uraniwa ni wa niwa, niwa ni wa niwa niwatori ga iru    "Souka. Yappari gaijin!"