🚀 go-pugleaf

Is there any word on 2.0? Going through previous mail from last
January, it appears that the original plan was to have something
out by now.

On a related note......
Because Khoros 1.5 does not make full use of Sun's virtual memory,
Mark Young posted a note about some updated read/write image routines.
(He also warned that they might break other things) Did anyone
get these routines from him?

Mathew Yeates

I got the following info from the waisserver at henry.jpl.nasa.gov,
port 1025, database=khorosmail


-------------------------------------------------------------------
From khoros-mailer  Wed Feb 26 14:06:01 1992
Return-Path: <khoros-mailer>
Received: by chama.eece.unm.edu (4.1/VISION/16Oct91)
	id <9202262107.AA15657@chama.eece.unm.edu>; Wed, 26 Feb 92 14:07:40 MST
Received: from blackstone.eece.unm.edu by chama.eece.unm.edu (4.1/VISION/16Oct91)
	id <9202262106.AA15643@chama.eece.unm.edu>; Wed, 26 Feb 92 14:06:01 MST
Received-Date: Wed, 26 Feb 92 14:06:01 MST
Received: by blackstone.eece.unm.edu (5.57/CLASSES/28Dec90)
	id <9202262104.AA05266@blackstone.eece.unm.edu>; Wed, 26 Feb 92 14:04:42 -0700
Posted-Date: Wed, 26 Feb 1992 14:04:40 -0700
Message-Id: <9202262104.AA05266@blackstone.eece.unm.edu>
From: young@blackstone.eece.unm.edu (Mark Young)
Date: Wed, 26 Feb 1992 14:04:40 -0700
In-Reply-To: klamer@mi.eltn.utwente.nl (Klamer Schutte)
       "Re: strange use of Sun virtual memory" (Feb  7,  9:04am)
X-Mailer: Mail User's Shell (7.2.3 5/22/91)
To: klamer@mi.eltn.utwente.nl (Klamer Schutte), khoros@chama.eece.unm.edu
Subject: Re: strange use of Sun virtual memory

Hi Mathew & Klamer,

   This is a discussion about three weeks ago about the use of readimage()
and why it is mallocing rather than simply attaching to the data pointer
provided by shared memory and sun virtual memory transports.  The following
is the letter sent of which both Mathew and Klamer responded about the use
of MAP_PRIVATE instead of MAP_SHARED when using Sun Virtual Memory.

   I also want to add that after receiveing mail i asked our local Sun office's
experts (Eric Liefeld and Scott Caruso) for more information about Sun's Virtual
Memory.  They kindly sent me two white papers about the implementation and
architecture design of Sun's Virtual Memory.  If anyone wants these papers
please mail me and i'll send you a copy.  

   Below is the original letter and then follows Klamer's and Mathew's
responses.

>On Feb 6,  6:54pm, Mathew Yeates wrote:
>} Subject: strange use of Sun virtual memory
>} From: mathew@elroy.jpl.nasa.gov (Mathew Yeates)
>} 
>} I changed my calls from readimage("foo") to readimage(sunvm=foo) but
>} noticed that I was still sucking up swap space. Here's why.
>} 
>} 
>} The net result of all this is the same as if you had just read from
>} the file descriptor. I can find no reason for doing this (is it faster
>} or something?) What I really want is something like
>} 
>} open foo and get file descriptor fd
>} map memory to foo, using fd, and get the handle addr
>} set imagedata = addr. NO MALLOC HERE.
>} 
>} then when I free the image I will unmap the pages. This seems more in the
>} spirit of Suns virtual memory and actually buys you something.  Mark,
>} am I missing something?
>} 
>} 
>} --- End of forwarded message from mathew@elroy.jpl.nasa.gov (Mathew Yeates)
>
>No you are exactly correct, but there is a subtle reason why we are still
>malloc'ing the data rather than directly attaching the memory mapped data
>or shared memory segment.  The transport routines are working correctly, it
>is the readimage()/writeimage() utilities that have not been updated.  This
>would not be very hard change to make, and i have a version that does exactly
>this.  As to the problem, it is the one that you have pointed out in the past
>The vipl routines were designed a while back so that an images input was
>(internally) re-used as it's output.  This is major reason why we will be
>changing all the ipl calling parameters for Khoros 2.0.
>
>Currently the transport mechanisms can attach to a sun virtual memory map or
>a shared memory segment, but it shouldn't allow a routine to re-use the input
>as the output.  What would happen is that it would directly change the image
>shared memory or in the case of sun virtual memory on disk.  I think most 
>would be frustrated to have their input image corrupted.  Or in the case of
>Sun Virtual Memory when a file is opened with kopen() for reading if you try
>to write to the data a bus error is generated.  For this reason we have decided
>to wait until Khoros 2.0, when we rewrite these routines, before adding this
>capibility.


} 
} From: mathew@elroy.jpl.nasa.gov (Mathew Yeates)
} Subject: Re: strange use of Sun virtual memory
} 
} Why not just call mmap with PRIVATE protection and open for reading and
} writing?
} 
} - -Mathew
} 
}
} From klamer@mi.eltn.utwente.nl Fri Feb  7 01:23:36 1992
} Subject: Re: strange use of Sun virtual memory
} 
} 
} I think this is not correct. When you map a file by mmap, and have flags
} (in the call to mmap) set to MAP_PRIVATE, then the pages of the file you
} write to will be yours! So the disk copy will >>not<< be disturbed!
} There might be another pitfall, however. Any attempt to free() the image
} data (at destruction of the image) is wrong! (Can be fixed by keeping this
} info in the image structure).
} 
} Klamer
} --
} Klamer Schutte		Tel: +31-53-892786	Fax: +31-53-340045
} Faculty of electrical engineering -- University of Twente, The Netherlands
} preferred: klamer@mi.el.utwente.nl     SMTP: klamer@utelmi01.el.utwente.nl
} 

I agree with both of you that the changing MAP_SHARED to MAP_PRIVATE will fix
this for Sun Virtual memory.  And that this should be a good compromise until
Khoros 2.0 comes out.  But i just wanted to add that although this works for
Sun Virtual Memory it will not work for shared memory.  The philosophy behind
the transport routine is to be able to share the data among multiple processes
so that if the data is available directly via shared memory or sun virtual
memory, multiple copies will not be made.  By changing the mode to MAP_PRIVATE
i believe you will end up creating multiple copies rather than sharing one
read only copy.  This will mean that if large data sets are used the system
performance will take a hit when multiple processes need to access the same
data set.  Also, with shared memory there is no way that i know of to invoke
the same MAP_PRIVATE feature.  That the shared memory data will be shared and
that there is no option to indicate that the system should make private copies.
I realize that the current situation is worse since although there maybe a
single copy in shared memory or sun virtual memory, each routine makes a
private copy by actually malloc'ing and coping the data.  Therefore, the
above compromise would improve the situation for sun virtual memory only.

Klamer's other point about freeimage() calling free() to free the image data is
also valid.  This can be fixed inside freeimage(), but maybe harder to fix for
the current IPL routines that currently call free() directly.  I'm not sure
what would happen if you try freeing memory that has not been malloc'ed, but
was allocated using the shmat() or mmap() system calls.  Therefore, unless we
can experiment more and come up with better understanding of how this will work
inside Khoros 1.0, i'm afraid to make changes in the system's readimage()
without being re-assured that these changes won't have determental side affects.

I realize that the implementation of the transport library has caused some
problems since some of the existing utilities cannot take full advantage of
the capibilities right away.  But rather than spending the time trying to get
the current system to work, we (and the organizations funding Khoros) would
rather put the effort into creating a better Khoros 2.0.  If you have comments
or suggestions please do not hesitate to mail me.

			thanks,
			    Mark

------------------------------------------------------------------
Mark Young                         young@blackstone.eece.unm.edu
 
Khoros Group                       (505) 277-6563  (work)
University of New Mexico           Albuquerque,    NM 87131
------------------------------------------------------------------