🚀 go-pugleaf

RetroBBS NetNews Server

Inspired by RockSolid Light RIP Retro Guy

Thread View: borland.public.cppbuilder.nativeapi
6 messages
6 total messages Started by Mike Collins Wed, 11 Jun 2008 17:53
Getting Started with Memory Mapped Files
#123
Author: Mike Collins
Date: Wed, 11 Jun 2008 17:53
38 lines
2693 bytes
Hi All, hope everyone is well...

I've got a situation where I need to *publish* some information from a 
service to a secondary application.  I have a service created and it does 
some monitoring.  I have an event log that is written to disk but I also 
want to maintain a list of the last 20 items that were in the log file. 
Currently I've been storing these in a STL vector but now I need to provide 
access to this list, to another process.

Having read a bit about this, it seems that a Memory Mapped file would be 
the easiest option.  So I'm just experimenting with Mem Mapped Files and 
there's a few things that I don't quite understand.  I need my service to be 
able to read and write to the MMF but I don't want any other process to be 
able to modify it...

1) Is CreateFileMapping() used to create a MMF and to provide initial access 
to the MMF?  What I mean is, does my service create the initial MMF with 
CreateFileMapping(), and does my secondary process get access to the created 
MMF with the same API call, CreateFileMapping?

2) I'm assuming that using CreateFileMapping(INVALID_HANDLE_VALUE,...) means 
that I don't actually have to physically create a file on disk initially?

3) Can I increase or decrease the size of the MMF once I've called 
CreateFileMapping() or should I try and determine the maximum size that it 
will ever be, initially?

4) How to I secure the MMF so only my service can access it?

5) I'm looking at a code example in VC++ and the author seems to have used a 
mutex to control access whilst reading or writing - is this necessary?

Sorry about the length of questions, and pointers or suggestions would be 
gratefully received.

Many thanks,

Mike 
Re: Getting Started with Memory Mapped Files
#124
Author: dhoke
Date: Wed, 11 Jun 2008 17:28
41 lines
2856 bytes
Another (? - don't know what you have) example that I've found to be a 
simple reference:
http://www.codeproject.com/KB/threads/csharedstruct.aspx

However, comments in the code specifically indicate that author had trouble 
trying to figure out how to use security ACLs to obtain results of the 
nature you desire (with some processes having only read-access and other(s) 
having more)...

But maybe it will clarify what area you need to look at.

(Also note that the class as presented there does not set its member 
variable m_bCreated, which might be useful in some cases.)

As to whether the mutex is necessary, that would depend on you - if you need 
to write multiple things that have to be "consistent" with one another, then 
a mutex probably would be necessary.  If any modified item is exclusively 
valid, than no mutex should be required (given that you want only one 
process to write to the area.)

"Mike Collins" <its@TheBottomOfThePost> wrote in message 
news:48500053@newsgroups.borland.com...
>
> Having read a bit about this, it seems that a Memory Mapped file would be 
> the easiest option.  So I'm just experimenting with Mem Mapped Files and 
> there's a few things that I don't quite understand.  I need my service to 
> be able to read and write to the MMF but I don't want any other process to 
> be able to modify it...
>

>
> 5) I'm looking at a code example in VC++ and the author seems to have used 
> a mutex to control access whilst reading or writing - is this necessary?
>
> Sorry about the length of questions, and pointers or suggestions would be 
> gratefully received.
>
> Many thanks,
>
> Mike
> 
Re: Getting Started with Memory Mapped Files
#125
Author: Remy Lebeau (TeamB)
Date: Wed, 11 Jun 2008 18:52
77 lines
3732 bytes
"Mike Collins" <its@TheBottomOfThePost> wrote in message 
news:48500053@newsgroups.borland.com...

> Currently I've been storing these in a STL vector but now I need
> to provide access to this list, to another process.
>
> Having read a bit about this, it seems that a Memory Mapped
> file would be the easiest option.

Maybe, maybe not.  Since your service is doing live logging, you would have 
to provide serialized access to the list so that other apps could not query 
it at the same time it is being updated.

It might be better to have the service expose a separate server connection, 
whether that be a named pipe or TCP/IP socket or COM server or other IPC 
mechanism, that other apps can connect to and query the data using a 
structured protocol.  Then the service does not have to expose direct access 
to its private memory.

> I need my service to be able to read and write to the MMF
> but I don't want any other process to be able to modify it...

The service can create the MMF with read/write access, and each application 
can open the MMF with read-only access.

> Is CreateFileMapping() used to create a MMF

Yes.

> does my service create the initial MMF with CreateFileMapping(),

Yes.

> and does my secondary process get access to the created MMF
> with the same API call, CreateFileMapping?

It can, though in your situation it would be better to have the secondary 
processes use OpenFileMapping() instead.

> I'm assuming that using CreateFileMapping(INVALID_HANDLE_VALUE,...)
> means that I don't actually have to physically create a file on disk 
> initially?

You don't create your own file at all, no.  The memory for such a MMF is 
backed by the system paging file, though.

> Can I increase or decrease the size of the MMF once I've called 
> CreateFileMapping()

No.  You would have to re-map it.

> should I try and determine the maximum size that it will ever be, 
> initially?

Yes.

> How to I secure the MMF so only my service can access it?

If you do not provide a value in the lpName parameter of 
CreateFileMapping(), then no other process will be able to find it to open 
it.

Alternatively, CreateFileMapping() has a lpFileMappingAttributes parameter, 
so you can construct a SECURITY_ATTRIBUTES that grants access to only your 
service process.

> I'm looking at a code example in VC++ and the author seems to have
> used a mutex to control access whilst reading or writing - is this 
> necessary?

Yes, if the contents of the MMF change dynamically.  You don't want a 
secondary processes to read the data while the service is currently writing 
it.  They could end up with incomplete/corrupted data.


Gambit 
Re: Getting Started with Memory Mapped Files
#127
Author: Remy Lebeau (TeamB)
Date: Wed, 11 Jun 2008 20:24
84 lines
3801 bytes
"Mike Collins" <its@TheBottomOfThePost> wrote in message 
news:485028a2@newsgroups.borland.com...

> Firstly, in my application, my service writes events to an event log
> but at the same time, it also adds them to a stl vector.  My service
> will be the ONLY process that should be able to write to the MMF.
> Is it still necessary to protect access with a mutex?

Yes, for the reason I already explained.  If you don't protect access, the 
the secondary process could read the MMF mid-write and the data it receives 
would be wrong.

> I'm not clear here how the mutex protects anything...
> I assum that the logic would go something like this:

Not quite.

> // To write to the MMF
> 1) Try to open Mutex_1.
> 2) if Mutex_1 cannot be openned (i.e. nothing is using the MMF), create 
> Mutex_1.

It is best to create the mutex ahead of time, such as when the MMF is 
created/opened, and just lock/unlock the mutex when writing.

> if Mutex_1 can be openned (i.e. something is using the MMF), do what? Wait 
> on it using WaitForSingleObject?

Yes.  For example:

    --- service ---

    hMapping = CreateFileMapping(INVALID_HANDLE_VALUE, NULL, PAGE_READWRITE, 
0, 256, "MyFileMapping");
    hMutex = CreateMutex(NULL, FALSE, "MyFileMappingMutex");
    ...
    // for each write operation...
    if( WaitForSingleObject(hMutex, 1000) == WAIT_OBJECT_0 )
    {
        // Perform write operation on the MMF...
        ReleaseMutex(hMutex);
    }
    ...
    CloseHandle(hMutex);
    CloseHandle(hMapping);


    --- application ---

    hMapping = OpenFileMapping(FILE_MAP_READ, FALSE, "MyFileMapping");
    hMutex = OpenMutex(SYNCHRONIZE, FALSE, "MyFileMappingMutex");
    ...
    // for each read operation...
    if( WaitForSingleObject(hMutex, 1000) == WAIT_OBJECT_0 )
    {
        // Perform read operation on the MMF...
        ReleaseMutex(hMutex);
    }
    ...
    CloseHandle(hMutex);
    CloseHandle(hMapping);

> would the same be try for reading.

Similarly, yes.  The secondary process can open the mutex when opening the 
MMF, and then lock/unlock the mutex when reading.

> surely there is some overhead in creating the mutex each time.

Yes, which is why you shouldn't be creating it on each write.  In fact, 
doing so will provide no protection for the MMF at all.

> Also, in the case of (6) what action do you take if the mutex exists?
> If you just wait on it with WaitForSingleObject, surely there is the
> potential to hang the service if another process creates the mutex
> and keeps it open?

Potentially, yes.  The WaitFor...() functions have a Timeout parameter you 
can use to address that, though.



Gambit 
Re: Getting Started with Memory Mapped Files
#126
Author: Mike Collins
Date: Wed, 11 Jun 2008 20:45
114 lines
5447 bytes
Ok, this makes a bit more sense now - if the MMF already exists then calling 
CreateFileMapping() basically acts like OpenFileMapping.

Two things are still confussing me.

Firstly, in my application, my service writes events to an event log but at 
the same time, it also adds them to a stl vector.  My service will be the 
ONLY process that should be able to write to the MMF.  Is it still necessary 
to protect access with a mutex?  I'm not clear here how the mutex protects 
anything... I assum that the logic would go something like this:

// To write to the MMF
1) Try to open Mutex_1.
2) if Mutex_1 cannot be openned (i.e. nothing is using the MMF), create 
Mutex_1.
   3) Perform write operation on the MMF.
   4) Release Mutex_1
6) if Mutex_1 can be openned (i.e. something is using the MMF), - do what? 
Wait on it using WaitForSingleObject?

- would the same be try for reading.  This seems a bit bashful - surely 
there is some overhead in creating the mutex each time.  Also, in the case 
of (6) what action do you take if the mutex exists?  If you just wait on it 
with WaitForSingleObject, surely there is the potential to hang the service 
if another process creates the mutex and keeps it open?

Secondly, I can't get my head around the File Mapping Security and Access 
Rights.  I just need to know how to secure the services side of the MMF 
creating so that other processes can only read the contents.

Sorry if this seems a bit layman and many thanks for the previous posts.

Mike

"Remy Lebeau (TeamB)" <no.spam@no.spam.com> wrote in message 
news:48501f3a$1@newsgroups.borland.com...
>
> "Mike Collins" <its@TheBottomOfThePost> wrote in message 
> news:48500053@newsgroups.borland.com...
>
>> Currently I've been storing these in a STL vector but now I need
>> to provide access to this list, to another process.
>>
>> Having read a bit about this, it seems that a Memory Mapped
>> file would be the easiest option.
>
> Maybe, maybe not.  Since your service is doing live logging, you would 
> have to provide serialized access to the list so that other apps could not 
> query it at the same time it is being updated.
>
> It might be better to have the service expose a separate server 
> connection, whether that be a named pipe or TCP/IP socket or COM server or 
> other IPC mechanism, that other apps can connect to and query the data 
> using a structured protocol.  Then the service does not have to expose 
> direct access to its private memory.
>
>> I need my service to be able to read and write to the MMF
>> but I don't want any other process to be able to modify it...
>
> The service can create the MMF with read/write access, and each 
> application can open the MMF with read-only access.
>
>> Is CreateFileMapping() used to create a MMF
>
> Yes.
>
>> does my service create the initial MMF with CreateFileMapping(),
>
> Yes.
>
>> and does my secondary process get access to the created MMF
>> with the same API call, CreateFileMapping?
>
> It can, though in your situation it would be better to have the secondary 
> processes use OpenFileMapping() instead.
>
>> I'm assuming that using CreateFileMapping(INVALID_HANDLE_VALUE,...)
>> means that I don't actually have to physically create a file on disk 
>> initially?
>
> You don't create your own file at all, no.  The memory for such a MMF is 
> backed by the system paging file, though.
>
>> Can I increase or decrease the size of the MMF once I've called 
>> CreateFileMapping()
>
> No.  You would have to re-map it.
>
>> should I try and determine the maximum size that it will ever be, 
>> initially?
>
> Yes.
>
>> How to I secure the MMF so only my service can access it?
>
> If you do not provide a value in the lpName parameter of 
> CreateFileMapping(), then no other process will be able to find it to open 
> it.
>
> Alternatively, CreateFileMapping() has a lpFileMappingAttributes 
> parameter, so you can construct a SECURITY_ATTRIBUTES that grants access 
> to only your service process.
>
>> I'm looking at a code example in VC++ and the author seems to have
>> used a mutex to control access whilst reading or writing - is this 
>> necessary?
>
> Yes, if the contents of the MMF change dynamically.  You don't want a 
> secondary processes to read the data while the service is currently 
> writing it.  They could end up with incomplete/corrupted data.
>
>
> Gambit
> 
Re: Getting Started with Memory Mapped Files
#132
Author: Jason Cipriani
Date: Fri, 13 Jun 2008 19:26
58 lines
3852 bytes
"Mike Collins" <its@TheBottomOfThePost> wrote in message 
news:48500053@newsgroups.borland.com...
> Hi All, hope everyone is well...
>
> I've got a situation where I need to *publish* some information from a 
> service to a secondary application.  I have a service created and it does 
> some monitoring.  I have an event log that is written to disk but I also 
> want to maintain a list of the last 20 items that were in the log file. 
> Currently I've been storing these in a STL vector but now I need to 
> provide access to this list, to another process.
>
> Having read a bit about this, it seems that a Memory Mapped file would be 
> the easiest option.  So I'm just experimenting with Mem Mapped Files and 
> there's a few things that I don't quite understand.  I need my service to 
> be able to read and write to the MMF but I don't want any other process to 
> be able to modify it...

There are a lot of other legitimate options that can be easier to deal with 
than sharing memory mapped files.

For instance, you can use named pipes (which also let you transparently 
publish data to applications existing on other machines over a network, 
simply by including the host name in the pipe name), start here:

http://msdn.microsoft.com/en-us/library/aa365150(VS.85).aspx

If you use a named pipe, you'd have the service create the pipe and act as a 
"pipe server", clients create pipes by name, connect, and on connect the 
server dumps the log info through the pipe. It can also keep sending updates 
through the pipe as well. No need to deal with locking access to memory 
mapped files with mutices or whatever -- processes aren't *sharing* the same 
resource so you don't have to worry about inter-process synchronization at 
all.

You can also use mailslots:

http://msdn.microsoft.com/en-us/library/aa365147.aspx

They only work if everything is on the same local machine, but they do 
provide a queue if nobody is connected, which may or may not be more 
appropriate for your situation.

You can also use sockets and send via TCP or UDP. Use the loopback network 
for communicating between processes on the same machine, and you get a 
transparent ability to send data to other machines on the network. Using 
sockets is going to be identical to using named pipes (although the pipes 
are nice since you can give them unique names and don't have to worry about 
conflicting ports or anything like that).

Yet another simple option is to just create a file in a well-known location. 
Dump the log info to the file, and interested applications can simply read 
the file. There's no shame in this, hehe.

Anyways, I suggest named pipes instead of memory mapped files. They're 
really easy to use, provide a lot of nice features, and you don't have to 
worry about inter-process synchronization either.

Jason
Thread Navigation

This is a paginated view of messages in the thread with full content displayed inline.

Messages are displayed in chronological order, with the original post highlighted in green.

Use pagination controls to navigate through all messages in large threads.

Back to All Threads