Thread View: borland.public.cppbuilder.nativeapi
6 messages
6 total messages
Started by Mike Collins
Wed, 11 Jun 2008 17:53
Getting Started with Memory Mapped Files
Author: Mike Collins
Date: Wed, 11 Jun 2008 17:53
Date: Wed, 11 Jun 2008 17:53
38 lines
2693 bytes
2693 bytes
Hi All, hope everyone is well... I've got a situation where I need to *publish* some information from a service to a secondary application. I have a service created and it does some monitoring. I have an event log that is written to disk but I also want to maintain a list of the last 20 items that were in the log file. Currently I've been storing these in a STL vector but now I need to provide access to this list, to another process. Having read a bit about this, it seems that a Memory Mapped file would be the easiest option. So I'm just experimenting with Mem Mapped Files and there's a few things that I don't quite understand. I need my service to be able to read and write to the MMF but I don't want any other process to be able to modify it... 1) Is CreateFileMapping() used to create a MMF and to provide initial access to the MMF? What I mean is, does my service create the initial MMF with CreateFileMapping(), and does my secondary process get access to the created MMF with the same API call, CreateFileMapping? 2) I'm assuming that using CreateFileMapping(INVALID_HANDLE_VALUE,...) means that I don't actually have to physically create a file on disk initially? 3) Can I increase or decrease the size of the MMF once I've called CreateFileMapping() or should I try and determine the maximum size that it will ever be, initially? 4) How to I secure the MMF so only my service can access it? 5) I'm looking at a code example in VC++ and the author seems to have used a mutex to control access whilst reading or writing - is this necessary? Sorry about the length of questions, and pointers or suggestions would be gratefully received. Many thanks, Mike
Re: Getting Started with Memory Mapped Files
Author: dhoke
Date: Wed, 11 Jun 2008 17:28
Date: Wed, 11 Jun 2008 17:28
41 lines
2856 bytes
2856 bytes
Another (? - don't know what you have) example that I've found to be a simple reference: http://www.codeproject.com/KB/threads/csharedstruct.aspx However, comments in the code specifically indicate that author had trouble trying to figure out how to use security ACLs to obtain results of the nature you desire (with some processes having only read-access and other(s) having more)... But maybe it will clarify what area you need to look at. (Also note that the class as presented there does not set its member variable m_bCreated, which might be useful in some cases.) As to whether the mutex is necessary, that would depend on you - if you need to write multiple things that have to be "consistent" with one another, then a mutex probably would be necessary. If any modified item is exclusively valid, than no mutex should be required (given that you want only one process to write to the area.) "Mike Collins" <its@TheBottomOfThePost> wrote in message news:48500053@newsgroups.borland.com... > > Having read a bit about this, it seems that a Memory Mapped file would be > the easiest option. So I'm just experimenting with Mem Mapped Files and > there's a few things that I don't quite understand. I need my service to > be able to read and write to the MMF but I don't want any other process to > be able to modify it... > > > 5) I'm looking at a code example in VC++ and the author seems to have used > a mutex to control access whilst reading or writing - is this necessary? > > Sorry about the length of questions, and pointers or suggestions would be > gratefully received. > > Many thanks, > > Mike >
Re: Getting Started with Memory Mapped Files
Author: Remy Lebeau (TeamB)
Date: Wed, 11 Jun 2008 18:52
Date: Wed, 11 Jun 2008 18:52
77 lines
3732 bytes
3732 bytes
"Mike Collins" <its@TheBottomOfThePost> wrote in message news:48500053@newsgroups.borland.com... > Currently I've been storing these in a STL vector but now I need > to provide access to this list, to another process. > > Having read a bit about this, it seems that a Memory Mapped > file would be the easiest option. Maybe, maybe not. Since your service is doing live logging, you would have to provide serialized access to the list so that other apps could not query it at the same time it is being updated. It might be better to have the service expose a separate server connection, whether that be a named pipe or TCP/IP socket or COM server or other IPC mechanism, that other apps can connect to and query the data using a structured protocol. Then the service does not have to expose direct access to its private memory. > I need my service to be able to read and write to the MMF > but I don't want any other process to be able to modify it... The service can create the MMF with read/write access, and each application can open the MMF with read-only access. > Is CreateFileMapping() used to create a MMF Yes. > does my service create the initial MMF with CreateFileMapping(), Yes. > and does my secondary process get access to the created MMF > with the same API call, CreateFileMapping? It can, though in your situation it would be better to have the secondary processes use OpenFileMapping() instead. > I'm assuming that using CreateFileMapping(INVALID_HANDLE_VALUE,...) > means that I don't actually have to physically create a file on disk > initially? You don't create your own file at all, no. The memory for such a MMF is backed by the system paging file, though. > Can I increase or decrease the size of the MMF once I've called > CreateFileMapping() No. You would have to re-map it. > should I try and determine the maximum size that it will ever be, > initially? Yes. > How to I secure the MMF so only my service can access it? If you do not provide a value in the lpName parameter of CreateFileMapping(), then no other process will be able to find it to open it. Alternatively, CreateFileMapping() has a lpFileMappingAttributes parameter, so you can construct a SECURITY_ATTRIBUTES that grants access to only your service process. > I'm looking at a code example in VC++ and the author seems to have > used a mutex to control access whilst reading or writing - is this > necessary? Yes, if the contents of the MMF change dynamically. You don't want a secondary processes to read the data while the service is currently writing it. They could end up with incomplete/corrupted data. Gambit
Re: Getting Started with Memory Mapped Files
Author: Remy Lebeau (TeamB)
Date: Wed, 11 Jun 2008 20:24
Date: Wed, 11 Jun 2008 20:24
84 lines
3801 bytes
3801 bytes
"Mike Collins" <its@TheBottomOfThePost> wrote in message
news:485028a2@newsgroups.borland.com...
> Firstly, in my application, my service writes events to an event log
> but at the same time, it also adds them to a stl vector. My service
> will be the ONLY process that should be able to write to the MMF.
> Is it still necessary to protect access with a mutex?
Yes, for the reason I already explained. If you don't protect access, the
the secondary process could read the MMF mid-write and the data it receives
would be wrong.
> I'm not clear here how the mutex protects anything...
> I assum that the logic would go something like this:
Not quite.
> // To write to the MMF
> 1) Try to open Mutex_1.
> 2) if Mutex_1 cannot be openned (i.e. nothing is using the MMF), create
> Mutex_1.
It is best to create the mutex ahead of time, such as when the MMF is
created/opened, and just lock/unlock the mutex when writing.
> if Mutex_1 can be openned (i.e. something is using the MMF), do what? Wait
> on it using WaitForSingleObject?
Yes. For example:
--- service ---
hMapping = CreateFileMapping(INVALID_HANDLE_VALUE, NULL, PAGE_READWRITE,
0, 256, "MyFileMapping");
hMutex = CreateMutex(NULL, FALSE, "MyFileMappingMutex");
...
// for each write operation...
if( WaitForSingleObject(hMutex, 1000) == WAIT_OBJECT_0 )
{
// Perform write operation on the MMF...
ReleaseMutex(hMutex);
}
...
CloseHandle(hMutex);
CloseHandle(hMapping);
--- application ---
hMapping = OpenFileMapping(FILE_MAP_READ, FALSE, "MyFileMapping");
hMutex = OpenMutex(SYNCHRONIZE, FALSE, "MyFileMappingMutex");
...
// for each read operation...
if( WaitForSingleObject(hMutex, 1000) == WAIT_OBJECT_0 )
{
// Perform read operation on the MMF...
ReleaseMutex(hMutex);
}
...
CloseHandle(hMutex);
CloseHandle(hMapping);
> would the same be try for reading.
Similarly, yes. The secondary process can open the mutex when opening the
MMF, and then lock/unlock the mutex when reading.
> surely there is some overhead in creating the mutex each time.
Yes, which is why you shouldn't be creating it on each write. In fact,
doing so will provide no protection for the MMF at all.
> Also, in the case of (6) what action do you take if the mutex exists?
> If you just wait on it with WaitForSingleObject, surely there is the
> potential to hang the service if another process creates the mutex
> and keeps it open?
Potentially, yes. The WaitFor...() functions have a Timeout parameter you
can use to address that, though.
Gambit
Re: Getting Started with Memory Mapped Files
Author: Mike Collins
Date: Wed, 11 Jun 2008 20:45
Date: Wed, 11 Jun 2008 20:45
114 lines
5447 bytes
5447 bytes
Ok, this makes a bit more sense now - if the MMF already exists then calling CreateFileMapping() basically acts like OpenFileMapping. Two things are still confussing me. Firstly, in my application, my service writes events to an event log but at the same time, it also adds them to a stl vector. My service will be the ONLY process that should be able to write to the MMF. Is it still necessary to protect access with a mutex? I'm not clear here how the mutex protects anything... I assum that the logic would go something like this: // To write to the MMF 1) Try to open Mutex_1. 2) if Mutex_1 cannot be openned (i.e. nothing is using the MMF), create Mutex_1. 3) Perform write operation on the MMF. 4) Release Mutex_1 6) if Mutex_1 can be openned (i.e. something is using the MMF), - do what? Wait on it using WaitForSingleObject? - would the same be try for reading. This seems a bit bashful - surely there is some overhead in creating the mutex each time. Also, in the case of (6) what action do you take if the mutex exists? If you just wait on it with WaitForSingleObject, surely there is the potential to hang the service if another process creates the mutex and keeps it open? Secondly, I can't get my head around the File Mapping Security and Access Rights. I just need to know how to secure the services side of the MMF creating so that other processes can only read the contents. Sorry if this seems a bit layman and many thanks for the previous posts. Mike "Remy Lebeau (TeamB)" <no.spam@no.spam.com> wrote in message news:48501f3a$1@newsgroups.borland.com... > > "Mike Collins" <its@TheBottomOfThePost> wrote in message > news:48500053@newsgroups.borland.com... > >> Currently I've been storing these in a STL vector but now I need >> to provide access to this list, to another process. >> >> Having read a bit about this, it seems that a Memory Mapped >> file would be the easiest option. > > Maybe, maybe not. Since your service is doing live logging, you would > have to provide serialized access to the list so that other apps could not > query it at the same time it is being updated. > > It might be better to have the service expose a separate server > connection, whether that be a named pipe or TCP/IP socket or COM server or > other IPC mechanism, that other apps can connect to and query the data > using a structured protocol. Then the service does not have to expose > direct access to its private memory. > >> I need my service to be able to read and write to the MMF >> but I don't want any other process to be able to modify it... > > The service can create the MMF with read/write access, and each > application can open the MMF with read-only access. > >> Is CreateFileMapping() used to create a MMF > > Yes. > >> does my service create the initial MMF with CreateFileMapping(), > > Yes. > >> and does my secondary process get access to the created MMF >> with the same API call, CreateFileMapping? > > It can, though in your situation it would be better to have the secondary > processes use OpenFileMapping() instead. > >> I'm assuming that using CreateFileMapping(INVALID_HANDLE_VALUE,...) >> means that I don't actually have to physically create a file on disk >> initially? > > You don't create your own file at all, no. The memory for such a MMF is > backed by the system paging file, though. > >> Can I increase or decrease the size of the MMF once I've called >> CreateFileMapping() > > No. You would have to re-map it. > >> should I try and determine the maximum size that it will ever be, >> initially? > > Yes. > >> How to I secure the MMF so only my service can access it? > > If you do not provide a value in the lpName parameter of > CreateFileMapping(), then no other process will be able to find it to open > it. > > Alternatively, CreateFileMapping() has a lpFileMappingAttributes > parameter, so you can construct a SECURITY_ATTRIBUTES that grants access > to only your service process. > >> I'm looking at a code example in VC++ and the author seems to have >> used a mutex to control access whilst reading or writing - is this >> necessary? > > Yes, if the contents of the MMF change dynamically. You don't want a > secondary processes to read the data while the service is currently > writing it. They could end up with incomplete/corrupted data. > > > Gambit >
Re: Getting Started with Memory Mapped Files
Author: Jason Cipriani
Date: Fri, 13 Jun 2008 19:26
Date: Fri, 13 Jun 2008 19:26
58 lines
3852 bytes
3852 bytes
"Mike Collins" <its@TheBottomOfThePost> wrote in message news:48500053@newsgroups.borland.com... > Hi All, hope everyone is well... > > I've got a situation where I need to *publish* some information from a > service to a secondary application. I have a service created and it does > some monitoring. I have an event log that is written to disk but I also > want to maintain a list of the last 20 items that were in the log file. > Currently I've been storing these in a STL vector but now I need to > provide access to this list, to another process. > > Having read a bit about this, it seems that a Memory Mapped file would be > the easiest option. So I'm just experimenting with Mem Mapped Files and > there's a few things that I don't quite understand. I need my service to > be able to read and write to the MMF but I don't want any other process to > be able to modify it... There are a lot of other legitimate options that can be easier to deal with than sharing memory mapped files. For instance, you can use named pipes (which also let you transparently publish data to applications existing on other machines over a network, simply by including the host name in the pipe name), start here: http://msdn.microsoft.com/en-us/library/aa365150(VS.85).aspx If you use a named pipe, you'd have the service create the pipe and act as a "pipe server", clients create pipes by name, connect, and on connect the server dumps the log info through the pipe. It can also keep sending updates through the pipe as well. No need to deal with locking access to memory mapped files with mutices or whatever -- processes aren't *sharing* the same resource so you don't have to worry about inter-process synchronization at all. You can also use mailslots: http://msdn.microsoft.com/en-us/library/aa365147.aspx They only work if everything is on the same local machine, but they do provide a queue if nobody is connected, which may or may not be more appropriate for your situation. You can also use sockets and send via TCP or UDP. Use the loopback network for communicating between processes on the same machine, and you get a transparent ability to send data to other machines on the network. Using sockets is going to be identical to using named pipes (although the pipes are nice since you can give them unique names and don't have to worry about conflicting ports or anything like that). Yet another simple option is to just create a file in a well-known location. Dump the log info to the file, and interested applications can simply read the file. There's no shame in this, hehe. Anyways, I suggest named pipes instead of memory mapped files. They're really easy to use, provide a lot of nice features, and you don't have to worry about inter-process synchronization either. Jason
Thread Navigation
This is a paginated view of messages in the thread with full content displayed inline.
Messages are displayed in chronological order, with the original post highlighted in green.
Use pagination controls to navigate through all messages in large threads.
Back to All Threads