🚀 go-pugleaf

To the operating system files have no structure other than simply being 
written from beginning of file to the end.  If you wish to insert something 
into the beginning of a file you could use SetFilePosition to move the 
read/write pointer to the beginning and write to the file.  However the data 
written will over-write what is already in that part of the file.

The way to insert something at a place other than the end of a file is to 
create a new file, stream in data from the old and out to the new until the 
insert point is reached, write the new data, then stream data in and out for 
the remaining portion of the file.

By designing your system to be last written at the beginning of the file you 
have arranged to not achieve the maximum speed that you desire.  A simpler 
method might be to save the data "upside down", with the last written at the 
end.  When reading back it is easy to seek to the end-of-file minus a record 
size and read that header from there.

A system with a strict need for minimum time to execute has conflicting 
requirements with the need to reconfigure the event log.  Better to include 
an 'deleted' bit in the record's header and to start another event file when 
the number of records reaches some predetermined number.  When the system 
reaches a state where a little time is available it is a simple task to then 
pass through the event files, compacting to a new, trimmed file.  The 
'deleted' bit also allows you to know enough about the event files' contents 
to get correct information even before compaction.

No comments on how to use for TFileStream as I see no value in using it for 
this task.  AnsiString and TStrings could be handy if you are saving data as 
characters but is conversion from raw data to characters consistent with 
your desire for very fast operation?

.  Ed

> Mike Collins wrote in message
> news:479b78fd@newsgroups.borland.com...
>
> Hi all, feel a little embarrassed to as such a seemingly easy question 
> but...
>
> I'm trying to create an event logging system.  I have a main thread that 
> monitors certain things and I need to log these in a text based event log. 
> Because the file i/o make take some time, I've created a new thread to do 
> the actual writing operations.  The main thread pushes events onto the 
> secondary threads stl stack.  The secondary thread should then process 
> these events in a FIFO order and write them to the event log.  So far i 
> think my logic is sound :-).
>
> However, I cannot find any reference or explanation on how to use the 
> WriteFile() api to insert a new piece of data at the start of the file.  I 
> want my events to be stored in ascending order of when they occurred. 
> I've used WriteFile a million times to create files or write data to the 
> end of them i.e. appending them. but i can't find anyway to write to the 
> start of the file.  Any suggestions.
>
> Secondly, I'm looking to achieve the best performance possible so with 
> this in mind, is it more resource intensive to write new events to the 
> start of a file as opposed to appending them to the end?
>
> Thirdly, I need to ensure that the event log does not exceed a certain 
> size. In either of the above cases (writing at the start or writing at the 
> end) how would i approach the problem of removing an entry from my event 
> log.  I would ideally like to perform a check to see if the log has 
> exceeded a certain size and then delete the oldest entry.
>
> Finally, are there any issues with performing file I/O from within a 
> thread? What happens if the thread's CPU time elapses before the write 
> operation has finalised?
>
> Sorry for all the questions, just really can't get my head around this.  I 
> had thought of maybe using one of the VCL wrappers i.e. TFileStream or the 
> TStrings but i need to ensure that each entry is written as it is 
> processed and not simple keep writing the whole stream or strings object 
> each time.
>
> Any help or advice would be gratefully received.