🚀 go-pugleaf

Use of FILE_WRITE_THROUGH does so far seem to be the answer.
Unfortunately it has a major impact on throughput. We have a source
device that returns generated data so can eliminate some overhead by
using that.

In our tests we are creating a 488MB file using that source device.

Without FILE_WRITE_THROUGH it takes between two and three minutes (ten
runs tried) but produces slightly truncated (between .5MB and 1.5MB)
output approx. two thirds of the time.

With FILE_WRITE_THROUGH we get no truncation but it takes around 11
minutes to produce the output.

With today's disk drives imaging already averages an hour per disk so
applying this fix is going to seriously impact our labs (and our
customers). It strongly suggests that we shouldn't use this flag unless
we really have to.

My next step is to find some way to detect file truncation. Then we can
at least provide a warning and suggest use of this option.

As for FlushFileBuffers() - well that just further confuses the issue.
Calling it adds a noticeable delay (a second or so) to file closure
/but doesn't prevent truncation/. I think that all FlushFileBuffers()
is doing is flushing out data that would have been written anyway. The
missing data is just being lost. Since we can read it back after
writing it it must be arriving at the cache but then...?

..and then we have the mystery of the code I posted yesterday. It is
only example code and the SetFilePointer() was just used in an attempt
to replicate our actualy I/O patterns but...it hardly ever truncates.
Last night I left it to generate a 300GB file and it generated it
successfully.

-- 
Andrue Cope [TeamB]
[Bicester, Uk]
http://info.borland.com/newsgroups/guide.html