🚀 go-pugleaf

Muhammad Momin Rashid wrote:

> Hi,
> 
> I remember someone telling me (on this very forum), that the optimal
> buffer
> size is 32k or 64k.

Something like that may have been valid in the old Dos days
when any buffer larger than 64KB would require a so called
Huge Pointer to access.  Nowadays the main trick for reading
large files is to use a large buffer, but not so large that
the OS has to swap to allocate the memory.  IOW, it does
no good to have a 64MB buffer for reading a large video file,
as example, if the OS is going to spend 5 minutes playing
with the swap file.  Best to get some system params, see
how much physical ram is available etc.. and take a guess
as to how much buffer to allocate.

Perhaps some benchmark scenarios tried out on various
machines with various amounts of ram under a few different
load conditions etc. would help.

There's no one answer.  Of course your approach should
vary depending on the task.  If you are reading, processing
and writing data out to file in a loop that would be
different than just trying to get the file in ram ASAP.

Try some bechmarks.  Another thing that might not occur
to you too is some people use external drives through
reletively slow interfaces such as USB.  You might
not think sucking up a few MB from a file would require
a "please wait" in the user interface until you see
somebody running your app through one of those! :)


-- 

"Only choice is an oxymoron."