🚀 go-pugleaf

In reply to a private email from a person who knows he can interrupt me. (but
should have added to the tread...)

Hi M,
>
> I saw your post in clpb on sorting:
>
> > Of course you've got a problem if your file contains more than N
> > records or if you've got limited memory. (With both the original
> > and the upcased name the record will have a size of 44 bytes,
> > limiting N to, depending on other global variables, somewhere
> > around the 1450)
> >
> > Any clues? Give it a try and come back if you're really stuck.
>
> How do you sort arrays larger than available memory?

You cannot have an array that is larger than available memory :-)

> I know it can be done, since I downloaded several sort programs from
> Simtel that do so. But if you can explain, then I (and probably a
> bunch of people in the newsgroup) would very much like to know how!

If your file exceeds available direct access memory, you will have to
settle for a multipass merge sort, in _very_ simplistic terms:

- read in as many records as allowed by available RAM
- sort them and write them out to file 1
- read in a next bunch of records
- sort them and write them out to file 2
- repeat above until entire original file has been processed.
- open files created
- read one record from each of them
- write out the smallest of these and read the next record from the
  file that delivered it
- repeat previous step until all files have reached eof

Of course you may in this scenario encounter other problems - too many
open files, in which case you would only merge the first so many files
into one, before repeating the exercise with the next bunch. And yes,
you can already do the first merge-pass with the data from file 1 and
the sorted data from the next bunch...

Of course it's far easier to download RPSRT102.ZIP and SPWNO413.ZIP and
use the latters EXEC(...) executing RPSORT to do your sorting... <G>

And if RPSORT isn't fast enough, it contains the full source, so you
can tweak it for even higher performance.

BTW, I once tested it with a 250 Mb file filled with garbage, RPSORT (multiple
keys!) took just over 10 minutes in an NT Dos box,  NTs SORT (single key):
zapped it after more than 30 minutes... Both were running as the only real
process.

Robert
--
Robert AH Prins
prinsra@williscorroon.com

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own