🚀 go-pugleaf

In article <7f4grb$7ik$1@nnrp1.dejanews.com>,
  Robert AH Prins <prinsra@williscorroon.com> wrote:
> In reply to a private email from a person who knows he can interrupt me. (but
> should have added to the tread...)
>
> Hi M,
> >
> > I saw your post in clpb on sorting:
> >
> > > Of course you've got a problem if your file contains more than N
> > > records or if you've got limited memory. (With both the original
> > > and the upcased name the record will have a size of 44 bytes,
> > > limiting N to, depending on other global variables, somewhere
> > > around the 1450)
> > >
> > > Any clues? Give it a try and come back if you're really stuck.
> >
> > How do you sort arrays larger than available memory?
>
> You cannot have an array that is larger than available memory :-)
>
> > I know it can be done, since I downloaded several sort programs from
> > Simtel that do so. But if you can explain, then I (and probably a
> > bunch of people in the newsgroup) would very much like to know how!
>
> If your file exceeds available direct access memory, you will have to
> settle for a multipass merge sort, in _very_ simplistic terms:
>
> - read in as many records as allowed by available RAM
> - sort them and write them out to file 1
> - read in a next bunch of records
> - sort them and write them out to file 2
> - repeat above until entire original file has been processed.
> - open files created
> - read one record from each of them
> - write out the smallest of these and read the next record from the
>   file that delivered it
> - repeat previous step until all files have reached eof
>
> Of course you may in this scenario encounter other problems - too many
> open files, in which case you would only merge the first so many files
> into one, before repeating the exercise with the next bunch. And yes,
> you can already do the first merge-pass with the data from file 1 and
> the sorted data from the next bunch...
>
> Of course it's far easier to download RPSRT102.ZIP and SPWNO413.ZIP and
> use the latters EXEC(...) executing RPSORT to do your sorting... <G>

Another option would be to write all the records to an indexed file and then
read them back out in sorted order according to the desired key(s).  If
you're looking for blazing speed this is not your best choice.   The
advantage of such an approach is that it's easy to write such a program (with
the proper tools) and you can define your own sorting rules.

Years ago I wrote such a utility using Borland's Database Toolbox.  I still
use it today.


>
> And if RPSORT isn't fast enough, it contains the full source, so you
> can tweak it for even higher performance.
>
> BTW, I once tested it with a 250 Mb file filled with garbage, RPSORT (multiple
> keys!) took just over 10 minutes in an NT Dos box,  NTs SORT (single key):
> zapped it after more than 30 minutes... Both were running as the only real
> process.



..

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own