Article View: comp.lang.pascal.borland
Article #336Private emails [Was: Sorting - Big files]
From: Robert AH Prins
Date: Fri, 16 Apr 1999 00:00
Date: Fri, 16 Apr 1999 00:00
82 lines
4257 bytes
4257 bytes
In article <7f4grb$7ik$1@nnrp1.dejanews.com>, Robert AH Prins <prinsra@williscorroon.com> wrote: > In reply to a private email from a person who knows he can interrupt me. (but > should have added to the tread...) And in response to this post I got another private email. I don't mind the occasional private email, and getting two is not exactly a deluge, but I would nevertheless appreciate it if you could ask your questions in the newsgroups. To reply to your first question: My name is Robert. (And I really dislike the ugly American habit of shortening names) > Hi M, > > > > I saw your post in clpb on sorting: > > > > > Of course you've got a problem if your file contains more than N > > > records or if you've got limited memory. (With both the original > > > and the upcased name the record will have a size of 44 bytes, > > > limiting N to, depending on other global variables, somewhere > > > around the 1450) > > > > > > Any clues? Give it a try and come back if you're really stuck. > > > > How do you sort arrays larger than available memory? > > You cannot have an array that is larger than available memory :-) > > > I know it can be done, since I downloaded several sort programs from > > Simtel that do so. But if you can explain, then I (and probably a > > bunch of people in the newsgroup) would very much like to know how! > > If your file exceeds available direct access memory, you will have to > settle for a multipass merge sort, in _very_ simplistic terms: > > - read in as many records as allowed by available RAM > - sort them and write them out to file 1 > - read in a next bunch of records > - sort them and write them out to file 2 > - repeat above until entire original file has been processed. > - open files created > - read one record from each of them > - write out the smallest of these and read the next record from the > file that delivered it > - repeat previous step until all files have reached eof > > Of course you may in this scenario encounter other problems - too many > open files, in which case you would only merge the first so many files > into one, before repeating the exercise with the next bunch. And yes, > you can already do the first merge-pass with the data from file 1 and > the sorted data from the next bunch... > > Of course it's far easier to download RPSRT102.ZIP and SPWNO413.ZIP and > use the latters EXEC(...) executing RPSORT to do your sorting... <G> > > And if RPSORT isn't fast enough, it contains the full source, so you > can tweak it for even higher performance. > > BTW, I once tested it with a 250 Mb file filled with garbage, RPSORT (multiple > keys!) took just over 10 minutes in an NT Dos box, NTs SORT (single key): > zapped it after more than 30 minutes... Both were running as the only real > process. URLs for RPSRT102.ZIP & SPWNO413.ZIP: If I need files, I use http://ftpsearch.lycos.com/?form=medium "Search type" - case insensitive substring match "Max hits" - I usually set this to 50 or 100, to get a few more closer to home sites and a decent amount of file dates, so that I can be sure I get the latest version of something. Don't enter an extension, as .ZIP is not universal for archives Robert -- Robert AH Prins prinsra@williscorroon.com -----------== Posted via Deja News, The Discussion Network ==---------- http://www.dejanews.com/ Search, Read, Discuss, or Start Your Own
Technical Details
Message-ID:
<7f6mfq$435$1@nnrp1.dejanews.com>Path:
arc1-ca.pugleaf.net!unknown!not-for-mailReferences:
<7f4grb$7ik$1@nnrp1.dejanews.com>Show Full Headers
Complete Article Headers
X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: 111659,e7de494902ed891c X-Google-Attributes: gid111659,public X-Google-Thread: 108ebd,e7de494902ed891c X-Google-Attributes: gid108ebd,public From: Robert AH Prins <prinsra@williscorroon.com> Subject: Private emails [Was: Sorting - Big files] Date: 1999/04/16 Message-ID: <7f6mfq$435$1@nnrp1.dejanews.com> X-Deja-AN: 467015327 References: <7f4grb$7ik$1@nnrp1.dejanews.com> To: bschor@vms.cis.pitt.edu X-Http-Proxy: 1.0 x10.dejanews.com:80 (Squid/1.1.22) for client 193.130.234.1 Organization: Deja News - The Leader in Internet Discussion X-Article-Creation-Date: Fri Apr 16 06:48:28 1999 GMT Newsgroups: comp.lang.pascal.borland,borland.public.turbopascal X-Http-User-Agent: Mozilla/4.05 [en] (WinNT; I)