Article View: comp.lang.pascal.borland
Article #337Re: Sorting - Big files
From: Mike Monett
Date: Sat, 17 Apr 1999 00:00
Date: Sat, 17 Apr 1999 00:00
55 lines
2711 bytes
2711 bytes
netnews@altavista.net wrote: > > In article <7f4grb$7ik$1@nnrp1.dejanews.com>, > Robert AH Prins <prinsra@williscorroon.com> wrote: > > In reply to a private email from a person who knows he can interrupt me. (but > > should have added to the tread...) 'Twas me - hit the wrong button in Netscape. Sorry. By the time I noticed, it was too late to do anything about it. [...] >> If your file exceeds available direct access memory, you will >> have to settle for a multipass merge sort, in very simplistic >> terms: >> - read in as many records as allowed by available RAM >> - sort them and write them out to file 1 >> - read in a next bunch of records >> - sort them and write them out to file 2 >> - repeat above until entire original file has been processed. >> - open files created >> - read one record from each of them >> - write out the smallest of these and read the next record from the >> file that delivered it >> - repeat previous step until all files have reached eof [...] Very good explanation. Thanks! The key is to "write out the smallest of these and read the next record *from the file that delivered it*". It didn't make any sense until that sunk in. The next problem is the number of file handles. This might be a good argument for not using the heap, which limits you to 64k chunks. Instead, allocate DOS memory with int21, function 48. This would allow sorting in ~500k chunks. If you had 30 file handles available, this would allow sorting files up to 15 megabytes before running out of file handles. Thanks to WIN9X, most machines come with 64 or 128 megs of ram. It seems you could use Himem to move data from DOS to XMS, and sort larger files by using XMS instead of writing to the hard disk. Of course, the ideal would be to use flat real mode and do the sort directly in ram! Default to using Himem only when in V86 mode. Best Regards, Michael R. Monett, Automated Production Test mailto:add@csolve.net
Technical Details
Message-ID:
<3718310F.1DB0@csolve.net>Path:
arc1-ca.pugleaf.net!unknown!not-for-mailReferences:
<7f4grb$7ik$1@nnrp1.dejanews.com> <7f53jk$nt8$1@nnrp1.dejanews.com>Show Full Headers
Complete Article Headers
X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: 111659,e7de494902ed891c X-Google-Attributes: gid111659,public X-Google-Thread: 108ebd,e7de494902ed891c X-Google-Attributes: gid108ebd,public From: Mike Monett <add@csolve.net> Subject: Re: Sorting - Big files Date: 1999/04/17 Message-ID: <3718310F.1DB0@csolve.net> X-Deja-AN: 467440436 Content-Transfer-Encoding: 7bit References: <7f4grb$7ik$1@nnrp1.dejanews.com> <7f53jk$nt8$1@nnrp1.dejanews.com> Content-Type: text/plain; charset=us-ascii Organization: UTS Mime-Version: 1.0 Newsgroups: comp.lang.pascal.borland,borland.public.turbopascal