Article View: borland.public.interbase.opensource
Article #160Re: Slow fetching of large record table
From: Dan Palley
Date: Mon, 12 Sep 2005 15:46
Date: Mon, 12 Sep 2005 15:46
23 lines
2082 bytes
2082 bytes
<Gerard Vanderveken Ghia_no_spam.pi.be> wrote in message news:43256774$1@newsgroups.borland.com... > > Hi, > I have a table with 200 000 entries (key number and data). Each morning I > receive a file with records and I need to verify with the database table > if keys are added or deleted, and if the data was changed on the other > records. > This process takes a very long time (2 hours). But I noticed that 2 times > 40 minutes are used in the fetching of the records. > I timed a fetchall. Also IBconsole takes this long time (Table - data - > scroll bottom) to load the complete table. > I tried to speed up by enlarging the bufferchunks to 50 000, but no avail. > The CPU (800MHz) stays always under 1%. What is it waiting for?It makes no > difference on running the program and database on the same or different > PC. > What can I do to speed up this and why isn't there more CPU time used? If possible, try reading the dataset without caching the results (Unidirectional=TRUE). A lot of CPU time is spent reallocating memory for buffers when caching lots of results. Dan
Technical Details
Message-ID:
<4325a2d4$1@newsgroups.borland.com>Path:
arc1-ca.pugleaf.net!g2news1.google.com!news3.google.com!border1.nntp.dca.giganews.com!nntp.giganews.com!pd7cy2so!pd7cy1no!shaw.ca!news-out.newsgroups.com!propagator2-sterling.newsfeeds.com!in.nntp.be!caladan.landsraad.net!newsgroups.borland.com!not-for-mailReferences:
<43256774$1@newsgroups.borland.com>Show Full Headers
Complete Article Headers
X-Google-Thread: 106772,9d3ef49be1dec05b X-Google-Attributes: gid106772,public X-Google-Language: ENGLISH,ASCII-7-bit From: "Dan Palley" <dan@trams.com> Newsgroups: borland.public.interbase.opensource References: <43256774$1@newsgroups.borland.com> Subject: Re: Slow fetching of large record table Date: Mon, 12 Sep 2005 08:46:24 -0700 Lines: 25 X-Priority: 3 X-MSMail-Priority: Normal X-Newsreader: Microsoft Outlook Express 6.00.2900.2670 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2670 X-RFC2646: Format=Flowed; Original NNTP-Posting-Host: 66.102.131.249 Message-ID: <4325a2d4$1@newsgroups.borland.com> X-Trace: newsgroups.borland.com 1126539988 66.102.131.249 (12 Sep 2005 08:46:28 -0700) Path: arc1-ca.pugleaf.net!g2news1.google.com!news3.google.com!border1.nntp.dca.giganews.com!nntp.giganews.com!pd7cy2so!pd7cy1no!shaw.ca!news-out.newsgroups.com!propagator2-sterling.newsfeeds.com!in.nntp.be!caladan.landsraad.net!newsgroups.borland.com!not-for-mail