🚀 go-pugleaf

RetroBBS NetNews Server

Inspired by RockSolid Light RIP Retro Guy

Article View: borland.public.interbase.opensource
Article #160

Re: Slow fetching of large record table

#160
From: Dan Palley
Date: Mon, 12 Sep 2005 15:46
23 lines
2082 bytes
<Gerard Vanderveken Ghia_no_spam.pi.be> wrote in message 
news:43256774$1@newsgroups.borland.com...
>
> Hi,
> I have a table with 200 000 entries (key number and data). Each morning I 
> receive a file with records and I need to verify with the database table 
> if keys are added or deleted, and if the data was changed on the other 
> records.
> This process takes a very long time (2 hours). But I noticed that 2 times 
> 40 minutes are used in the fetching of the records.
> I timed a fetchall. Also IBconsole takes this long time (Table - data - 
> scroll bottom) to load the complete table.
> I tried to speed up by enlarging the bufferchunks to 50 000, but no avail.
> The CPU (800MHz) stays always under 1%. What is it waiting for?It makes no 
> difference on running the program and database on the same or different 
> PC.
> What can I do to speed up this and why isn't there more CPU time used?

If possible, try reading the dataset without caching the results 
(Unidirectional=TRUE).  A lot of CPU time is spent reallocating memory for 
buffers when caching lots of results.

Dan 
Technical Details
Message-ID: <4325a2d4$1@newsgroups.borland.com>
Path: arc1-ca.pugleaf.net!g2news1.google.com!news3.google.com!border1.nntp.dca.giganews.com!nntp.giganews.com!pd7cy2so!pd7cy1no!shaw.ca!news-out.newsgroups.com!propagator2-sterling.newsfeeds.com!in.nntp.be!caladan.landsraad.net!newsgroups.borland.com!not-for-mail
References: <43256774$1@newsgroups.borland.com>

Show Full Headers
Complete Article Headers
X-Google-Thread: 106772,9d3ef49be1dec05b
X-Google-Attributes: gid106772,public
X-Google-Language: ENGLISH,ASCII-7-bit
From: "Dan Palley" <dan@trams.com>
Newsgroups: borland.public.interbase.opensource
References: <43256774$1@newsgroups.borland.com>
Subject: Re: Slow fetching of large record table
Date: Mon, 12 Sep 2005 08:46:24 -0700
Lines: 25
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.2900.2670
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2670
X-RFC2646: Format=Flowed; Original
NNTP-Posting-Host: 66.102.131.249
Message-ID: <4325a2d4$1@newsgroups.borland.com>
X-Trace: newsgroups.borland.com 1126539988 66.102.131.249 (12 Sep 2005 08:46:28 -0700)
Path: arc1-ca.pugleaf.net!g2news1.google.com!news3.google.com!border1.nntp.dca.giganews.com!nntp.giganews.com!pd7cy2so!pd7cy1no!shaw.ca!news-out.newsgroups.com!propagator2-sterling.newsfeeds.com!in.nntp.be!caladan.landsraad.net!newsgroups.borland.com!not-for-mail