Slow fetching of large record table

Article View: borland.public.interbase.opensource

Article #159

Back to borland.public.interbase.opensource Previous Next

#159

From: Gerard Vanderveken Ghianospam.pi...
Date: Mon, 12 Sep 2005 11:33

8 lines
1717 bytes

Hi,
I have a table with 200 000 entries (key number and data). Each morning I receive a file with records and I need to verify with the database table if keys are added or deleted, and if the data was changed on the other records.
This process takes a very long time (2 hours). But I noticed that 2 times 40 minutes are used in the fetching of the records.
I timed a fetchall. Also IBconsole takes this long time (Table - data - scroll bottom) to load the complete table.
I tried to speed up by enlarging the bufferchunks to 50 000, but no avail.
The CPU (800MHz) stays always under 1%. What is it waiting for?It makes no difference on running the program and database on the same or different PC.
What can I do to speed up this and why isn't there more CPU time used?

Technical Details

Message-ID: <43256774$1@newsgroups.borland.com>

Path:

arc1-ca.pugleaf.net!g2news1.google.com!news3.google.com!border1.nntp.dca.giganews.com!nntp.giganews.com!feed6.newsreader.com!newsreader.com!news-out1.kabelfoon.nl!newsfeed.kabelfoon.nl!xindi.nntp.kabelfoon.nl!194.25.134.62.MISMATCH!newsfeed00.sul.t-online.de!newsfeed01.sul.t-online.de!t-online.de!newsgroups.borland.com!not-for-mail

Show Full Headers

Complete Article Headers

X-Google-Thread: 106772,9d3ef49be1dec05b,start
X-Google-Attributes: gid106772,public
X-Google-Language: ENGLISH,ASCII-7-bit
From: "Gerard Vanderveken" Ghia_no_spam.pi.be
Sender: "Gerard Vanderveken" Ghia_no_spam.pi.be
Reply-To: "Gerard Vanderveken" Ghia_no_spam.pi.be
Subject: Slow fetching of large record table
Newsgroups: borland.public.interbase.opensource
X-User-Info: 212.35.96.130     
NNTP-Posting-Host: 207.105.83.66
Message-ID: <43256774$1@newsgroups.borland.com>
Date: 12 Sep 2005 04:33:08 -0700
X-Trace: newsgroups.borland.com 1126524788 207.105.83.66 (12 Sep 2005 04:33:08 -0700)
Lines: 8
Path: arc1-ca.pugleaf.net!g2news1.google.com!news3.google.com!border1.nntp.dca.giganews.com!nntp.giganews.com!feed6.newsreader.com!newsreader.com!news-out1.kabelfoon.nl!newsfeed.kabelfoon.nl!xindi.nntp.kabelfoon.nl!194.25.134.62.MISMATCH!newsfeed00.sul.t-online.de!newsfeed01.sul.t-online.de!t-online.de!newsgroups.borland.com!not-for-mail