🚀 go-pugleaf

Does the file you receive contain all 200,000 records? If not, how many
records does a typicla file contain.

It sounds like you are bringing all 200,000 records to the client. How
long that  will take depends on the speed of and load on your network
but it will take a long time.

Either of the following approaches should be faster.

1) Read the flat file one row at a time and SELECT the matching row
from the database using the primary key. Then process the record on the
client. To select the row from the database use an IBSQL component with
a parameterized SELECT.

2) Convert the flat file to fixed length format. Create an external
table in the database that matches the fixed lenght file. INSERT the
data from the external table into a table in the database (this would
be a good place to use a temporary table if you are using IB 7.5.1). Do
the processing on the server.

-- 
Bill Todd (TeamB)


"Gerard Vanderveken" Ghia_no_spam.pi.be wrote:

> 
> Hi,
> I have a table with 200 000 entries (key number and data). Each
> morning I receive a file with records and I need to verify with the
> database table if keys are added or deleted, and if the data was
> changed on the other records.  This process takes a very long time (2
> hours). But I noticed that 2 times 40 minutes are used in the
> fetching of the records.  I timed a fetchall. Also IBconsole takes
> this long time (Table - data - scroll bottom) to load the complete
> table.  I tried to speed up by enlarging the bufferchunks to 50 000,
> but no avail.  The CPU (800MHz) stays always under 1%. What is it
> waiting for?It makes no difference on running the program and
> database on the same or different PC.  What can I do to speed up this
> and why isn't there more CPU time used?