🚀 go-pugleaf

"Bill Todd" <no@no.com> wrote:
>Does the file you receive contain all 200,000 records? If not, how many
>records does a typicla file contain.

Yes, it does. Every day there are between 100-500 modifications (keys added, keys deleted, data changed). 

>It sounds like you are bringing all 200,000 records to the client. How
>long that  will take depends on the speed of and load on your network
>but it will take a long time.

Yes, I'm doing a First and then a while not eof do processing and next loop.
I'm wondering why it takes so long. CPU load stays under 1% average at all times (peaks are 3 or 4 %)
Obviously, the CPU is more waiting then running. Is there not a way to make it more CPU intensive? 
I should think 100 minutes at 1% could be done in 1 minute at 100%?

>Either of the following approaches should be faster.
>
>1) Read the flat file one row at a time and SELECT the matching row
>from the database using the primary key. Then process the record on the
>client. To select the row from the database use an IBSQL component with
>a parameterized SELECT.

So, you think running 200 000 separate selects should execute faster? 
This will not be a one row select, because I have also to verify if there are database records in the gap between two flat file record numbers. Eg. select * from table where number between last_flat_file_number + 1 and flat_file_number order by number 

>2) Convert the flat file to fixed length format. Create an external
>table in the database that matches the fixed lenght file. INSERT the
>data from the external table into a table in the database (this would
>be a good place to use a temporary table if you are using IB 7.5.1). 

I'm using 6.02 (the freeware version).
Can this file for filling the table come from a stream? I preprocess the records already in a stringlist.

>Do the processing on the server.

You mean with a stored procedure?

>-- 
>Bill Todd (TeamB)
>
Thanks for looking into my problem.
Gerard Vanderveken.