🚀 go-pugleaf

RetroBBS NetNews Server

Inspired by RockSolid Light RIP Retro Guy

Article View: borland.public.cppbuilder.database.ado
Article #1284

Re: Slow performance with ADOQuery

#1284
From: Viatcheslav V. Vassiliev
Date: Tue, 14 Sep 2004 08:50
308 lines
10694 bytes
I found some code. In Delphi it looks as (replace BetterADODataSet with
ADOQuery):

//-----------------------------------------------------------------------
unit Unit1;

interface

uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, DB, ADODB, BetterADODataSet, ComCtrls, StdCtrls;

type
 TCacheItem = record
  Data: array of OleVariant;
   end;

 TCache = record
  Start: Integer;
  Items: array of TCacheItem;
   end;

  TForm1 = class(TForm)
 ListView1: TListView;
 ADOConnection1: TADOConnection;
 BetterADODataSet1: TBetterADODataSet;
 Button1: TButton;
 procedure ListView1Data(Sender: TObject; Item: TListItem);
 procedure Button1Click(Sender: TObject);
 procedure ListView1DataHint(Sender: TObject; StartIndex,
   EndIndex: Integer);
  private
 { Private declarations }
 FCache: TCache;
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation

uses
 ADOInt;

{$R *.dfm}

function OleVarToStr(const V: OleVariant): string;
begin
 if VarIsStr(V) or VarIsNumeric(V) or (VarType(V) = varDate) then Result :=
V
   else Result := '';
end;

procedure TForm1.ListView1Data(Sender: TObject; Item: TListItem);
var
 i, p: Integer;
begin
 p := Item.Index + 1 - FCache.Start;
 if (p >= Low(FCache.Items)) and (p <= High(FCache.Items)) then begin
  for i := 0 to ListView1.Columns.Count-1 do begin
   if i = 0 then
    Item.Caption := OleVarToStr(FCache.Items[p].Data[i])
     else
    Item.SubItems.Add(OleVarToStr(FCache.Items[p].Data[i]));
    end;
   end;
end;

procedure TForm1.Button1Click(Sender: TObject);
var
 i: Integer;
 Column: TListColumn;
begin
 BetterADODataSet1.Active := True;
 ListView1.Items.Count := BetterADODataSet1.RecordCount;

 for i:=0 to BetterADODataSet1.Fields.Count-1 do begin
  Column := ListView1.Columns.Add();
  Column.Caption := BetterADODataSet1.Fields[i].FieldName;
   end;
end;

procedure TForm1.ListView1DataHint(Sender: TObject; StartIndex,
  EndIndex: Integer);
var
 rs: _Recordset;
 i, j: Integer;
begin
 if (StartIndex >= FCache.Start) and (EndIndex < FCache.Start +
Length(FCache.Items)) then Exit;
 if not BetterADODataSet1.Active then Exit;

 FCache.Start := StartIndex;
 SetLength(FCache.Items, 0);
 SetLength(FCache.Items, EndIndex - StartIndex + 1);

 rs := BetterADODataSet1.Recordset;
 rs.MoveFirst();
 rs.Move(StartIndex, EmptyParam);

 for i:=0 to High(FCache.Items) do begin
  with FCache.Items[i] do begin
   SetLength(Data, rs.Fields.Count);
   for j:=0 to High(Data) do
    Data[j] := rs.Fields[j].Value;
    end;

  rs.MoveNext();
  if rs.EOF then begin
   SetLength(FCache.Items, i+1);
   Break;
    end;
   end;
end;

end.

//--------------------------------------------------------------------------
-----
Regards,
Vassiliev V. V.
http://www.managed-vcl.com - using .Net objects in Delphi for Win32 +
ADO.Net
http://www.oledbdirect.com - The fastest way to access MS SQL Server,
MS Jet (Access) and Interbase (through OLEDB)


"Viatcheslav V. Vassiliev" <support@oledbdirect.com> �������/�������� �
�������� ���������: news:4141ef48@newsgroups.borland.com...
> You may also reduce number of calls to MoveBy() or RecNo by implementing
> cache for rows (and values of all fields) that are about to display using
> TListView::OnDataHint() event. In OnData you just read from cache. It is
> similar to way TDBGrid works and could significantly speedup displaying. I
> remeber that I did this in one project and this helped.
>
> //------------------------------------------
> Regards,
> Vassiliev V. V.
> http://www.managed-vcl.com - using .Net objects in Delphi for Win32 +
> ADO.Net
> http://www.oledbdirect.com - The fastest way to access MS SQL Server,
> MS Jet (Access) and Interbase (through OLEDB)
>
> "Vic" <victor@espainformatica.com> �������/�������� � �������� ���������:
> news:414191f5@newsgroups.borland.com...
> > I'm trying to use bookmarks, First() and MoveBy(XX), but I only obtain a
> > slowest scroll.
> > Can you show me some example about how and in which listview's events i
> must
> > use the bookmarks and the other functions ?
> >
> > "Viatcheslav V. Vassiliev" <support@oledbdirect.com> escribi� en el
> mensaje
> > news:41417741$1@newsgroups.borland.com...
> > > Client-side cursor caches all data in memory, so it is like copy
> recordset
> > > to memory. You may try to scroll recordset and cache bookmarks, and
> > instead
> > > of RecNo use Bookmark. Also calling First(), then MoveBy(XX) may
> increase
> > > speed because another mechanism is used.
> > >
> > > DBGrid caches some data for visible rows and this could speed up
> > displaying
> > > data.
> > >
> > > 40 000 records could occupy quite much memory (RAM in case of client
> > cursor)
> > > if there are many columns or there are BLOBs. If this is the problem,
> you
> > > may create temp Access db, copy data from SQL Server into this Access
db
> > > (with SELECT .. INTO from Access db) and use use data from this
database
> > > with server-side cursor (bookmarks are preffered over RecNo). This
> should
> > > work if your data will be up to ~1 GB.
> > >
> > > General recommendation is avoid using so large recordset with UI.
> > >
> > > //------------------------------------------
> > > Regards,
> > > Vassiliev V. V.
> > > http://www.managed-vcl.com - using .Net objects in Delphi for Win32 +
> > > ADO.Net
> > > http://www.oledbdirect.com - The fastest way to access MS SQL Server,
> > > MS Jet (Access) and Interbase (through OLEDB)
> > >
> > > "Vic" <victor@espainformatica.com> �������/�������� � ��������
> ���������:
> > > news:4140ac2b@newsgroups.borland.com...
> > > > > What is your DBMS?
> > > > > > Info : I'm using builder 6 and sqlserver 2000 in windows xp sp1.
> > > > SQL Server 2000 :)
> > > >
> > > > > What is cursor type (server or client)?
> > > > Client.
> > > >
> > > > > Is db server on local computer or remote?
> > > > db server is on a remote computer
> > > >
> > > > If you need more information ask me what you need?
> > > >
> > > > "Viatcheslav V. Vassiliev" <support@oledbdirect.com> escribi� en el
> > > mensaje
> > > > news:414095c6@newsgroups.borland.com...
> > > > > What is your DBMS? What is cursor type (server or client)? Is db
> > server
> > > on
> > > > > local computer or remote?
> > > > >
> > > > > //------------------------------------------
> > > > > Regards,
> > > > > Vassiliev V. V.
> > > > > http://www.managed-vcl.com - using .Net objects in Delphi for
Win32
> +
> > > > > ADO.Net
> > > > > http://www.oledbdirect.com - The fastest way to access MS SQL
> Server,
> > > > > MS Jet (Access) and Interbase (through OLEDB)
> > > > >
> > > > > "Vic" <victor@espainformatica.com> �������/�������� � ��������
> > > ���������:
> > > > > news:414078e5@newsgroups.borland.com...
> > > > > > I have a problem with the TADOQuery. In my project I need to
read
> > > > records
> > > > > > ( about 40000 ) from some tables and show them in a list. For do
> it,
> > I
> > > > use
> > > > > a
> > > > > > virtual TListview (OwnerData = true) and the ADOQuery with a SQL
> > > > statement
> > > > > > to read from database. The process for reading records from the
> > > database
> > > > > to
> > > > > > the query is fast, it gets about 3 sec., and I use the event
> OnData
> > of
> > > > the
> > > > > > listview to show the records directly form the query.
> > > > > > Here is the code :
> > > > > >
> > > > > > void __fastcall TForm1::ListView1Data(TObject *Sender, TListItem
> > > *Item)
> > > > > > {
> > > > > >    ADOQuery->RecNo = Item->Index + 1;
> > > > > >
> > > > > >    for ( int i = 0; i < ListView1->Columns->Count; i++ )
> > > > > >    {
> > > > > >       if ( i == 0 )
> > > > > >             Item->Caption =
> ADOQuery->FieldByName("CODIGO")->Value;
> > > > > >       else
> > > > > >
> > > > > >
Item->SubItems->Add(ADOQuery->FieldByName("NOMBREFISCAL")->Value);
> > > > > >    }
> > > > > > }
> > > > > >
> > > > > > Before show the records directly from the query, I tried to copy
> the
> > > > query
> > > > > > to my own list, and show the records from the list, but that
> process
> > > was
> > > > > > very very very slow. Then I decide to avoid the 'QueryToMemory'
> > > process.
> > > > > >
> > > > > > The problem that I have now is when I scroll the listview. While
> at
> > > the
> > > > > top
> > > > > > of the listiew the scrolling is fast, when I go to
> > > > > > the bottom of it the scrolling is much slower.
> > > > > >
> > > > > > I think that the problem should be here :
> > > > > >    ADOQuery->RecNo = Item->Index + 1;
> > > > > > because if I comment this line, the scroll process returns to
the
> > > normal
> > > > > > performance, but ofcourse I can't comment this line.
> > > > > >
> > > > > > How can I solve that problem ?
> > > > > > Is it any other faster way to do what I need ?
> > > > > >
> > > > > > Thank you in advance.
> > > > > >
> > > > > > Info : I'm using builder 6 and sqlserver 2000 in windows xp sp1.
> > > > > > --
> > > > > > Victor
> > > > > > ESPA Inform�tica.
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
Technical Details
Message-ID: <4146b0ee@newsgroups.borland.com>
Path: arc1-ca.pugleaf.net!g2news1.google.com!news2.google.com!proxad.net!fr.ip.ndsoftware.net!newsfeed00.sul.t-online.de!newsfeed01.sul.t-online.de!t-online.de!newsgroups.borland.com!not-for-mail
References: <414078e5@newsgroups.borland.com> <414095c6@newsgroups.borland.com> <4140ac2b@newsgroups.borland.com> <41417741$1@newsgroups.borland.com> <414191f5@newsgroups.borland.com> <4141ef48@newsgroups.borland.com>

Show Full Headers
Complete Article Headers
X-Google-Thread: 10698b,f914d51d26714f04
X-Google-Attributes: gid10698b,public
X-Google-Language: ENGLISH,ASCII
From: "Viatcheslav V. Vassiliev" <support@oledbdirect.com>
Newsgroups: borland.public.cppbuilder.database.ado
References: <414078e5@newsgroups.borland.com> <414095c6@newsgroups.borland.com> <4140ac2b@newsgroups.borland.com> <41417741$1@newsgroups.borland.com> <414191f5@newsgroups.borland.com> <4141ef48@newsgroups.borland.com>
Subject: Re: Slow performance with ADOQuery
Date: Tue, 14 Sep 2004 12:50:53 +0400
Lines: 310
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.2800.1437
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441
NNTP-Posting-Host: 213.234.218.42
Message-ID: <4146b0ee@newsgroups.borland.com>
X-Trace: newsgroups.borland.com 1095151854 213.234.218.42 (14 Sep 2004 01:50:54 -0700)
Path: arc1-ca.pugleaf.net!g2news1.google.com!news2.google.com!proxad.net!fr.ip.ndsoftware.net!newsfeed00.sul.t-online.de!newsfeed01.sul.t-online.de!t-online.de!newsgroups.borland.com!not-for-mail