🚀 go-pugleaf

RetroBBS NetNews Server

Inspired by RockSolid Light RIP Retro Guy

Thread View: borland.public.cppbuilder.database.ado
17 messages
17 total messages Started by Patrick Wong Tue, 24 Apr 2007 03:31
TADOQuery Memory Leak
#156
Author: Patrick Wong
Date: Tue, 24 Apr 2007 03:31
103 lines
3755 bytes
Dear all,

Does anyone ever experience the captioned problem?  My application is having 
this problem,  after being run for several hours the memory usage grew 
significantly, error pops up with "Virtual memory too low" and the app has 
to be restarted again.  The ADOQuery is created and run in a thread.  Code 
excerpts are something like below:

__fastcall TThreadTestDB::TThreadTestDB(bool CreateSuspended)
 : TThread(CreateSuspended)
{
  m_ADOConnectionTest = new TADOConnection(NULL);
  m_ADOConnectionTest->LoginPrompt = false;
  m_ADOConnectionTest->ConnectionString = 
AnsiString("Provider=Ifxoledbc.2;Password=bar;Persist Security 
Info=True;User ID=foo;Data Source=abc@dbserver");
  m_ADOQueryTest = new TADOQuery(NULL);
  m_ADOQueryTest->Connection = m_ADOConnectionTest;
}
//---------------------------------------------------------------------------

void __fastcall TThreadTestDB::Execute()
{
  AnsiString strSQL = "select name from table1 where account=\'1234\'";
  AnsiString strRtn;

 //---- Place thread code here ----
  CoInitialize(NULL);

  if(!m_ADOConnectionTest->Connected)
    m_ADOConnectionTest->Open();

  do
  {
   if(m_ADOQueryTest->Active)
     m_ADOQueryTest->Close();

    m_ADOQueryTest->SQL->Clear();
   m_ADOQueryTest->SQL->Add(strSQL);
   m_ADOQueryTest->Prepared = true;
   m_ADOQueryTest->Open();

   if(m_ADOQueryTest->RecordCount > 0)
   {
     m_ADOQueryTest->First();

      do
      {
       strRtn = m_ADOQueryTest->FieldByName("name")->AsString;
      m_ADOQueryTest->Next();
     } while(!m_ADOQueryTest->Eof);
   }
  }
  ::Sleep(100);
 Application->ProcessMessages();
  } while (!Terminated);

  if(m_ADOQueryTest->Active)
    m_ADOQueryTest->Close();

  CoUninitialize();
}
//---------------------------------------------------------------------------

__fastcall TThreadTestDB::~TThreadTestDB(void)
{
  if(m_ADOQueryTest != NULL)
  {
   m_ADOQueryTest->Close();
    delete m_ADOQueryTest;
    m_ADOQueryTest = NULL;
  }

  if(m_ADOConnectionTest != NULL)
  {
    m_ADOConnectionTest->Close();
    delete m_ADOConnectionTest;
    m_ADOConnectionTest = NULL;
  }
}


Each time the m_ADOQueryTest->Open(); is executed the memory usage increases 
by 10-20k, without ever drops when the query is closed.  I also tried 
deleting the and recreating the ADOQuery in every iteration in the do..while 
loop, but it does not help.


Googling can find many threads discussing this problem, one of which is:

http://groups.google.de/group/borland.public.delphi.thirdpartytools.general/feed/atom_v1_0_topics.xml?num=50

But most of them ends without concrete solution.

I tested in both BCB6 and BDS2006 with most updated patches.  I am running 
the app in Windows XP.


Hope can get some kind advice here.


Cheers,
Patrick 
Re: TADOQuery Memory Leak
#158
Author: Vassiliev V. V.
Date: Tue, 24 Apr 2007 10:09
119 lines
4670 bytes
Try to place code from constructor and destructor into Execute method - this 
way they will be executed in your thread, not in main thread (or thread 
where you create TThreadTestDB object).

//------------------------------------------
Regards,
Vassiliev V. V.
http://www.managed-vcl.com - using .Net objects in Delphi for Win32 +
ADO.Net
http://www.oledbdirect.com - The fastest way to access MS SQL Server,
MS Jet (Access) and Interbase (through OLEDB)


"Patrick Wong" <patrickwong@computer.org> �������/�������� � �������� 
���������: news:462d7a19$1@newsgroups.borland.com...
> Dear all,
>
> Does anyone ever experience the captioned problem?  My application is 
> having this problem,  after being run for several hours the memory usage 
> grew significantly, error pops up with "Virtual memory too low" and the 
> app has to be restarted again.  The ADOQuery is created and run in a 
> thread.  Code excerpts are something like below:
>
> __fastcall TThreadTestDB::TThreadTestDB(bool CreateSuspended)
> : TThread(CreateSuspended)
> {
>  m_ADOConnectionTest = new TADOConnection(NULL);
>  m_ADOConnectionTest->LoginPrompt = false;
>  m_ADOConnectionTest->ConnectionString = 
> AnsiString("Provider=Ifxoledbc.2;Password=bar;Persist Security 
> Info=True;User ID=foo;Data Source=abc@dbserver");
>  m_ADOQueryTest = new TADOQuery(NULL);
>  m_ADOQueryTest->Connection = m_ADOConnectionTest;
> }
> //---------------------------------------------------------------------------
>
> void __fastcall TThreadTestDB::Execute()
> {
>  AnsiString strSQL = "select name from table1 where account=\'1234\'";
>  AnsiString strRtn;
>
> //---- Place thread code here ----
>  CoInitialize(NULL);
>
>  if(!m_ADOConnectionTest->Connected)
>    m_ADOConnectionTest->Open();
>
>  do
>  {
>   if(m_ADOQueryTest->Active)
>     m_ADOQueryTest->Close();
>
>    m_ADOQueryTest->SQL->Clear();
>   m_ADOQueryTest->SQL->Add(strSQL);
>   m_ADOQueryTest->Prepared = true;
>   m_ADOQueryTest->Open();
>
>   if(m_ADOQueryTest->RecordCount > 0)
>   {
>     m_ADOQueryTest->First();
>
>      do
>      {
>       strRtn = m_ADOQueryTest->FieldByName("name")->AsString;
>      m_ADOQueryTest->Next();
>     } while(!m_ADOQueryTest->Eof);
>   }
>  }
>  ::Sleep(100);
> Application->ProcessMessages();
>  } while (!Terminated);
>
>  if(m_ADOQueryTest->Active)
>    m_ADOQueryTest->Close();
>
>  CoUninitialize();
> }
> //---------------------------------------------------------------------------
>
> __fastcall TThreadTestDB::~TThreadTestDB(void)
> {
>  if(m_ADOQueryTest != NULL)
>  {
>   m_ADOQueryTest->Close();
>    delete m_ADOQueryTest;
>    m_ADOQueryTest = NULL;
>  }
>
>  if(m_ADOConnectionTest != NULL)
>  {
>    m_ADOConnectionTest->Close();
>    delete m_ADOConnectionTest;
>    m_ADOConnectionTest = NULL;
>  }
> }
>
>
> Each time the m_ADOQueryTest->Open(); is executed the memory usage 
> increases by 10-20k, without ever drops when the query is closed.  I also 
> tried deleting the and recreating the ADOQuery in every iteration in the 
> do..while loop, but it does not help.
>
>
> Googling can find many threads discussing this problem, one of which is:
>
> http://groups.google.de/group/borland.public.delphi.thirdpartytools.general/feed/atom_v1_0_topics.xml?num=50
>
> But most of them ends without concrete solution.
>
> I tested in both BCB6 and BDS2006 with most updated patches.  I am running 
> the app in Windows XP.
>
>
> Hope can get some kind advice here.
>
>
> Cheers,
> Patrick
> 
Re: TADOQuery Memory Leak
#159
Author: Nige
Date: Wed, 25 Apr 2007 13:01
137 lines
4790 bytes
Hi Patrick,

Memory management in BCB is very bad. Bytes are leaked here and there
throughout the running of any BCB application.

You can minimise the problem by creating as few objects as needed. So could
your TADOConnection and TADOQuery be created outside of this thread object,
and only pointers to these objects passed in? This will help your memory
problem a lot.

However I'm not sure how this will affect things since you are using a
thread. You might have to synchronise access to the query and connection so
your main thread doesn't use these at the same time as the new thread.


Nigel

PS To tidy up your code a bit, you don't have to call Close() before
deleting your TADOQuery. Also, it's quicker to say query->SQL->Text =
"select..." instead of query->SQL->Clear() then
query->SQL->Add("select...").

"Patrick Wong" <patrickwong@computer.org> wrote in message
news:462d7a19$1@newsgroups.borland.com...
> Dear all,
>
> Does anyone ever experience the captioned problem?  My application is
having
> this problem,  after being run for several hours the memory usage grew
> significantly, error pops up with "Virtual memory too low" and the app has
> to be restarted again.  The ADOQuery is created and run in a thread.  Code
> excerpts are something like below:
>
> __fastcall TThreadTestDB::TThreadTestDB(bool CreateSuspended)
>  : TThread(CreateSuspended)
> {
>   m_ADOConnectionTest = new TADOConnection(NULL);
>   m_ADOConnectionTest->LoginPrompt = false;
>   m_ADOConnectionTest->ConnectionString =
> AnsiString("Provider=Ifxoledbc.2;Password=bar;Persist Security
> Info=True;User ID=foo;Data Source=abc@dbserver");
>   m_ADOQueryTest = new TADOQuery(NULL);
>   m_ADOQueryTest->Connection = m_ADOConnectionTest;
> }
>
//--------------------------------------------------------------------------
-
>
> void __fastcall TThreadTestDB::Execute()
> {
>   AnsiString strSQL = "select name from table1 where account=\'1234\'";
>   AnsiString strRtn;
>
>  //---- Place thread code here ----
>   CoInitialize(NULL);
>
>   if(!m_ADOConnectionTest->Connected)
>     m_ADOConnectionTest->Open();
>
>   do
>   {
>    if(m_ADOQueryTest->Active)
>      m_ADOQueryTest->Close();
>
>     m_ADOQueryTest->SQL->Clear();
>    m_ADOQueryTest->SQL->Add(strSQL);
>    m_ADOQueryTest->Prepared = true;
>    m_ADOQueryTest->Open();
>
>    if(m_ADOQueryTest->RecordCount > 0)
>    {
>      m_ADOQueryTest->First();
>
>       do
>       {
>        strRtn = m_ADOQueryTest->FieldByName("name")->AsString;
>       m_ADOQueryTest->Next();
>      } while(!m_ADOQueryTest->Eof);
>    }
>   }
>   ::Sleep(100);
>  Application->ProcessMessages();
>   } while (!Terminated);
>
>   if(m_ADOQueryTest->Active)
>     m_ADOQueryTest->Close();
>
>   CoUninitialize();
> }
>
//--------------------------------------------------------------------------
-
>
> __fastcall TThreadTestDB::~TThreadTestDB(void)
> {
>   if(m_ADOQueryTest != NULL)
>   {
>    m_ADOQueryTest->Close();
>     delete m_ADOQueryTest;
>     m_ADOQueryTest = NULL;
>   }
>
>   if(m_ADOConnectionTest != NULL)
>   {
>     m_ADOConnectionTest->Close();
>     delete m_ADOConnectionTest;
>     m_ADOConnectionTest = NULL;
>   }
> }
>
>
> Each time the m_ADOQueryTest->Open(); is executed the memory usage
increases
> by 10-20k, without ever drops when the query is closed.  I also tried
> deleting the and recreating the ADOQuery in every iteration in the
do..while
> loop, but it does not help.
>
>
> Googling can find many threads discussing this problem, one of which is:
>
>
http://groups.google.de/group/borland.public.delphi.thirdpartytools.general/feed/atom_v1_0_topics.xml?num=50
>
> But most of them ends without concrete solution.
>
> I tested in both BCB6 and BDS2006 with most updated patches.  I am running
> the app in Windows XP.
>
>
> Hope can get some kind advice here.
>
>
> Cheers,
> Patrick
>
>
Re: TADOQuery Memory Leak
#160
Author: Clayton Arends
Date: Wed, 25 Apr 2007 16:34
8 lines
1311 bytes
> Memory management in BCB is very bad. Bytes are leaked here and there
> throughout the running of any BCB application.

I know of plenty of areas in BCB that leak memory but your claim that all 
programs leak is nonsense.  Most memory leaks are the result of sloppy 
programming.

- Clayton 
Re: TADOQuery Memory Leak
#161
Author: Clayton Arends
Date: Wed, 25 Apr 2007 18:17
13 lines
1244 bytes
Since the ADO objects are being constructed in the main application thread 
you most likely need to invoke OleInitialize() for your main thread.  Place 
the following in either your main form's constructor or in WinMain().

  OleInitialize(NULL);

> Application->ProcessMessages();

Don't call this from within any thread other than the main application 
thread.

HTH,
- Clayton 
Re: TADOQuery Memory Leak
#162
Author: Clayton Arends
Date: Wed, 25 Apr 2007 18:20
5 lines
1129 bytes
I forgot to ask.  Another point of memory leak could be the destruction of 
your threads.  You didn't show how you are deleting them.  Are you setting 
FreeOnTerminate to true or are you deleting the thread in some other way?

- Clayton 
Re: TADOQuery Memory Leak
#163
Author: Patrick Wong
Date: Thu, 26 Apr 2007 00:53
30 lines
1807 bytes
"Clayton Arends" <nospam_claytonarends@hotmail.com> wrote in message 
news:462f9bbe$1@newsgroups.borland.com...
>I forgot to ask.  Another point of memory leak could be the destruction of 
>your threads.  You didn't show how you are deleting them.

In my test only one thread is created.  The memory leak happens during the 
thread runs (every ADOQuery->Open() is called).  I explicitely terminate and 
delete it from the main thread:

void __fastcall TForm1::Button3Click(TObject *Sender)
{
  if(pDBThread != NULL)
  {
    pDBThread->Terminate();
    pDBThread->WaitFor();

    delete pDBThread;
    pDBThread = NULL;
  }
}

At that time the memory usage stops to grow.


> Are you setting FreeOnTerminate to true or are you deleting the thread in 
> some other way?
>

The FreeOnTerminate is set to false. 
Re: TADOQuery Memory Leak
#164
Author: Patrick Wong
Date: Thu, 26 Apr 2007 01:02
22 lines
1704 bytes
"Clayton Arends" <nospam_claytonarends@hotmail.com> wrote in message 
news:462f9b39$1@newsgroups.borland.com...
> Since the ADO objects are being constructed in the main application thread 
> you most likely need to invoke OleInitialize() for your main thread. 
> Place the following in either your main form's constructor or in 
> WinMain().
>
>  OleInitialize(NULL);

The TADOConnection and TADOQuery objects are declared as private members of 
the thread and created at the thread's constructor.  Is it still an issue 
calling CoInitalize() and CoUninitialize() in the thread's context?

>
>> Application->ProcessMessages();
>
> Don't call this from within any thread other than the main application 
> thread.
>

Comment out that does not help. 
Re: TADOQuery Memory Leak
#165
Author: Patrick Wong
Date: Thu, 26 Apr 2007 02:30
9 lines
1232 bytes
"Vassiliev V. V." <support@oledbdirect.com> wrote in message 
news:462dd7ee$1@newsgroups.borland.com...
> Try to place code from constructor and destructor into Execute method - 
> this way they will be executed in your thread, not in main thread (or 
> thread where you create TThreadTestDB object).
>

Regrettably that does not help. 
Re: TADOQuery Memory Leak
#166
Author: Clayton Arends
Date: Thu, 26 Apr 2007 03:13
24 lines
2055 bytes
> The TADOConnection and TADOQuery objects are declared as private members
> of  the thread and created at the thread's constructor.  Is it still an
> issue calling CoInitalize() and CoUninitialize() in the thread's context?

I'm not exactly sure what you mean by "is it still an issue".  Any code
that is executed in a thread's constructor and destructor are executing in
the context of the main application thread.  TThread descendants don't start
executing in their own context until Execute() is invoked.

However, I took another look at my test app that replicates your behavior.
I thought the by additionally calling OleInitialize() in the main
application thread the memory leak went away.  But, it appears that I was
mistaken.  I am currently trying to figure out what is going on here.

>>> Application->ProcessMessages();
>>
>> Don't call this from within any thread other than the main application 
>> thread.
>
> Comment out that does not help.

I didn't say it would help ... just don't do it. ;-)

- Clayton
Re: TADOQuery Memory Leak
#168
Author: Nige
Date: Thu, 26 Apr 2007 15:26
16 lines
1512 bytes
"Clayton Arends" <nospam_claytonarends@hotmail.com> wrote in message
news:462f8315$1@newsgroups.borland.com...
> I know of plenty of areas in BCB that leak memory but your claim that all
> programs leak is nonsense.  Most memory leaks are the result of sloppy
> programming.

I was referring to the unstoppable* memory-eating monster that is the
Borland memory manager, when used in conjunction with a program that
repeatedly creates and deletes objects during the course of its running -
for example, the code Patrick posted.

So not every program - just quite a few of them.

Nige

*: Replacement memory managers are available.
Re: TADOQuery Memory Leak
#169
Author: Clayton Arends
Date: Mon, 30 Apr 2007 14:42
408 lines
11368 bytes
I found time off and on during the week to test this issue.  I made two test 
applications.  One in BCB6 and one in BDS2006 Delphi personality.  Both 
applications were built standalone.  Both applications yielded almost 
identical results.  Memory consumption was measured using a custom memory 
usage function and the Windows Task Manager.  I have included my BCB6 test 
code at the end of this post if anyone wishes to double-check my test and 
results.

The application algorithm constructs 10 threads.  After 500 ms it terminates 
those threads and then creates 10 more threads.  There are 3 testing levels 
that perform progressively more tasks:

- Test 1: Construction and destruction of the ADO objects does not result
  in memory leaks.

- Test 2: Opening the ADO connection results in a very slow memory leak.

- Test 3: Opening the ADO query results in a slightly faster yet still
  slow memory leak.

On test 3 I let the application run over night.  Over 900,000 pairs of ADO 
objects were created, opened, and destroyed.  Memory use was around 95MB and 
continued to jump a few K every few minutes.

At one point, to rule out the VCL implementation of ADO, I rewrote the test 
to use the raw ADO interface classes (_Connection, and _RecordSet) with 
similar memory leak results.

I performed some general searches for ADO memory leaks and found some 
scattered references.  Many threads concluded that the leaks were caused by 
misuse of the ADO objects.  Some concluded that certain versions of ADO 
drivers were causing the leaks.  A majority had no conclusions.  It is hard 
to know for sure what is going on here.  However, the fact that the leak 
doesn't happen until the ADO objects are put to use tells me that the leak 
is somewhere outside of the domain of the VCL memory manager.

- Clayton

  // Unit1.h
  #ifndef Unit1H
  #define Unit1H
  //-----------------------------------------------------------------------
  #include <Classes.hpp>
  #include <Controls.hpp>
  #include <ExtCtrls.hpp>
  #include <StdCtrls.hpp>
  //-----------------------------------------------------------------------
  class TForm1 : public TForm
  {
    __published:
    TLabel *Label1;
    TLabel *Label2;
    TButton *Level1Btn;
    TButton *Level2Btn;
    TButton *Level3Btn;
    TButton *StopBtn;
    TTimer *Timer1;
    void __fastcall Level1BtnClick(TObject *Sender);
    void __fastcall Level2BtnClick(TObject *Sender);
    void __fastcall Level3BtnClick(TObject *Sender);
    void __fastcall StopBtnClick(TObject *Sender);
    void __fastcall Timer1Timer(TObject *Sender);
    private:
      void SetLabel1Caption();

    public:
      __fastcall TForm1(TComponent* Owner);
  };

  extern TForm1 *Form1;

  #endif


  // Unit1.cpp
  #include <vcl.h>
  #pragma hdrstop

  #include "Unit1.h"
  #pragma package(smart_init)
  #pragma resource "*.dfm"
  //-----------------------------------------------------------------------
  #include <ADODB.hpp>
  #include <DateUtils.hpp>
  #include <SyncObjs.hpp>
  //-----------------------------------------------------------------------
  #include <memory>
  //-----------------------------------------------------------------------
  #include <ComObj.hpp>
  //-----------------------------------------------------------------------

  TForm1 *Form1;

  class auto_CoInitializeEx
  {
    public:
      auto_CoInitializeEx()
      { ::CoInitializeEx(NULL, COINIT_MULTITHREADED); }
      ~auto_CoInitializeEx()
      { ::CoUninitialize(); }
  };

  class TThreadTestDB : public TThread
  {
    private:
      TADOConnection* fConnection;
      TADOQuery* fQuery;

      int TestLevel;

      void CreateObjects();
      void DeleteObjects();

    protected:
      virtual void __fastcall Execute();

    public:
      __fastcall TThreadTestDB(int ATestLevel);
      __fastcall ~TThreadTestDB();
  };

  TCriticalSection* Sync;
  unsigned __int64 CreatedObjectCount = 0;
  unsigned __int64 ThreadCount = 0;

  __fastcall TThreadTestDB::TThreadTestDB(int ATestLevel)
    : TThread(true), TestLevel(ATestLevel)
  {
    Sync->Acquire();
    ++ThreadCount;
    Sync->Release();

    FreeOnTerminate = true;
  }

  __fastcall TThreadTestDB::~TThreadTestDB()
  {
    Sync->Acquire();
    --ThreadCount;
    Sync->Release();

    DeleteObjects(); // In case of early termination due to exception
  }

  void TThreadTestDB::CreateObjects()
  {
    Sync->Acquire();
    ++CreatedObjectCount;
    Sync->Release();

    fConnection = new TADOConnection(NULL);
    fConnection->LoginPrompt = false;
    fConnection->ConnectionString =
      AnsiString(
        "Provider=Microsoft.Jet.OLEDB.4.0;"
        "Data Source=Chapter2.mdb;"
        "Persist Security Info=False");

    fQuery = new TADOQuery(NULL);
    fQuery->Connection = fConnection;
    fQuery->ParamCheck = false;
  }

  void TThreadTestDB::DeleteObjects()
  {
    delete fQuery, fQuery = NULL;
    delete fConnection, fConnection = NULL;
  }

  void __fastcall TThreadTestDB::Execute()
  {
    auto_CoInitializeEx auto_coinit;

    while (!Terminated)
    {
      CreateObjects();

      if (TestLevel > 1)
      {
        fConnection->Open();

        if (TestLevel > 2)
        {
          fQuery->SQL->Text = "select * from product";
          for (fQuery->Open();
               !fQuery->Eof && !Terminated;
               fQuery->Next())
          {
          }

          fQuery->Close();
        }

        fConnection->Close();
      }

      if (TestLevel == 1)
      { // Only sleep if performing the first test
        Sleep(10);
      }

      DeleteObjects();
    }
  }

  /* GetMemoryInUse
   *
   * Retrieves the rough amount of memory in use by the application.
   */
  unsigned __int64 GetMemoryInUse()
  {
    unsigned __int64 memInUse = 0;
    MEMORY_BASIC_INFORMATION mbi;
    void* address = NULL;
    memset(&mbi, 0, sizeof(MEMORY_BASIC_INFORMATION));
    while (::VirtualQuery(address, &mbi, sizeof(mbi)) == sizeof(mbi))
    {
      if (mbi.State == MEM_COMMIT && mbi.Type == MEM_PRIVATE)
        memInUse += mbi.RegionSize;
      address = (void*) (((int) mbi.BaseAddress) + mbi.RegionSize);
      memset(&mbi, 0, sizeof(MEMORY_BASIC_INFORMATION));
    }
    return memInUse;
  } // GetMemoryInUse

  static const MaxThreads = 10;
  TThreadTestDB* Threads[MaxThreads];

  void StartEm(int Level)
  {
    for (int index = 0; index < MaxThreads; ++index)
    {
      Threads[index] = new TThreadTestDB(Level);
      Threads[index]->Resume();
    }
  }

  void StopEm()
  {
    for (int index = 0; index < MaxThreads; ++index)
    {
      TThread* thread = Threads[index];
      if (thread)
      {
        Threads[index] = NULL;
        thread->Terminate();
      }
    }
  }

  __fastcall TForm1::TForm1(TComponent* Owner) : TForm(Owner)
  {
    Sync = new TCriticalSection;
  }

  void __fastcall TForm1::Level1BtnClick(TObject *Sender)
  {
    SetLabel1Caption();
    Timer1->Tag = 1;
  }

  void __fastcall TForm1::Level2BtnClick(TObject *Sender)
  {
    SetLabel1Caption();
    Timer1->Tag = 2;
  }

  void __fastcall TForm1::Level3BtnClick(TObject *Sender)
  {
    SetLabel1Caption();
    Timer1->Tag = 3;
  }

  void TForm1::SetLabel1Caption()
  {
    Label1->Caption = "Started: " + Now().DateTimeString();
  }

  void __fastcall TForm1::StopBtnClick(TObject *Sender)
  {
    Timer1->Tag = 0;
    StopEm();
  }

  __int64 lastmax = 0;
  __int64 lastmin = 0;
  TDateTime lastmaxtime(0);

  void __fastcall TForm1::Timer1Timer(TObject *Sender)
  {
    if (Timer1->Tag > 0)
    {
      Sync->Acquire();
      int threadCount = ThreadCount;
      Sync->Release();

      StopEm();
      if (threadCount > 0)
      {
        Timer1->Interval = 100;
      }
      else
      {
        StartEm(Timer1->Tag);
        Timer1->Interval = 500;
      }
    }

    __int64 mem = GetMemoryInUse();
    Caption = String(ThreadCount) + " active threads : " +
      FormatFloat("#,##0", mem) + " bytes";

    static String ratemsg;
    if (mem > lastmax)
    { // Determines the minimum used since the last time the max changed
      ratemsg =
        "Last Min: " + FormatFloat("#,##0", lastmin) +
            " Max: " + FormatFloat("#,##0", mem) +
        "\nLast Increase: " + lastmaxtime.TimeString();

      lastmaxtime = Now();
      lastmax = mem;
      lastmin = 0;
    }

    if (lastmin == 0 || mem < lastmin)
    {
      lastmin = mem;
    }

    Label2->Caption = ratemsg +
        "\nCreated Objects: " + String(CreatedObjectCount);
  }


  // Unit1.dfm
  object Form1: TForm1
    Left = 374
    Top = 114
    Width = 435
    Height = 264
    Caption = 'Form1'
    Color = clBtnFace
    Font.Charset = DEFAULT_CHARSET
    Font.Color = clWindowText
    Font.Height = -11
    Font.Name = 'MS Sans Serif'
    Font.Style = []
    OldCreateOrder = False
    PixelsPerInch = 96
    TextHeight = 13
    object Label1: TLabel
      Left = 24
      Top = 12
      Width = 32
      Height = 13
      Caption = 'Label1'
    end
    object Label2: TLabel
      Left = 24
      Top = 72
      Width = 32
      Height = 13
      Caption = 'Label2'
    end
    object Level1Btn: TButton
      Left = 24
      Top = 36
      Width = 75
      Height = 25
      Caption = 'Test Level 1'
      TabOrder = 0
      OnClick = Level1BtnClick
    end
    object Level2Btn: TButton
      Left = 108
      Top = 36
      Width = 75
      Height = 25
      Caption = 'Test Level 2'
      TabOrder = 2
      OnClick = Level2BtnClick
    end
    object Level3Btn: TButton
      Left = 192
      Top = 36
      Width = 75
      Height = 25
      Caption = 'Test Level 3'
      TabOrder = 3
      OnClick = Level3BtnClick
    end
    object StopBtn: TButton
      Left = 272
      Top = 36
      Width = 75
      Height = 25
      Caption = 'Stop'
      TabOrder = 1
      OnClick = StopBtnClick
    end
    object Timer1: TTimer
      Interval = 500
      OnTimer = Timer1Timer
      Left = 4
      Top = 4
    end
  end 
Re: TADOQuery Memory Leak
#170
Author: Dmitry
Date: Tue, 01 May 2007 06:12
13 lines
1353 bytes
Clayton,

> However, the fact that the leak
> doesn't happen until the ADO objects are put to use tells me that the leak
> is somewhere outside of the domain of the VCL memory manager.

I have read the topik but still not sure weather problem has solution.
Could you give more explanation.
Has code given by you a memory leaks or not?

Thank you.
Dmitry
Re: TADOQuery Memory Leak
#171
Author: Clayton Arends
Date: Tue, 01 May 2007 17:33
17 lines
1751 bytes
Running on my computer there appears to be a leak.

If the leak is in the VCL then anybody should be able to duplicate the 
result.

If the leak is in my code then somebody might be able to find it.  Though, I 
seriously doubt my code is at fault unless there is some initialization 
routine that needs to be called.

If the leak is somewhere in the ADO drivers then it will depend on the 
driver versions whether the leak exists or not.

When I find some more time I think I will try this test again without 
threads to see what happens.  I'm slapping my head right now for not trying 
that before posting my previous results.

- Clayton 
Re: TADOQuery Memory Leak
#172
Author: Patrick Wong
Date: Thu, 03 May 2007 03:58
36 lines
2955 bytes
"Clayton Arends" <nospam_claytonarends@hotmail.com> wrote in message 
news:4636005c$1@newsgroups.borland.com...
>I found time off and on during the week to test this issue.  I made two 
>test applications.  One in BCB6 and one in BDS2006 Delphi personality. 
>Both applications were built standalone.  Both applications yielded almost 
>identical results.  Memory consumption was measured using a custom memory 
>usage function and the Windows Task Manager.  I have included my BCB6 test 
>code at the end of this post if anyone wishes to double-check my test and 
>results.
>
> At one point, to rule out the VCL implementation of ADO, I rewrote the 
> test to use the raw ADO interface classes (_Connection, and _RecordSet) 
> with similar memory leak results.
>

Grateful for your effort.  Since the root cause is still a mystery, I 
rewrote the database access routine with DBExpress for trial.  I also let 
the application run over night and the test shows that using DBExpress the 
significant memory growth over time does not happen anymore.

>
> I performed some general searches for ADO memory leaks and found some 
> scattered references.  Many threads concluded that the leaks were caused 
> by misuse of the ADO objects.  Some concluded that certain versions of ADO 
> drivers were causing the leaks.  A majority had no conclusions.  It is 
> hard to know for sure what is going on here.
>

In this thread 
http://groups.google.ru/groups?hl=ru&lr=&threadm=41ed54ce%241%40newsgroups.borland.com&rnum=6&prev=/groups%3Fas_q%3DTADOCommand%26as_ugroup%3D*borland*%26as_scoring%3Dd%26lr%3D%26num%3D20%26hl%3Dru

some people said they spot the problem, while other claimed that the fix was 
already applied to Delphi 7.  But I find both my BCB6 and BDS2006 are still 
having the old code.  If anyone can kindly have a trial please let me know 
the result.  Thanks in advance. 
Re: TADOQuery Memory Leak
#173
Author: Arnie
Date: Thu, 03 May 2007 13:47
51 lines
3356 bytes
"Clayton Arends" <nospam_claytonarends@hotmail.com> wrote in message 
news:463779b7$1@newsgroups.borland.com...
> Running on my computer there appears to be a leak.
>
> If the leak is in the VCL then anybody should be able to duplicate the 
> result.
>
> If the leak is in my code then somebody might be able to find it.  Though, 
> I seriously doubt my code is at fault unless there is some initialization 
> routine that needs to be called.
>
> If the leak is somewhere in the ADO drivers then it will depend on the 
> driver versions whether the leak exists or not.
>
> When I find some more time I think I will try this test again without 
> threads to see what happens.  I'm slapping my head right now for not 
> trying that before posting my previous results.
>
> - Clayton

BCB5, ADOExpress

I don't think it's the threads.  We have a service that runs 24/7.  The work 
is done in a thread other than the main service thread.  This thread is 
created when the service starts and only terminates when the service is 
stopped.

The thread sleeps and wakes up every one minute.  It has a list of one or 
more database connections.  When the thread wakes up it creates the needed 
DB objects, opens a connection to a DB and looks for work to do (one or more 
queries).  Then, it closes the connection and destroys all DB objects.  If 
there are more DB connections defined, it repeats the process within the 
same wake cycle.

There are memory leaks and the amount leaked depends on the DB in use.  We 
support Oracle, SQL Server and DB2.  We use ADO over ODBC for DB2.  The 
others use their own OLEDB drivers.

I can see that some of the leaks are due to losing unnamed semaphore handles 
when looking at the service with Process Explorer.  I'm guessing it's either 
ADO or OLEDB that's losing them but can't tell for sure.

I have reproduced the problem in a simple console app.  Loop in main(), 
create the TADOConnection, open, close and delete it.  Sleep for 20 seconds, 
rinse and repeat.

The bottom line is that our customers have to restart the service every day 
or two, a real pain.  So, I'd be interested to know if you find out anything 
further.

- Arnie 
Re: TADOQuery Memory Leak
#174
Author: Clayton Arends
Date: Thu, 03 May 2007 15:12
18 lines
2087 bytes
> I have reproduced the problem in a simple console app.  Loop in main(), 
> create the TADOConnection, open, close and delete it.  Sleep for 20
> seconds, rinse and repeat.

It looks like you've done that part of the test for me.

Your application situation is very similar to mine.  I have a service app 
that is part of a 3-tier database solution.  Memory leaks have been a 
problem for us since we use ADO exclusively at that level.  The problem is 
so pervasive that we are building into the app a scheduled nightly "restart" 
of our service so that the OS can clean up all of the memory leaks.

The next test I can try (or someone else can) would be to import the ADO COM 
directly and use those interfaces rather than the ones supplied by Borland. 
This test will rule in/out the VCL as being the root of the cause.  I'll try 
to get to that this weekend.

- Clayton
Thread Navigation

This is a paginated view of messages in the thread with full content displayed inline.

Messages are displayed in chronological order, with the original post highlighted in green.

Use pagination controls to navigate through all messages in large threads.

Back to All Threads