🚀 go-pugleaf

RetroBBS NetNews Server

Inspired by RockSolid Light RIP Retro Guy

Article View: borland.public.cppbuilder.internet.web
Article #564

Re: How to use TIdHTTP::Get() to get 'login-protected' website-contents (when knowing the user-name/password)?

#564
From: MR
Date: Fri, 31 Mar 2006 06:06
94 lines
4107 bytes
"Remy Lebeau \(TeamB\)" <no.spam@no.spam.com> schrieb:
>> This sounds good, but how can I get the downloaded
>> address-content into a string (for saving and analyzing
>> for further links)?
>
>That has been answered many times before.  Please go to http://www.deja.com
>and search through the newsgroup archives.

I found a fitting website at

http://www.bytesandmore.de/rad/index.htm?http://www.bytesandmore.de/rad/cpp/snipp/sc08019.php
which contains even a working demo-project.

Unfortunately when implementing it, I got some strange behaviour,
which does not occur in the demo.

The function I'm interested in was 

bool ExtractHtmlCodeOfCurrentPage(TCppWebBrowser* pCppWebBrowser,
TStrings* pOutputStrings, bool blHtml).

I called that from a global function (my code):
  
a1 AnsiString __fastcall GetHTMLTextCode(AnsiString _web_address, bool
_html_flag)
a2 {
a3 static TCppWebBrowser *cpp_web_browser=NULL;
a4  if(cpp_web_browser==NULL)
a5    cpp_web_browser=new TCppWebBrowser((TComponent *)NULL);
a6  cpp_web_browser->Navigate(WideString(_web_address));
a7  AnsiString ret=GetHTMLTextCode(cpp_web_browser, _html_flag);
a8  return(ret);
a9 }

with 

b01 AnsiString __fastcall GetHTMLTextCode(TCppWebBrowser
*_cpp_web_browser, bool _html_flag)
b02 {
b03 AnsiString ret;
b04 while(_cpp_web_browser->Busy) {
b05   Sleep(500/*ms*/);
b06    Application->ProcessMessages();
b07  }

b08  TMemo *memo=new TMemo((TComponent *)NULL);
b09  memo->Visible=false;
b10  memo->Parent=Application->MainForm;
b11  bool success=ExtractHtmlCodeOfCurrentPage(_cpp_web_browser,
memo->Lines, _html_flag);
b12  if(success) {
b13    for(size_t i=0; i<memo->Lines->Count; i++)
b14      ret=ret+(i==0 ? "" : "\n")+memo->Lines->Strings[i];
b15  }
b16  delete memo;
b17  return(ret);
b18}

The current main-problem is that I do not get all lines of the
html-source. They are omitted after some lines. Despite of testing
TCppWebBrower's IsBusy-property...

But there also occur some strange behavior, which might depend with
this problem somehow:

When trying to use a TCppWebBrowser already in the form (instead of
creating a new one (a03-a05) , as soon as I try to call Navigate()
(a6) I get an exception:

  Project ... raised an exception class EAccessViolation with message
'Access violation at address 00321986 in module 'bcbie100.bpl'. Read
of address 000003D0'.

~~

If I create a TCppWebBrowser as local (i.e. non-static) and delete it
at the end of the function a whole bunch of IE-Instances are opened as
side-effect. (So it seems to me that the iniitalization of
TCppWebBrowser has not been finished yet and because the
TCppWebBrowser-handle is no more valid after delete IE itself is
opened: Perhaps this is the reason for the incompletion of the
html-code).
  
~~

The strange thing is: in the recompiled demo-source all this works
fine, including the Navigate() from a TCppBuilder-component inside the
form...

What might be wrong here?

Thanks a lot,

Michael
Technical Details
Message-ID: <5ofp22p5s6oil5igobqj56cpolfjpcleor@4ax.com>
Path: arc1-ca.pugleaf.net!g2news1.google.com!news1.google.com!news3.google.com!border1.nntp.dca.giganews.com!nntp.giganews.com!newsfeed00.sul.t-online.de!newsfeed01.sul.t-online.de!t-online.de!newsgroups.borland.com!not-for-mail
References: <ea9f22d43tn11hsanfouueqru7oo3lige6@4ax.com> <44282eda$1@newsgroups.borland.com> <j5tg22pjjfk846c5okc6iqplntcvn5tlh3@4ax.com> <44287752$3@newsgroups.borland.com>

Show Full Headers
Complete Article Headers
X-Google-Thread: feeec,3b4cf1b4db85efc1
X-Google-Attributes: gidfeeec,public
X-Google-Language: ENGLISH,ASCII-7-bit
From: <MR>
Newsgroups: borland.public.cppbuilder.internet.web
Subject: Re: How to use TIdHTTP::Get() to get 'login-protected' website-contents (when knowing the user-name/password)?
Date: Fri, 31 Mar 2006 08:06:44 +0200
Message-ID: <5ofp22p5s6oil5igobqj56cpolfjpcleor@4ax.com>
References: <ea9f22d43tn11hsanfouueqru7oo3lige6@4ax.com> <44282eda$1@newsgroups.borland.com> <j5tg22pjjfk846c5okc6iqplntcvn5tlh3@4ax.com> <44287752$3@newsgroups.borland.com>
X-Newsreader: Forte Free Agent 3.3/32.846
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
NNTP-Posting-Host: 84.57.21.199
X-Trace: newsgroups.borland.com 1143785268 84.57.21.199 (30 Mar 2006 22:07:48 -0700)
Lines: 94
Path: arc1-ca.pugleaf.net!g2news1.google.com!news1.google.com!news3.google.com!border1.nntp.dca.giganews.com!nntp.giganews.com!newsfeed00.sul.t-online.de!newsfeed01.sul.t-online.de!t-online.de!newsgroups.borland.com!not-for-mail