🚀 go-pugleaf

<MR> wrote in message news:ea9f22d43tn11hsanfouueqru7oo3lige6@4ax.com...

> Because this content is more than one page and I want to access
> to it offline, I tried to get the same pages with TIdHTTP's Get()-
> methode (hoping, because login-information is already stored in the
> cookie (or somewhere else on my computer), I would get access
> to it without providing any password information. But unfortunately
> I was wrong.

Indy does not persistently store cookies automatically.  That is your own
responsibility to manage.  Indy only stores cookies in memory on a
per-instance basis.  Once the HTTP's cookie manager is destroyed, so are the
cookies that it currently has.

> Comparing IE's source and TIdHTTP::Get()-result is also quite
> different: In the first one is my login-information, in the second one
> (when displayed with IE afterwards) quite the same page I get in
> IE if I would have not provided my login-information.

How exactly are they different?  What does your actual code look like?  What
does the downloaded HTML look like between IE and TIdHTTP?

Do note that many servers send different content to different browser types.
Many servers prefer IE, and send something else to non-IE browsers.  You can
use the TIdHTTP's UserAgent property to make the server think that IE is
connecting to it, when it really isn't.

> So, how can I download the required pages. Can I use TIdHTTP::Get()
> at all? Can I provide the Get()-command with some login/password
> information.

> The very best solution would be, if there would be a property in TidHTTP,
> which would allow having access to IE's cookies ;-)

That is not going to happen.  For starters, not everyone uses IE to begin
with.  Second, the formats used for storing cookies on the hard drive are
not standardized at all.  Every browser stores cookies differently.  If you
want access to IE's cookies, then you have to parse them yourself and then
put the extracted data into the TIdHTTP's cookie manager manually.

> Or can I perhaps call IE from inside my program and let it
> download and store the required pages automatically?

You can use Borland's TCppWebBrowser component for that.  It is a wrapper
for IE's ActiveX control.


Gambit

"Remy Lebeau \(TeamB\)" <no.spam@no.spam.com> schrieb:

>> Or can I perhaps call IE from inside my program and let it
>> download and store the required pages automatically?
>
>You can use Borland's TCppWebBrowser component for that.  It is a wrapper
>for IE's ActiveX control.

This sounds good, but how can I get the downloaded address-content
into a string (for saving and analyzing for further links)? 

Thanks,

Michael

<MR> wrote in message news:j5tg22pjjfk846c5okc6iqplntcvn5tlh3@4ax.com...

> This sounds good, but how can I get the downloaded
> address-content into a string (for saving and analyzing
> for further links)?

That has been answered many times before.  Please go to http://www.deja.com
and search through the newsgroup archives.


Gambit

"Remy Lebeau \(TeamB\)" <no.spam@no.spam.com> schrieb:
>> This sounds good, but how can I get the downloaded
>> address-content into a string (for saving and analyzing
>> for further links)?
>
>That has been answered many times before.  Please go to http://www.deja.com
>and search through the newsgroup archives.

I found a fitting website at

http://www.bytesandmore.de/rad/index.htm?http://www.bytesandmore.de/rad/cpp/snipp/sc08019.php
which contains even a working demo-project.

Unfortunately when implementing it, I got some strange behaviour,
which does not occur in the demo.

The function I'm interested in was 

bool ExtractHtmlCodeOfCurrentPage(TCppWebBrowser* pCppWebBrowser,
TStrings* pOutputStrings, bool blHtml).

I called that from a global function (my code):
  
a1 AnsiString __fastcall GetHTMLTextCode(AnsiString _web_address, bool
_html_flag)
a2 {
a3 static TCppWebBrowser *cpp_web_browser=NULL;
a4  if(cpp_web_browser==NULL)
a5    cpp_web_browser=new TCppWebBrowser((TComponent *)NULL);
a6  cpp_web_browser->Navigate(WideString(_web_address));
a7  AnsiString ret=GetHTMLTextCode(cpp_web_browser, _html_flag);
a8  return(ret);
a9 }

with 

b01 AnsiString __fastcall GetHTMLTextCode(TCppWebBrowser
*_cpp_web_browser, bool _html_flag)
b02 {
b03 AnsiString ret;
b04 while(_cpp_web_browser->Busy) {
b05   Sleep(500/*ms*/);
b06    Application->ProcessMessages();
b07  }

b08  TMemo *memo=new TMemo((TComponent *)NULL);
b09  memo->Visible=false;
b10  memo->Parent=Application->MainForm;
b11  bool success=ExtractHtmlCodeOfCurrentPage(_cpp_web_browser,
memo->Lines, _html_flag);
b12  if(success) {
b13    for(size_t i=0; i<memo->Lines->Count; i++)
b14      ret=ret+(i==0 ? "" : "\n")+memo->Lines->Strings[i];
b15  }
b16  delete memo;
b17  return(ret);
b18}

The current main-problem is that I do not get all lines of the
html-source. They are omitted after some lines. Despite of testing
TCppWebBrower's IsBusy-property...

But there also occur some strange behavior, which might depend with
this problem somehow:

When trying to use a TCppWebBrowser already in the form (instead of
creating a new one (a03-a05) , as soon as I try to call Navigate()
(a6) I get an exception:

  Project ... raised an exception class EAccessViolation with message
'Access violation at address 00321986 in module 'bcbie100.bpl'. Read
of address 000003D0'.

~~

If I create a TCppWebBrowser as local (i.e. non-static) and delete it
at the end of the function a whole bunch of IE-Instances are opened as
side-effect. (So it seems to me that the iniitalization of
TCppWebBrowser has not been finished yet and because the
TCppWebBrowser-handle is no more valid after delete IE itself is
opened: Perhaps this is the reason for the incompletion of the
html-code).
  
~~

The strange thing is: in the recompiled demo-source all this works
fine, including the Navigate() from a TCppBuilder-component inside the
form...

What might be wrong here?

Thanks a lot,

Michael