Thread View: borland.public.cppbuilder.internet.socket
17 messages
17 total messages
Started by Stefano Bonifazi
Tue, 07 Nov 2006 09:18
Help: HTTP and Unicode web pages
Author: Stefano Bonifazi
Date: Tue, 07 Nov 2006 09:18
Date: Tue, 07 Nov 2006 09:18
35 lines
2158 bytes
2158 bytes
Hi all! :)
I need to download web pages that contain Unicode text!
I thought to use Indy HTTP component as Indy is known as the best
Internet suite for BCB... I use version 9, the latest I think for BCB6.
In my early tests I wish only to show the downloaded page in a TntMemo
component (Tnt components are capable of handling Unicode text).
I firstly tried to use simply the return of the Get(AnsiString URL)
function ... but in the Memo appaired strange characters other than
those in my web page (a cyrillic page).
Mr Lebeau told me to use the overloaded function and put all in raw
stream! I used TTntMemoryStream
TTntMemoryStream* pstream=new TTntMemoryStream;
IdHTTP1->Get("http://someRussianwebpage.htm",(TStream*)pstream);
then if I write that stream on file
pstream->SaveToFile("c:\\prova.htm");
in that page there are again those different characters something like:
����� ��������� ��������� ������� ������ ��������
if use TntMemo1->Lines->LoadFromStream(pstream); nothing is shown
if i load in a WideString this way
WideString ws=WideString((char*)pstream->Memory);
TntMemo1->Text=ws;
the same ugly characters are showed :(
Is there anyone who can help me?
Thank U very much in advance!
Stefano B
Re: Help: HTTP and Unicode web pages
Author: Hans Galema
Date: Tue, 07 Nov 2006 10:33
Date: Tue, 07 Nov 2006 10:33
15 lines
1572 bytes
1572 bytes
Stefano Bonifazi wrote: > In my early tests I wish only to show the downloaded page in a TntMemo > component (Tnt components are capable of handling Unicode text). So you only want to show the source of that page. If you wanted to display the page you could use a T(Cpp)WebBrowser component. Is the page after download correctly displayed in IE/Firefox? > ����� ��������� ��������� ������� ������ �������� Now before I say things that I cannot make true: can you give the url of such a page on the internet so we could have a look? Hans.
Re: Help: HTTP and Unicode web pages
Author: Stefano Bonifazi
Date: Tue, 07 Nov 2006 11:10
Date: Tue, 07 Nov 2006 11:10
33 lines
2332 bytes
2332 bytes
Hi Mr. Galema! Thank You very much for Your reply! Hans Galema wrote: > So you only want to show the source of that page. If you wanted to display > the page you could use a T(Cpp)WebBrowser component. No I will want to use the text I got in future.. for now I've been only testing .. the first steps, I need to parse the Unicode text then! >Is the page after > download correctly displayed in IE/Firefox? Very good question! After saving a page as htm, by using TTntMemoryStream::SaveToFile method few pages are then loaded fine with Opera browser, most simply display those ugly characters instead! The pages loaded fine with opera anyway were in php.. mayb some code simply reloaded the text from internet connection whel loaded with opera! > Now before I say things that I cannot make true: can you > give the url of such a page on the internet so we could have > a look? No problem... I will need my code to work with common pages .. so I just used google to get some Russian pages because I thought the problem was with php pages I was using as tests... but then I tried even a simple htm page ( http://scream3.film.ru/story.htm ) .. of course this is only a test page! I won't work with that! I used it as a test page ;) Hope You can help me!!! Thanks again!! Stefano
Re: Help: HTTP and Unicode web pages
Author: Stefano Bonifazi
Date: Tue, 07 Nov 2006 11:31
Date: Tue, 07 Nov 2006 11:31
28 lines
2316 bytes
2316 bytes
Hi again! continuing with the test I have noticed some interesting thing that mayb could be useful to u in order to help me! I tried to load http://scream3.film.ru/story.htm with Firefox from the web ant it showed the same symbols instead the cyrillic ones!!!!!!! Again I saved that page displayed correctly by opera with this browser, if I try to reload it with the same Opera, now it shows those ugly characters instead!!! The same happens if I try to load it with Frontpage.. while MS Word load it fine!!!!! Mystery of Unicode? Help I'm going crazy!! Stefano Hans Galema wrote: > Stefano Bonifazi wrote: >> In my early tests I wish only to show the downloaded page in a TntMemo >> component (Tnt components are capable of handling Unicode text). > > So you only want to show the source of that page. If you wanted to display > the page you could use a T(Cpp)WebBrowser component. Is the page after > download correctly displayed in IE/Firefox? > >> ����� ��������� ��������� ������� ������ �������� > > Now before I say things that I cannot make true: can you > give the url of such a page on the internet so we could have > a look? > > Hans.
Re: Help: HTTP and Unicode web pages
Author: Hans Galema
Date: Tue, 07 Nov 2006 11:33
Date: Tue, 07 Nov 2006 11:33
16 lines
1722 bytes
1722 bytes
Stefano Bonifazi wrote: > ... but then I tried even a simple > htm page ( http://scream3.film.ru/story.htm ) As far as I can see that page does not contain 16 bit unicode characters. It does contain cyrillic characters and I have to adjust my browser to cyrillic characters before they are displayed correct. When I save the page and look in it with a hexeditor there are only 8bit characters. The saved page does display correct again in Firefox. Now to be sure that the browser while saving does not change things I would like you to grab that page with TIdHTTP, save it and post it in the attachmentsgroup. Hans.
Re: Help: HTTP and Unicode web pages
Author: Hans Galema
Date: Tue, 07 Nov 2006 11:35
Date: Tue, 07 Nov 2006 11:35
10 lines
1268 bytes
1268 bytes
Stefano Bonifazi wrote: > I tried to load http://scream3.film.ru/story.htm with Firefox from the > web ant it showed the same symbols instead the cyrillic ones!!!!!!! You have to tell Firefox to use cyrillic: View | Character Encoding | Cyrillic Hans.
Re: Help: HTTP and Unicode web pages
Author: Stefano Bonifazi
Date: Tue, 07 Nov 2006 11:58
Date: Tue, 07 Nov 2006 11:58
26 lines
1992 bytes
1992 bytes
Hans Galema wrote: > Stefano Bonifazi wrote: >> ... but then I tried even a simple >> htm page ( http://scream3.film.ru/story.htm ) > > As far as I can see that page does not contain 16 bit unicode characters. > It does contain cyrillic characters and I have to adjust my > browser to cyrillic characters before they are displayed correct. > > When I save the page and look in it with a hexeditor there are only > 8bit characters. The saved page does display correct again in Firefox. > > Now to be sure that the browser while saving does not change things > I would like you to grab that page with TIdHTTP, save > it and post it in the attachmentsgroup. I think there is no need to attach as I tried to lad the saved page with FireFox too and told that to use cyrillic.. and all was displayed fine! So now how can I tell Tnt to display them fine.. or load fine that code in a widestring? Thank You very much for the quick answer!! Stefano B.
Re: Help: HTTP and Unicode web pages
Author: Hans Galema
Date: Tue, 07 Nov 2006 12:14
Date: Tue, 07 Nov 2006 12:14
16 lines
1562 bytes
1562 bytes
Stefano Bonifazi wrote: > I think there is no need to attach .... > > So now how can I tell Tnt to display them fine.. or load fine that code > in a widestring? TRichEdit can display them too. I can type here cyrillic text in TRichEdit, Wordpad and Notepad. Quite possible that that was only possible after I installed Russian as inputlanguage on XP. But ... If you want me to help you then do attach. Hans.
Re: Help: HTTP and Unicode web pages
Author: Stefano Bonifazi
Date: Tue, 07 Nov 2006 12:53
Date: Tue, 07 Nov 2006 12:53
20 lines
1660 bytes
1660 bytes
Ok I post that in the attachments group Thank U again! Hans Galema wrote: > Stefano Bonifazi wrote: > >> I think there is no need to attach .... >> >> So now how can I tell Tnt to display them fine.. or load fine that >> code in a widestring? > > TRichEdit can display them too. > > I can type here cyrillic text in TRichEdit, Wordpad and Notepad. > Quite possible that that was only possible after I installed Russian > as inputlanguage on XP. > > But ... If you want me to help you then do attach. > > Hans.
Re: Help: HTTP and Unicode web pages
Author: Hans Galema
Date: Tue, 07 Nov 2006 14:01
Date: Tue, 07 Nov 2006 14:01
35 lines
2411 bytes
2411 bytes
Stefano Bonifazi wrote:
> Ok I post that in the attachments group
> TTntMemoryStream* pstream=new TTntMemoryStream;
> IdHTTP1->Get("http://scream3.film.ru/story.htm",(TStream*)pstream);
> WideString ws=WideString((char*)pstream->Memory);
> pstream->SaveToFile("c:\\Test.html");
> TntMemo1->Text=ws;
> delete pstream;
So you meant:
TTntMemoryStream* pstream=new TTntMemoryStream;
IdHTTP1->Get("http://scream3.film.ru/story.htm",(TStream*)pstream);
pstream->SaveToFile("c:\\Test.html");
delete pstream;
The attached page has nearly the same size as when saved with Firefox.
Also here the text in cyrillic are characters in 8bit. (Cyrillic: Windows 1251).
What is the reason that you use TTntMemoryStream? You could as well
use TMemoryStream I think.
TMemoryStream* pstream=new TMemoryStream;
IdHTTP1->Get("http://scream3.film.ru/story.htm",(TStream*)pstream);
pstream->SaveToFile("c:\\Test.html");
delete pstream;
Just converting 8bit to Widestring will not do I think to display all normal.
During the conversion from 8 to 16 bit you should tell the converter that
-some- character encoding is 8bit cyrillic.
Sory. I'm not familiar with that.
Hans.
Re: Help: HTTP and Unicode web pages
Author: Bob Gonder
Date: Tue, 07 Nov 2006 15:57
Date: Tue, 07 Nov 2006 15:57
23 lines
1955 bytes
1955 bytes
Stefano Bonifazi wrote: > The same happens if I try to load it with >Frontpage.. while MS Word load it fine!!!!! Mystery of Unicode? It doesn't sound like it is Unicode. It just sounds like it is a CodePage thing. Set the code page of your Memo to Cyrillic, and it should display ok. See http://en.wikipedia.org/wiki/CP866 But the data looks more like http://en.wikipedia.org/wiki/ISO/IEC_8859-5 But that might not help your unicode parsing problem. This is the hex of a text portion of that html: (bh is ByteHex) <bh:f0><bh:ed><bh:ee><bh:e9> <bh:ca><bh:e0><bh:eb><bh:e8><bh:f4><bh:ee><bh:f0><bh:ed><bh:e8><bh:e8>. <bh:ca><bh:ee><bh:f8><bh:ec><bh:e0><bh:f0><bh:fb> <bh:e2><bh:ee><bh:e7><bh:e2><bh:f0><bh:e0><bh:f9><bh:e0><bh:fe><bh:f2><bh:f1><bh:ff> <bh:ea> <bh:ed><bh:e5><bh:e9> <bh:f1><bh:ed><bh:ee><bh:e2><bh:e0> <bh:e8> <bh:f1><bh:ed><bh:ee><bh:e2><bh:e0>,
Re: Help: HTTP and Unicode web pages
Author: Stefano Bonifazi
Date: Tue, 07 Nov 2006 17:00
Date: Tue, 07 Nov 2006 17:00
16 lines
1600 bytes
1600 bytes
Hans Galema wrote: > Just converting 8bit to Widestring will not do I think to display all > normal. > During the conversion from 8 to 16 bit you should tell the converter that > -some- character encoding is 8bit cyrillic. > > Sory. I'm not familiar with that. > > Hans. Thank You very much for Ur help! Where should I post that matter then? Best regards, Stefano B.
Re: HTTP and Unicode web pages
Author: Remy Lebeau (TeamB)
Date: Tue, 07 Nov 2006 17:42
Date: Tue, 07 Nov 2006 17:42
24 lines
1643 bytes
1643 bytes
"Stefano Bonifazi" <stefboombastic@email.it> wrote in message news:45504f4b@newsgroups.borland.com... > in that page there are again those different characters something like: > > ����� ��������� ��������� ������� ������ �������� That is because that is what the server is actually sending to begin with. You can verify that for yourself using a packet sniffer, such as Ethereal ()http://www.ethereal.com, to look at the data being transmitted. > if use TntMemo1->Lines->LoadFromStream(pstream); nothing is shown Did you remember to reset the stream's Position back to 0 after Get() exits? > if i load in a WideString this way <snip> > the same ugly characters are showed :( As they should be. Gambit
Re: Help: HTTP and Unicode web pages
Author: Stefano Bonifazi
Date: Tue, 07 Nov 2006 17:50
Date: Tue, 07 Nov 2006 17:50
14 lines
1472 bytes
1472 bytes
Thank You for You reply! Bob Gonder wrote: > Stefano Bonifazi wrote: > Set the code page of your Memo to Cyrillic, and it should display ok. how? that is should I change the font Charset (I've already done without success!)? ... what is a Memo codepage? > But that might not help your unicode parsing problem. And so . what should I do? Thank U again! Stefano B.
Re: HTTP and Unicode web pages
Author: Stefano Bonifazi
Date: Tue, 07 Nov 2006 18:14
Date: Tue, 07 Nov 2006 18:14
26 lines
1797 bytes
1797 bytes
Thank You Mr. Lebeau for Your reply!! Remy Lebeau (TeamB) wrote: > That is because that is what the server is actually sending to begin with. > You can verify that for yourself using a packet sniffer, such as Ethereal > ()http://www.ethereal.com, to look at the data being transmitted. Thanks for the advice that will help me in all internet Tests ;) > >> if use TntMemo1->Lines->LoadFromStream(pstream); nothing is shown > > Did you remember to reset the stream's Position back to 0 after Get() exits? No i forgot... what a silly! > >> if i load in a WideString this way > <snip> >> the same ugly characters are showed :( > > As they should be. And then how can I convert them to their proper characters when I display them by using TNT components? Thank U in advance!!! Stefano B.
Re: HTTP and Unicode web pages
Author: Remy Lebeau (TeamB)
Date: Tue, 07 Nov 2006 19:17
Date: Tue, 07 Nov 2006 19:17
13 lines
1398 bytes
1398 bytes
"Stefano Bonifazi" <stefboombastic@email.it> wrote in message news:4550cd15@newsgroups.borland.com... > And then how can I convert them to their proper characters > when I display them by using TNT components? They are already proper characters to begin with. There is nothing wrong with the data you have shown. So it has to be a problem with how Tnt is interpreting the data. Gambit
Re: Help: HTTP and Unicode web pages
Author: Bob Gonder
Date: Wed, 08 Nov 2006 15:02
Date: Wed, 08 Nov 2006 15:02
14 lines
1569 bytes
1569 bytes
Stefano Bonifazi wrote: >Bob Gonder wrote: >without success!)? ... what is a Memo codepage? See "SETTEXTEX Structure" and the EM_SETTEXTEX message for RichEdit controls. >> But that might not help your unicode parsing problem. > >And so . what should I do? Did you try MultiByteToWideChar() with the proper codepage?
Thread Navigation
This is a paginated view of messages in the thread with full content displayed inline.
Messages are displayed in chronological order, with the original post highlighted in green.
Use pagination controls to navigate through all messages in large threads.
Back to All Threads