🚀 go-pugleaf

Eduardo Jauch escreveu:
> I search on help but not find any error code 6 returned by 
> WSAGetLastError...

I put the WSAGetLastError after the line that causes the error...
Now it allways return 0... What seens more correct, because recv is 
returning 0 and not -1...

But I'm curious now...

Why is recv returning 0?
It wouldn't wait until receive some data?

Eduardo Jauch escreveu:
> Why is recv returning 0?
> It wouldn't wait until receive some data?

I was reading that this can mean that the connection was closed...
If this is true, I need to check for 0 bytes received and 
reconnect/send/recv(in loop) again?

Thanks!

Eduardo Jauch escreveu:
> Eduardo Jauch escreveu:
>> Why is recv returning 0?
>> It wouldn't wait until receive some data?
> 
> I was reading that this can mean that the connection was closed...
> If this is true, I need to check for 0 bytes received and 
> reconnect/send/recv(in loop) again?
> 
> Thanks!

Well... The answer to my question is yes...
The server is returning 500 (Internal Server Error)...
Maybe the server is set to refuse connection if the client try many 
"recv" in a short time...

I'm trying to reconnect after X milliseconds...
Reconnect imediatly don't work...

Eduardo Jauch escreveu:
> 
> Well... The answer to my question is yes...
> The server is returning 500 (Internal Server Error)...
> Maybe the server is set to refuse connection if the client try many 
> "recv" in a short time...
> 
> I'm trying to reconnect after X milliseconds...
> Reconnect imediatly don't work...

No...

even if i wait 4000 milliseconds, the server does not respond...
But if I kill the thread and create it again (what takes far few from 
4000 milliseconds) the server turn to respond again...

I don't think my thread is the "problem"...
But take a look anyway...

Someone have this problem before?

thanks! :)

"Eduardo Jauch" <eduardo.jauch@gmail.com> wrote in message
news:45e24baa@newsgroups.borland.com...

> one of the recv return 0 bytes

That means that the other party has closed the connection gracefully.

> i'm not setting the socket to nonbloking and the connection does
> not time out by server not responding nor something like that.

recv does not return 0 in non-blocking mode when a timeout occurs.  It
returns SOCKET_ERROR (-1) and then WSAGetLastError() returns
WSAETIMEOUT.

> I'm using the recv to return only the HEAD (not using GET).

You are probably not taking into account that the server will likely
close the connection after sending its reply to any command.  That is
the default behavior for HTTP 1.0 requests.  The connection is only
left open when a keep-alive is negotiated between the client and the
server as part of the command/response exchange.  I'm guessing that
you are not reading the HTTP protocol documentation carefully enough,
as this is covered.

> When i use WSAGetLastError, always return code 6
> I search on help but not find any error code 6 returned by
> WSAGetLastError...

WSAGetLastError() is only meaningful when a real error occurs.  recv()
returning 0 is not an error condition.


Gambit

"Eduardo Jauch" <eduardo.jauch@gmail.com> wrote in message
news:45e2b430$1@newsgroups.borland.com...

> The server is returning 500 (Internal Server Error)...
> Maybe the server is set to refuse connection if the client
> try many "recv" in a short time...

No.  Your HTTP request was malformed to begin with, so the server did
not know how to process it.  500 is a valid HTTP reply.  The server is
then likely closing the connection after sending its reply.


Gambit

Remy Lebeau (TeamB) escreveu:
> Your HTTP request was malformed to begin with, so the server did
> not know how to process it.  

No. My HTTP request is perfect.
Is after more than 15 iterations of the loop that I get the error. And i 
DON'T change the request any time.

I'm using this form:

char *first = "GET 
/agendamento-web/agendamento.do?task=forwardCalendario&forward=confirmarInformacoesDeVisto&requerenteAgendado[0].informacoesChecadas=true&agendamentoVO.agenciadorGrupoVO.seqAgenciadorGrupo=&locale=pt_BR&com.telecom.tipo_usuario=AGENCIA+DE+VIAGEM 
";
char final[1024];

strcpy(final, "HTTP/1.1\r\nHost: www.visto-eua.com.br\r\nCookie: ");
strcat(final, cookies);
strcat(final, "\r\n\r\n");

cookies is a char [] variable that holds the cookies (hehe) that I get 
on the first connect.

> 500 is a valid HTTP reply.  The server is
> then likely closing the connection after sending its reply.

Yes, it is. And told me that the connection is closed.


You think that the server could change the coockies from time to time? 
There a gigantic code for each session...

I'll change the code to take in account this, so refresh the cookies 
variable if the cookies returned changed.

Anyway, if I put a Sleep(5000) after test the return of recv to 0, yet I 
continue getting the error. But if I kill the thread and re-create it 
(what takes no more tnah ONE second), all work's fine...

Any Idea?

Remy Lebeau (TeamB) escreveu:
> "Eduardo Jauch" <eduardo.jauch@gmail.com> wrote in message
> news:45e24baa@newsgroups.borland.com...
> 
>> one of the recv return 0 bytes
> 
> That means that the other party has closed the connection gracefully.
> 

Yes. I find it. :)
The HEAD returned too says "Connection: closed"

> recv does not return 0 in non-blocking mode when a timeout occurs.  It
> returns SOCKET_ERROR (-1) and then WSAGetLastError() returns
> WSAETIMEOUT.
>

I'm a litle confused now...
I must use non-bloking or bloking to make the recv WAIT for the response?

> You are probably not taking into account that the server will likely
> close the connection after sending its reply to any command.  

No. I'm not. But is because the server don't do this. Only after many 
iterations (requests) it do that.

> That is the default behavior for HTTP 1.0 requests.  The connection is only
> left open when a keep-alive is negotiated between the client and the
> server as part of the command/response exchange.  I'm guessing that
> you are not reading the HTTP protocol documentation carefully enough,
> as this is covered.

Oh, I'm reading carefully, yet I don't understand all of it of course 
but I'm only begining ;)

It's exactly because this that I choose to use HTTP 1.1.
On the documentation says that the connection will keep alive for 
default. But if i misunderstood, plese, tell me.

> WSAGetLastError() is only meaningful when a real error occurs.  recv()
> returning 0 is not an error condition.

Yes. Now that I put it after the line on the recv that return the 0 
byte, it always return 0.

Because i'm not using HTTP 1.0 and the request (HEAD and GET) don't 
change every looping, I'm guessing that or the server is programed to 
close connections that make too requests within a few time, or it is 
changing the session cookies at that time...
I'm more inclinated to believe on the first.


Anyway, it don answer why I need to kill the thread to make things 
working again, even if I try to reconnect and begin the whole proccess 
from begining.

It's seens that my code have a problem that I not yet find...
Still looking for it...

Thanks :)

"Eduardo Jauch" <eduardo.jauch@gmail.com> wrote in message
news:45e3429c$1@newsgroups.borland.com...

> No. My HTTP request is perfect.

If it were, you would not be getting that error from the server in the
first place.  That error is almost always reported due to an incorrect
request, usually becvause you left required fields out, or formatting
them wrong.

> Is after more than 15 iterations of the loop that I get the error.

That does not mean that the requests were perfect every time to begin
with.

> char final[1024];
>
> strcpy(final, "HTTP/1.1\r\nHost: www.visto-eua.com.br\r\nCookie: ");
> strcat(final, cookies);
> strcat(final, "\r\n\r\n");

I would STRONGLY advise you to re-think the approach you are taking
with your socket programming.  You are leaving your self open to all
kinds of possible errors.  The first, and most common, one being
buffer overflows.  You would be much better off if you wrap your
socket code with more general operations (write a line, write a list
of lines, read a line, etc).  Your code would become a lot more solid,
and a lot easier to work with.

> You think that the server could change the coockies from time to
time?

Of course it can.  You have to look at the cookies of EVERY reply and
then update your cache accordingly.

I seriously think you need to stop trying to implement HTTP from
scratch and use a pre-made library.  You would save yourself a LOT of
hassle and problems.

> Anyway, if I put a Sleep(5000) after test the return of recv to 0,
> yet I continue getting the error.

The server's error reply has NOTHING to do with HOW you are
sending/receiving the data, and EVERYTHING to do with WHAT you are
sending.  Please show your complete request that is reporting the
error.


Gambit

"Eduardo Jauch" <eduardo.jauch@gmail.com> wrote in message
news:45e34585$1@newsgroups.borland.com...

> I'm a litle confused now...
> I must use non-bloking or bloking to make the recv WAIT for the
response?

Obviously, you have not been reading up on how to use sockets properly
in the first place, or how the HTTP protocol actually works.  Which is
all the more reason that you should be using a pre-made HTTP library
to do all of the hard work for you.

It does not matter whether you use a blocking or non-blocking socket.
You must continue reading from the socket until you receive a complete
blank line.  Then you parse the headers that were received (in
particular the "Content-Length" and "Transfer-Encoding" headers) to
figure out how much of the remaining reply data, if any, you need to
finish reading.

If you use a blocking socket, than recv() will wait for any bytes to
arrive before exiting.

IF you use a non-blickong socket, then you have to use select() to
know when bytes are available before then calling recv().  If you do
not, then recv() will return immediately and WSAGetLastError() will
return WSAEWOULDBLOCK.

In either case, it is your responsibility to cache any data that does
get read, and then parse that data to know whether more data has to be
read again.  You keep doing that in a loop until all intended data has
been received, or until an error occurs.

> No. I'm not. But is because the server don't do this. Only after
> many iterations (requests) it do that.

You still need to parse the reply headers.  The server will tell you
whether it is keeping the socket open on its end or not.  You have to
then decide whether to keep the socket open on your end as well, or to
close it for the next request.  This is basic HTTP behavior that you
have to implement properly.  You are sending a v1.1 request, so the
default behavior is to keep the socket open, but the server will send
a "Connection" header to confirm or reject that.

> Because i'm not using HTTP 1.0 and the request (HEAD and GET)
> don't change every looping, I'm guessing that or the server is
programed
> to close connections that make too requests within a few time

Very unlikely.  The whole point of using keep-alives is to allow
clients to make more, and faster, requests to the same server because
there is no overhead of tearing down a connection and reconnecting for
each new request.

> or it is changing the session cookies at that time...

That is a possibility.

> Anyway, it don answer why I need to kill the thread to make
> things working again

Because you are likely managing the socket incorrectly in the first
place.  That is frequently the case for people who choose to program
at the socket API level without first understanding what they are
actually doing.  Socket programming is not trivial to understand, it
takes awhile to grasp it correctly.


Gambit

Remy Lebeau (TeamB) escreveu:
>> No. My HTTP request is perfect.
> 
> If it were, you would not be getting that error from the server in the
> first place.  That error is almost always reported due to an incorrect
> request, usually becvause you left required fields out, or formatting
> them wrong.
> 

Maybe the cookies are set differently by server. This is the only thing
that could be wrong in my request.

>> Is after more than 15 iterations of the loop that I get the error.
> 
> That does not mean that the requests were perfect every time to begin
> with.

If I don't change anything at my request on every step. If the cookies
are different, so, it may be this that get incorrectly at some point...

> I would STRONGLY advise you to re-think the approach you are taking
> with your socket programming.  You are leaving your self open to all
> kinds of possible errors.  The first, and most common, one being
> buffer overflows.  You would be much better off if you wrap your
> socket code with more general operations (write a line, write a list
> of lines, read a line, etc).  Your code would become a lot more solid,
> and a lot easier to work with.

Sorry, but I don't understand... How was this? What you mean with "write
a line, write a list of lines, read a line"?

> Of course it can.  You have to look at the cookies of EVERY reply and
> then update your cache accordingly.

I'll change the code to update the cookies.

> I seriously think you need to stop trying to implement HTTP from
> scratch and use a pre-made library.  You would save yourself a LOT of
> hassle and problems.
> 

I have an application to do what I'm trying to acomplish using sockets,
that is working, using WinHTTP. I do the same application using
TCppWebBrowser. :)

With TCppWebBrowser I have a problem with memory, because IE don't free
them.

The other, using WinHTTP work perfectly, yet from time to time stop for
10-20 seconds, waiting for the server replay, and turn to work again...

>> Anyway, if I put a Sleep(5000) after test the return of recv to 0,
>> yet I continue getting the error.
> 
> The server's error reply has NOTHING to do with HOW you are
> sending/receiving the data, and EVERYTHING to do with WHAT you are
> sending.  Please show your complete request that is reporting the
> error.

This is how i send my request:

bool __fastcall TPesquisa::agendar_recv(void)
{
   int bytes;
   const char *first = "GET 
/agendamento-web/agendamento.do?task=forwardCalendario&forward=confirmarInformacoesDeVisto&requerenteAgendado[0].informacoesChecadas=true&agendamentoVO.agenciadorGrupoVO.seqAgenciadorGrupo=&locale=pt_BR&com.telecom.tipo_usuario=AGENCIA+DE+VIAGEM 
HTTP/1.1\r\nHost: www.visto-eua.com.br\r\nUser-Agent: Mozilla/5.001 
(windows; U; NT4.0; en-us) Gecko/25250101\r\nCookie: ";
   char info_checked[2049];

   strcpy(info_checked, first);
   strcat(info_checked, cookies); //<<<== cookies is: char cookies[2048]
   strcat(info_checked, "\r\n\r\n");

   for(;!Terminated;)
   {
     html[0] = '\0';
     bytes = send(sockfd, info_checked, strlen(info_checked), 0);
     acessos++;
     if(bytes == SOCKET_ERROR)
     {
       error_code = WSAGetLastError();
       closesocket(sockfd);
       return false;
     }
     else if(bytes == 0)
     {
       zero_on_send++;
       closesocket(sockfd);
       if(Login() != 0)
       {
         return false;
       }

       strcpy(info_checked, first);
       strcat(info_checked, cookies); //<<== New cookies
       strcat(info_checked, "\r\n\r\n");
     }
     else
       break;
   }

   for(;!Terminated;)
   {
     bytes = recv(sockfd, recv_buffer, sizeof(recv_buffer) - 1, 0);
     if(bytes == SOCKET_ERROR)
     {
       error_code = WSAGetLastError();
       closesocket(sockfd);
       return false;
     }
     else if(bytes == 0)
     {
       zero_on_recv++;
       closesocket(sockfd);
       if(Login() != 0)
       {
         return false;
       }
       if(CadastrarRequerente() == -1)
       {
         return false;
       }

       for(ptr_recv = &recv_buffer[0]; *ptr_recv != '\0';)
       {
         if(*ptr_recv == 'S')
         {
           ptr_recv++;
           if(*ptr_recv == 'e')
           {
             ptr_recv++;
             if(*ptr_recv == 't')
             {
               ptr_recv++;
               if(*ptr_recv == '-')
               {
                 ptr_recv++;
                 if(*ptr_recv == 'C')
                 {
                   ptr_recv++;
                   if(*ptr_recv == 'o')
                   {
                     ptr_recv++;
                     if(*ptr_recv == 'o')
                     {
                       ptr_recv += 6;
                       for(ptr_cookies = &cookies[0]; *ptr_recv != '\r'; 
ptr_cookies++, ptr_recv++)
                         *ptr_cookies = *ptr_recv;

                       *ptr_cookies = '\0';
                       break;
                     }
                   }
                 }
               }
             }
           }
         }
         else
           ptr_recv++;
       }

       for(; !Terminated ;)
       {
         html[0] = '\0';
         bytes = send(sockfd, info_checked, strlen(info_checked), 0);
         if(bytes == SOCKET_ERROR)
         {
           error_code = WSAGetLastError();
           closesocket(sockfd);
           return false;
         }
         else if(bytes == 0)
         {
           zero_on_send++;
           closesocket(sockfd);
           if(Login() != 0)
           {
             return false;
           }

           strcpy(info_checked, first);
           strcat(info_checked, cookies); //<<== New cookies
           strcat(info_checked, "\r\n\r\n");
         }
         else
           break;
       }
     }

     if(Terminated)
       return false;

     recv_buffer[bytes] = '\0';
     ptr_recv = &recv_buffer[bytes - 5];

     strcat(html, recv_buffer);

     if(strcmp(ptr_recv, "0\r\n\r\n") == 0)
       break;
   }

   return true;
}


The function agendar_rec is inside a looping.
Now, after rewriting the code (new code above), It's working.

I as missing the "CadastrarRequerente()" that must be called after every 
"Login()" to "agendar_recv()" work.

But yet i'm disconnected.

I used "zero_on_send" and "zero_on_recv" together with "acessos", and 
find that at each 91 requests the connection is closed...

Now I save the cookies every time recv is performed... It never changes.
The truth is that I find that the cookies are ALWAYS the same. Never 
changes. So, this is not the problem. I tried on other machines (with 
different IP's) and it was all the same, for the last 3 days at last...

Or is my system (windows) closing my connection for some reason, or the 
server is programmed to close the connection when reach this counter...

What do you think?

"Eduardo Jauch" <eduardo.jauch@gmail.com> wrote in message
news:45e4284e$1@newsgroups.borland.com...

> It comes a first part, that contains the header and an hexa that say
> the number of bytes to read. Then on the next loop, I receive a new
> chunk with a new hexa that says the length of the new chunk, a
> third time this occour and on the fourth, the signal that the send
of
> chunked data ends is present "0\r\n\r\n". So I quit the loop because
> there not more data to read...

That is how chunked transfers work, but that is not how your earlier
code was actually reading chunked data.  You were not reading the
chunked sizes at all.  You were reading the incoming data in arbitrary
byte sizes into a buffer without any care whatsoever to the actual
content of the transfer, and then trying to verify the last chunk
only.  A proper transfer should be performing the following sequence
instead:

    send the request
    read a single line
    verify the response code in the line
    loop
        read a line
        if line is empty then break
        store the line into a header list
    end
    if list contains 'Transfer-Encoding: chunked' header then
    begin
        loop
            read a line
            extract chunk size from the line
            if chunk size is 0 then break
            read number of bytes specified
            store bytes into buffer
            read an empty line
        end
        read an empty line
    end
    else if list contains 'Content-Length' header then
    begin
        read number of bytes specified
        store bytes into buffer
    end
    else
    begin
        loop
            read as many bytes as are currently available
            if disconnected then break
            store bytes into buffer
        end
    end

> And what I must to do?

See above.


Gambit

Remy Lebeau (TeamB) escreveu:
> "Eduardo Jauch" <eduardo.jauch@gmail.com> wrote in message
> news:45e4284e$1@newsgroups.borland.com...
> 
>> It comes a first part, that contains the header and an hexa that say
>> the number of bytes to read. Then on the next loop, I receive a new
>> chunk with a new hexa that says the length of the new chunk, a
>> third time this occour and on the fourth, the signal that the send
> of
>> chunked data ends is present "0\r\n\r\n". So I quit the loop because
>> there not more data to read...
> 
> That is how chunked transfers work, but that is not how your earlier
> code was actually reading chunked data.  You were not reading the
> chunked sizes at all.  You were reading the incoming data in arbitrary
> byte sizes into a buffer without any care whatsoever to the actual
> content of the transfer, and then trying to verify the last chunk
> only.  A proper transfer should be performing the following sequence
> instead:
> 
>     send the request
>     read a single line
>     verify the response code in the line
>     loop
>         read a line
>         if line is empty then break
>         store the line into a header list
>     end
>     if list contains 'Transfer-Encoding: chunked' header then
>     begin
>         loop
>             read a line
>             extract chunk size from the line
>             if chunk size is 0 then break
>             read number of bytes specified
>             store bytes into buffer
>             read an empty line
>         end
>         read an empty line
>     end
>     else if list contains 'Content-Length' header then
>     begin
>         read number of bytes specified
>         store bytes into buffer
>     end
>     else
>     begin
>         loop
>             read as many bytes as are currently available
>             if disconnected then break
>             store bytes into buffer
>         end
>     end
> 
>> And what I must to do?
> 
> See above.
> 
> 
> Gambit
> 
> 
I understand.

I'm not readind the chunk length. But this is because the char[] where I 
store the page is far bigger than the page I receive...

I know... Bad pratice.

but to do this is time consuming. I first do a code that does something 
similarly... But the "speed" was slower that if I "force" to do like I'm 
doing now :)

But I'll implement the code with the knowledge and tips that you show 
me, at last to verify if anything changed...

For now at last.

But all you said was good advise of course, and I thank you :)

I'll do tests with the code for long time anyway, and problably will 
implement most of what was discussed here :)

Thanks Gambit.

"Eduardo Jauch" <eduardo.jauch@gmail.com> wrote in message
news:45e4a68c$1@newsgroups.borland.com...

> I'm not readind the chunk length. But this is because the char[]
> where I store the page is far bigger than the page I receive...

Doesn't matter.  You still have to process the chunks properly.  The
way you are reading the data, you are leaving all of that chunk
information inside your buffer.  You will have to strip off all of
that info anyway when you go to process the contents of the downloaded
data later on.  If you process the chunks while they are downloading,
as described earlier, then they won't be in your buffer at all when it
comes time to process it.

> but to do this is time consuming.

Not as much as you think.  Besides, you are going to have to take the
time to process the chunk info anyway, so you may as well do it as
soon as they arrive.

> I first do a code that does something similarly... But the "speed"
was
> slower that if I "force" to do like I'm doing now :)

You probably did not optimize the code once it was running.


Gambit

Remy Lebeau (TeamB) escreveu:
> Doesn't matter.  You still have to process the chunks properly.  The
> way you are reading the data, you are leaving all of that chunk
> information inside your buffer.  You will have to strip off all of
> that info anyway when you go to process the contents of the downloaded
> data later on.  If you process the chunks while they are downloading,
> as described earlier, then they won't be in your buffer at all when it
> comes time to process it.
> 

What I do is:

Receive the whole data first. All of them is loaded into "html", that is 
a char [].

What I really do is search the first "<select " on the page. When I find 
it, I look for the second "<option " (if it exists) and so on until I 
find the first "</select>"

The only things I "use" is the "value=xxxxx" on the options...

If I find a value interesting, I execute another GET (not show in my 
previous posts). Now, I do the same, but with the second "<select " on 
the page.

These select's stay on the end of the html.

When I first enter the page (via GET), the server load all the 
information for the 3 selects. But show only the information for the 
first. When I execute the second GET, what I'm doing is asking to server 
to fill the second select based on my selection (of the first).

All the information don't change, even if one of the options is 
discarded or used by another user... So, I can iterate on the values 
until I find something useful and send a GET saying to the server that I 
want that information.If no one ask for it before me, I get, if not, the 
server say that the information dos not exist anymore.

The "speed" to a desired info "vanish" is fewer than 1 second 
sometimes... BEcause all data is loaded by the server for me on the 
first GET, Only this is timing consuming, the other three GETS to ask 
the desired information togheder occour in fewer time than the first. 
So, If I can do a code that do this fewer than 1 sec I got more chances 
to achive the info.

This is why I do such horrible code... And why I don't mind to read the 
chunked data etc...

And because the information that I want always come only with the last 
chunk, I don't bother with the rest. In fact, I only keep the rest for 
security reasons. Because sometimes the data that came is bigger than 
normal, and some of that come in the chunk right before the last.

It's a page generated automatically. Always the same, except for the 
options on the three selects.


>> but to do this is time consuming.
> 
> Not as much as you think.  Besides, you are going to have to take the
> time to process the chunk info anyway, so you may as well do it as
> soon as they arrive.
> 

Because I look for data that will come only 7000 bytes offset the begin 
of the body, specific data, and I don't bother if the page changes and 
my app will stop working for a few days until I rewrite the code, this 
is not so important exactly now.


>> I first do a code that does something similarly... But the "speed"
> was
>> slower that if I "force" to do like I'm doing now :)
> 
> You probably did not optimize the code once it was running.
> 

Problably :)

"Eduardo Jauch" <eduardo.jauch@gmail.com> wrote in message
news:45e4c107$1@newsgroups.borland.com...

> Receive the whole data first. All of them is loaded into "html",
> that is a char [].

Which is STILL the wrong thing to do!!!  You are not even ensuring
that your char[] really is large enough to hold everything the server
may send to you, and at worse you are overallocating more memory than
you really need right now.

> What I really do is search the first "<select " on the page. When I
> find it, I look for the second "<option " (if it exists) and so on
until
> I find the first "</select>"

That is very error prone.  Because of the nature of chunked transfers,
you are not even guaranteed to have such complete strings in your
buffer because you are not stripping out the chunked information
first.  You could have "<sel"+chunk+"ect" and the like, which would
cause your searching algorithm to fail.  You MUST handle chunked
transfers correctly in order to ensure the integrity of the data you
want to parse.  I don't know how to make this any clearer to you.

> This is why I do such horrible code...

Your code is horrible not because of HOW UGLY it is to look at, but
because how INCOMPLETE it is to begin with.  There are so many things
you are doing wrong with it that you are not guaranteed to end up with
the data that you are expecting.

> And why I don't mind to read the chunked data etc...

The presence of chunking is not an issue.  Your handling of chunking,
on the other hand, is very much an issue.

> And because the information that I want always come only
> with the last chunk, I don't bother with the rest.

That is also not a guarantee, either.  You really don't seem to be
grasping what HTTP is actually sending you.  So you are going to keep
handling it all wrong.

> sometimes the data that came is bigger than normal, and some of that
> come in the chunk right before the last.

All the more reason why you NEED to handle chunking, and in fact any
HTTP transfer in general, properly from the very beginning.  But you
are refusing to do that much.  Obviously, you want a broken,
error-prone, dangerous implementation, so that is your choice.  I'm
done debating this discussion any further.


Gambit

Remy Lebeau (TeamB) escreveu:
> Which is STILL the wrong thing to do!!!  You are not even ensuring
> that your char[] really is large enough to hold everything the server
> may send to you, and at worse you are overallocating more memory than
> you really need right now.

I know that.

> 
>> What I really do is search the first "<select " on the page. When I
>> find it, I look for the second "<option " (if it exists) and so on
> until
>> I find the first "</select>"
> 
> That is very error prone.  Because of the nature of chunked transfers,
> you are not even guaranteed to have such complete strings in your
> buffer because you are not stripping out the chunked information
> first.  You could have "<sel"+chunk+"ect" and the like, which would
> cause your searching algorithm to fail.  You MUST handle chunked
> transfers correctly in order to ensure the integrity of the data you
> want to parse.  I don't know how to make this any clearer to you.

You're right. I forget the problem of the hex code of the chunk be 
inside the data.

> 
>> This is why I do such horrible code...
> 
> Your code is horrible not because of HOW UGLY it is to look at, but
> because how INCOMPLETE it is to begin with.  There are so many things
> you are doing wrong with it that you are not guaranteed to end up with
> the data that you are expecting.
> 

I know that.

>> And why I don't mind to read the chunked data etc...
> 
> The presence of chunking is not an issue.  Your handling of chunking,
> on the other hand, is very much an issue.
> 

I'm rewritting the code since yesterday to take care of many of the 
issues that you target. Including strip the chunk information of the 
data you pointed now. The code I show you on the beggining of the 
discussion was rewrite many times since them.

>> And because the information that I want always come only
>> with the last chunk, I don't bother with the rest.
> 
> That is also not a guarantee, either.  You really don't seem to be
> grasping what HTTP is actually sending you.  So you are going to keep
> handling it all wrong.
> 
>> sometimes the data that came is bigger than normal, and some of that
>> come in the chunk right before the last.
> 
> All the more reason why you NEED to handle chunking, and in fact any
> HTTP transfer in general, properly from the very beginning.  But you
> are refusing to do that much.  Obviously, you want a broken,
> error-prone, dangerous implementation, so that is your choice.  

Sometimes, the broken, error-prone, dangerous implementation, even if it 
really fails from time to time, is a better choice than the opposite...

> I'm done debating this discussion any further.
> 

ok. Was a very instructive (yet may you don't believe) discussion to me. :)

Thanks. :)

"Eduardo Jauch" <eduardo.jauch@gmail.com> wrote in message
news:45e4d8f6@newsgroups.borland.com...

> I'm rewritting the code since yesterday to take care of many of the
> issues that you target. Including strip the chunk information of the
> data you pointed now. The code I show you on the beggining of the
> discussion was rewrite many times since them.

You should have said so earlier and saved us alot of aggravation.

> Sometimes, the broken, error-prone, dangerous implementation, even
> if it really fails from time to time, is a better choice than the
opposite...

Not when the opposite is total failure and crashing.

> ok. Was a very instructive (yet may you don't believe) discussion to
me. :)

Glad you got something out of it.


Gambit