🚀 go-pugleaf

Joe Black wrote:

> .... The problem seems to be when I send 
> several packets straight after each other, the packets are all sent 
> within a thread that activates every 35 ms and it sends one packet at a 
> time.

But what is the problem ?

> For some reason though several packets seem to get concatenated together 
> to form one larger packet, if I slow down the transmit thread to say 
> every 200ms this problems dissapears, 

Yes that is normal. Winsock will send it's buffer when it is full or
a certain amount of time has elapsed.

> this sort of time delay however is 
> unacceptable. What I would like to know is:
> 
> Is there anyway to determine whether the last packet was sent before I 
> attempt to send the next one 

Don't think so. But why would you? You have to program the receiving
side in such a way that it can handle multiple 'packets' in one read event.
Just as the receiving side should concatenate read-events until
a complete (big) message is received.

FYI: it is confusing that you use the word packet for the message
that you send. TCP/IP uses packets already.

Hans.

Hans Galema wrote:
> Joe Black wrote:
> 
>> .... The problem seems to be when I send several packets straight 
>> after each other, the packets are all sent within a thread that 
>> activates every 35 ms and it sends one packet at a time.
> 
> 
> But what is the problem ?

Several packets get concatenated together.

> 
>> For some reason though several packets seem to get concatenated 
>> together to form one larger packet, if I slow down the transmit thread 
>> to say every 200ms this problems dissapears, 
> 
> 
> Yes that is normal. Winsock will send it's buffer when it is full or
> a certain amount of time has elapsed.

This is what I was wondering, I was hoping I could send data when it was 
empty.

> 
>> this sort of time delay however is unacceptable. What I would like to 
>> know is:
>>
>> Is there anyway to determine whether the last packet was sent before I 
>> attempt to send the next one 
> 
> 
> Don't think so. But why would you? You have to program the receiving
> side in such a way that it can handle multiple 'packets' in one read event.
> Just as the receiving side should concatenate read-events until
> a complete (big) message is received.

I have a fairly simple app and I was hoping to make it easier, if this 
is the way to do it then I will code it to as you suggest.

> 
> FYI: it is confusing that you use the word packet for the message
> that you send. TCP/IP uses packets already.

Appologies, the data I send is in a packet format, hence i use the term 
packet. I am fairly new to the Sockets side of things so my terminology 
may be confusing.

> 
> Hans.

Many thanks, you seem to have answered everything I needed to know

Joe

Hello!
You wrote  on Tue, 25 Oct 2005 17:17:37 +0100:

 JB> I have a client/server set of apps, the client uses a TClientSocket and
 JB> the server the TServerSocket, all works great data sent back and
 JB> forward fine, apart from one or two places. The problem seems to be
 JB> when I send several packets straight after each other, the packets are
 JB> all sent within a thread that activates every 35 ms and it sends one
 JB> packet at a time.

The problem is that TCP is not packet-oriented, but stream-oriented. If you
call send() 2 times, this doesn't mean that there will be 2 TCP packets on
the other end. There can be anything between 1 and 20 (or more) TCP packets.
If you want message-based communication, you can use, for example,
MsgConnect (http://www.eldos.com/msgconnect/)

With best regards,
Eugene Mayevski

"Joe Black" <nospam@newsgroup.com> wrote in message
news:435e5931$1@newsgroups.borland.com...

> For some reason though several packets seem to get
> concatenated together to form one larger packet

That is perfectly normal socket behavior.  A socket buffers all outgoing
data until either the buffer is full or until the socket has no pending data
waiting to be sent.

> if I slow down the transmit thread to say every 200ms this
> problems dissapears

Yes, because you are allowing the socket to have some time to clear its
internal buffer before more data is passed to it.

> Is there anyway to determine whether the last packet was
> sent before I attempt to send the next one

No.  But that is not needed in the first place.  TCP is a stream transport.
It doesn't care what the contents of the data is.  It sends a stream of
arbitrary bytes.  So you are responsible for formatting your outgoing data
in such a way that the receiver knows where one packet ends and the next
begins.  There are three ways to do that:

1) send messages that are all the same byte size.  The receiver reads the
necessary number of bytes and then processes them.

2) preceed each message with a fixed-sized length value.  The receiver first
reads the necessary number of bytes to receive the length and then continues
reading the specified number of bytes to finish the message.

3) Place a unique terminator sequence of bytes at the end of a message.  The
terminator cannot appear in the message contents at all.  This way, the
receiver continues reading bytes until the terminator is encountered.

In all three situations, you have to code your reading to handle three
different scenerios when a read event occurs:

1)  no bytes are available at all

2) fewer bytes are received than are needed for a complete message,
requiring you to wait for another read event in order to receive more bytes
for that message.

3) bytes from multiple messages are received at one time.

In all read events, you should buffer the received bytes and then process
and remove any complete messages that are in buffer, leaving any incomplete
messages in the buffer for later processing.


Gambit

Eugene Mayevski wrote:
> Hello!
> You wrote  on Tue, 25 Oct 2005 17:17:37 +0100:
> 
>  JB> I have a client/server set of apps, the client uses a TClientSocket and
>  JB> the server the TServerSocket, all works great data sent back and
>  JB> forward fine, apart from one or two places. The problem seems to be
>  JB> when I send several packets straight after each other, the packets are
>  JB> all sent within a thread that activates every 35 ms and it sends one
>  JB> packet at a time.
> 
> The problem is that TCP is not packet-oriented, but stream-oriented. If you
> call send() 2 times, this doesn't mean that there will be 2 TCP packets on
> the other end. There can be anything between 1 and 20 (or more) TCP packets.
> If you want message-based communication, you can use, for example,
> MsgConnect (http://www.eldos.com/msgconnect/)
> 
> With best regards,
> Eugene Mayevski
> 

Thanks Eugene,

I will have a look at the MsgConnect as you suggested, this app was 
meant to be a simple thing but the addition of sockets has baked my 
noodle a bit :-(.

Cheers Joe

Remy Lebeau (TeamB) wrote:
> "Joe Black" <nospam@newsgroup.com> wrote in message
> news:435e5931$1@newsgroups.borland.com...
> 
> 
>>For some reason though several packets seem to get
>>concatenated together to form one larger packet
> 
> 
> That is perfectly normal socket behavior.  A socket buffers all outgoing
> data until either the buffer is full or until the socket has no pending data
> waiting to be sent.
> 
> 
>>if I slow down the transmit thread to say every 200ms this
>>problems dissapears
> 
> 
> Yes, because you are allowing the socket to have some time to clear its
> internal buffer before more data is passed to it.
> 
> 
>>Is there anyway to determine whether the last packet was
>>sent before I attempt to send the next one
> 
> 
> No.  But that is not needed in the first place.  TCP is a stream transport.
> It doesn't care what the contents of the data is.  It sends a stream of
> arbitrary bytes.  So you are responsible for formatting your outgoing data
> in such a way that the receiver knows where one packet ends and the next
> begins.  There are three ways to do that:
> 
> 1) send messages that are all the same byte size.  The receiver reads the
> necessary number of bytes and then processes them.
> 
> 2) preceed each message with a fixed-sized length value.  The receiver first
> reads the necessary number of bytes to receive the length and then continues
> reading the specified number of bytes to finish the message.
> 
> 3) Place a unique terminator sequence of bytes at the end of a message.  The
> terminator cannot appear in the message contents at all.  This way, the
> receiver continues reading bytes until the terminator is encountered.
> 
> In all three situations, you have to code your reading to handle three
> different scenerios when a read event occurs:
> 
> 1)  no bytes are available at all
> 
> 2) fewer bytes are received than are needed for a complete message,
> requiring you to wait for another read event in order to receive more bytes
> for that message.
> 
> 3) bytes from multiple messages are received at one time.
> 
> In all read events, you should buffer the received bytes and then process
> and remove any complete messages that are in buffer, leaving any incomplete
> messages in the buffer for later processing.
> 
> 
> Gambit
> 
> 


Hi Remy,

Many thanks for your response, it covers all the areas I'm having the 
trouble with, I had an feeling I was going to have to put the data back 
together after I recieved it but I couldn't find the right sort of 
information about it, my googleing skills must be lacking at the moment :-).

Thanks again

Joe

Hello!
You wrote  on Wed, 26 Oct 2005 09:05:05 +0100:

 JB> I will have a look at the MsgConnect as you suggested, this app was
 JB> meant to be a simple thing but the addition of sockets has baked my
 JB> noodle a bit :-(.

Yes, this is the place where the thing like MsgConnect can help, since it
removes complexities of socket-related stuff from your project.

With best regards,
Eugene Mayevski

In addition to Remys excelent suggestions, if you are having a timing
problem on the receiving end, that is, if you need the 35 ms to
process the data, but when the data is concatinated, you run out of
processing time.....
Then take a look at the
	setsockopt( ) function, and the  TCP_NODELAY setting.
This turns off buffering for that socket.
This makes the socket less efficient in terms of bandwidth, but more
responsive to real-world events.

Great post , it helped me too...

Data concatenation is only for Socket->SendBuf(...) or Socket->SendText(...) or both ?
Thank you
Alfred

Alfred wrote:

> Data concatenation is only for Socket->SendBuf(...) or Socket->SendText(...) or both ?

Both of course. The socket doesn't know if you're sending text or random 
data.

Michel
-- 
----------------------------------------
Michel Leunen
mailto: see my homepage.
C++Builder, BCC5.5.1 Web site:
http://www.leunen.com/
----------------------------------------

>Both of course. The socket doesn't know if you're sending text or random 
>data.

LOL I am very luchy then.. I got no problem with sendtext till now.. Ok I will modify all my code then..

thank you very much
Alfred

"Alfred" <alf@no.spam.not.valied.email> wrote in message
news:435fc087$1@newsgroups.borland.com...

> Data concatenation is only for Socket->SendBuf(...) or
Socket->SendText(...) or both ?

Both.  SendText() passes the AnsiString to SendBuf().


Gambit

>Both.  SendText() passes the AnsiString to SendBuf().

Ah thank you..
indy have the same problem ?
Thank you very much
alfred

"Alfred" <alf@no.spam.not.valied.email> wrote in message
news:4360fa19$1@newsgroups.borland.com...

> indy have the same problem ?

No.  Indy hides the concatenation and splitting of packets from you
transparently.  When you read or send something, Indy does not return
control to your code until the data has been read/sent in full, or until an
actual error occurs.


Gambit