Thread View: borland.public.cppbuilder.internet.socket
18 messages
18 total messages
Started by Maurice Anderson
Fri, 23 Dec 2005 18:09
TWinSocketStream Read in BYTE buffer
Author: Maurice Anderson
Date: Fri, 23 Dec 2005 18:09
Date: Fri, 23 Dec 2005 18:09
15 lines
1467 bytes
1467 bytes
Hello, I have a BYTE buffer and Stream defined as: BYTE rbuf[256]; TWinSocketStream* str = new TWinSocketStream(ClientSocket, 5000); Once the data is read from str into the buffer, how do I go about using the buffer? I am not familiar with the BYTE type except to know that it's a 8 bit unsigned integer. So lets say I send a file over, how would I 'convert' the file back to it's original format and save it to say the C drive. Or lets say I sent a simple text message over as "Hello World". How do I recover the messages from rbuf? Thanks
Re: TWinSocketStream Read in BYTE buffer
Author: Remy Lebeau (TeamB)
Date: Fri, 23 Dec 2005 20:39
Date: Fri, 23 Dec 2005 20:39
171 lines
5442 bytes
5442 bytes
"Maurice Anderson" <mauriceanderson@hotmail.com> wrote in message
news:43ac3d34$1@newsgroups.borland.com...
> So lets say I send a file over, how would I 'convert' the file back
> to it's original format and save it to say the C drive.
Simply open a new file on the drive and then write the buffer into that
file. I strongly recommend that you send the file size along with the file
data so that the receiver knows ahead of time how much data to expect. For
example:
bool __fastcall ReadBuf(TWinSocketStream *Stream, void *Buffer, int
BufSize)
{
BYTE *pBuf = (BYTE*) Buffer;
int NumRead;
while( BufSize > 0 )
{
NumRead = Stream->Read(pBuf, BufSize);
if( NumRead < 1 )
return false;
pBuf += NumRead;
BufSize -= NumRead;
}
return true;
}
bool __fastcall WriteBuf(TWinSocketStream *Stream, void *Buffer, int
BufSize)
{
BYTE *pBuf = (BYTE*) Buffer;
int NumSent;
while( BufSize > 0 )
{
NumSent = Stream->Write(pBuf, BufSize);
if( NumSent < 1 )
return false;
pBuf += NumSent;
BufSize -= NumSent;
}
return true;
}
// sending the file ...
{
BYTE rbuf[256];
int bufsize;
// open a file ...
int datasize = ...; // get the file size
if( WriteBuf(str, &datasize, sizeof(int)) )
{
while( datasize > 0 )
{
bufsize = datasize;
if( bufsize > sizeof(rbuf) )
bufsize = sizeof(rbuf);
// read from the file into rbuf, up to no more than bufsize
bytes ...
// update bufsize with the number of bytes actually read ...
if( !WriteBuf(str, rbuf, bufsize) )
break;
datasize -= bufsize;
}
}
// close the file ...
}
// reading the file ...
{
BYTE rbuf[256];
int bufsize, datasize = 0;
if( ReadBuf(str, &datasize, sizeof(int)) )
{
bool error = false;
// open a file ...
// preallocate the file to datasize bytes ...
while( datasize > 0 )
{
bufsize = datasize;
if( bufsize > sizeof(rbuf) )
bufsize = sizeof(rbuf);
if( !ReadBuf(str, rbuf, bufsize) )
{
error = true;
break;
}
// write rbuf to the file, up to no more than bufsize bytes
...
datasize -= bufsize;
}
// close the file ...
if( error )
// delete incomplete file ...
}
}
> Or lets say I sent a simple text message over as "Hello World".
> How do I recover the messages from rbuf?
If the contents of the buffer are null-terminated, then you can cast the
buffer to a char*:
AnsiString msg = (char*) rbuf;
If it is not null-terminated, you can pass the buffer to one of AnsiString's
constructors that takes a length as a second parameter.
AnsiString msg((char*) rbuf, number of characters in buffer);
Again, I highly recommend that you send the string length along with the
string data. For example:
bool __fastcall WriteString(TWinSocketStream *Stream, const AnsiString
&S)
{
int length = S.Length();
if( WriteBuf(Stream, &length, sizeof(int)) )
return WriteBuf(Stream, S.c_str(), length);
return false;
}
bool __fastcall ReadString(TWinSocketStream *Stream, AnsiString &S)
{
int length = 0;
if( ReadBuf(Stream, &length, sizeof(int)) )
{
S.SetLength(length);
return ReadBuf(Stream, S.c_str(), length);
}
return false;
}
// sending a message ...
if( !WriteString(Stream, "Hello World!") )
// error ...
// reading a message ...
AnsiString msg;
if( ReadString(Stream, msg) )
// use msg as needed ...
else
// error ...
Gambit
Re: TWinSocketStream Read in BYTE buffer
Author: Maurice Anderson
Date: Sat, 24 Dec 2005 03:48
Date: Sat, 24 Dec 2005 03:48
32 lines
2028 bytes
2028 bytes
Thanks for the reply Remy, but I am a little confused on some things:
Is ReadBuf the function in the server thread to read data from the client?
Is WriteBuf also meant to bein the server thread to write the contents of
the buffer into a new file name?
> I strongly recommend that you send the file size along with the file data
so that the receiver knows ahead
of time how much data to expect.
How do I send the byte ahead of time when I launch the server thread? As in
SocketThread = new TMyServerThread(true,ClientSocket);?
I have something like this in the server thread's Execute method:
TWinSocketStream* str = new TWinSocketStream(ClientSocket, timeout);
while( (!Terminated) && (ClientSocket->Connected) )
{
if (str->WaitForData(6000))
{
if( ReadBuf(str, &datasize, sizeof(int)) )
{
//....
}
}
}
delete str;
I am sorry, I am still somewhat totally confused on what to do.
Sorry ... Thanks
Re: TWinSocketStream Read in BYTE buffer
Author: Remy Lebeau (TeamB)
Date: Sun, 25 Dec 2005 08:43
Date: Sun, 25 Dec 2005 08:43
24 lines
1812 bytes
1812 bytes
"Maurice Anderson" <mauriceanderson@hotmail.com> wrote in message news:43acc4c5$1@newsgroups.borland.com... > Is ReadBuf the function in the server thread to read data from the client? You can call it from both sides. Assuming you want to implement two-way communication. > Is WriteBuf also meant to bein the server thread to write the > contents of the buffer into a new file name? No. WriteBuf() is for sending data on the socket. It takes a TWinSocketStream as a parameter. I intentionally did not include specific code for handling the files, as I would leave that up to you to decide how to manage. I gave you the socket code only. > How do I send the byte ahead of time when I launch the server thread? I already gave you the code to do that. Please re-read my earlier message again. Gambit
Re: TWinSocketStream Read in BYTE buffer
Author: Maurice Anderson
Date: Sun, 25 Dec 2005 18:59
Date: Sun, 25 Dec 2005 18:59
12 lines
1361 bytes
1361 bytes
> I already gave you the code to do that. Please re-read my earlier message again. Ok, I'll look it over. One more question: How do I concantenate a BYTE and another type such as char. For example how would I concantenate these two? BYTE pBuf[256]; char test[256]; And thanks again for your help!!
Re: TWinSocketStream Read in BYTE buffer
Author: Remy Lebeau (TeamB)
Date: Sun, 25 Dec 2005 19:44
Date: Sun, 25 Dec 2005 19:44
14 lines
1545 bytes
1545 bytes
"Maurice Anderson" <mauriceanderson@hotmail.com> wrote in message news:43aeebe1$1@newsgroups.borland.com... > One more question: How do I concantenate a BYTE and another > type such as char. For example how would I concantenate these two? Allocate a larger buffer and then copy the contents of the two into it. But why would you want to concatenate the two types? What EXACTLY are you trying to accomplish? Gambit
Re: TWinSocketStream Read in BYTE buffer
Author: Maurice Anderson
Date: Sun, 25 Dec 2005 20:54
Date: Sun, 25 Dec 2005 20:54
38 lines
2657 bytes
2657 bytes
> Allocate a larger buffer and then copy the contents of the two into it.
How would I do that?
pBuf += test; <- doesn't work
test+= pBuf ; <- doesn't work
> But why would you want to concatenate the two types? What EXACTLY are you
trying to accomplish?
I am "trying" to implement your idea to send the type(binary or text) and
byte size to the server for pre-processing so that it knows how to handle
the data sent. So for binary data, I want to server to save the data back
as a file in it's original format and for text, I want to display it as a
popup message or something. So the 'frame' would look something like this
format: [type code][data length][file name][data]. So my code would look
something like this:
char type[5] = "0x01"; <- type code
char length[25] = "192"; <- type code (in bytes)
char filename[15] = "boot.ini"; <- type code (in bytes)
TMemoryStream *DataFromFile = new TMemoryStream;
DataFromFile ->LoadFromFile("c:\\boot.ini");
/* Convert it to BYTES and concantenate the fields into 1 'frame' */
BYTE *buffer2 = (BYTE *)DataFromFile ->Memory;
buffer2 += ( type + length + filename);
/* Display it in a memo to view how the frame looks */
Memo1->Lines->Add((char *)Strm->Memory);
ClientSocket1->Socket->SendBuf(buffer2, x); // x = number of total bytes to
send over
I know I don't have it all together here, but I am hoping that whatever
example code I get, I can just learn from and modify it to fit my
application. I am pretty good at doing that.
Re: TWinSocketStream Read in BYTE buffer
Author: Jonathan Benedicto
Date: Tue, 27 Dec 2005 20:38
Date: Tue, 27 Dec 2005 20:38
110 lines
4449 bytes
4449 bytes
Maurice Anderson wrote:
> I am "trying" to implement your idea to send the type(binary or text)
> and byte size to the server for pre-processing so that it knows how
> to handle the data sent. So for binary data, I want to server to
> save the data back as a file in it's original format and for text, I
> want to display it as a popup message or something. So the 'frame'
> would look something like this format: [type code][data length][file
> name][data]. So my code would look something like this:
I'd make a "virtual" frame, where the data is never written as a frame, but
it written in a more space efficient method. Like this:
// untested.
To write a file "frame" :
int TypeCode = 0x01;
int FileNameLength;
int DataStreamLength;
AnsiString FileName;
TMemoryStream *DataStream;
FileName = "test.txt";
DataStream = new TMemoryStream;
DataStream->LoadFromFile( "C:\test.txt" );
FileNameLength = FileName.Length();
DataStreamLength = DataStream->Size;
WriteBuf( str, &TypeCode, sizeof( int ) );
WriteBuf( str, &FileNameLength, sizeof( int ) );
WriteBuf( str, FileName.c_str(), FileName.Length() );
WriteBuf( str, &DataStreamLength, sizeof( int ) );
WriteBuf( str, DataStream->Memory, DataStream->Size );
To write a text "frame" :
int TypeCode = 0x02;
int TextLength;
AnsiString Text;
Text = "Hello";
TextLength = Text.Length();
WriteBuf( str, &TypeCode, sizeof( int ) );
WriteBuf( str, &TextLength, sizeof( int ) );
WriteBuf( str, Text.c_str(), Text.Length() );
Now, to read this back would be a touch more difficult.
void __fastcall ReadData( TWinSocketStream *PStream )
{
int TypeCode;
if( !ReadBuf( PStream, &TypeCode, sizeof( int ) ) )
return;
if( TypeCode == 0x01 )
{
// A file.
int FileNameLength;
int DataStreamLength;
AnsiString FileName;
TMemoryStream *DataStream;
if( ReadBuf( PStream, &FileNameLength, sizeof( int ) ) &&
FileNameLength > 0 && FileNameLength < MAX_PATH )
{
FileName.SetLength( FileNameLength );
if( ReadBuf( PStream, FileName.c_str(), FileName.Length() ) &&
FileName != "" )
{
if( ReadBuf( PStream, &DataStreamLength, sizeof( int ) &&
DataStreamLength > 0 )
{
DataStream->Size = DataStreamLength;
if( ReadBuf( PStream, DataStream->Memory,
DataStream->Size ) )
{
// file received, so save it.
DataStream->SaveToFile( FileName );
}
}
}
}
}
else if( TypeCode == 0x02 )
{
// Text.
int TextLength;
AnsiString Text;
if( ReadBuf( PStream, &TextLength, sizeof( int ) ) && TextLength >
0 )
{
Text.SetLength( TextLength );
if( ReadBuf( PStream, Text.c_str(), Text.Length() ) && Text !=
"" )
{
// text received.
ShowMessage( Text );
}
}
}
}
HTH
Jonathan
http://jomitech.com/mtbcc32.shtml - Multi-threaded compilation for BCB
http://jomitech.com/forum - JomiTech Forums
Re: TWinSocketStream Read in BYTE buffer
Author: Remy Lebeau (TeamB)
Date: Wed, 28 Dec 2005 08:57
Date: Wed, 28 Dec 2005 08:57
174 lines
6582 bytes
6582 bytes
"Maurice Anderson" <mauriceanderson@hotmail.com> wrote in message
news:43af06d7$1@newsgroups.borland.com...
> How would I do that?
What exactly was unclear about my earlier statement? Allocate a new buffer
that is the same as, or larger than, the sum of the number of bytes of the
other two buffers combined. And then copy the bytes of the two buffers into
the new one.
> I am "trying" to implement your idea to send the type(binary or text) and
> byte size to the server for pre-processing so that it knows how to handle
> the data sent.
Why aren't you using the code I already gave you? I showed you exactly how
to send the byte size of the data that is being sent. Just expand on that
code to send an additional value in front of the byte size to indicate the
type of data being sent. For example:
// sending a text message ...
{
char type = 0x01; // text
if( WriteBuf(str, &type, sizeof(char)) )
{
if( WriteString(str, "Hello World!") )
// text message sent ok ...
else
// text message error ...
}
else
// text message error ...
}
// sending a file ...
{
// open a file ...
int datasize = ...; // get the file size
char type = 0x02; // file
if( WriteBuf(str, &type, sizeof(char)) )
{
if( WriteString(str, "the filename") )
{
if( WriteBuf(str, &datasize, sizeof(int)) )
{
BYTE rbuf[256];
int bufsize;
while( datasize > 0 )
{
bufsize = datasize;
if( bufsize > sizeof(rbuf) )
bufsize = sizeof(rbuf);
// read from the file into rbuf, up to no more than
bufsize bytes ...
// update bufsize with the number of bytes actually
read ...
if( !WriteBuf(str, rbuf, bufsize) )
break;
datasize -= bufsize;
}
}
}
}
// close the file ...
}
// read a packet and decide how to handle it ...
{
char type = 0x00;
if( ReadBuf(str, &type, sizeof(char)) )
{
switch( type )
{
case 0x01: // text
{
AnsiString msg;
if( ReadString(str, msg) )
// use msg as needed ...
break;
}
case 0x02: // file
{
AnsiString filename;
if( ReadString(str, filename) )
{
BYTE rbuf[256];
int bufsize, datasize = 0;
bool error = false;
if( ReadBuf(str, &datasize, sizeof(int)) )
{
// open a file ...
// preallocate the file to datasize bytes ...
while( datasize > 0 )
{
bufsize = datasize;
if( bufsize > sizeof(rbuf) )
bufsize = sizeof(rbuf);
if( !ReadBuf(str, rbuf, bufsize) )
{
error = true;
break;
}
// write rbuf to the file, up to bufsize
bytes ...
datasize -= bufsize;
}
// close the file ...
if( error )
// delete incomplete file ...
}
}
break;
}
default: // unknown type
{
// unknown data received, ignore it ...
int datasize = 0;
if( ReadBuf(str, &datasize, sizeof(int)) )
{
BYTE rbuf[256];
int bufsize;
while( datasize > 0 )
{
bufsize = datasize;
if( bufsize > sizeof(rbuf) )
bufsize = sizeof(rbuf);
if( !ReadBuf(str, rbuf, bufsize) )
break;
datasize -= bufsize;
}
}
break;
}
}
}
}
> So the 'frame' would look something like this format:
> [type code][data length][file name][data].
There is no point in putting a filename into a text message. But in
general, it is good to frame your packets, yes.
> So my code would look something like this:
No, it would not. Not even close.
Gambit
Re: TWinSocketStream Read in BYTE buffer
Author: newsgroups.borland.com
Date: Thu, 29 Dec 2005 19:41
Date: Thu, 29 Dec 2005 19:41
42 lines
2590 bytes
2590 bytes
TMemoryStream *DataStream = new TMemoryStream;
.
.
.
> while( datasize > 0 )
> {
> bufsize = datasize;
> if( bufsize > sizeof(rbuf) )
> bufsize = sizeof(rbuf);
>
> if( !ReadBuf(str, rbuf, bufsize) )
> {
> error = true;
> break;
> }
>
> // write rbuf to the file, up to bufsize
bytes ...
>
> datasize -= bufsize;
> }
DataStream->Write(rbuf, bufsize);
<----- What I added
DataStream->SaveToFile( "maurice.ini" );
<----- What I added
>
> // close the file ...
>
The contents of 'maurice.ini', is garbled. It looks like this:
�� �|�w �X|8� o� U �w(%�w����
Any ideas why this is happening?
Also, I have a quick question about function ReadBuf(TStream *Strm, void
*Buffer, int Count). In the function you have
pBuf += NumRead. Why are you adding the number of bytes read to our data?
Sorry if it's a dumb question, but I am still learning..
Thanks.
Re: TWinSocketStream Read in BYTE buffer
Author: Hans Galema
Date: Thu, 29 Dec 2005 21:09
Date: Thu, 29 Dec 2005 21:09
19 lines
2022 bytes
2022 bytes
newsgroups.borland.com wrote: Please can you change that name to a more appropriate one ? > DataStream->Write(rbuf, bufsize); > <----- What I added > DataStream->SaveToFile( "maurice.ini" ); > <----- What I added > > The contents of 'maurice.ini', is garbled. It looks like this: > �� �|�w �X|8� o� U �w(%�w���� Well if that was what the sender sent then it is ok. You did not tell what you expected. And you checked it late. Before Write(rbuf), did you check the contents of rbuf ? For instance with a ShowMessage ( rbuf ); ? Hans.
Re: TWinSocketStream Read in BYTE buffer
Author: Remy Lebeau (TeamB)
Date: Thu, 29 Dec 2005 21:32
Date: Thu, 29 Dec 2005 21:32
315 lines
10263 bytes
10263 bytes
"newsgroups.borland.com" <mauriceanderson@hotmail.com> wrote in message
news:43b43a5a$1@newsgroups.borland.com...
> TMemoryStream *DataStream = new TMemoryStream;
Why are you using TMemoryStream instead of TFileStream?
> DataStream->Write(rbuf, bufsize);
> <----- What I added
You added it in the wrong place. You did not put it where I told you to put
it. I put comments in the code to tell you where to perform various
operations, but you seem to be ignoring them completely.
Here is a more complete code snippet for you:
bool __fastcall ReadBuf(TWinSocketStream *Stream, void *Buffer, int
BufSize)
{
BYTE *pBuf = (BYTE*) Buffer;
int NumRead;
try
{
while( BufSize > 0 )
{
NumRead = Stream->Read(pBuf, BufSize);
if( NumRead < 1 )
return false;
pBuf += NumRead;
BufSize -= NumRead;
}
}
catch(const Exception &)
{
return false;
}
return true;
}
bool __fastcall WriteBuf(TWinSocketStream *Stream, void *Buffer, int
BufSize)
{
BYTE *pBuf = (BYTE*) Buffer;
int NumSent;
try
{
while( BufSize > 0 )
{
NumSent = Stream->Write(pBuf, BufSize);
if( NumSent < 1 )
return false;
pBuf += NumSent;
BufSize -= NumSent;
}
}
catch(const Exception &)
{
return false;
}
return true;
}
bool __fastcall ReadString(TWinSocketStream *Stream, AnsiString &S)
{
int length = 0;
if( ReadBuf(Stream, &length, sizeof(int)) )
{
if( length > 0 )
{
try
{
S.SetLength(length);
}
catch(const Exception &)
{
return false;
}
return ReadBuf(Stream, S.c_str(), length);
}
else
{
S = "";
return true;
}
}
return false;
}
bool __fastcall WriteString(TWinSocketStream *Stream, const AnsiString
&S)
{
int length = S.Length();
if( WriteBuf(Stream, &length, sizeof(int)) )
return WriteBuf(Stream, S.c_str(), length);
return false;
}
bool __fastcall SendTextMessage(TWinSocketStream *Stream, const
AnsiString &Msg)
{
char type = 0x01;
if( WriteBuf(Stream, &type, sizeof(char)) )
return WriteString(Stream, Msg) )
return false;
}
bool __fastcall SendFile(TWinSocketStream *Stream, const AnsiString
&FileName)
{
bool error = false;
try
{
TFileStream *FS = new TFileStream(FileName, fmOpenRead |
fmShareDenyWrite);
try
{
char type = 0x02;
if( WriteBuf(str, &type, sizeof(char)) )
{
if( WriteString(str, ExtractFileName(FileName)) )
{
int size = FS->Size;
if( WriteBuf(str, &size, sizeof(int)) )
{
BYTE rbuf[256];
int bufsize;
while( size > 0 )
{
bufsize = size;
if( bufsize > sizeof(rbuf) )
bufsize = sizeof(rbuf);
bufsize = FS->Read(rbuf, bufsize);
if( bufsize < 1 )
{
error = true;
break;
}
if( !WriteBuf(Stream, rbuf, bufsize) )
{
error = true;
break;
}
size -= bufsize;
}
}
}
}
}
__finally
{
delete FS;
}
}
catch(const Exception &)
{
error = true;
}
return !error;
}
bool __fastcall ReadFile(TWinSocketStream *Stream, AnsiString &FileName)
{
AnsiString fn;
if( ReadString(Stream, fn) )
{
fn = ("C:\\Some Folder\\" + ExtractFileName(fn));
int size = 0;
if( ReadBuf(str, &size, sizeof(int)) )
{
bool error = false;
try
{
TFileStream *FS = new TFileStream(fn, fmCreate);
try
{
FS->Size = size;
BYTE rbuf[256];
int bufsize;
while( size > 0 )
{
bufsize = size;
if( bufsize > sizeof(rbuf) )
bufsize = sizeof(rbuf);
if( !ReadBuf(rbuf, bufsize) )
{
error = true;
break;
}
if( FS->Write(rbuf, bufsize) < 1 )
{
error = true;
break;
}
size -= bufsize;
}
}
__finally
{
delete FS;
}
}
catch(const Exception &)
{
error = true;
DeleteFile(fn);
}
if( !error )
{
FileName = fn;
return true;
}
}
}
return false;
}
bool __fastcall ReadPacket(TWinSocketStream *Stream)
{
char type = 0x00;
if( ReadBuf(Stream, &type, sizeof(char)) )
{
switch( type )
{
case 0x01: // text
{
AnsiString Msg;
if( ReadString(Stream, Msg) )
{
// use msg as needed ...
return true;
}
break;
}
case 0x02: // file
{
AnsiString FileName;
if( ReadFile(Stream, FileName) )
{
// use FileName as needed...
return true;
}
break;
}
default: // unknown type
{
// unknown data received, do something...
break;
}
}
}
return false;
}
> The contents of 'maurice.ini', is garbled.
You did not show the code you used to send the data. But since you are
writing the buffer to the stream in the wrong place anyway, you are likely
writing a buffer that has not been initialized yet.
> Also, I have a quick question about function ReadBuf(TStream *Strm,
> void *Buffer, int Count). In the function you have
> pBuf += NumRead. Why are you adding the number of bytes
> read to our data?
I am incrementing the pointer to the data, not appending the number to the
data itself. The pointer is incremented to point to a new memory location
inside the Buffer.
For example, say a 10-byte Buffer is passed to ReadBuf(). Then say that
TWinSocketStream::Read() receives 5 bytes the first time it is called. The
first 5 bytes of the Buffer now contain data. pBuf is then incremented by 5
bytes so that the next call to TWinSocketStream::Read() will receive the
next block of incoming data beginning at the 6th byte of the Buffer. And so
on.
The same applies to WriteBuf() as well. Again, with a 10-byte Buffer passed
in, if TWinSocketStream::Write() sends 5 bytes the first time, then the next
call to Write() has to start at the 6th byte in the Buffer. And so on.
Gambit
Re: TWinSocketStream Read in BYTE buffer
Author: newsgroups.borland.com
Date: Fri, 30 Dec 2005 21:43
Date: Fri, 30 Dec 2005 21:43
41 lines
2703 bytes
2703 bytes
Guys, it works!!
> but you seem to be ignoring them completely.
My bad, I am not ignoring them it's just that I somewhat misunderstood your
directions.
> Here is a more complete code snippet for you:
Thanks, the extra code really helped fill in the gap
> I am incrementing the pointer to the data, not appending the number to the
> data itself. The pointer is incremented to point to a new memory location
> inside the Buffer.
Oh, I interpreted it the way an int variable is incremented by '+=', as in
int a; a += 1; Again, my mis-understanding.
Also, I discovered an error in the code that caused files sent over to the
server to be pre-padded with 'blank' bytes equal to the original file/data
size over. So a file that is 192 Bytes on the client become 384 Bytes on
the server. The error is where you pre-initialized the size of FS to the
size of the data sent over. As in:
ReadFile(TWinSocketStream *Stream, AnsiString &FileName)
{
......
try
{
TFileStream *FS = new TFileStream(fn, fmCreate);
try
{
FS->Size = size; <- Problem here
....
}
Once I commented out the line, it worked perfectly!!
Gambit, I want to thank you again for your time and effort on this one ...
again, your help really got me out of a ditch. The same goes for Jonathan
and Hans, your insights help me better grasp how the data is managed and
sent.
Re: TWinSocketStream Read in BYTE buffer
Author: Remy Lebeau (TeamB)
Date: Mon, 02 Jan 2006 23:07
Date: Mon, 02 Jan 2006 23:07
36 lines
2831 bytes
2831 bytes
"newsgroups.borland.com" <mauriceanderson@hotmail.com> wrote in message
news:43b5a868$1@newsgroups.borland.com...
> Oh, I interpreted it the way an int variable is incremented by '+=',
> as in int a; a += 1; Again, my mis-understanding.
A pointer is an integer as far as the CPU is concerned, just one whose value
is a memory address. Any mathematical operation can be performed on a
pointer since it is just a number like any other.
> Also, I discovered an error in the code that caused files sent over
> to the server to be pre-padded with 'blank' bytes equal to the
> original file/data size over. So a file that is 192 Bytes on the client
> become 384 Bytes on the server. The error is where you pre-initialized
> the size of FS to the size of the data sent over.
I did not realize it at the time, but setting the Size of a TFileStream
changes the Position to the new Size as well. That is a bug in the VCL, in
my opinion. Setting the Size should no change the Position. You will have
to set the Position back to 0 after setting the Size before then writing the
data to the file.
FS->Size = size;
FS->Position = 0; // <-- add this
> Once I commented out the line, it worked perfectly!!
It is good to set the Size beforehand when you know the amount of data being
written (which you do). It allows the OS to optimize the writing to the
harddrive since it will not have to hunt around for available space on each
call to TFileStream::Write() - all of the needed space is pre-allocated
ahead of time.
Gambit
Re: TWinSocketStream Read in BYTE buffer
Author: Maurice Anderson
Date: Thu, 05 Jan 2006 10:04
Date: Thu, 05 Jan 2006 10:04
154 lines
6552 bytes
6552 bytes
Also found an issue in ReadBuf which causes TFileStream *FS to write fewer
bytes to the file than what is in the stream. For example:
(1) We have a file that is hdrfx1.jpg(63,820) bytes sent from the client
http://www.sonystyle.com/intershoproot/eCS/Store/en/imagesProducts/650x650/H
DRFX1.jpg
(2) When it hits my server it first goes to ReadPacket(..)
where it is determined to be a file.
(3) From there it gets sent to ReadFile(..).
(4) ReadFile calls Readbuf in a while loop to read the contents
from the stream into a buffer 256 bytes at a time
TFileStream *FS = new TFileStream(fn, fmCreate);
.....
BYTE rbuf[256];
.....
while( size > 0 )
{
bufsize = size;
if( bufsize > sizeof(rbuf) )
bufsize = sizeof(rbuf);
if( !ReadBuf(rbuf, bufsize) )
{
error = true;
break;
}
if( FS->Write(rbuf, bufsize) < 1 )
{
(5) So ReadBuf is called 249 times to read 63,744 Bytes.
And each time ReadBuf returns, FS faithfully writes the contents
to itself the same 249 times. This leaves the last 76 Bytes to go.
(6) All is good so far until we have to read the last 76 Bytes
to complete the file. For this we have to take a look
at ReadBuf.
bool __fastcall ReadBuf(TWinSocketStream *Stream, void *Buffer, int
BufSize)
{
BYTE *pBuf = (BYTE*) Buffer;
int NumRead;
while( BufSize > 0 )
{
NumRead = Stream->Read(pBuf, BufSize);
if( NumRead < 1 )
return false;
pBuf += NumRead;
BufSize -= NumRead;
}
return true;
}
(7) You see, once the Stream reads those 76 Bytes into pBuf, it cycles
back once more. On this last cycle in encounters 0 bytes to read
so it kicks back a false as the return variable:
if( NumRead < 1 )
return false;
(8) The false return goes back to ReadFile where it gets completely kicked
out of the while loop before those last 76 Bytes are read by FS.
if( !ReadBuf(rbuf, bufsize) )
{
error = true;
break; < - exists loop with 76 bytes
left in rBuf
}
if( FS->Write(rbuf, bufsize) < 1 ) <- Never sees
the last 76 bytes
{
(9) So we end up with a file written short those 76 Bytes.
(10) To resolve this I change around ReadBuf to return the number of
bytes read. If the bytes read from ReadBuf are < 1, then I kick
out
of the while loop.
Therefore, FS gets to read those last 76 Bytes before the while
loop
is exited and we have our complete file transferred.
int __fastcall TBinaryTransfer::ReadBuf(TStream *Stream, void *Buffer,
int BufSize)
{
BYTE *pBuf = (BYTE*) Buffer;
int NumRead;
int NumRead2 = 0;
try
{
while( BufSize > 0 )
{
NumRead = Stream->Read(pBuf, BufSize);
if( NumRead < 1 ){
return NumRead2;
}
NumRead2 = NumRead; <- Holds the last byte amount read
in case we have
to return when we hit zero.
pBuf += NumRead;
BufSize -= NumRead;
}
}
catch(const Exception &)
{
return 0;
}
return 256;
}
ReadFile:
while( size > 0 )
{
if( bufsize > sizeof(rbuf) )
bufsize = sizeof(rbuf);
numread = ReadBuf(Stream, rbuf, bufsize, 0);
if( numread < 1 )
{
error = true;
break; <-- exits only when we
are sure there
is nothing
left in rbuf
}
if( FS->Write(rbuf, numread) < 1 )
{
error = true;
break;
}
size -= bufsize;
}
Thanks again, you are a life saver.......... troubleshooting the code help
me really understood how you read and processed the contents of the stream.
You are truly very talented in this area and a blessing to many people. And
your insights are always appreciated.
-M-
Re: TWinSocketStream Read in BYTE buffer
Author: Remy Lebeau (TeamB)
Date: Thu, 05 Jan 2006 18:50
Date: Thu, 05 Jan 2006 18:50
268 lines
9916 bytes
9916 bytes
"Maurice Anderson" <mauriceanderson@hotmail.com> wrote in message
news:43bcedb2$1@newsgroups.borland.com...
> Also found an issue in ReadBuf which causes TFileStream *FS
> to write fewer bytes to the file than what is in the stream.
In order to ensure that all data gets written to the file, you have to look
at the return value of TFileStream::Write(), just like you have to with
TWinSocketStream::Write(), so that you know how many bytes were actually
written to the stream. That way, you can call Write() again if needed to
save any remaining unwritten bytes.
The easiest way to do that is to change ReadBuf() and WriteBuf() to take a
TStream* parameter instead of a TWinSocketStream* parameter, since the inner
logic already handles incomplete reads/writes. That way, you can pass both
TWinSocketStream and TFileStream to the same processing code:
bool __fastcall ReadBuf(TStream *Stream, void *Buffer, int BufSize)
{
BYTE *pBuf = (BYTE*) Buffer;
int NumRead;
try
{
while( BufSize > 0 )
{
NumRead = Stream->Read(pBuf, BufSize);
if( NumRead < 1 )
return false;
pBuf += NumRead;
BufSize -= NumRead;
}
}
catch(const Exception &)
{
return false;
}
return true;
}
bool __fastcall WriteBuf(TStream *Stream, void *Buffer, int BufSize)
{
BYTE *pBuf = (BYTE*) Buffer;
int NumSent;
try
{
while( BufSize > 0 )
{
NumSent = Stream->Write(pBuf, BufSize);
if( NumSent < 1 )
return false;
pBuf += NumSent;
BufSize -= NumSent;
}
}
catch(const Exception &)
{
return false;
}
return true;
}
Then you can update ReadFile() and SendFile() accordingly:
bool __fastcall SendFile(TWinSocketStream *Stream, const AnsiString
&FileName)
{
bool error = false;
try
{
TFileStream *FS = new TFileStream(FileName, fmOpenRead |
fmShareDenyWrite);
try
{
char type = 0x02;
if( WriteBuf(Stream, &type, sizeof(char)) )
{
if( WriteString(Stream, ExtractFileName(FileName)) )
{
int size = FS->Size;
if( WriteBuf(Stream, &size, sizeof(int)) )
{
BYTE rbuf[256];
int bufsize;
while( size > 0 )
{
bufsize = size;
if( bufsize > sizeof(rbuf) )
bufsize = sizeof(rbuf);
if( !ReadBuf(FS, rbuf, bufsize) ) // <--
here
{
error = true;
break;
}
if( !WriteBuf(Stream, rbuf, bufsize) )
{
error = true;
break;
}
size -= bufsize;
}
}
}
}
}
__finally
{
delete FS;
}
}
catch(const Exception &)
{
error = true;
}
return !error;
}
bool __fastcall ReadFile(TWinSocketStream *Stream, AnsiString &FileName)
{
AnsiString fn;
if( ReadString(Stream, fn) )
{
fn = ("C:\\Some Folder\\" + ExtractFileName(fn));
int size = 0;
if( ReadBuf(Stream, &size, sizeof(int)) )
{
bool error = false;
try
{
TFileStream *FS = new TFileStream(fn, fmCreate);
try
{
FS->Size = size;
FS->Position = 0;
BYTE rbuf[256];
int bufsize;
while( size > 0 )
{
bufsize = size;
if( bufsize > sizeof(rbuf) )
bufsize = sizeof(rbuf);
if( !ReadBuf(Stream, rbuf, bufsize) )
{
error = true;
break;
}
if( !WriteBuf(FS, rbuf, bufsize) ) // <-- here
{
error = true;
break;
}
size -= bufsize;
}
}
__finally
{
delete FS;
}
}
catch(const Exception &)
{
error = true;
DeleteFile(fn);
}
if( !error )
{
FileName = fn;
return true;
}
}
}
return false;
}
> (5) So ReadBuf is called 249 times to read 63,744 Bytes.
> And each time ReadBuf returns, FS faithfully writes the contents
> to itself the same 249 times. This leaves the last 76 Bytes to go.
That is not a correct assessment.
The size value that ReadFile() receives will indicate the full size of the
file. In this case, the size should be coming through as 63820, not 63744.
ReadFile() will keep reading until all 63,820 bytes have been received.
That is because of this line:
while( size > 0 )
The size of the buffer being used does not matter. In this case, since the
buffer size is 256, after 249 iterations of the loop, the size variable will
be 76, and ReadFile() will then read the last 76 bytes instead of 256 bytes
and then exit the loop since the size variable falls to 0.
> (7) You see, once the Stream reads those 76 Bytes into pBuf, it cycles
> back once more. On this last cycle in encounters 0 bytes to read
> so it kicks back a false as the return variable:
That is not a correct assessment, either.
On input, the BufSize parameter is set to 76. ReadBuf() loops only as long
as there is data expected to be received:
while( BufSize > 0 )
Every time pStream->Read() returns without error, the actual number of bytes
received is substracted from the BufSize parameter:
BufSize -= NumRead;
After 76 bytes have eventually been received, the BufSize parameter has
fallen to 0 and the loop exits without calling pStream->Read() again. At
which time, a 'true' value is returned, not 'false'.
> if( !ReadBuf(rbuf, bufsize) )
> {
> error = true;
> break; < - exists loop with 76
bytes
> left in rBuf
> }
No, it does not. ReadBuf() return true, not false, on the last 76 bytes.
> if( FS->Write(rbuf, bufsize) < 1 ) <- Never
sees
> the last 76 bytes
> {
Yes, it will.
> (10) To resolve this I change around ReadBuf to return the number of
> bytes read. If the bytes read from ReadBuf are < 1, then I kick
> out of the while loop.
You should not have done that. There is nothing wrong with the ReadBuf()
code I have given you.
Not only that, but you have introduced bugs into ReadBuf() now. If
successful, you are returning a hard-coded 256 rather than the number of
bytes actually read. If failed, you are returning the number of bytes
received in the last read, not the total number of bytes received before the
error occured.
Gambit
Re: TWinSocketStream Read in BYTE buffer
Author: Maurice Anderson
Date: Fri, 06 Jan 2006 09:45
Date: Fri, 06 Jan 2006 09:45
50 lines
3641 bytes
3641 bytes
> Not only that, but you have introduced bugs into ReadBuf() now. ..
Ok, I see your points and undid my changes.
Couple things that are not clear with me though:
(1) What chunk sizes does TClientSocket and
TServerSocket use to send/receive bytes over
the network? 4K, 8K, 16K, etc?
(2) What is the transfer speed on say a 100 /mbits LAN?
(3) What is the limit size in terms of Byes that that the Stream
can hold? And once this limit is reached, what happens if
I don't do a ClientSocket1->Socket->SendBuf(..);
(4) Using your ReadBuf/writeBuf methods at what point is the
data considered transfered? For example on the Client end
we have: ClientSocket1->Socket->SendBuf(..) and on the
server we have: NumRead = Stream->Read(pBuf, BufSize);
Once Socket->SendBuf returns, is the data considered 'sent'
over or is it considered sent when we read from the Stream in:
Stream->Read(pBuf...)?
(5) Does the whole 25 MB file already exists in the stream ready
to be written to pBuf (and by extension fully copied from the client)
or is the ClientSocket holding it's horses ready to transmit another
chunk as soon as NumRead = Stream->Read(pBuf, BufSize) is completed?
(6) Pertaining to the sending/receiving of bytes in #4, how can I count
said bytes transferred I basically want to add some sort of data
transfer BYTE counter to let the user know file/data copy progress.
(7) What is the most efficient BYTE buffer size to allocate for the reading
and
writing functions for small, medium and large files? Or does it
matter?
Currently, I have BYTE rbuf[256];, what happens if I set it to say:
BYTE rbuf[1024], etc?
(8) How can I get the Client to open its own thread for sending data?
I use TServerClientThread to receive connections, but how can I
use it to open a connection to a server and use it to transmit?
(9) How can I track each individual connection received on the server
to say get the source IP address?
(10) How an I convert a BYTE to binary 1s and 0s?
Thanks
Re: TWinSocketStream Read in BYTE buffer
Author: Remy Lebeau (TeamB)
Date: Fri, 06 Jan 2006 19:42
Date: Fri, 06 Jan 2006 19:42
184 lines
8432 bytes
8432 bytes
"Maurice Anderson" <mauriceanderson@hotmail.com> wrote in message
news:43be3aac@newsgroups.borland.com...
> What chunk sizes does TClientSocket and TServerSocket use to
> send/receive bytes over the network? 4K, 8K, 16K, etc?
That is managed by the socket internally, and by the network hardware. The
VCL does not control that.
> What is the transfer speed on say a 100 /mbits LAN?
That is entirely dependant on the speed of your code when sending/reading
data, as well as the amount of other traffic on the network at the time.
Speed will vary.
> What is the limit size in terms of Byes that that the Stream can hold?
TWinSocketStream does not hold any data. It is just a passthrough to the
socket. The socket itself has separate internal buffers for incoming and
outgoing data. The VCL does not control those sizes, but you can use the
Win32 API getsockopt() function to retreive the sizes, and setsockopt() to
change the sizes.
> And once this limit is reached, what happens if I don't do a
> ClientSocket1->Socket->SendBuf(..);
The data remains buffered inside the socket. Eventually, the inbound buffer
will fill up and the socket will not be able to accept any further data
until buffer space is cleared. While the buffer is full, the sender will
likely receive an error, such as WSAENOBUFS (10055) or
WSAEWOULDBLOCK(10035).
> Using your ReadBuf/writeBuf methods at what point is the data
> considered transfered?
Inbound data is "transferred" as soon as it appears in your target buffer.
As soon as decide not to read any more data, you have finished transferring
everything.
Outbound data is "transferred" as soon as the socket accepts it. The socket
will buffer the data internally and then transmit it over the network at its
leisure. TWinSocketStream::Write() will return how many bytes the socket
accepted into its buffer. There is no way to know when the data is actually
transmitted over the network. The only thing you can do is assume that it
will be transmitted *at some point in time* if the socket accepted it, and
move on.
> For example on the Client end we have: ClientSocket1->Socket->SendBuf(..)
> and on the server we have: NumRead = Stream->Read(pBuf, BufSize);
> Once Socket->SendBuf returns, is the data considered 'sent' over or is it
> considered sent when we read from the Stream in: Stream->Read(pBuf...)?
On the client side, it is considered "sent" when SendBuf() returns,
indicating that the socket accepted the data into its internal buffer.
On the server side, it is considered "received" when the data appears in
pBuf.
> Does the whole 25 MB file already exists in the stream ready to be
> written to pBuf (and by extension fully copied from the client)
The buffers inside a socket are not large enough to hold that much data at
one time.
> is the ClientSocket holding it's horses ready to transmit another
> chunk as soon as NumRead = Stream->Read(pBuf, BufSize) is completed?
Yes.
> Pertaining to the sending/receiving of bytes in #4, how can I
> count said bytes transferred
That is what the return value of the stream's Read/Write() methods is for,
as are the return value of the socket's SendBuf/ReceiveBuf() methods. The
return value is the number of bytes actually received/accepted at that
particular moment.
> What is the most efficient BYTE buffer size to allocate for the
> reading and writing functions for small, medium and large files?
That is entirely up to you to choose. Like I told you earlier, the code I
gave you does not care what sizes are used. What you can do, however, is
use get/setsockopt() to have your buffer sizes match the socket's internal
buffer sizes.
> Currently, I have BYTE rbuf[256];, what happens if I set it to
> say: BYTE rbuf[1024], etc?
All that does is lets more data be buffered in memory at a time. Of course,
the larger the buffer, the more space that is available when receiving data
from a socket, which will likely allow the socket to clear its incoming
buffer more quickly. But the size of the buffer for outgoing data is still
limited to the size of the socket's internal outgoing buffer.
> How can I get the Client to open its own thread for sending data?
TClientSocket is not a threaded component at all. You will have to manage
the thread yourself manually. Derive a class from TThread and override its
Execute() method. Pass the TClientSocket's Socket property to the thread,
such as a parameter to the thread's constructor:
class TClientThread : public TThread
{
private:
TCustomWinSocket *FSocket;
TWinSocketStream *FStream;
protected;
virtual void __fastcall AfterConstruction();
virtual void __fastcall Execute();
public:
__fastcall TClientThread(TCustomWinSocket *ASocket);
__fastcall ~TClientThread();
};
__fastcall TClientThread::TClientThread(TCustomWinSocket *ASocket)
: TThread(true)
{
FSocket = ASocket;
FStream = new TWinSocketStream(FSocket, 5000);
}
void __fastcall TClientThread::AfterConstruction()
{
Resume();
}
__fastcall TClientThread::~TClientThread()
{
delete FStream;
}
void __fastcall TClientThread::Execute()
{
while( (!Terminated) && (FSocket->Connected) )
{
// use FStream as needed ...
}
}
Then you can start the thread after you have connected the TClientSocket to
the server:
TClientThread *FThread = NULL;
void __fastcall TForm1::ClientSocket1Connect(TObject *Sender,
TCustomWinSocket *Socket)
{
FThread = new TClientThread(Socket);
}
void __fastcall TForm1::ClientSocket1Disconnect(TObject *Sender,
TCustomWinSocket *Socket)
{
if( FThread )
{
FThread->Terminate();
FThread->WaitFor();
delete FThread;
FThread = NULL;
}
}
> I use TServerClientThread to receive connections, but how can
> I use it to open a connection to a server and use it to transmit?
You cannot use TServerClientThread on the client side. It only works on the
server side. Use TThread instead on the client side, as shown above.
> How can I track each individual connection received on the server
> to say get the source IP address?
TServerSocket has ActiveConnections and Connections[] properties available.
Or else you can maintain your own list manually, such as by grabbing the IP
address at the beginning of the TServerClientThread's ClientExecute() method
and then add it to your own thread-safe list somewhere, and then remove the
address at the end of ClientExecute().
> How an I convert a BYTE to binary 1s and 0s?
It is already stored in a binary format. Everything in memory is binary.
Gambit
Thread Navigation
This is a paginated view of messages in the thread with full content displayed inline.
Messages are displayed in chronological order, with the original post highlighted in green.
Use pagination controls to navigate through all messages in large threads.
Back to All Threads