🚀 go-pugleaf

Carl Danley wrote:
> Actually, this sounds very interesting... My only thing is this: As I 
> dont have the greatest experience with the whole process for sending 
> voice packets over the internet, i would have to catch up on what you 
> already know. I would be happy to work with someone on implementing this 
> feature into a program. I'm not sure what I could help with but I'd love 
> to help. In reference to receiving the voice packets all cut and choppy, 
> out of order, I believe we would have to implement something to put them 
> back into order... Sounds obvious, but this would imply some way to send 
> a packet with the ordering so that when all of the voice packets have 
> been received, the receiver can put them back into place... Anyways, I 
> would love to work with you on this project, if I am capable, so that we 
> may both further our understanding of the transmission of voice packets. 
> I dont have MSN Messenger, but I can go get one if needed.
> 
> Carl :)

Hey,
I'm really sorry, I was unexpectedly thrown out the Internet for the 
past two weeks. To make a long story short, I moved out on July 1st and 
my ISP forgot about my request. I got the Internet back only this morning.

I just read the entire thread. I'm going to comment randomly to all of 
the discussion (sorry if I didn't keep track of who said what).

What I was doing in my system (Aleck) and all of the other testing 
programs before, was this:

Used WaveIn/WaveOut functions (from the Win API). Those are really easy 
to use. Basically, you have at least two buffers (more is better, but I 
found out through my tests that 2 were fine on my system, which was then 
a P3-500). Through the WaveIn functions, you give those buffers to the 
sound card and ask it to start recording. The sound card notifies as 
soon as one of the buffer is filled. You process the buffer and put it 
back in the sound card's queue. The reason why you need more than one 
buffer is that the sound card does not have any other internal level of 
buffer of any kind, so if you have just one buffer, you would miss some 
of the recording as you process the single buffer before you give it 
back. So basically, the sound card start filling the second buffer while 
you are processing the one which is complete. For playing back, you just 
give the sound card some filled buffers (using WaveOut functions) and 
ask it to play them. Again, you need to have more than one, so it can 
play the next buffer as you are preparing a new one.

Of course, the interesting part is in between the recording and playing 
back steps (cause well, you are not going to listen to yourself).

Once you have one filled buffer (a waveform with some of your pretty 
voice in it) you have to compress it (uncompressed high quality wave is 
too heavy for regular Internet use, although it might be acceptable on a 
local network). There is a number of audio compression algorythms 
available out there, the most well known being MP3. However, MP3 is 
designed for music. It is not optimized for voice-only and thus gives a 
big file for a nominal quality level. In my system I used Speex, it's a 
high quality, free (open source), audio codec specialised for voice. It 
is in fact the default codec used by TeamSpeak (a 
voice-over-the-internet software especially designed for used while 
playing games over the net). It is relatively easy to use, has a low 
latency (when used correctly), very resistent to packet loss (you can 
loss 70% of the packets and still play back a voice message that is 
surprisingly clear).

Once you have a buffer containing some compressed audio data, comes the 
time when you send the buffer to your peer through the Internet.

At the other end, the receiving program will uncompressed the audio data 
back to a waveform and play it back using WaveOut.

The main goal is to reduce the latency (that is, the time between the 
moment you speak into your microphone and the moment your voiced is 
played back on the other computer) to a minimum. As such, you want to 
create as small sound buffers as possible (the length of the buffer is 
the first latency cause... that is, a one-second buffer makes a 
one-second delay). IIRC, Speex used 30 ms buffers. Then, you want to 
process the buffer (apply any sound effect [normalize volume, add echo, 
...], compress, and queue to be sent to the peer) as fast as possible. 
Here some optimization (and a good choice of codec and algorithm and a 
good machine) is important. This is the second cause of latency. The 
third cause of latency is the Internet (here you can't do much). 
Decompression and possibly more audio processing is the last latency cause.

Last, for the playback to be smooth, the receiver needs to build some 
form of queue and impose a small delay before playing back the first 
buffer, because the Internet does not guarantee that the buffers will be 
received at a regular interval, Internet delay would cause playback 
cuts. And this is precisely where I had problems. When I tested with the 
sender and receiver on the same machine, the playback was perfect. But 
the there was that beast called the Internet in between, there were no 
way to recreate a clear voice playback, no matter the length of the 
delay imposed. And let me tell you, you hear isn't fine enough to follow 
a conversation, 30 ms at a time.

BTW, I always used TCP during all of my tests (no UDP), so packet order 
was never a problem. But of course, if you use UDP (or some other 
protocol where the packets are not guaranteed to be received in the 
order sent) you need some synchronization mechanism, the most obvious 
being to simply number the packets and replace them in order as they are 
received.

So well, this is pretty much were I've been. Aleck has been stopped for 
over 3 years now, after several months of development and a lot of efforts.

If you wish to contact me, try MSN Messenger at tom@tom420.com (not 
valid as an e-mail address -- specify who you are when adding to avoid 
rejection). Just don't make assumptions. I have projects of my own right 
now, so even thou you convince me to help you, there is little chance 
that I will be able to join a project in the short-term. That said, I'd 
really like to talk with you nevertheless, we never know where that 
would take us :)