🚀 go-pugleaf

In <1jvjuvINN8l6@ftp.UU.NET> news@gvc.com (News Administrator) writes:

> [ modem compression on or off for news? ]

>I'm most interested in hard data on PEP, TurboPEP and V32bis/V42bis.

I run a Sun3 news system at home.  Here's what I discovered:

Compressing a compressed file results in a 50% slow-down.  Why?
Because compression is not magic, it is the removal of entropy and once
you remove all of the entropy you can't remove any more.
[ How's that for over-simplying a very difficult mathematical concept? ]

(BTW, I knew that was going to be the result but I was trying to figure
out if PEP would automatically shut off compression if the compression
ratio was really bad.  It doesn't.  Maybe TurboPEP does? ]

So, who should lead us to the question, "So, should my Sun3 do the
compression or should I leave it up to my modem?"

I expected it to be better to off-load the dirty work to the modem,
since my Sun3 is so slow.  However, I found that the 2x bytes going
through the serial port slowed things down worse than running
"compress" on the Sun3.

That's my view.  Your mileage may vary.

Tom
-- 
Tom Limoncelli -- tal@warren.mentorg.com (work) -- tal@plts.uucp (play)

Unix is a trademark of, WHATT????

In article <1jvjuvINN8l6@ftp.UU.NET> PTomblin@gvc.com writes:
[Is modem compression a good idea when transferring news?]
>I'm most interested in hard data on PEP, TurboPEP and V32bis/V42bis.

Well, I don't have any hard data to give you, but I can tell you a bit
about modem compression.

First of all, there are two types of compression commonly used over
dialup modem links: MNP5 and V.42bis.  Of these two, V.42bis is the
one to use if you have a choice, since MNP5 has the rather undesirable
quality of expanding precompressed data, and thus *reducing*
throughput.  V.42bis will notice when the compressed version of the
data is bigger than the uncompressed version, and send that packet
uncompressed.

So, in a sense, your friend is right -- using MNP5 compression on a
link *will* slow down throughput when you're sending compressed news
batches.  Using V.42bis compression, on the other hand, shouldn't slow
anything down -- unless you or your friend have a cheap underpowered
modem that can't run the data pump at full speed and do data
compression at the same time.

The compression algorithm used by PEP, while not similar to MNP5,
shares with it the quality of not checking the compressed data's size
before sending it.  So, PEP compression should be turned off when
transferring compressed data as well.  TurboPEP compression is just
V.42bis, so that's okay; I'm not sure, however, whether it's possible
to only turn on compression for a TurboPEP connection.

So, if you can use a compression mode that won't expand your
precompressed data, you might want to try using it.  Otherwise, don't
worry about it unless you have a large amount of uncompressed traffic
(i.e., mail) flowing over the link; if you do, you might want to look
into using compressed BSMTP instead of regular rmail.

It's been my experience, though, that most of the throughput problems
encountered by UUCP users stem from the awful parameters used by most
UUCP implementations, not from compression problems.  64-byte packets
are fine if you're sending data at 1200bps or even 2400bps, but at
V.32bis and PEP speeds it really creates a problem.  The latency of
fast modem links is such that the ACKs can't get back to the sending
computer before it runs out of data in the window, and thus the it
winds up sitting around for the ACK to travel over the wire back to
your computer.  In addition, using small packets means that UUCP has
to do numerous small individual writes, generating a lot of overhead
and context switches.

What you want to do is get a version of UUCP on both ends that can use
big -- 512-byte or 1024-byte -- packets.  By using larger packets, you
eliminate the latency problems, and also increase efficiency by
allowing uucico to make fewer write(2) system calls.

Unfortunately, most stock versions of UUCP don't support big packets.
To my knowledge, the only stock version that *does* is the HDB UUCP
distributed with SVR4.  If you don't have SVR4, you'll have to get
Taylor UUCP and build it for your OS.  Of course, there are lots of
other reasons to get Taylor UUCP; I replaced the SVR4 UUCP on my
system even though it *could* handle big packets.

Okay, I lied: Here are some hard numbers.  Using 1024-byte packets and
7 windows, I'm getting in the neighborhood of 1700cps over a V.32bis
modem link with V.42 and V.42bis.  My computer is a 486/33 running
SVR4 and Taylor UUCP v1.04gamma; his computer is a NeXTstation running
NeXTOS 3.0 and an unknown version of Taylor UUCP.  Neither machine is
noted for being a real computing powerhouse, but my machine manages to
keep the data pipe full most of the time (i.e., the modem has to
throttle my machine by dropping CTS before the entire window is sent).
Another site, a DECstation 5000 running Ultrix 4.3, using 64-byte
packets, 7 windows, and V.32 (n.b., not V.32bis) with MNP5, is lucky
to get 800cps.  Yet another site, a 386SX/25 running SCO Unix 3.2v4.1,
using 64-byte packets, 7 windows, and V.32bis with V.42bis, gets
850-875cps.

-- 
Marc Unangst, N8VRH         | "Of course, in order to understand this you
mju@mudos.ann-arbor.mi.us   |  have to remember that the nucleus of the atom
                            |  is squishy."
                            |    -W. Scheider, from a Physics lecture

In article <1k18fcINN274@ftp.UU.NET> tom_limoncelli@Warren.MENTORG.COM (Tom Limoncelli) writes:
>I expected it to be better to off-load the dirty work to the modem,
>since my Sun3 is so slow.  However, I found that the 2x bytes going
>through the serial port slowed things down worse than running
>"compress" on the Sun3.

I have measured the CPU used and running compress actually saves cycles
because later uuxqt and uucico have half the data to process.  The time
saved copying, checksumming and servicing interups is slightly more than
that used by the compress program.

That was for sending.  On receive the savings should be even greater
as receiving typically results in more interupts and uncompress uses
less cycles than compress.

In article <1k1930INN2nl@ftp.UU.NET> mju@mudos.ann-arbor.mi.us (Marc Unangst) writes:

	[ ... remaining preceeding discussion deleted ... ]

>It's been my experience, though, that most of the throughput problems
>encountered by UUCP users stem from the awful parameters used by most
>UUCP implementations, not from compression problems.  64-byte packets
>are fine if you're sending data at 1200bps or even 2400bps, but at
>V.32bis and PEP speeds it really creates a problem.  The latency of
>fast modem links is such that the ACKs can't get back to the sending
>computer before it runs out of data in the window, and thus the it
>winds up sitting around for the ACK to travel over the wire back to
>your computer.  In addition, using small packets means that UUCP has
>to do numerous small individual writes, generating a lot of overhead
>and context switches.
>
>What you want to do is get a version of UUCP on both ends that can use
>big -- 512-byte or 1024-byte -- packets.  By using larger packets, you
>eliminate the latency problems, and also increase efficiency by
>allowing uucico to make fewer write(2) system calls.
>
>Unfortunately, most stock versions of UUCP don't support big packets.
>To my knowledge, the only stock version that *does* is the HDB UUCP
>distributed with SVR4.  If you don't have SVR4, you'll have to get
>Taylor UUCP and build it for your OS.  Of course, there are lots of
>other reasons to get Taylor UUCP; I replaced the SVR4 UUCP on my
>system even though it *could* handle big packets.

	This advice may be good for general use, but not in this individual
case.  Remember that the original poster was asking about the case of a
Sun-3 as his news machine.  There have been many debates in various
comp.sys.sun.* newsgroups about the serial ports and the drivers in the
Suns. If you go to a uucp that allows 7 windows and 1K packets (and are
running 19200 baud), you are probably going experience a *drop* in
throughput.  The problem is that the combination of chip and drivers does
not properly handle the case of the CPU needing to tell the modem that it is
about to overflow its buffers.  You will get lots of "silo overflow"
messages, and lots of retries.  I forget the actual packet size, but in one
of the discussions, it was shown that a large enough packet size at 19200
will ensure that there is a 0% effective throughput, becuase it will
overflow the internal buffers every time.  (Well, very small files will get
through, but anything using the full buffer will just waste connect time.)

	This is made worse by compression on files that acutally work,
especially with higher port speeds, because while the modem is moving only a
certain number of bytes/second, if it is uncompressing those bytes, and then
spitting them at the CPU, the load is even worse.

	If you really need reliable uucp transfers at high speeds, it seems
that the choices are to use some of the SCSI-interfaced serial ports, or an
ethernet terminal server, both of which bypass the internal serial drivers
and chips.  (It is not sure whether a better driver is worth the trouble
with the existing chips, and the chips are part of the design of the CPU
card, rather than being a plug-in card.  (Apparently, even the s-bus cards
for the SPARC machines are not that much of an improvement.)

	( As I am also using a Sun-3 as a news server at home, I may be in
the market for a SCSI serial board or a ethernet terminal server when I see
the first full month's bill from uunet. :-)

	Good Luck
		DoN.
-- 
 Email:   <dnichols@d-and-d.com>  |  ...!uunet!ceilidh!dnichols 
		 <dnichols@ceilidh.beartrack.com>
 Donald Nichols (DoN.)  |   Voice (Days): (703) 704-2280 (Eves): (703) 938-4564
	--- Black Holes are where God is dividing by zero ---

In article <1k1vlpINNi7u@ftp.UU.NET> jerry@olivea.ATC.Olivetti.Com (Jerry Aguirre) writes:
>In article <1k18fcINN274@ftp.UU.NET> tom_limoncelli@Warren.MENTORG.COM (Tom Limoncelli) writes:
>>I expected it to be better to off-load the dirty work to the modem,
>>since my Sun3 is so slow.  However, I found that the 2x bytes going
>>through the serial port slowed things down worse than running
>>"compress" on the Sun3.

>I have measured the CPU used and running compress actually saves cycles
>because later uuxqt and uucico have half the data to process.  The time
>saved copying, checksumming and servicing interups is slightly more than
>that used by the compress program.

>That was for sending.  On receive the savings should be even greater
>as receiving typically results in more interupts and uncompress uses
>less cycles than compress.

Furthermore, my tests have shown that some of the compression modes
used by modems seem to kill uucp protocol g throughput.  On large,
plain ASCII files, with my ZyXEL V32bis/V42bis modem, I get around
550 cps with MNP5 on, and 1000cps with MNP5 off.  Remember that since
this has no host-based compress either way, it means that the same
file takes almost twice as long to transfer with MNP5 on - and since
it's probably optimal for compression, precompressed data could only
be even worse thru MNP5.  This appears to be due to packetization
used by MNP5 interfering with g protocol's packetization.  My tests
have also shown that V42bis doesn't seem to make much difference from
no compression at all (ie: still around 1000cps), tho this may be more
an artifact of my DTE speed.

I seem to get optimal throughput with MNP4 and MNP5 disabled.
V42bis doesn't seem to make any difference.

One of my feeds is a Sun3, and I routinely get 1100 CPS on large compressed
files - he is using a ZyXEL too.
-- 
Chris Lewis; clewis@ferret.ocunix.on.ca; Phone: Canada 613 832-0541
Psroff 3.0 info: psroff-request@ferret.ocunix.on.ca
Ferret list: ferret-request@ferret.ocunix.on.ca