🚀 go-pugleaf

In article <6d713t$k3n$1@Mars.mcs.net>, Karl Denninger <karl@MCS.COM> wrote:
>>http://www.boardwatch.com/mag/98/mar/bwm24.html
[...]
>1)	They never signed into anywhere.  Therefore, there are several
>	problems with the so-called "data":

Actually that's false.  If you'd read the article you'd notice that while
the first-pass statistics were gathered with just the raw connect rate of
the modem, they also did a smaller number of trials with actual connections
(and actual downloads).  Although this sample set is much smaller and
thus has less statistical significance, it did pretty closely mirror the 
results of the initial connect speed test.

>	a)	There's no independant verification possible from the other
>		side of the connections.  Without RADIUS logs, there's just
>		no way to know if we're really getting good data from
>		Rickard or not.

So your defense is 'maybe Boardwatch magazine made up the data'.
Interesting.  Do you have some sort of evidence, or is that just a wild
accusation?

>	b)	There is no indication of what devices were on the other 
>		end of the connections, because Rickard didn't bother
>		coordinating these tests with the other side of the link.
>		There is ALSO no indication of what FIRMWARE is on the other
>		end of the links, and that, for some manufacturers, is
>		critically important.

Of course, this applies equally to both X2 and K56.  The tests were done to
42 different X2 dialups and 61 different K56 dialups.  It's probably safe to
say that the hardware/firmware he tested against is reasonably representative
of what's out there.

If you think that your K56 is so much better, why not ask him to run his
tests against it?

>2)	USR is known to connect at a higher rate than it can actually
>	support, and then downshift.
[...]
>3)	K56Flex is known to typically (in most implementations anyway)
>	connect at a LOWER rate than it can actually support, and then
>	upshift over the next couple of minutes.
[...]

These _exact_ issues are discussed extensively in the article.  The fact
remains that even when they tested actual usable download bandwidth
X2 won by a large margin.  This essentially negates the "USR lies about
connect speeds" argument.

>4) 	Long-distance connections are NOT typical of the usual ISP user
>	profile.

Again, this is discussed a lot in the article.  

> Yet that is *ALL* that Rickard tested.  This gives insane

False.  If you'd have actually READ the article, you'd notice that he
tested both local and long-distance in the connect-speed test.  In the
download throughput, tests were to local ISPs.

>	advantages, for example, to the carrier that the tester uses for IXC
>	carriage!  Why?  Because there is at least one "crossing" between
[...]
>	to ignore alarms for literally months, leading you to wonder if they 
>	even wire the ALARM output on their switches to anything!

First, a much more likely scenario for why long distance calls have an
advantage is explained in the article (it goes through a simpler pad)

Second, what does this have to do with K56/X2?  K56 and X2 were
both tested with local and long-distance connections.

>5)	In the realm of K56Flex, Rickard doesn't know about the S202=32 
>	init switch (primarily because he never called any of the tested 
>	ISPs tech desks).  That's an undocumented, but VERY important 
>	switch to turn on in some Rockwell versions.
>
>	The Zoom and TDK CyberExpress5600 modems, in particular, DO work 
>	in a 2-bit robbed environment.  I know this, because I get these
>	connections *all the time* and the firmware reports 2 or even 3 bits
>	robbed a good bit of the time.  Rickard and whoever he claims to have
>	talked to are simply wrong on this.  However, you *DO* need S202=32
>	in the init for a whole bunch of those firmware revisions for it to 
>	work in a 2-bit robbed environment.

Well, according to the article, a senior engineer at Rockwell spent two
days at the test site trying to make the K56 work.  He failed.  Now you claim
basically that neither Jack Rickard (longtime editor of Boardwatch and
well-known expert in modem-based communication) nor an actual engineer from
Rockwell knew enough to make a Rockwell-based modem work acceptably.

So according to YOU, the Rockwell-based modems can't be made to work
reasonably by Rockwell's own engineers.  Yet you recommend them to
consumers.  I think everyone would do well to reflect on that little
pearl of wisdom from Karl.

>6)	For those who buy their time from others, Rickard just billed them
>	for hundreds of minutes of time without their consent and not as
>	their customer.  This looks an awful lot like theft of service,
>	absent informed consent.  If I were paying by the minute, I'd be
>	looking for someone to be indicted over this.

First, this is again irrelevant to the fact that your pet modem technology
fell on its face.

Second, it's not like he was calling 1-800 dialins.  He made 140,000 calls,
most of which were long distance.  That's a fair amount of money (thousands
of dollars no matter how you count it) out of his pocket.  So it's not like
he's shifting all the load of his tests to the ISPs.

Third, do you have any proof that any of the ISPs he was calling were actually
paying by the minute for their incoming phone service?

Fourth, do you honestly expect me to believe that you'd be nearly as outraged
if K56 hadn't lost?

>7)	We, and many others, just don't see anything close to what Rickard
>	claims is happening.  We have hard logs in our DBMS for every call
>	- including ACTUAL data transmission rates.  I've got data going
>	back to April of 1996, when we first switched to RADIUS from our
>	proprietary system, on both V.34 ASCEND and K56Flex
>	Livingston/Lucent hardware.

In this article he practically is begging for K56 shops to give him some
of this real world data.  Boardwatch was able to get their hands on a
significant amount of real world connection rate data from a big X2 shop
and reported the results six months ago.  As of the time this new article
was written, all K56 shops that planned on giving him connect speed data
haven't come through.  Odd, huh?

And even if they did get K56 data you'd have to take it with a grain of
salt.  If you look at Rickard's connect speed data for K56 (all of this
with K56-capable modems connected with PCM-capable phone lines to K56-capable
dialins) at:
	http://www.boardwatch.com/mag/98/mar/pg35a.jpg
you'll see that the biggest problem wasn't so much that it connected slow,
but that in the majority of cases it failed to work AT ALL.  The speed
peak falls *below* 33.6.  If you only count connections >33.6 then K56
looks only about half as bad as it really is.

>	His data is simply non-representative of the real world, and I know 
>	of a whole lot of other ISPs who are saying the same thing.  We're 
>	not alone on this.

Now if you and those "many others" would like to give Rickard some actual
data, then maybe we could see.  So far all of the data that's come forward
has been pretty supportive of X2.  If you have connection speed data
to the contrary I'd suggest sending it to Boardwatch.

My personal analysis of the situation is this - perhaps in some cases
K56 isn't as bad as these tests indicate.  However, there are good reasons
it lost.  K56 loses in three ways:

  1. USR was the first to have a working PCM modem.  They've simply had
     longer to perfect their product.  K56 is playing catch-up.

  2. K56flex is actually a marriage of convenience between two different
     modem technologies - Rockwell's K56plus and Lucent's V.flex2.  When
     it was obvious to everyone that both Rockwell and Lucent were going
     to get creamed in the marketplace they suddenly decided to team
     up on the K56flex 'standard'.  They then each rushed their separate
     implementations out the door in order to ONLY be a month later than
     X2.

  3. USR has what is undoubtedly the most thorough real-world testing
     methodology in the industry.  By far.  People always wonder
     why ExecPC are such USR fanatics.  The answer is quite simple.
     For years ExecPC ran the world's largest BBS system.  In that
     time, the folks here saw pretty much every brand of modem that
     existed in the last 15 years.  They've also seen pretty much every
     phone-line imperfection in action.  When you see this much real-world
     modem use, you tend to become very impressed by USR.

     And no wonder.  The Boardwatch article went on and on about USR's
     testing methodology, and they're right.  The folks at USR are absolutely
     maniacs about testing.  In this case it shows.

     In the past the lab testing was ALMOST sufficient to cover most
     situations.  Try connections with some different levels of noise and
     a few different attenuations and call it a day.  PCM technologies break
     this horribly.  Suddenly the whole issue of 'noise' suddenly isn't
     just a frequency response.  Does the telco equipment rob bits?  Which
     ones?  What pad is on the codec?  What exact encoding is the codec
     using?  This is on top of the normal analog modem issues.  Suddenly
     making a sufficient set of tests to perform at the lab is a daunting
     task.  The only way to really find what works on what combination of
     telco equipment in the field is to try it everywhere.  This is USR's
     forte.

So on one hand you have USR with a head-start on the competition AND the
testing system in place to pull it off.  Oh yeah, plus they've been using
flash upgradable DSP-based designs for years, so they didn't even have to
redo their basic design.  (Aside to Karl: any of your customers using K56
with modems they bought two years ago?  Didn't think so.)

On the other hand you have the K56 camp.  First they start behind.  Then
they have to come up with entirely new chipset designs to support PCM.
Then they decide to merge their standards into something new.  Then they
have to rush the whole mess out the door so they don't become completely
irrelevant.  To top it off, two different implementations have to
interoperate.  What do you get?

You get EXACTLY what you'd expect.  On an EXCEPTIONAL phone line (where PCM
is really no big trick) you get a horse race.  Some tests even say that
K56 edges out X2 by a *slight* margin.  Maybe that's true, it probably varies
from line to line.

However, once you get to the average PCM-capable line (maybe a little far
from the CO, maybe there's a bit robbed here or there, maybe the CODEC has
floated a tad off from mu-law, etc) and suddenly you're playing to USR's
strengths.  You find that K56 sometimes works sorta.  X2 just determines
what's going on and adapts to the line conditions it sees.  Whatever your
telco has thrown at your line, it's probably seen before in testing.  It
has nothing to do with the technology per se, but everything to do with
implementation.

That's the lesson that the Boardwatch article demonstrates - on some
real-world phone lines that were by all accounts pretty clean but nothing
superb X2 won easily.  USR's legendary testing system and their mastery
of DSP modem design comes together and handles the line conditions
beautifully.  To those companies which know how the modem industry works
this was no surprise.  It is nice to have some vindication though :-)

>Finally, its all irrelavent - V.90 is due out on the central site side for
>virtually everyone's gear within 60 days, and these results, even if they
>ARE "correct" for some set of conditions, will have nothing to do with V.90
>results!
>
>Rickard likes to claim that USR will have the "best" V.90.  We'll see...

That's exactly why it's all NOT irrelevant!

In the PCM world, almost everything is client-side controlled - it's the
only one with access to the analog side of the connection so it determines
what type of encodings will be used for the PCM connection.

The equipment at the ISP side will become increasingly irrelevant to
connection speed under v.90.  However, the customer-side will continue
to be very important.  On most phone lines, the customer's modem will
have to do alot of careful work to determine exactly what pieces of the 64k
digital spectrum it can and cannot decode.  This will determine the
bandwidth available.  Rickard's data shows that USR currently handles
this quite a lot better than the competition.

>I'll put *ACTUAL* throughput *AND* connection rates of our K56flex plant 
>up against X2 any day of the week, and anyone who would like to run a REAL
>test is welcome to get ahold of us.

You've got Rickard's email address.  He usually seems up for a challenge.

-Mitchell Blank Jr
 mitch@execpc.com
 ExecPC, Systems Administration