🚀 go-pugleaf

In article <1991Sep1.195029.16374@aber.ac.uk> pcg@aber.ac.uk (Piercarlo Grandi) writes:
>scogen@xenitec.on.ca (scogen@xenitec.on.ca) writes:
>
>> We are trying to configure an SCO Unix system running some type of
>> IP, (maybe TCP, maybe UPD) to operate at maximum throughput. Our goal
>> is to get up to _6_ Megabits/Sec.
>
>Well, nearly any modern Ethernet card will deliver 6 Megabits per second.
>After all it's only around 600KB/s. SCO's TCP protocol stack will most
>probably not be able to keep up with the card though.
>
>> We realize that we need EISA bus machines with 32-bit Ethernet cards to do
>> this.
>
>Why ever? The 16 bit ISA bus runs at 8Mhz and transfers 16 bit across.

Some run at 10 as will some controllers, you may be able to match
a pair that give that xtra bit....

>Taking account of all hw protocol overheads it can sustain between 5-6 MB per
>second. Here you are proposing to load it with just 600KB/s, which is one
>tenth of its top capacity. Think of this: my VGA card will, during continuous
>graphics operation, sustain 2MB/s over the ISA bus, and my SCSI controller
>will get 1.4MB/s sustained from a SCSI disk.
>

But while you are using the bus...

EISA offers 32bit DMA so you can DMA into user space directly, whatever
its address. EISA Bus Master DMA also holds the bus for a shorter time
(upto 33MHz) and being Bus Master reduces the cpu involvement...
(I know PCAT Bus Master boards exist, but they are finicky on memory
timings).

>32 bit EISA Ethernet cards are an expensive joke, as is any "turbo" interface
>between a 40MB/s 32 bit bus (EISA) and a 0.8MB/s 1 bit one (Ethernet).

Cuts response time to the controller requests though and gives
better burst speed to the controller. Expense is probably releated to
current sales volumes. Advent of Intel BMIC usage will bring
costs down...

>
>    Same goes for EISA SCSI disk controllers (the most expensive SCSI disks
>    top out at 4MB/s, and most cannot do more more than 1.5MB/s off the
>    platter). In practice EISA is only needed for very high speed graphics
>    (good luck with that) and for disk arrays, or, theoretically, for very
>    high bandwidth tapes.

Mmm, well with the new Seagate 1.25GB disks that spin at 5400+ rpm, ~3.0MBytes/s
is possible, and with more than one device 5.0 MBytes/s is not
impossible.

Even if the sustained xfer rate is not this, it allows quicker reponse to
simultaneous 2 device requests and reduces bus contention. This all adds
upto better system throughput.

Also, modern drives have caches which allow better than 1.5MByte/s xfers
for < cache size xfers.

>
>    The real benefits of EISA, for most people, are not in bandwidth, but
>    in configurability and interrupt sharing and multiple mastering, where
>    the ISA bus is badly misdesigned.

This is indeed true, but for power users everything helps. EISA also means
up to 16 slots on some boards...

EISA Bus also allows more upmarket systems to be designed which still
allow those systems to use standard ISA boards for less performance
critical uses - providing a more expanded product offering.

EISA gives actual bus timings as a standard, as opposed to "if-it-works-
with-an-IBM-PCAT-then-it-must-be-ok" standards.

>
>> Another important detail is that each computer will be talking over a
>> Cisco router with one ethernet card per computer.
>
>Note that you need an Ethernet card per computer anyhow :-).
>
>Probably what you mean is that you will have a separate Ethernet cable per
>computer. This of course means that each machine would have its own 0.8MB/s
>Ethernet dedicated to itself. The problem is that I doubt very much that
>this would buy you anything at all, as I would expect the CISCO router not
>to be able to sustain more than 0.8MB/s aggregate anyhow, as routers are
>normally not designed to sustain the maximum combined bandwidth of all the
>Ethernets they link. Does anybody know for sure what is the maximum thruput
>of a CISCO router?
>
>I expect that in practice having a star configuration with a router in the
>middle and an Ethernet to each machine will probably buy you nothing at all,
>except having a single active failure point.
>
>> This way we completely eliminate packet collisions.
>
>This is probably completely unnecessary. Collisions are very rarely a
>problem, the Ethernet will in practice share out its capacity between
>machines in only slightly less than linear fashion, if packet sizes are not
>too small.
>
>If you really want for some funny reason a star configuration, with an
>active hub, and no collisions and guaranteed point-to-point capacity,
>consider seriously getting a Token Ring.  At least Token Ring active hubs
>are designed for this task, and not as routers, and tend to be fairly fail
>safe too.
>
>> They also told us about their ES3210 card, which they claim can
>> operate at _13_ Megabits/Sec. I thought that the cable can't operate
>> faster than 10 Megabits/Sec. Am I missing something?
>
>Well, they are telling you nice numbers out of thin air.
>
>> How do I get real numbers that will tell me what to expect in the field?
>
>Toss a coin... :-)
>
>> I don't know how obvious my lack of network expertise is,
>
>Very obvious. :-). But at least you know. Some other poster who also replied
>to your article is not in a much better position than yourself, but does not
>know. :-)
>
>> but if you see any questions that I'm not asking, I need to know that
>> also. I'm sure I'm not asking everything I need to know.
>
>Well, now that I have answered your questions, let's tell you something more
>useful.
>
>First of all, the good news: as I have said almost any modern Ethernet card
>will deliver substantially all of the Ethernet bandwidth, and this bandwidth
>is at worst one fifth of the ISA bus bandwidth, so you don't need any EISA
>machine.  Collisions will be in practice not a problem. You probably don't

Collisions may be a problem. You have to get the DATA from somewhere and
put it somewhere too, unless all data is RAM to RAM only. The longer you
have the Bus the less time there is for disk i/o and handling other ISA
xfers you need. EISA will help reduce this risk. An Ethernet card
that handles the kind of bandwidth you want is probably dumb in the
sense that no protocol handling at link or IP layers and above is
handled out on the board, but in the kernel drivers. You are correct below
that the protocol stacks will probably get you...

EISA will give you 32bit DMA address space, so your drivers can (if
nicely written), give you DMA from user space to controller (or controller
to user space) without copying, whatever your memory size. This is
especially important for high volume disk i/o on SVR4 where the
buffer cache is through the normal VM page fault / free page pool
mechanism and can therefore come potentially from any real memory
address (and not just below 16MB where the SVR4 buffer cache is likely to be).

>need a CISCO router at all, you can put all the machines on the same
>Ethernet, if you don't need 6Mb/s sustained bandwidth between each machine,
>but only aggregate.
>
>The bad news are that the TCP/UDP/IP sw on the host, and in particular
>SCO's, will have such huge overheads and misdesigns that in practice you
>will not be able to use more than a fraction of the nominal speed of the
>hardware. Somebody (Van Jacobson and others) have spent some serious effort
>in tuning up certain implementations of TCP/UDP/IP and can boast of being
>able of getting TCP/UDP/IP end-to-end connections that exploit 90-95% of the
>hardware capacity, but figures like 30-50% are far more common. I don't
>remember exactly, but I seem to remember that you'd be lucky to get more than
>300-400KB/s end-to-end on a TCP/UDP/IP connection from SCO's Internet
>protocol stack.

And even slower for file system to net to file system xfers.
Ftp in BINARY/IMAGE mode for big files probably will even give
only 100kbytes/s of so on a 386/486 type of PC running a fairly standard
Un*x and TCP/IP stack. Tftp won't give much better either.

>
>
>Finally, what you should tell us is what is your problem, e.g. how many
>machines you have got to link, how far apart they are, what is the average
>packet length, and what is the maximum packet rate that you expect from each
>machine, and other characteristics. Once you have told this to us, we can
>tell you whether Ethernet makes any sense at all. Possible alternatives are
>running TCP/UDP/IP over SCSI or over several point-to-point links, e.g.
>synch, or RS-422, for example.

If you are going to do SCSI, ie using host controllers both as
target and initiator, the reliability will be higher than Ethernet
and a simpler protocol layering than TCP/IP may be in order for
efficiency. (ALso bigger block sizes could be used than normal
datagrams).

>-- 
>Piercarlo Grandi                   | ARPA: pcg%uk.ac.aber@nsfnet-relay.ac.uk
>Dept of CS, UCW Aberystwyth        | UUCP: ...!mcsun!ukc!aber-cs!pcg
>Penglais, Aberystwyth SY23 3BZ, UK | INET: pcg@aber.ac.uk

Martin Cryer
Unisys Europe Africa
Unix SVR4 Development
Unisys Salt Lake City

All opinions are my own and not those of my employer.