🚀 go-pugleaf

*********************************************************
On what I've said in my previous mails there are two words I 
need to explain - "endianess" and "coherency bag".
*********************************************************

ENDIANESS

In comp.arch, David James (dvj@apple.com) and Glen Stone 
(stone@apple.com) wrote:
> ...
> So, with this background, please reconsider your
> "no-endian-order" statement.
> ...

We wanted to say "endian independent" NOT "no-endian-order".  
I agree that we didn't make it clear and thanks for pointing 
it out.  I will be careful on this word.

Specifically...
Making the data path 8-bit wide and by sequencing the bytes 
temporaly in ascending address order, one does not worry 
about spatial positioning of the bytes on the transfer path.  
The registers within Anet ports bi-endian with endianess 
established at system initialization. Yes, the most 
significant address byte starts first in the packet making 
Anet channel protocol to be big-addressan.


COHERENCY BAG

In comp.arch, I wrote:
> QuickRing is Apple's highspeed I/O solution and it
> is a derivative of SCI.  Our rational behind Anet
> is very much in-line with SCI guys', except we
> wanted a solution that will be
> A. for I/O (we don't want to carry the coherency bag)
                                     ^^^^^^^^^^^^^^^^^
And Haakon O. Bugge (Dolphin SCI Tech. A.S.) responded:
> You can certainly use SCI without any coherency.  You
> can perform read/write/lock 1-16 bytes, or read/write
> of 64-byte entities.  None of these have anything to
> do with coherency.  The addition length of the 64-byte
> response packet (due to forwId, backId, stat.cStat) is
> 6.25%.
> Based on these facts, I would very much like to know
> what you mean with "the coherency bag".

Also, in comp.arch, David B. Gustavson (SLAC) responded:
> 1) Coherency in SCI costs you aobut 2 bits in the packet
> header, to allow for more command codes.  Otherwise, it
> costs you nothing if you don't use it.

I was trying to refer to
(a) the overhead in the node controller logic to deal with
    the command codes and the node buffers, and
(b) in the transfer protocol.
The overhead that I will elaborate below is not only due to 
cache/coherency but it is one of the major contributors, the 
others being due to the MP issues (starvation, deadlock, 
etc.)  I am responding specifically to those comments on 
SCI.  The following responses may not apply to other SCI 
derivatives.

Command Codes
The command codes are designed to do many things, 
coherency, broadcast, address range, data symbol length.  
Currently, 10 commands are reserved for cache related 
transactions.

Node Buffers
A SCI packet can hold 16-, 64- or 256-byte data.  A send 
packet has at least 16-byte overhead (address, command, 
CRC, etc.).  The minimum packet data size for each 
coherent node is 64 bytes (cache line size).  Hence the 
FIFOs and queues need to be at least 80-byte long.  If a 
non-coherent node is to work together with coherent nodes 
in a SCI topology, it's FIFOs need to be at least 80-byte 
long.  Cache-to-cache (cread00, cread64, cwrite64) and 
extended coherent memory read (mread00, mread64) 
transactions have an extended header, which adds another 
16-bytes to the FIFOs.  There may be several of these 
FIFOs and queues.  That's why SCI chips have "70K-100K 
gates, most of which are used to implement fast 2-port 
memory cells!".  (An Anet port requires only a few 
thousand gates.)

Protocol Efficiency
Yes, the "unused symbol overhead" (USO - I made this one 
up) for the coherent transaction status (stat.cStat, fid 
and bid) is 5-byte long and it takes only 6.25% for 64-
byte data transfers using a 64-byte command.  But,
	- for 16-byte non-cached data transfers using a 16-byte
	  command, the USO will be 15.63%.

The total USO can go up even more (a lot more) if there 
are "dead" data symbols.  For example,
	- for 34-byte data transfers using three 16-byte commands
	  or one 64-byte command the USO will be 30.21% or
	  43.75%, respectively,  and
	- for 2-byte data transfers using a 16-byte command the
	  USO will be 59.38%.

Note that USO does not take into account the other 
overheads associated with each packet, such as, targetId, 
command, sourceId, control and CRC (and idle symbols).  
What would be the data efficiency if all the overheads 
are considered?


*********************************************************
This is not Compaq's announcement or specification.  This is 
only an example case to point out the cost and performance 
effectiveness of Anet as an I/O interface.
*********************************************************

Assume we are designing an Ethernet NIC for a chosen I/O 
subsystem.  What will be the (IC or card) cost, if the I/O 
protocol is Anet, SCI, QuickRing, RamLink and other SCI 
derivatives?

[We can do an Anet Ethernet NIC for much cheaper than any 
existing wide-bus implementation (including ISA).  This is 
just one of the many places we can shave cost by using Anet 
and offer reasonably high performance on the interface.  
Anet card form factor can even smaller than PCMCIA if we so 
desired.]