🚀 go-pugleaf

New concepts on Multibus II

Klaus Eimermacher, ATM Computer GmbH

1. Introduction

Approximately 10 years ago INTEL introduced a complete new bus concept -
the MULTIBUS II system. It consists of 4 different bus systems, which can
be implemented in one environment:

- local bus extension (LBX II)
- parallel system bus (PSB II)
- serial system bus   (SSB)
- single board extension (SBX)

The most commonly used bus of Multibus II is the parallel system bus
(PSB II). This bus is optimized for communication between intelligent
bus agents and has numerous LAN - like features.

This backplane-LAN is commonly called a "LAN in a box".

Today, boards are usually connected to Multibus II using the Message
Passing Coprocessor (MPC), which is available from INTEL and VLSI.
This ensures that boards from different vendors have the same hardware-
and firmware - interface to the bus. Therefore, no compatibility problems
arise when connecting to Multibus II. The Message Passing Coprocessor needs
a DMA controller for Multibus II-transfers. The DMA controller moves data
from memory to MPC or vice versa (see ff).

Currently the Multibus Manufacturers Group (MMG) is extending the Multibus II
specification from 40 MBytes to provide a bus bandwidth of 80 Mbyte/sec and
"hot board insertion". These new features have already been shown in demo
systems. The work on these new technologies has shown that new concepts are
needed in the future in order to take advantage of these new Multibus II
features.

One reason for this is that the currently available DMA controllers have
reached their performance limits, thus limiting board performance. Another
disadvantage of current board designs is the high percentage of CPU
performance needed to use the on-board I/O.

ATM Computer GmbH now has solved both of these problems on the new MBS/433-ST 
board by using additional new RISC processors on a 486 single board computer.
The result is a design which achieves higher I/O-rates and higher Multibus II
data rates than can be achieved by standard CPU/DMA designs.

2. Message Passing on Multibus II

Multibus II supports, besides the standard address ranges "memory" and "I/O",
2 other address ranges, which are not utilised in traditional bus systems:

1) the geographic addressing of agents and of configuration registers on
these agents, the so called interconnect address range.

The interconnect address range consists of up to 512 control- or configuration
registers per bus agent.  These registers can be defined to be accessible
"globally" - that is, they can be accessed by any busagent - or "locally".
The definition of the interconnect registers and of valid values can be found
in Intel's "interconnect specification". It is accepted world-wide as a
defacto Multibus II standard.  The registers allow self recognition of system
configurations by software and - by reprogramming the interconnect registers
with different parameters - other configurations can very easily be programmed
into a board.

2) the Message address range.

The message address range" supports 2 different types of messages:

- the unexpected or unsolicited message, and
- the expected or solicited message.

The unsolicited message is used as a high speed interrupt message and used
to transport small amounts of data. The unsolicited message is also used for
negotiation of a long data transfer - the solicited message. The solicited
message supports data transfers of up to 16 Mbytes of data. The data is packed
and unpacked using a special "packet protocol". This is all done by the MPC
during send or receive operations.

A message consists of a header, which has to be compliant with the Multibus II
"Message Transport Protocol", and of user data.

Error conditions, such as parity error or timeout, are automatically handled
by the MPC without involving the CPU.

2.1. "Traditional" board designs

Most of the available Multibus II boards are designed as follows:

	 +---------+      +---------+  +-----------+
	 | Memory  |      |   CPU   |  | (A)DMA-   |
	 +----+----+      +----+----+  | Controller|
	      |                |       +-----+-----+
	      |                |             |
       -------+----------------+-------------+--------------
			  local CPU bus
       --------+--------------------------------------------
	       |
	+------+------+        +---------------+
	| Message -   |        | Interconnect- |
	| Passing -   +--------+ Controller    |
	| Coprocessor |        | 87xx          |
	+------+------+        +---------------+
	       |
    -----------+-------------
       Multibus II (PSB II)
    -------------------------

The local bus is shared by the CPU and a DMA controller. The main memory
which holds data and messages is also connected to the local bus. To send
a message to another PSB bus agent, the CPU has to assemble the message in
memory, to program its DMA controller, and to program the MPC. The CPU then
writes an unsolicited message with type "buffer request" into the MPC registers
which is sent by the MPC to an MPC on a receiving agent.  After receiving
the "buffer grant" message, the data transfer is automatically performed by
the MPC and DMA controller, without involving the CPU on the transmitting or
receiving agents.

If the transfer was successful, a completeness-interrupt is signalled to the
CPU. Error conditions are signalled immediately, to give the CPU the chance
to handle the error condition. Some kinds of error conditions are handled
independently by the MPC using retry mechanisms, e.g. PSB parity error.

Most of today's Multibus II boards use the 82258(A) ADMA controller, which is
manufactured by SIEMENS and INTEL. This controller has a 16 bit bus interface
and is available in CMOS up to 20 MHz. Older board designs use the 10 or 12 MHz NMOS version.
With a special board design, the controller can perform 32 bit bus cycles
from memory to MPC or vice versa, so called "flyby cycles". This works only
if data is 32 bit aligned. Otherwise, the DMA controller has to do two 16 bit
cycles, the "two cycle mode".

The maximum address range for the ADMA without additional on-board circuitry
is 16 Mbytes. For operating systems, this limits the accessible memory for
message transfers to 16 MB on most of the boards.

The software protocol for Multibus II message passing and programming of the
DMA and MPC are all handled by software drivers, which are run by the CPU.
To do this, the CPU has to service a lot of MPC and DMA interrupts. The task
switch overhead involved - saving the old system state, creating a new
environment, servicing the interrupt, restoring the old system state consumes
a significant amount of the available CPU performance, thus unnecessarily
slowing down application programs and important tasks inside the operating
system.  Although the message passing protocol is the most important example
for the drawbacks of current designs, the above statements hold true for any
kind of I/O handled by the main CPU.

3. A New Concept

To achieve higher throughput, the CPU has obviously to be freed from most of
the overhead involved in handling message passing and I/O in general. To
achieve this, ATM Computer has implemented the following concept:

- use of an additional, independent RISC - I/O processor, which is running
  the complete Multibus II transport software.

- decoupling of this RISC processor from the CPU memory bus and implementation
  of an independent I/O bus (two bus system).

- separate code memory for the I/O processor on the I/O bus

- use of a high performance 32 bit DMA controller

This results in a configuration as follows:

   +---------+      +---------+
   | Memory  |      |486 CPU  |
   +----+----+      +----+----+
	|                |
	|                |
 -------+----------------+----------------------------
	  local CPU bus
 ----------+------------------------------------------
	   |
 +---------+--------+           +---------------+
 |   Busarbiter     |           |    Memory     |
 +---------+--------+           +-------+-------+
	   |                            |
 ----------+----------------------------+-------------
		   E/A Bus
 --------+--------------------------------------+-----
	 |                                      |
  +------+------+  +---------------+     +------+---------+
  | Message -   |  | Interconnect- |     | 80960 CA       |
  | Passing -   +--+ Controller    |     | 33 Mhz RISC -  |
  | Coprocessor |  | 87xx          |     | Processor/DMA  |
  +------+------+  +---------------+     +----------------+
	 |
 --------+-------------
  Multibus II (PSB II)
 ----------------------

(simplified block diagram of MBS/433-ST)

4. Implementation

For the I/O processor, ATM Computers chose INTEL's high performance 80960 CA
RISC processor.  It not only gives the necessary CPU power to run message
passing and I/O, but also solves the problem of implementing a very high
performance 32-bit DMA controller. The chip has an integrated 4 channel 32
bit DMA controller with data rates up to 53 Mbyte/sec.

256 KB of SRAM memory are connected to the I/O bus, this memory holds the
programs and data of the 960 CPU.  With this hardware configuration, it is
now possible to run the complete Multibus II Transport protocol software
independently on the RISC I/O processor, without influencing the main CPU
(486). The main CPU gets only one single interrupt, either after successful
completion of a full message transfer, or if an unresolvable error has been
detected. Most of the minor errors or special conditions can be resolved by
the transport software run by the 960, with only a few exceptions, e.g. a
"receiver not ready" condition.

Communication between the 486 main CPU and the 960 I/O CPU is implemented
using a 32 bit communication register and queues in memory. The objective
was to implement a very fast interface.
Commands to do message transfers or I/O are given to the 960 I/O processor
via a command queue.  The 960 reads and executes the commands, and writes a
completion status in a status queue. Then, the 486 is interrupted. The format
of the queues is an enhanced version of INTELs "Shared Memory Interface
specification".

The communication register is used during initialization to pass information
to the 960, e.g. the start address of the 960 initialization code. After
power up, the 486 CPU downloads the 960 I/O program code into the 960 memory.
The programs are either stored in EPROM or on disc.

A bus arbiter manages the bus cycles from the 486 bus to the I/O bus and vice
versa. The bus priority can be fixed or "round robin". Due to the instruction
caches inside both processors, normally those accesses do not collide with the
program execution on the processors.

The software on the 960CA RISC processor is based on the realtime kernel iRMK
960. Message passing is implemented as tasks within iRMK 960. The software can
easily be extended with additional I/O tasks.

ATM Computer GmbH sells a configured ready to use version of the 960 software
including full message passing transport software in combination with its real-
time operating system REAL/IX.

4.1 Features of the single board computer MBS/433-ST

In addition to the above mentioned I/O-processor, the MBS/433-ST board offers
the following I/O interfaces:

4.1.1 I/O connected to the 486 CPU bus

- two channel RS232 (V24) serial controller using Z85230 (20MHz)
- two 8259A compatible interrupt controllers with up to 15 interrupt sources
- one 8254 compatible timer with three 16 bit timers
- one interrupt multiplexer, which allows a flexible interrupt configuration
  without jumpering or wrapping.
- an 8 KByte non volatile data RAM

4.1.2 I/O connected to the 960 I/O bus

The MBS/433-ST is designed to run operating systems which use high SCSI
performance, such as REAL/IX. Therefore, the 2 MIPS SCSI controller 53C710
(33 MHz) is connected to the I/O bus. This device has a RISC CPU kernel and
allows for very flexible control of SCSI devices by executing "scripts".
These scripts are written in a special programming language tailored to the
requirements and the special behaviour of the SCSI bus. The scripts have to
be stored in the SRAM connected to the I/O bus. Thus, code fetches by the
SCSI controller do not interfere with the main CPU.

The controller supports data rates up to 10 Mbytes/sec on the SCSI bus in
synchronous mode. The 53C710 has a timer designed to detect timeout conditions
on the SCSI bus of greater than 250ms.  For a good realtime system, this
interval is too long. To meet the realtime system requirements, an additional
8254 timer was connected to program shorter SCSI timeout intervals. Normal SCSI
devices respond 8-10ms after a command. Shorter timeouts avoid long bus locks
on the SCSI-bus.  Additionally the 8254 supplies the timer to interrupt for
iRMK960.

In order to connect additional I/O devices, the board offers two extension
interfaces:

a) a 32 bit interface compliant to OME Version 2 on connector P2

The OME 2 interface is connected via the backplane, compliant with the OME
specification version 2 using a 128 pin DIN connector on P2. The backplane is
available with 3, 4, and 5 slots.  This is more suitable for Industrial
requirements than the "piggy-back" interface which was defined in OME version 1.Connecting via P2 on the backplane brings many
advantages, in particular easier
board handling during maintenance of a system.

ATM Computer offers a broad range of extension modules on OME 2.

b) a serial fiber optic interface with currently 50 Mbit/s net data rate
  (increasable to 125 Mbit/s)

The serial fiber optic interface is based on the AMD TAXI technology. The
interface has 1 Kbyte transmit- and receive-FIFOs on each side and can be
operated with interrupts. The net data rate of 50Mbit/s is sufficient to
connect I/O subsystems, or to directly link two boards.  Access to the
interface is via standard ST connectors.

5. Future Multibus II enhancements

The technical committee of the Multibus Manufacturers Group (MMG) is currently
working on a specification for Multibus II performance enhancement. The proposal
will set the maximum bus burst data rate to 80 Mbyte/sec. Technologically, this
is no big deal, because new faster technologies for MPC chips, and faster ABT
bus drivers can be used.  These components are available now, and therefore it
is easy to double the bus clock today.

With "traditional" board concepts, the Multibus II performance enhancement will
half the packet transfer time on the bus, but will not change the overall
system performance.  The potential bottleneck are the slow DMA controllers,
which limit data throughput. ATMs new concept  using the 960 RISC CPU/DMA will
be able to take advantage of the increased Multibus II performance.

Another MMG subcommittee is working on the specification of "hot board
insertion", that is disconnecting boards from and connecting boards to the PSB
with "power on". To make this work, electrical problems have to be solved, as
well as problems like "booting and self-testing replaced boards".  The main
goal is to be able to use currently available boards without any, or with only
minor changes.

A demo system was shown at the BUSCON WEST in February 1992 at Long Beach.
This system included a special backplane which could run with 20 MHz, and
which also supported "hot board insertion". Currently, the MMG plans to submit
these two extensions to the Multibus II specification, and to obtain the
approval of IEEE, in 1993.

6. Conclusion

Enhancements in chip technology today enable the performance of Multibus II
to be doubled. To take advantage of the performance of this "new" Multibus II,
new concepts in board development and board design are necessary. Only by
using new processor technology based on RISC, in combination with standard
CISC processors like the 80x86 family and their operating systems, and by
implementing advanced on-board bus concepts, can the system profit by the
doubled bus speed.

The "hot board insertion" feature will extend the range of Multibus II
applications in the industrial, and communications market.

ATM's MBS/433-ST and Concurrent Technologies' CP 486/960 single board
computers are the first high performance Multibus II CPUs to designed to take
advantage of the new Multibus II features!

used abbreviations:

OME   -  On Board Module Extension
CPU   -  Central Processing Unit
MMG   -  Multibus Manufacturers Group
I/O   -  Input / Output
RISC  -  Reduced Instruction Set CPU
CISC  -  Complex Instruction Set CPU
MPC   -  Message Passing Coprocessor
FIFO  -  First-in, First-out
DMA   -  Direct Memory Access

Trademarks:

iRMK960  - Fa. INTEL
UNIX     - Fa. AT&T
REAL/IX  - Fa. Modcomp / ATM
TAXI    - Applied Micro Devices

-----------
If you have any questions regarding the new Multibus II concepts or
regarding the boards please do not heasitate to contact me!

---

------------------------------------------------------------------------snip-
  Klaus Eimermacher    | ATM Computer GmbH | Email: macher@atmkn.uucp  
   _    _____  _    _  |     Abt. E53      |        (..!unido!atmkn!macher)  
 /   \ |_   _||  \/  | | Buecklestr. 1 - 5 | Voice: 07531/807-332      
| AEG |  | |  |      | |  W-7750 Konstanz  | Fax:   07531/807-280
|_| |_|  |_|  |_|  |_| |     Germany       | Ampr.net: 44.130.55.1
                       |                   |           dl1gek.ampr.org 
-----------------------------------------------------------------------------