🚀 go-pugleaf

Mark H. Wilkinson asks:
>Does anyone know the details of how Acorn managed to get RISCiX to work?
>(Apart from a couple of guys at Acorn who did the hacking!)

Hmmmm.  I'm one of those guys.  Does that mean I'm not allowed to
answer?  I will anyway... :-).

>There seem to be a number of conflicts between the architecture of the
>ARM and the assumptions that BSD makes about the machine it will run on.
>For example:
>
>* BSD prioritises interrupts and disables all interrupts below a certain
>  priority to acheive mutual exclusion (using x=spltty(); ...; splx(x);).
>  The ARM doesn't prioritise interrupts unless you start twiddling bits
>  in the IOC's interrupt mask.

The IOC interrupt masks provide a more general interrupt control
mechanism than BSD's original home architecture (the VAX, on which
interrupts are prioritised in a fixed way defined by the hardware).
It is therefore easy to implement a prioritised scheme in software,
precisely by "twiddling bits" in the masks in both IOC and the
expansion card ("podule") backplane.  RISC iX does just that.  

At any particular spl, only those devices whose priorities are defined
(by software) to be higher than that spl have their associated bits
set in the IOC IRQ(A,B) or backplane interrupt mask registers.  There
is a little cpu cost in implementing the software interrupt priority
scheme: any splxxx() function call requires execution of about 30
instructions, rather than a single in-lined special machine
instruction as happens on the VAX.  Overall though, we find this
scheme to be both efficient (though it could be tweaked further), and
very effective in maximising interrupt performance.

For example, the current RISC iX kernel can handle continuous 19,200
baud serial line input with no lost characters (assuming bulk read()
calls with no complex kernel or user-level character interpretation).
This implies a latency for the serial line interrupt of less than 0.5
milliseconds.  This is achieved even on an ARM2-powered R140 with no
hardware serial line FIFO, and with neither hardware nor software flow
control in use.  It is only possible by using a very high priority
hardware interrupt routine which just stuffs the data into a software
FIFO and ensures that a lower-priority (spltty) software interrupt
will be activated to do the main tty input processing.  Without
prioritisation we would have no hope of getting anywhere near that
performance.


>* BSD uses one stack (if I remember rightly); the kernel and interrupt
>  handlers use the user mode stack to hold temporaries and such. The ARM
>  gives the kernel, IRQ and FIQ handlers separate stack pointers
>  which, while looking like a good idea, forces the kernel to allocate
>  memory for two or more stacks rather than one.

Not quite right on BSD's behaviour: BSD kernel foreground context uses
part of the protected "u-area" pages for its stack.  The u-area does
lie next to the user stack and can be paged out with the user's own
pages, but is not part of the user stack per se.  I've forgotten how
interrupts are handled but I think this is done on on a separate
stack.

RISC iX operates in SVC mode for practically all non-user-mode
activities.  The IRQ-mode and FIQ-mode versions of R13 are not used as
stack pointers.  As for BSD, the foreground kernel-mode stack, used in
handling system calls, lies within the u-area which again is adjacent
(in logical address space) to the base of the user stack region.

Interrupts are serviced on a separate stack, but in SVC mode: on IRQ
entry, the mode is switched back to SVC mode soon after saving the
return PC/PSR held in R14_IRQ; all processing up to the return from
IRQ is then done in SVC mode.  The previous SVC-mode stack pointer
value is preserved across IRQs.

FIQs are serviced in very short low level handlers which do not use a
stack (though they may use a register dump area), and which remain in
FIQ mode; any higher priority FIQ sources are re-enabled during
servicing, but IRQs stay disabled.


>* The kernel executes from virtual (i.e. logically mapped) memory. On ARM
>  based machines this could be inefficient as the MEMC would have to be
>  reprogrammed to page the interrupt handler in (for example) and it would
>  seem more logical to place the kernel in the physical memory space.

The available 32 MBytes of logical address space (LAS) is shared
between user and kernel needs.  Page 0 is protected and controlled by
the kernel since it contains the exception and interrupt vectors.  In
RISC iX 1.2, user space (code then data at low end, stack coming down
from high end) extends from 0x00008000 through 0x017F7FFF.  The kernel
controls the per-process u-area page at address 0x017F8000, and the
kernel proper starts at 0x01800000.  Areas beyond the kernel code and
data up to 0x02000000 are used in various special ways, such as kernel
heap, alternative temporary u-area addresses, physical I/O mapping
space, sound and screen buffers, etc.

The kernel controlled parts of LAS are permanently mapped in and their
MEMC entries do not change.  Although keeping the kernel all in
physical address space might release a small amount of LAS for user
mode, we already provide more VM per process than than one can have
physical memory in the machine, so there seems little real point.  In
addition, running the kernel in physical address space (PAS) does not
save any real memory.  Having said that, we do in fact keep the file
system buffers in PAS, for two reasons: (a) to reduce ARM3 cache
impact during transfers to/from user buffers (PAS is never cached);
and (b) to remove the pressure on the kernel part of the LAS,
particularly when you want to configure lots of buffers.

For the user part of LAS, only those pages whose mapping or protection
needs to change across a context switch actually have their MEMC
entries re-programmed.  Since most processes only require a few pages
(user code/data/stack, and the u-area), this does not usually cost
much.  In addition, user process context, including MEMC mappings, is
only switched as often as it has to be; for example if a process
sleeps for a short time but no other process is or becomes runnable
until it wakes up again, no context switch will occur.  Large
processes do obviously cause more work since they have more pages
whose mapping needs to change, but overall things work quite
efficiently.


>* Is MEMC's operating system mode used? (The one that stops access to IO
>  devices while leaving the processor in SVC mode.)

No.  The UNIX execution model can make no sensible use of this mode.


>* 32k is a huge page size for a Unix system; what effect does this have on
>  the size of executables and how does the vm system cope with it?

32KB is indeed big, compared to most other UNIX machines.  In
consequence, executables are typically large (96KB normal minimum) in
terms of nominal ("ls -l") size.  However the UNIX file system
conveniently allows un-used parts of files to be stored as "holes"
which take up no space on disc.  As a result, a typical small program
without symbol information will actually occupy only 24KB on an
8KB-block filesystem (1 block of header, 1 block of code area, 1 block
of data) and only 16KB on a 4KB-block filesystem (1 block of header, 1
block of code area, 1 block of data, 1 block for the indirect block
which is unfortunately required in this case :-().  It's more than we
would really like, but it's not that painful.  At least RISC iX's
shared libraries help to keep typical image files small, and squeezing
helps reduce the size of larger programs.

The VM system copes tolerably well.  Not surprisingly, some of the
original BSD code which tried to cope with the VAX's *small* pages
(512b) is not used in RISC iX - clustering and klustering are not
particularly useful with 32KB pages!


> Anyone care to illuminate the grey areas?
>
> -Mark.

Hoping this is useful...

Mark.