🚀 go-pugleaf

In an earlier message [Subject: Re: the good old "tools vs. integrated
systems" debate] I suggested that we needed some thought on the business
of communicating between programs using ports and messages.  So here's the
way I've been thinking, anyway...  Apologies for it getting a mite long.

By the way, I forgot to mention AREXX last time, and everybody promptly set
me straight on that.  I'm not sure that AREXX is quite what I'm looking
for, though.  I'm not even sure how relevant it is to the points that I'm
concerned with at the moment.  It's hard to tell, because I haven't had a
chance to look at it yet.  All I know is what I can remember from Bill
Hawes' talk at BADGE, and from reading a REXXText.  It's obviously a very
comprehensive command language, for situations where you need that sort of
thing [but NOT a pretty language... my impressions on its structure from
the manual were definitely un-positive].

I don't think it's necessarily particularly incompatible with my concerns
here either. The thing I feel is most important is that when one program
gets a message from another -- via AREXX or wherever -- there should be a
standard way for it to know what to expect as the contents of that message
and how to handle it.  Remember that we're not just talking command lines
here: there are all sorts of things we might want to pass between programs.


I was saying before that there are several levels that we have to look at,
from the basic use of ports up to maybe very complex information protocols.
Let's start at the bottom and work up, as even the basic matter of messages
and ports seems to need some standardization.

First of all, people still get it wrong.  VideoScape-3D is an example: it
supposedly can be synchronized to an external task if it finds a couple of
public ports declared; the trouble is, both task- and reply-ports have to
be created (and therefore owned) by the external task, and so there's no
way that the Waiting V-3D can be signalled when the task replies!
Obviously if even a top notch guy like Allen can slip up (he admits he was
in a hurry...(:-)) there's need for an expansion on the good ol' RKM.  Even
the KickStart Guide skimps a bit here, I think.


Things aren't helped by some limitations in the design of the message/port
system. It's fine for a group of tasks that are well behaved towards each
other, but there are loopholes when two independent programs want to
communicate with each other.  Consider one program that has a desire to
send a message to another program's named port, IF THAT EXISTS; one
sequence you might think of is:

            ....
            his_port = FindPort("his_port");
            ....

            while (appropriate) {
                .....
                if (his_port != NULL)
                    PutMsg(his_port,mymsg);
                ....
            }
            ....

You can see that this has problems, because "his_port" could go away at any
time and the program wouldn't know about it.  Even if it did a FindPort
each time, this still isn't clobber-proof because it COULD happen that the
port went away just at that moment.  The only perfect method is to always
use the sequence:

            Forbid();
            his_port = FindPort("his_port");
                if (his_port != NULL)
                    PutMsg(his_port,mymsg);
            Permit();

This is cumbersome, and has quite a bit of overhead each time, because
FindPort has to search.  Obviously we want something a little better for a
general protocol.  I can suggest two possible paths to take.  The first is
straightforward; the second needs more work, but I can see advantages.


The simple approach is to decide on a standard protocol in which, to open
communication, the sending program does something like:

            Forbid();
            his_port = FindPort("his_port");
                if (his_port != NULL)
                    PutMsg(his_port,LOCKMSG);
            Permit();

The program owning "his_port" notes (and replies to, naturally) the
LOCKMSG, and will promise not to go away until it receives a corresponding
RELEASEMSG from the sender.  Then the sender can use the his_port handle
with impunity until it wants to end the conversation.  It would be a good
idea to also reserve a special cut-off message for the "his_port" owner to
send back if it wanted to end the conversation, to keep things symmetrical.

By implication we have two ports, "his_port" and the reply port of the
first program.  This should be all we need for two-way communication, as
long as the reply port is prepared to handle messages other than replies.
I don't see any difficulties here, and we don't want more ports around than
we need.


That's the first approach, and I can't really see any reasons not to adopt
it for situations where two programs want to talk directly to each other,
but I keep visualizing scenarios which beg for a little more flexibility.
I think I'm going to leave expanding on these scenarios for another time
(I'd like to avoid too many topics in one heading -- there's probably
already too much here, but nevah mind...). I'll just say for now that I
think they need some kind of central "message broker" process.  This could
either be a process with a single recognized message port or a resident
library.


There's one other thing to look at first, and that's the format and content
of the messages being passed around.  As I said in that earlier note, I
don't think we need anything very elaborate for a basic structure --
something like:

    struct IPCMsg {
        struct Message  ipc_Msg;
        ULONG           ipc_ID;
        /* Data follows... */
    };

ipc_ID would indicate the type of data concerned; ipc_Msg.mn_Length would
be (4 + length_of_data).  I suggest that the ID should normally be a four
character string pretty much like an IFF ID word -- in fact we should
reserve "FORM" etc. for messages that ARE IFF objects.  To cover all
situations, one should be allowed less than four characters, but always
left-justified (padding could be spaces or nulls -- I see no preference);
on the other hand if the first byte was ZERO, the ID would be a "private"
one agreed between the programs involved; maybe we should also reserve
codes 00000000-000000FF for universally agreed shorthands; two of these
could well be the LOCKMSG and RELEASEMSG above.  A couple of basic IDs that
immediately come to mind are "TEXT", for simple ASCII, and "CLI ", for DOS
command lines (to a shell). Decisions on particular IDs would be handled in
the same way as IFFs are now -- through a clearing house.

The rest of the structure should be filled in properly, too, and we should
be sure that the standard rules are followed.  (I presume -- but haven't
checked -- that PutMsg sets the ln_Type to NT_MESSAGE, and ReplyMsg sets it
to NT_REPLYMSG.  Anyone know for sure?)  The receiving program should be
able to assume that if mn_Reply is NULL it should dispose of the message's
memory when done; if it is non-zero, of course, the message MUST be
replied (possibly with data and/or ID changed).

Yeah.. well that's quite enough for now.  Nothing very exotic, but maybe
it'll be a start on a foundation we can build a flexible IPC protocol from.
I'll churn out my rationale for a message broker as soon as I can.
Meanwhile, I'll sit back and wait for the flak on this lot.

                                            -- Pete --