🚀 go-pugleaf

In article <0776.AA0776@julie> mcr@julie.UUCP (Michael Richardson) writes:
 >Someone said: (beats me, my copy of Amiga Usenet isn't up that type of thing)
 >[Reading in the message and deleting everything but the Subject: and
 >the Message-ID: line and then changing Message-ID: to References: works
 >nicely. Anyway--- ] SOMEONE SAID:
 >>>>>The shell has the responsibility of unloading (or not unloading
 >>>>>in the case of resident segments) the programs.

     ^^^^^  Randell Jesup said.

 >>>>
 >>>>But does this hold true when the shell is NOT running under the CLI?
 >>>>It will be running as a DOS Process, but NOT as a CLI process.  The

    ^^^^^  I said.  (Alternating between him and I.)

 >  I don't have any real AmigaDOS docs (just Exec docs) and I've yet to
 >completely understand just what is in the Process structure that is
 >not found in the Task structure....

 ^^^^^  You said.  :-)

You can look at the Process structure in <libraries/dosextens.h> for
starters...

 >>>>> Shells are shells, period.  The Workbench is a visual shell.
 >>>Exit() itself does almost nothing, except return control to the "shell"
 >>>as if the program had exited.
 >  But this also implies (correctly) that CLIs are NOT shells. They are
 >special. That is the very first thing that strikes people about Unix,
 >and where most if not all of its power comes from. Programs are programs.

Well...  I wouldn't say that.  The CLI *is* a shell.  A clumsy and
quite poor one, but it is.  It should be a cleaner implementation.

 >>I'm willing to use AddTask if it will still operate as a DOS process
 >>and be able to call dos.library routines.  How should I go about doing
 >>this?  Also, what makes pr_SegList bizare?  (aside from usage of
 >>BPTRs?)
 >  So am I!!!!

I've been thinking about it, and I'm not really sure I *do* want to
run as a DOS Process.  Methinks maybe as a task instead.

 >>Hmm.  Looking at "The AmigaDOS manual," (AmigaDOS V1.1, I believe;
 >>published February 1986.) it lists the Process structure starting with
 >>BPTR SegArray, describing it as an array of SegList pointers with its
 >>size in the first longword.  [seems consistent with BSTR's.]
 >  I really have to get this a copy.

It could be better, but it's definitely better than nothing.

 >>On the other hand, in the <libraries/dosextens.h> include file, the
 >>Process struct is defined with pr_task, pr_MsgPort, pr_Pad, and then
 >				 ^^^^^^^  ^^^^^^^^^^
 >  I would assume that the pr_MsgPort has something to do with receiving
 >packets back from the file system task. (For Reads and Writes)

Indeed it does.  I wasn't questioning *what* pr_MsgPort was for...

 >>Hmm.  This seems the only reasonable way to go; replace CreateProc(),
 >>and use AddTask().  I take it the extra segments in the SegArray
 >>(pr_SegList) are such as that cleanup code for CreateProc() which are
 >>not to be unloaded.  May I safely assume that if I replace
 >>CreateProc() with my own function which sets up a structure starting
 >  What stuff do you intend on adding?

Not entirely decided yet.  Probably will have some Unixy things like
maybe UID's/GID's, etc...  I'll have to think about it.

 >>output of the executed program, or set them to zero and define the
 >>standard input, output and error channels in my own structure?  (I
 >>want stderr as a passed file descriptor, along with stdin and
 >>stdout...)  I want to break dependency from the CLI, yet still be able
 >			      ^^^^^^^^^^^^^^^^^^^^^^^
 >>to run programs which depend on the CLI themselves.  (possibly by
 >>using or replacing the AmigaDOS RUN command.)
 >  I disagree. CA= won't document a lot of things because they say
 >  "it will break in 1.4"
 >  but they won't change things because
 >  "too many things would break"

I suppose there is some of that, but it IS a rather difficult position
to be in whatever they say.  Don't be so hard on them.

 >  I think the problem is:
 >    "will user x be able to run program y under your shell"
 >  (tell them to start a CLI)
 >   the problem is
 >    "will program x be able to run under the CLI or your shell"

 >  That way, you can encourage people to write programs that conform
 >to your standard (generic programs will just need a relinking with
 >a new startup) instead of letting them run all their old
 >code unmodified. If they need the old stuff, "C=+ESC" gets most
 >people a CLI... 

NO.  The idea here is to improve the shell and simplify life for the
programmer and the user.  Saying "What, you need the CLI?  Well, go
start one, then!" doesn't help at all.  I can see adding a program,
command or special syntax which would run the specified program in a
CLI-type environment, if that's what it needs, but telling the user to
f*ck off and die just because he wants to run a CLI-dependant program
is not what I'm after.  That sort of attitude is all too prevalent
already.

I believe in doing things right the first time.  (Or as best as
reasonable.)  If I'm going to do it, I want it done right.  If I
didn't care if it were done right, I wouldn't bother doing it myself.
There's plenty of programs done poorly out there.

 >>Similarly, should I zero pr_ConsoleTask and pr_FileSystemTask or not?
 >>(I intend to have read(), write(), etc. calls...  functionally similar
 >>to Lattice's, but not dependant on the Lattice compiler or lc.lib
 >>library.)
 >  Wish read(), write(), were better documented. I don't really understand
 >what they do... (internally)

I'm not worried about how AmigaDOS Read() and Write() calls work
internally.  They're written in BCPL anyhow.  (Ugh.)  I can
extrapolate well enough from the function's functionality (hmm - time
to join the redundancy department of redundancy department...) and the
data structures (i.e. the file system format on-disk) they access.  I
don't need to know exactly how it was done.  (Though I'm always
curious.)

 >>I have programmed quite a bit under Unix and rather prefer the Unix
 >>file system and system calls to many of those available on the Amiga.
 >>I don't much care for AmigaDOS or BCPL.  Some of the user interface
 >>issues between Unix and AmigaDOS are trivial, like "../" vs. "/" for
 >>parent directory.

 >>Other differences, such as the way directories are implemented, differ
 >>more significantly.  Unix-style directories are far more efficient at
 >>directory listing (but no the hash-lookup searching AmigaDOS does)
 >>than AmigaDOS with its backpointers.  Also, the lack of links in
 >>AmigaDOS is a big loss.
 >  Rah! Rah! Rah!

Gee, wow.  A supporter.  :-)

 >>I want to write a shell which will be more cleanly implemented than
 >>the CLI (i.e. no BCPL) and which will start programs with argv AND
 >>envp arrays, and have Unix-type calls available.
 >  Get out my head! That was my idea! :-)

Foo.  My idea first.  It was part of my motivation behind the whole
thing.  (I just hadn't bothered to mention it yet when I first posted
to the net for information...)

 >  Actually that would be a good initial difference for this shell.
 >Set a flag (pr_CIN and pr_COS==NULL? hmm. )

Eeew.  Yuck.  No.  I decided against that sort of approach already.
Smacks of kludginess (what a word) and will probably be incompatible
later on.  (oh, but of course.)

 >and then pass the argv and envp. The program which
 >is executed could then check for the flag and (if not set) do the "standard"
 >startup or just use the argv and envp. (Or things can be linked
 >with a startup that doesn't have this. Programmer choice)

Questionable.  I'd rather not introduce such complications into the
program startup; it goes against everything I'm trying to accomplish
here.  I'm looking for a cleaner implementation of both the shell and
generally for running programs as well.  Also, keeping the startup
simple and whatnot will help keep program executable sizes small,
which IS important to me.  Besides, directly supporting the CLI
environment for new programs like that not only encourages the
kludginess already extant, but it also discourages people actually
making full use of the setup I'm building.  Not a help.

 >  How to pass argv and envp? This is a message passing operating
 >system right? Well, send the task a message... How do you tell when
 >the task is finished? Well, exit() returns this message (with an error
 >code) and then does a Wait(). The exec.library (woops that name
 >is taken. The Execute.library then--- I guess I mean Execute.device see
 >below) then RemTasks it, and deallocates it (if it isn't on the resident
 >list) and makes the error code available to the task.

I may well use message passing internally to the functions, but I do
NOT want to force the programs to deal with message passing just to
run.  Workbench does.  I don't want to.  I'll keep it internal; that's
fine.  I'm trying to make things EASIER here.

 >  What else? How about a standard tc_ExeceptionCode?
 >  Make SIGBREAKF_CTRL_C, and ^E exceptions. ^C will
 >cause an "abort()" routine to be run. (signal() or sigvec()'able of course)
 >The standard abort() should be able to run some number of clean up
 >functions, and then it should call "exit()"

I'm not so sure about that.  I've considered such an idea.  But, I
don't want to tie this to any compiler, first off.  (abort() and
exit() ARE compiler-dependant, though ought to be fairly consistent
across compilers.)  And I don't want to use Exit(), certainly.  (And I
CAN'T if the task isn't a dos process.)  But there's no telling what
you might be interrupting with it.  And what cleanup code might be
missed...  Needs more thought.

 >  Also, when ^E is received, the task does a "Wait(SIGBREAKF_CTRL_F)"
 >and PRESTO, Job control!!! (Or course, ^E is signal()/sigvec() settable too.)
 >When the shell wishes the job to continue, just send a ^F signal. (Or
 >any other agreed upon signal)

aCk.  I don't want to use ^E and ^F regardless; if I implement any
sort of job control, I'll try to use ^Z if I can.  I'm not sure what
the way to go is...

 >  This assumes something about the shell though: that it isn't part of the
 >same task as the programs it runs. This is fine, as this CLI "subroutine"
 >stuff bugs me a lot.

Ugh.  Wasn't it clear from the start that the programs would be
separate tasks from the shell?  Perhaps not.  Well, they most
certainly would be separate tasks.  I hate the "subroutine" structure
of the CLI; it makes it seem like you've got a bunch of virtual MS-DOS
(or some such simgle-tasking system) machines in one instead of a true
multitasking system.  Everyone's answer to wanting to run multiple
programs at once is "That's easy!  Just start a new CLI!"  Can you say
screen and system clutter?  It's also clumsy as all hell, and totally
unnecessary to boot.

And WHY do I get this sinking feeling that about half the Amiga
programmers out there (i.e. the mediocre ones) pulled their
programming practices straight from the C-64??  Argh.  Yeah, I know.
They probably DID.  *sigh*

 >  It ISN'T fork()/exec(), but it does make a much more orthogonal
 >system, shells send messages to "execute" servers, much the same
 >way that comm programs send messages to "serial" devices...

What you're describing is basically Workbench in text form.

 >>What I would like to do is write a file system based in part on Minix,
 >>and in part on AmigaDOS (but *no* BCPL!) which would work more
 >>effectively with floppy-based systems.  (i.e. Mount file systems like
 >>in Minix/Unix, but volume-oriented, as in AmigaDOS.)
 >  Have you talked to Colin (Plumb?) (microsoft!w-colinp)
 >  His extent based file system looks quite nice...

No, but I have noted his postings about his file system with some
interest.  It's a bit limited, however, being tied to floppies.  But
it could work well.  Myself, I'd just assume stick with a
block-oriented format, as it is much more portable.  (like, to hard
drives and file servers and whatever.)

 >  I suggest that you start by writting a file system handler that
 >understands the AmigaDOS file system in C.  Then rewrite it
 >(with a different file system type) that is completely C. (Get
 >rid of BSTR, etc... Incompatible, but that's ok.)

What I will do is write a file system handler in C which will interface
with AmigaDOS behind the scenes, handling the conversions for BPTRs
and BSTRs so as to provide a clean and consistent interface for
programs to use.  I *could* write a handler which directly accesses
AmigaDOS disks, but then I'd still have to deal with BCPL crap and I
would merely be tying myself to a specific version of AmigaDOS.  I
couldn't use it for FFS, for example.  No, just as well to have
AmigaDOS play with its disks itself and coordinate that instead.

 >>I DO rather like the low level Exec.  The ROM Kernal is quite well
 >>designed.  I consider AmigaDOS to be rather poor, on the other hand.
 >>It DOES contain some rather clever design features; assigned devices
 >>are useful (though not good as a general replacement for environment
 >>variables) and being able to mount devices is a definite plus.
 >  Unmount is need though.

It would be good to have, for consistency, but I don't see it as a
particular problem; devices which are mounted and unused don't really
use much in the way of resources.  And it shouldn't be too hard to
write an unmount; Forbid() and dig around in the AmigaDOS structures
and remove the entries mount put in.  But it's not a nice thing to do.
:-)

 >>But what I DO want now is the fork() and exec() system calls from
 >>Unix.  (execve() for purists)  And I'm willing to take a stab at
 >>writing them myself.
 >  Why? In a way I rather like the idea of calling CreateTask with
 >an entry point. I think that the underlying mechanism could be
 >changed to support a more fork() like system, but in how many
 >cases is fork() not followed immediately by an exec()?

Many cases.  Simplest example, to implement redirection.

 >  In this case, the vfork() call was invented, and also the
 >"copy-on-write" MMU stuff (although that is very usefull for
 >shared libraries too)

vfork() is more efficient than fork() when *soon* followed by an
exec().  a fork() *immediately* followed by an exec() is functionally
identical to a task spawning function like CreateProc.  The point is,
*between* the fork() and exec(), you can set up the environment for
the new process in the context of that new process.  It is a valuable
ability.

 >>Fork() would start with a FindTask(0L) call, allocate space for a new
 >>Task structure, copy the data over, and similarly duplicate the text
 >>and data segments, modifying the appropriate fields to point to the
 >				   ^^^^^^^^^^^
 >>copies, and link in the new task to Exec with either AddTask()
 >>(obviously preferable) or manually if necessary.
 >  As has been pointed out, this is VERY difficult without an MMU.

Not difficult, just slow and sorta kludgy.  As Randell said, it
involves swapping the stack and data segments around, at every task
switch.  (of the forked task, that is.)  If it is *soon* followed by
an exec(), everything will be dandy.

 >>void AddTask(struct Task *, char *, char *);
 >		task         initialPC, finalPC

Yeah, I looked it up after I posted.

 >>Anyhow, I see several points of possible difficulty.
 >  So do I.

 >>First, is whether this fork() routine would need to disable
 >>multitasking (interrupts seem fine) at all, such as when copying the
 >  I don't think so.

Nor do I.

 >>Task structure of the current process.  Or, is it perfectly safe and
 >>consistent (no race conditions) to let multitasking continue
 >>unhindered while it initializes the new Task structure?  (Clearly, it
 >		       ^^^^^^^^^^^
 >  Nobody but you knows about the task structure until you AddTask.
 >And at that point, you had better be finished fiddling with it, because
 >the process is ready to run.

Indeed.  I don't think it's a problem, really.

 >>Second, how should the fork() call make execution begin at the return
 >>point of said fork() call?  (Perhaps this question will answer itself
 >>when I look up AddTask().)
 >  Look up the return address of the fork(). Put the return value under
 >it, and give the address of a routine that will just do an "RTS".
 >This stack is, of course a freshly copied stack, having all
 >the frame pointers, changed. Even if you leave all the data and
 >code shared, you must at least duplicate the stack.

Must duplicate all, really.  I don't know whether or not it's really
worth implementing vfork() also.  Maybe.  As for getting the return
address, a JSR to a point within the function code coupled with a JMP
seems cleanest.

 >>Finally, (I hope) how to handle file descriptors.  The solution for
 >>this would seem to be to have file i/o operations in this library -
 >>open(), close(), read(), write(), etc. and have a table of file
 >>descriptors, file pointers, and AmigaDOS file handles (for now, at
 >>least.) that the "parent" and "child" share.  I don't know whether or
 >>not to try to preserve the parent/child relationship as Unix does, or
 >>try some other setup.  I'll have to think about it.

 >  The fexec() library function would lookup the highest FileHandle assigned
 >to ask (kept in an Exec list of course to eliminate that 20 files open bit)
 >and duplicate each one. This list would be part of the standard stuff
 >passed to the execute.device (along with argv, envp and the name of the
 >program and/or address at which to execute)

WTF?  No.  The list of file descriptors is NOT to be duplicated.  It
is to be shared.  (As it is under Unix.)

 >  Parent/child is preserved without being forced.

How is it preserved, pray tell?

 >  (How do you do redirection? Well, I guess one has to write
 >a custom fexec() as one couldn't go around changing the your own
 >FileHandles, there being no seperation like fork().  --- We don't need
 >to duplicate a running process structure, just get a new
 >"OSInterface" structure. Modify that, and pass it to the execute.device.

???  Why?

 >Actually, that could be interesting:
 >OpenDevice("Execute.device","ls",&lsInterface,0) and then modify the
 >lsInterface (instance of OSInterface) to change the environment, etc...
 >and then do a "DoIO()" on it with a CMD_START (or Send/BeginIO() for
 >asynchronous processes...))

It is NOT going to be implemented as a "device".  End of subject.  It
is NOT a device, nor does it pretend to be a device in any way, shape
or form.  And it is NOT going to use OpenDevice and DoIO to start a
new process.  Do you have ANY clue how f*cking complicated Exec device
I/O IS to anyone new to the Amiga???  You have to do quite a bit to
get something simple done.  Fine if you already know it, but a supreme
bitch to deal with the first 2 dozen times around.  This does not NEED
to be implemented as a device, not SHOULD it.  It won't be.
Externally OR internally.  It WILL be a Library, though whether
link-time or run-time (as in OpenLibrary, that is) I don't yet know.

 >[lots of stuff on exec() deleted]
 >>How to handle resource deallocation of the old process presents yet
 >>another difficulty.  (oh, but of course!)

 >>allocated by the prior process need to be released.  One solution (the
 >>cheap and easy way out) is to free malloc() allocated memory (if I
 >>write a malloc(), etc. set of routines) and leave other resources
 >>allocated, (such as memory gained from AllocMem) to allow resources to
 >>be passed to the called program.
 >  This makes sense to me. If you need to pass the resource, you
 >are already Amiga specific, use the amiga specific call...

Basically.

 >>initialization program ("Unix" bootstrap program) which would install
 >>the library in ram, start an "init" process, and then hang around as
 >>the "system task" to coordinate everything.

 >  I called it "Execute.device"

Wrong.  Like I said, it will NOT be any "Execute.device".  I
understand the logic behind it, but it just isn't the way to do it.

 >>Someone... anyone...  please reply.  I wouldn't want this to be a
 >>wasted effort.  (And more information/ideas is always helpful.)
 >>
 >>Enough for now.
 >  I was going to send this as a reply (and some people may wish that
 >I had) but I changed my mind...

It's ok by me; been having trouble with mailers to reply, and there
may well be lurkers which are actually interested in this trash.  :-)

And those that don't care are welcome to hit 'n' or 'k'.

Oh, and I meant I wanted someone, anyone to reply...  either to me
directly, OR to the net...

Deven
--
------- shadow@pawl.rpi.edu ------- Deven Thomas Corzine ---------------------
Cogito  shadow@acm.rpi.edu          2346 15th Street            Pi-Rho America
ergo    userfxb6@rpitsmts.bitnet    Troy, NY 12180-2306         (518) 272-5847
sum...     In the immortal words of Socrates:  "I drank what?"     ...I think.