🚀 go-pugleaf

Quoting sdm7g@aemsun.med.Virginia.EDU (Steven D. Majewski) from comp.lang.perl:
>What is the format of a unix mailbox file and where can the 
>definition be found. ( I didn't see any pointers in the man
>page for 'mail'. )

The format of the *unix* mail box (i.e. the format in which the mail
delivery subsystem stores the mail in the system mail box) is really
simple:

It starts with a From line describing the sender of the message (idented
for clarity only):

	From <internet_address> <date string> [remote from <site>]

For instance:

	From ram@lyon.eiffel.com  Wed Aug 26 16:02:12 1992

The "remote from" part is used by UUCP and apparently by SYSV R4 mail.
It is not recognized by old mail programs, I think.

Then the RFC-822 mail follows. It is composed of a set of headers, a
blank line (single newline) which marks the end of header, and then the
body of the mail itself. Note that the end of the message is marked by the
end of the file, or the beginning of another message. This is why it is
important to escape the leading "From" lines in the body of a message, or
some simple mail handling tools are confused. Usually, appending a '>' as in
>From is sufficient.

>How is the status of new, read, etc. indicated in the file? 

This is a mail user-agent convention, and it may vary. But traditional
/usr/bin/mailx (aka /usr/ucb/Mail) simply add a Status: line in the header
of the message.

>BTW: the reason for the question is that I need to write a filter
>to split/merge and generally reorganize my mailboxes. If anyone has
>any hints on a good program to do this, I'm listening. And that's why
>I've tacked 'comp.lang.perl' onto the Newsgroups line. I would expect
>someone must have done something like this in Perl, already. 

Sorry, I only have a perl program to sort message *before* they arrive into
your mailbox. For post-processing, I use MH, which stores messages in
individual files -- makes life *much* easier as you may use traditional
shell scripts to process your mail and don't even have to use perl! :-)

Good luck!
-- 
Raphael Manfredi <ram@eiffel.com>
Interactive Software Engineering Inc.
270 Storke Road, Suite #7                      / Tel +1 (805) 685-1006 \
Goleta, California 93117, USA                  \ Fax +1 (805) 685-6869 /

*** Old business: ***

Many thanks to Raphael Manfredi <ram@eiffel.com> ( who posted a follow-up ) 
and to the many other folks who mailed me answers to my question. 

I had wondered if the null chars ( 0x00 ) I had observed in the mailbox
file were functional ( and necessary ) separators. The consensus seems
to agree with  arensb@kong.gsfc.nasa.gov (Andrew Arensburger - RMS):

>	As far as I can tell, they are produced by broken mail software
>and grudgingly tolerated by sendmail/SMTP. Or at any rate (9600 bps? :-) ),
>when a cron script sends bogus junk to mail, I usually get a complaint
>from daemon saying
>	Mail: Ignoring NULL characters in mail.


I also got numerous suggestions for programs to filter/pre-process mail, 
including procmail, filter ( in elm ), mh, etc. ( And investigating this
is one of the things that prompted my other questions about POP vs. IMAP: 
If I filter my incoming mail on my unix host, then I can't access that 
mail with POP - which can only access the default mailbox ( usually 
/var/spool/mail/_user_ ). - Someone else has already posted a summary
of POP/IMAP/PCMAIL clients and servers in another thread. I will post
a followup when I have something more to add. ) 


*** New Business: *** 

I note that elm, besides a Status of New,Read, etc., also has a 
"permanent" status of Urgent, Form-letter, etc. 

Is this also encoded in the "Status:" header field? 

Is it "legal" or "well-behaved" to SEND a "Status:" line in a header,
or is this always inserted by the UA software ? 

i.e. Am I allowed to send a message with "Status: Urgent" and does 
any software besides 'elm' know what to do with that field? ) 

Are there recognized values across different mailers for that Status field? 
( and what are they? ) 


======== "If you have a hammer, find a nail" - George Bush,'91  =========
 Steven D. Majewski		University of Virginia Physiology Dept.
 sdm7g@Virginia.EDU		Box 449 Health Sciences Center
 Voice: (804)-982-0831/32	1300 Jefferson Park Avenue
 FAX:   (804)-982-1616		Charlottesville, VA 22908

In an article, sdm7g@aemsun.med.Virginia.EDU (Steven D. Majewski) writes:
>What is the format of a unix mailbox file and where can the 
>definition be found. ( I didn't see any pointers in the man
>page for 'mail'. )

Which mail system are you using?  There is the V7 mail format that
has "From " as the first line in the message header; the header and
body is plain text.  There is also MMDF which inserts four CTRL-A's
before and after each message, and there is MH in which each message
is a file, and the folder is a directory (this makes the most sense,
and is more in line with the UNIX philosophy, IMHO.)

The status of the message is determined by the 'Status:' line in
the message header.

-- 
Mike Stefanik  mike@pacsoft.com  ...!uunet!pacsoft!mike
Pacific Software Group, Riverside, CA

In article <1992Aug26.203046.25591@murdoch.acc.Virginia.EDU>, sdm7g@aemsun.med.Virginia.EDU (Steven D. Majewski) writes:
> 
> What is the format of a unix mailbox file and where can the 
> definition be found. ( I didn't see any pointers in the man
> page for 'mail'. )

To my knowledge, it isn't documented anywhere. This is is part because
it  depends partly on what flavor of Unix mailer you have.  The "pure"
uucp mailer (which isn't so very pure on most systems these  days  ;-)
had a very simple format back in the old days. A mail file was started
either by the start of the file or by the sequence:
   "\n\nFrom "
Note the two line feeds before the "From", and the SPACE after it. But
as  I've  noted,  there are some variant mailers that do clever things
that shoot down attempts to match on this simple string.  The most fun
are the ones that add their own headers before an existing "\n\nFrom "
byte sequence.

> How is the status of new, read, etc. indicated in the file? 

What does that mean?  You are clearly talking about some mailer  other
than the usual ones that come with Unix.  The basic Unix mailers don't
add anything but a "From " line to the start of  a  message.   But  of
course lots of add-on mail packages do all sorts of clever things with
their headers.  None of this is very portable.  Well, if you  are  the
sort  that  thinks  that the only mailers that God wants us to use are
the brand X mailers (where X mostly == Internet  in  the  BSD  world),
then  you  probably think that some particular headers are "standard".
You are in for some rude surprises when the Real World starts to  send
you mail.

> BTW: the reason for the question is that I need to write a filter
> to split/merge and generally reorganize my mailboxes. If anyone has
> any hints on a good program to do this, I'm listening. And that's why
> I've tacked 'comp.lang.perl' onto the Newsgroups line. I would expect
> someone must have done something like this in Perl, already. 

Yup; I've been doing just that. I have a little perl program that does
the  useful  task of chopping up the mailbox into mail files (and only
occasionally mistakes one message for N when some turkey  mailer  adds
headers  with  embedded blank lines followed by a "From" line ;-).  It
looks for interesting (to me) subjects and/or senders, and  does  some
appropriate  (for  me)  things when it gets a match, running vi on all
the files that don't match.  I also have a little perl program that  I
feed the headers via "!}rpl" and it generates headers for a reply. And
so on.  It's fun; everyone should try it.  You rapidly get a real feel
for how truly demented the various mailers out there can be.

The most frustrating thing about figuring out how to chop up a mailbox
file is knowing that the mail arrives in separate files, just like you
want it.  The idiots who wrote the mail packages decided to join these
files  into  one big one, eating up lots of cpu time to do so, just so
that you and I would have to write programs to try to reconstruct  the
original  separate files, eating up yet more cpu time.  If they'd just
made the "mailbox" a directory rather than a  file,  they  could  have
delivered  the mail via a link, and no chopping would be necessary.  I
keep getting tempted to write a new mailer that does it this way ...

BTW, when it reached here, it didn't have comp.lang.perl anywhere.  So
maybe you should repost it there ...

> ======== "If you have a hammer, find a nail" - George Bush,'91  =========

The nail that sticks up gets hammered down.
   -- Old Japanese saying.
-- 
All opinions Copyright (c) 1992 by John Chambers. Inquire for licensing at:
1-617-647-1813 ...!{bu.edu,harvard.edu,eddie.mit.edu,ruby.ora.com}!minya!jc 
--
Pensu tutmonde; agu loke.

In article <1350@minya.UUCP> jc@minya.UUCP (John Chambers) writes:
>In article <1992Aug26.203046.25591@murdoch.acc.Virginia.EDU>, sdm7g@aemsun.med.Virginia.EDU (Steven D. Majewski) writes:
>> What is the format of a unix mailbox file and where can the 

>had a very simple format back in the old days. A mail file was started
>either by the start of the file or by the sequence:
>   "\n\nFrom "

The ones I see only have one \n. The last mail message ends on the line
before the ^From.

>> How is the status of new, read, etc. indicated in the file? 
>
>What does that mean?  You are clearly talking about some mailer  other
>than the usual ones that come with Unix.  The basic Unix mailers don't
>add anything but a "From " line to the start of  a  message.   

No, he is thinking about the usual mail user agent. The MTA generally
adds only the Received-From headers: the From header is already in the
mail. The MUA manages the status of the messages by adding the Status:
line, which contains things like O for old and R for read. "Pine" uses
the X-Status: line.

To a normal system user, there is no distinction between MTA and MUA --
they are both part of "mail".

>The most frustrating thing about figuring out how to chop up a mailbox
>file is knowing that the mail arrives in separate files, just like you
>want it.  

Mail arrives as input on STDIN, which can be read using the same calls
as reading a file but isn't the same thing. For one thing, you can't
rewind.

>The idiots who wrote the mail packages decided to join these
>files  into  one big one, eating up lots of cpu time to do so, just so
>that you and I would have to write programs to try to reconstruct  the
>original  separate files, eating up yet more cpu time.  

Of course, it is too much effort to use the documented features that
those "idiots" put into the mail system so that you can handle the
incoming mail as separate files. It is SO much easier to flame them.
Why not just take the mail as "separate files" as it comes in, writing
your own file locking code to handle simultaneous arrivals, and leave
the gratuitous insults at home? 

Apparently you are interested in taking incoming mail and placing it
in separate files so you can manipulate it there. MH already does
that.  The 'inc' command (for incorporate) reads your /usr/spool/mail
and places files in  a directory - like~/Mail/inbox.  ONce it's in
there you can use anything you want on it.  You can use MH commands in
a shell script also.  The FAQ was recently posted in comp.mail.mh.
If all else fails, you can e-mail me for more info.
-Dave
saranen@epx.cis.umn.edu