🚀 go-pugleaf

dan@cubmol.bio.columbia.edu (Daniel Zabetakis) wrote:
>
>    We are having a problem with a full mail spool. Some users have
>large amounts of mail kept there. I am pretty sure that much of it is
>from mailing lists that they don't want to dump into thier account mbox,
>but don't really want to delete. Well, it is going to have to go now.

I run into the same problem about once each semester.

>   My question in regard to policy is whether it is OK to threaten to
>remove the mail. I intend to write to the active users by e-mail and ask
>them to remove excessive mail (or explain why it is neccessary to keep
>it). If they fail to respond to a reasonable attempt, I intend to delete
>the mail. (probably won't happen).

Our general policy sets a size limit of 150 Kb on user mailboxes.  I check 
the mailbox sizes (rougly) biweekly.   I also wrote a quick shell script,
named "mailsize", that will tell users the size of their mailbox and issue
a gentle reminder if they're over the limit.

If a user mailbox is over the limit for two consecutive checks (which cover
a month), I send a quick piece of mail, asking them to delete/save mail from 
their mailbox.  If they fail to do so, and if they are still over the 150 Kb 
limit after two weeks, the entire mailbox is moved into their home directory.  
(If the mailbox is *really* large, I'll compress it) I then send a piece of 
mail (which will be the first one they see) explaining my action and telling 
them how to access the old mailbox (the -f option to mailx).

I consider this a decent solution for our users, because:
	- The user has up to 6 weeks to deal with the situation
	- The mail is not destroyed
	- The mail is still accessible
	- The diskspace consumption counts against their resource quotas
	  (For our users, resource quotas are a constant goad to responsibility)

>   The real question comes up for the occasional users. Some people only
>log on every few months, if that. I don't want to have to track them
>down and explain the problem to them individually. How much effort should
>I take to get in contact with them before deleting the mail as unwanted.
>We don't often remove accounts (being a depertmental machine with not too
>many users).

The same scenario applies to all user mailboxes.

>   What about runaway mailing lists? I think some of our users who don't
>log in often have joined mailing lists, but never read the mail. Can I
>look at the mail to determine if this is the case? What should I do if it
>is?

I don't think you should look at the actual mail.  However, you can (and
I do) examine the mailer logs, which only show date, time, sender, desti-
nation, and size.  (Alternatively, you can extract the From: headers with
any grep-like tool)  I believe that this approach minimizes your intrusion
into user's confidentiality.

If an large but untouched mailbox has a lot of mailing list traffic, AND if 
the user ignores (or misses) my earlier messages, I'll contact the mailing 
list owner and request that the user be removed.

I've only had to do this about 6 times in 3 years.......

--Wes
-- 
MORGAN@UKCC         |       Wes Morgan       |        ...!ukma!ukecc!morgan 
morgan@ms.uky.edu   | Engineering  Computing |   morgan@wuarchive.wustl.edu
morgan@engr.uky.edu | University of Kentucky | JWMorgan@dockmaster.ncsc.mil
  Mailing list for AT&T StarServer S/E  - starserver-request@engr.uky.edu

In article <1992Nov5.194200.3673@news.columbia.edu> dan@cubmol.bio.columbia.edu (Daniel Zabetakis) writes:
>
>    We are having a problem with a full mail spool. Some users have
>large amounts of mail kept there. I am pretty sure that much of it is
>from mailing lists that they don't want to dump into thier account mbox,
>but don't really want to delete. Well, it is going to have to go now.
>

Here at OSU, we have approximately 8,000 accounts and a one gigabyte
news spool directory in our MAGNUS cluster. We also have a dedicated
mail machine in the cluster. We use home directory quotaing for all of our 
accounts.  We got into the same sticky situation you mention regarding mail
spools and, thus far, we handle it as follows:

1.) We automatically (via "cron") run a script twice a week which sends out 
    warning notices to folks who have more than 128K of mail spool in use 
    (they get *really* tired of seeing these if they are repeat
    "overutilizers"). Of course, this has the downside of
    generating still more mail traffic and spool volume :-(.
    The warning message points out the spool usage policy and
    how to go about accessing it (actually, it quotes the 
    relevant section).

2.) We periodically "migrate" the mail of users who have had over 256K of
    mail spool in use and have gotten at least two warnings. We migrate
    this mail into their home directory as a compressed file and
    send them a note (to their now empty 
    mail spool file) telling them how to decompress the file and how to 
    access the file from the common mail packages. The home directory 
    "quotaing" comes into effect in that they can be driven over quota 
    by not reading & handling their mail. This migration and notification
    is accomplished via a script. This script is run manually when 
    the mail spool usage has become a concern (i.e. >75%). We don't want
    *anyone* losing their incoming mail if we can possibly avoid it.

>   My question in regard to policy is whether it is OK to threaten to
>remove the mail. I intend to write to the active users by e-mail and ask
>them to remove excessive mail (or explain why it is neccessary to keep
>it). If they fail to respond to a reasonable attempt, I intend to delete
>the mail. (probably won't happen).

We try to never irretreivably delete mail. We are presently keeping monthly 
incremental backups in perpetuity. I guess that one thing you could
do as a last resort is have a last step of "migrating" their mail
to tape. Ideally, this sort of thing is done here for accounts inactive for
extended periods or never accessed since creation. It makes a lot
of sense to do this periodically via a script and try to bundle a lot 
of "expirations" on one tape (we use 1.2 GB DATs).

One downside of the "migration" approach is that sometimes we are
at a critical utilization level on the user disks which hold the
home directories. At these times, we can't afford the space on the
user disks to migrate (even compressed) mail to the users home
directories. One of the problems with user quotas is that they never
clean up if they never log in, so the space usage for a user
can exceed even their "hard" quota (the script which migrates
compressed spool mail to the users directory runs as "root" 
so no mail is lost). Of course the answer to this "never login"
problem is the last stage migration to tape & deletion of unused
accounts outlined above. 

Incidently, "popmail" accounts have the potential to complicate 
the whole situation enormously (especially
if the POP client user sets the client to leave the mail at the host, or 
if they never use their POP account). Our tenetive solution to this is the 
addition of a popmail server machine with its' own mail spool.

As you can see, for a large system or cluster of systems, the problem and
solutions to same can become quite complex and involve *many* 
previously unforseen factors.

>   The real question comes up for the occasional users. Some people only
>log on every few months, if that. I don't want to have to track them
>down and explain the problem to them individually. How much effort should
>I take to get in contact with them before deleting the mail as unwanted.
>We don't often remove accounts (being a depertmental machine with not too
>many users).

We get some folks here who go on sabitical for months or *years* and yet 
still want their accumulated e-mail when they get back. It is a problem.

>
>   What about runaway mailing lists? I think some of our users who don't
>log in often have joined mailing lists, but never read the mail. Can I
>look at the mail to determine if this is the case? What should I do if it
>is?

*Very* common problem I suspect. Most of the complaints I've
seen from folks who get "mail spool usage" warnings come from
folks who like to accumulate vast quantities of mailing list
spawned goodies to read in the indeterminate future (they are
generally afraid of missing an unspecifiable "something" :-). 
Usually, migrating the mail into their quotaed home directory causes
them to reevalute what is important enough to keep on disk
in semi-permanent retention.

I hope this helps identify some of the issues and a potential
(and admittedly somewhat creaky) series of solutions. Maybe
one of these days, I'll write a "spool master" package which
incorporates a portable multi-platform set of solutions to these
problems and make my fortune ;-). (Or maybe I'll just go for
"fame" and distribute it freely :0).

Of course, the usual disclaimers concerning the inadvisability
of following my advice/examples/blatherings apply and in no
way should any of this be considered as endorsed by OSU,
it management, or its' system janatorial staff.
-- 
==========================================================================
              Disclaimer: These are my views, not the U's

        "If it's in the paper it must be true!" --- D. Doright 

dan@cubmol.bio.columbia.edu (Daniel Zabetakis) writes:


>    We are having a problem with a full mail spool. Some users have
>large amounts of mail kept there. I am pretty sure that much of it is
>from mailing lists that they don't want to dump into thier account mbox,
>but don't really want to delete. Well, it is going to have to go now.

At UIUC we set 200K quotas for the /usr/spool/mail partition on our
student account machines (uxa has 8800 accounts, ux4 has 5500).
/bin/mail has also been modified to check quotas before delivery and
will bounce a message if the user is over quota.  It's simple, efficient,
and encourages users not to store mail in the system spool area.  It
does have the side-effect of making users vulnerable to mail bombing,
but that can be handled on a per-case basis.  

/pbp
-- 
"A well-schooled electorate, being necessary to the security of a free State,
the right of the People to keep and read Books shall not be infringed."
	-- J. Neil Schulman

In the referenced article, dan@cubmol.bio.columbia.edu (Daniel Zabetakis) writes:
>
>    We are having a problem with a full mail spool. Some users have
>large amounts of mail kept there. I am pretty sure that much of it is
>from mailing lists that they don't want to dump into thier account mbox,
>but don't really want to delete. Well, it is going to have to go now.
>
>   My question in regard to policy is whether it is OK to threaten to
>remove the mail.

I would not do this if I were you.  A couple of stopgap
solutions I have found are:

1.  Peer pressure.  Inform the other users that if everyone
    doesn't clean up their mail, NOBODY gets mail any more
    (due to the filesystem being out of space), and simply
    list all users with their mailbox sizes.  Let them argue
    it out amongst themselves.

2.  For the occasional users, compressing their mail will buy
    you some extra space, and it will still be there if they
    log in and yelp for it.  This also has the advantage that
    you don't have to read their mail (which I avoid at all costs).

    In real emergencies, you can dump their mail off to tape
    until they come back and ask for it.  Again, you haven't read
    their mail or deleted it, so their privacy is protected, but you
    get the space you need.

Wendy Nather
Swiss Bank Corporation, Zurich
	pooh@apple.com
	wendy@il.us.swissbank.com

Daniel Zabetakis (dan@cubmol.bio.columbia.edu) wrote:
: 
:    What about runaway mailing lists? I think some of our users who don't
: log in often have joined mailing lists, but never read the mail. Can I
: look at the mail to determine if this is the case? What should I do if it
: is?
: 

That's getting a real problem here too. I feel that a responsible
automatic mail server should be auditing its readership once in a
while and asking readers to specifically re-subscribe. I also feel
that not enough is done to give sysadmins facilities to unsubscribe to
mailing lists on behalf of dead accounts. [My apologies if such
facilities do exist, its just that it can be difficult to locate the
address for unsubscription without prying into a user's mailbox].

: DanZ
: 
: -- 
--
Norman Bridges

email cca09@keele.ac.uk  ||  tel (0782) 583073

Keele University::Computer Centre::Keele::Newcastle,Staffs::ST5 5BG::UK

In article <BxwzoA.325@gabriel.keele.ac.uk> cca09@keele.ac.uk (Norman Bridges) writes:
>Path: austin.onu.edu!pacific.mps.ohio-state.edu!zaphod.mps.ohio-state.edu!darwin.sura.net!spool.mu.edu!agate!doc.ic.ac.uk!uknet!keele!seq1!cca09
>From: cca09@keele.ac.uk (Norman Bridges)
>Newsgroups: comp.admin.policy
>Subject: Re: Full mail spool policy?
>Message-ID: <BxwzoA.325@gabriel.keele.ac.uk>
>Date: 18 Nov 92 13:48:09 GMT
>References: <1992Nov5.194200.3673@news.columbia.edu>
>Sender: news@gabriel.keele.ac.uk (UseNet News Service)
>Organization: Keele University, England

 I also feel
>that not enough is done to give sysadmins facilities to unsubscribe to
>mailing lists on behalf of dead accounts. [My apologies if such
>facilities do exist, its just that it can be difficult to locate the
>address for unsubscription without prying into a user's mailbox].

>Norman Bridges
>
>email cca09@keele.ac.uk  ||  tel (0782) 583073
>
>Keele University::Computer Centre::Keele::Newcastle,Staffs::ST5 5BG::UK

Why should the administrator have to unsubscribe the dead account.  If the 
account is removed, the mail bounces back to the listserv and the list owner 
can remove the subscriber.
__
     +-------------------------------------------------------+
     |  David R. Warner, Jr.       |          (419) 772-2227 |
     |  Professor of Law           |     FAX: (419) 772-1875 |
     |  Pettit College of Law      | Internet:               |
     |  Ohio Northern University   |   warner@austin.onu.edu |
     |  Ada, Ohio 45810            |                         |
     +-------------------------------------------------------+

In <BxwzoA.325@gabriel.keele.ac.uk> cca09@keele.ac.uk (Norman Bridges) writes:
>That's getting a real problem here too. I feel that a responsible
>automatic mail server should be auditing its readership once in a
>while and asking readers to specifically re-subscribe.

Sorry, but that's just not going to happen.  Even with an automated
system, this is going to create a collosal headache for the admin
and the subscribers.

>                                                        I also feel
>that not enough is done to give sysadmins facilities to unsubscribe to
>mailing lists on behalf of dead accounts. [My apologies if such
>facilities do exist, its just that it can be difficult to locate the
>address for unsubscription without prying into a user's mailbox].

The usual mechanism is to remove the account.  Most decent lists will
have bounces directed to an admin address.  Others will bounce to the
poster.  Sometimes the poster will forward these to the admin.  Otherwise
the list admin will post a message periodically to catch bounces.

Apropos to the original subject, many lists employ a system where mail
to a user bounces if their mail file is over some limit.  Some mail
systems (Sendmail+IDA for example) allow you to setup customized
per user bounce messages.  Examples include "joeuser has not logged
in in over 2 weeks; mail rejected" or "joeuser account has been
disabled; mail rejected".  

I've received mail from some admins asking to please remove so-and-so 
from your list, so obviously some admins will go through the users mail 
file.  I don't particularly like this approach though, as it may be in
violation of the user's privacy.  I would recommend the following
approaches to these problems:

Problem: users often have multi-megabyte mail spool files, and the
spool area is in danger of filling up.

Solution: Impose a mail spool size limit.  When a user reaches the
limit, mail the user a note along the lines of "Your incoming mail
file has exceeded out xxxKB limit.  Incoming mail will be disabled
until you trim your file to less than this limit." then bounce all
additional mail to that user back with the message "joeuser has
exceeded the local mail file size limit." until the file size shrinks.

Problem: users subscribe to mailing lists but then don't log in for
several weeks.

Solution: If you are only concerned with the file sizes, the above
size limit should be sufficient.  If user en abstentia is your main
problem, you could start bouncing mail if a user hasn't logged in in
a while.  This especially seems to be a problem over the summer
vacation.

Problem: user's account has been removed or disabled

Solution: It is usually sufficient to just delete the account.  However,
no such user bounces don't always mean that the user is gone.  Sometimes
it means oops our NIS just broke or oops our system is going bonkers.
ocf.berkeley.edu is notorious for periodically bouncing mail with the
'no such user' message when really it was just having a bad day.  I'll
usually allow bounces to continue for a few days in case the problem
was temporary, but if it was bounced back with the specific message that
the user no longer has an active account, then I'd delete the address
immediately.

The advantages of these systems is that once they are set up, if your
software allows it, then everything is basically on auto-pilot.  You
also don't have to rifle through private files either.

--- jim@ferkel.ucsb.edu --- Jim Lick --- jim@tcp.com --- jIngOrO@CaveMUCK ---
------- perfect little dream the kind that hurts the most - |\| | |/| -------
---  CaveMUCK is back!  Telnet to cave.tcp.com (128.95.10.106) port 2283  ---

>
>Why should the administrator have to unsubscribe the dead account.  If the 
>account is removed, the mail bounces back to the listserv and the list owner 
>can remove the subscriber.

   Our site is used mostly for research. When a user leaves, thier account
is usually not removed. After a while we kill thier passwd so noone can log
in, but we don't delete any of thier files. 
   Each user here works for a lab. It may be that others in the lab will need
access to the files of a user after (perhaps long after) they have left.
Things only get deleted after a -long- period. So accounts can still get mail.

DanZ


-- 
This article is for entertainment purposes only. Any facts, opinions,
narratives or ideas contained herein are not necessarily true, and do
not necessarily represent the views of any particular person.  

      All these administrative issues go away if you charge real money.
Netcom gives me the following message at logon:

> NETCOM On-line Communication Services, Inc.
>  
>   >>>>   News will be offline for a few hours today, so that we can
>   >>>>   upgrade to a faster machine.
>  
> Your account balance is:       19.50
>  
> This disk usage summary is for the last 20 days.
> Your average usage to date is:     5.86 meg
> At this rate your disk charge will be: $   2.14

       The first 5MB of disk space is covered with the monthly charge
for the account, and amounts over 5MB are billed at a modest rate.
This eliminates any reason for the system administration to be concerned
with file content, which is appropriate to a commercial service.

       Netcom is constantly upgrading their hardware, as usage and revenue
increase.  For a commercial service, this is what you want to happen.
Yet the rates are much lower than those of a competing service, the WELL,
which has underpowered hardware and a much larger staff.

					John Nagle

I have been a postmaster for 11 years now and I have found the following
to work:

- the system automatically purges mail older than 30 days.  Of course, you
need to inform your users of this policy.

- When spool fills up I have found that it is because some user(s) have
subscribed themselves to about 5-6 high volume lists, and have stopped
using the computer.  After 30 days, these users have 500+ pieces of mail
which the system then prunes slightly due to the 30 day criteria mentioned
above.  I check the last login time/date and I sometimes find people who
have not logged in for a year or two but continue to get some obscure
mail list.  It is then my responsibility to contact the moderator and
get the user removed.  I do this about 3-4 times a year and scan for all
users with more than 100 unread pieces of mail and then go and check their
last login.

- If the user mentioned above does login, but is using the mail spool
space for storage of large amounts of data and files, I send them mail
and explain the situation and they generally are understanding and comply.
The users don't do it out of malice but rather out of ignorance.

Hank Nussbacher
Israel