Article View: demon.ip.developers
Article #1097Re: Robust design for dosdis expire
From: walter briscoe
Date: Sun, 28 Apr 1996 00:00
Date: Sun, 28 Apr 1996 00:00
77 lines
4543 bytes
4543 bytes
In article <830590991.2944.0@thokk.demon.co.uk>
Malcolm@demon.net "Malcolm Muir" writes:
> walter briscoe (walter@wbriscoe.demon.co.uk) wrote:
> > In article <830462963snz@tsys.demon.co.uk>
> > tomw@tsys.demon.co.uk "Tom Wheeley" writes:
>
> > > This program would also skip gracefully any newsgroups it cannot expire,
> > > without messing up indices. It should also process the newsgroups in size
> > > order (smallest first), to try to prevent disk full errors. This wonder
> > > utility should also cause porcine creatures to become airborne... ;)
>
> The stats at the summary shoudl also remain accurate.
They do!
>
> > I now have an expire which:
> > 1) Does not corrupt active if the disk does not have enough room.
> > 2) Skips newsgroups where the expiry time is negative
> > 3) Skips newsgroups where the first article is younger than the expiry
> > time
> > 4) Optionally, keeps unread articles where there is only one user.
>
> > It is a lot faster as a result of 2&3.
>
> Are the summary stats still accurate ? 2 and 3 were done to preserve
> this accuracy. I'd be the first to accept there are better ways to
> gain the information than run the group through the expiry code, that
> was a decision of the original author that I never got round to
> addressing when I added the code to skip expiring a group. The
> original expire did not support the negative expiry time option. In
> fact it did not support different expiry times for different groups.
The size of a group from which no articles are removed is the size of
the article file for the group. The number of articles is the
difference between 2 figures in ACTIVE.
>
> > BTW, I read an earlier article where Philib McNab suggested that expire
> > should take note of prune files. I would not have thought there was a
> > problem with the current expire program because it does not renumber
> > articles but merely beheads the list in each newsgroup.
>
> But has the constraint that numbers must be sequential with no gaps.
> This is the problem with keeping or pruning articles out of sequence.
Serves me right for sending a message after midnight. I was wrong!
Renumbering of articles - required where an earlier article is kept and
a later one deleted currently requires renumbering of those later
articles. What is the advantage of requiring the article sequence to be
monotonic. ACTIVE contains 2 numbers which describe the sequence. I
can see that it is necessary for one of those numbers to describe the
last number used - this allows the next number to be assigned.
Currently the other number is one less than the minimum article number.
Does it need to have this logic? Article numbers are available in the
IDX files. The consistency check that it allows would disappear. e.g.
expire: In alt.msdos.programmers article 25 found when 28 expected
The sequence could be checked as increasing and discrepancies between
active and the idx files could still be noted.
I have added a check to deal with snzprune actions. If the first
article to be kept is older than the expiry time and a prune action is
pending for that group, expiry is not done on that group and the user is
given an explanation and the relevant command. e.g.
expire: For alt.msdos.programmers, first do: snzprune altmsdp
I have been thinking about the mechanism used to protect articles
against deletion by expire. Keeping unread articles is one mechanism.
I wonder if marking articles as protected by negative inversion of their
times would do the job. This would require mechanisms in SNEWS.EXE to
invert the protection on a message and to show that messages were
protected. The action of EXPIRE of making times a non-decreasing
sequence would also have to go.
--
walter briscoe
Technical Details
Message-ID:
<830682895snz@wbriscoe.demon.co.uk>Path:
arc1-ca.pugleaf.net!unknown!not-for-mailReferences:
<829700843snz@tsys.demon.co.uk> <830170486snz@wbriscoe.demon.co.uk> <830215383snz@wash.demon.co.uk> <830462963snz@tsys.demon.co.uk> <830566222snz@wbriscoe.demon.co.uk> <830590991.2944.0@thokk.demon.co.uk>Show Full Headers
Complete Article Headers
X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: 10bb05,69cce58401bcf88 X-Google-Attributes: gid10bb05,public From: walter briscoe <walter@wbriscoe.demon.co.uk> Subject: Re: Robust design for dosdis expire Date: 1996/04/28 Message-ID: <830682895snz@wbriscoe.demon.co.uk> X-Deja-AN: 151885304 x-mail2news-path: wbriscoe.demon.co.uk x-nntp-posting-host: wbriscoe.demon.co.uk references: <829700843snz@tsys.demon.co.uk> <830170486snz@wbriscoe.demon.co.uk> <830215383snz@wash.demon.co.uk> <830462963snz@tsys.demon.co.uk> <830566222snz@wbriscoe.demon.co.uk> <830590991.2944.0@thokk.demon.co.uk> reply-to: walter@wbriscoe.demon.co.uk newsgroups: demon.ip.developers