🚀 go-pugleaf

In article <829700843snz@tsys.demon.co.uk>
           tomw@tsys.demon.co.uk "Tom Wheeley" writes:

> Problem:
> 
> Being of low and fluctuating free disk space,  ...

[snip]

> Proposal:
> 
> Firstly, make expire check for free disk space for each newsgroup.  If there
> is unsufficient, then warn and skip the newsgroup.
> 
> Secondly, instead of expiring on order of ACTIVE (or whatever), expire in
> order of increasing disk usage, thus increasing the chances of there being
> enough disk space for the larger groups to be dealt with.

I would like that proposal to be postponed until their is not a problem
of expire corrupting the snews database.  I keep my news on a compressed
drive and am sceptical about the reliability of free space figures.

I also operate with a disk which is nearly full.

I have just got a MUCH bigger system and, before I move to it, I thought
I would sort out the problem of expire corrupting the news database.

I am in correspondence with Malcolm Muir and John Washington on the work
and hope to deliver it for official release.

In the meantime, I am able to explain the mechanism and offer a
workround which destroys no information rather than re-indexing a
newsgroup which tends to mark all articles as unread. (I have yet to
explain that problem.)

When you expire a newgroup, the article file, the index file and ACTIVE
are updated.  EXPIRE handles the article and index files safely.
However ACTIVE is updated in an unsafe manner.

After EXPIRE fails to update a group, you get a message like:
"Can't open <groupname> got <n> when <m> expected."
The usual reason for this is that ACTIVE has been updated and the
articles database has not.  The fix is to edit ACTIVE.  This is a fairly
safe operation as each record has a normal DOS line terminator (\r\n)
rather than a UNIX (\n).  You need not worry about your editor following
the CP/M convention of marking a file end with a CTRL-Z(0x1a).  The fix
is to set the message number to one less than the expected value.

e.g.
  Before you try any of this, take a copy of ACTIVE.  %SNEWS%\ACTIVE.BAK
  is a BAD name to use because that name is likely to be used for
  workfiles.  ACTIVE.BAK in a directory away from your demon files is ok.

  Suppose you get the message
      snews: in article demon.ip.developers 700 found when 724 expected
  and the corresponding line in ACTIVE is
      demon.ip.developers 05476554 00000723 00000751 y
  you remove the problem by changing the 3rd field on that line so it
  now reads
      demon.ip.developers 05476554 00000699 00000751 y

  i.e. If the 3rd field in the relevant line is one less than the
  "found" field in the snews failure message, change that 3rd field to
  be one less than the "expected" number in the snews message.

  Try the fix.  If it clears the situation, you are ahead.  If not,
  restore the saved file and reindex and you are not behind apart from
  the effort you spent trying to avoid REINDEX.

I have a new version of EXPIRE.C which I am working on.  It solves the
problem above.  It does not get disk problems with groups which have
negative expiry times.  However, it is sensitive to power-failure and
user interrupts.  My next version will reduce the window of sensitivity
to the time taken to rename 3 files.

BTW, EXPIRE has an alternative mode of working
which gives me the screaming heeby-jeebies.  It takes a parameter and
applies that to all groups. e.g. expire 0 will delete all news in all
groups - even those with -ve expiry times.  Opinions on
changing/removing that feature?  Opinions on adding an option not to
expire unread articles.  I would like to mark FAQs etc as articles not
to be expired.  Making them unread would do that for me with the revised
specification.

Malcolm has expressed concerns that EXPIRE should not be any slower than
it is now.  I shall check out the effect of reading article files using
2 4k buffers rather than a line at a time.  IMHO, the 1st criterion of
expire is that it should NEVER destroy the integrity of information.
Speed is secondary.  I have reduces the monolithic tendency of the
file.  The maximum function length in the original file is more than
400 lines.  I have reduced that to less than 100 lines at the cost of
introducing some data with static linkage.  My version of EXPIRE.C is
about a third bigger than that in SNEWS 1.30.  I shall try and reduce
that bloat.  I am not happy that my current code has several copies of
"rollback" logic.

Sorry, this is long.
--
walter briscoe