🚀 go-pugleaf

In article <870540359snz@pdlmail.demon.co.uk>
           pete@pdlmail.demon.co.uk "Pete" writes:

>Just poking around the source (as one does...) I came across
>a problem in the 'remlist()' func in smtpserv.c - that is,
>the official one on ftp.demon.  Knowing how tight ka9q is
>for mem, it strikes me it would be a good idea to fix it.

Yup, but the only part of the alias expansion that uses it is itself
broken for multiple RCPT TOs - much better to remove the function
entirely.  (IIRC, the way I read it, expansion of the default alias
doesn't match the description in DISPC.TXT either).

>And talking of lists, some memory might be reclaimed by
>rationalising them across ka9q:  I've only looked so far at
>smtpserv and nntpcli with a view to producing a unified
>kill/keep facility and there is duplication of effort even
>in the live code :(  smtpserv.c has a 'struct list ..' and
>nntpcli.c has a (similar) 'struct textline ..' - both with
>their own sets of functions to manipulate them. 

Bear in mind that SMTP and NNTP can be configured in separately.

>And another thing: 'struct textline' is defined in nntpcli.c
>as:
>
>struct textline
>    {
>    struct textline *nextline;      /* pointer to next text line */
>    char text[NNTPMAXLEN];   /* array holding text (shrunk to fit) */
>    };
>
>which is a profligate waste of mem - most kill entries will
>be less than 512 bytes long (one hopes..) so it should be more
>efficient to define 'text' as char * and replace the 'strcpy'
>with a 'strdup' even with the extra 'free' call afterwards.

Ah, but it never actually does a malloc (sizeof (struct textline)) -
the comment is misleading as it's not shrunk but never fully allocated
in the first place - see the MINTEXTLINESIZE #def (using offsetof()
from stddef.h instead would be more portable and protect against
alignment problems).  It saves 4 bytes per list entry and a level of
indirection so it is slightly more efficient than the standard linked
list struct.

The NNTP batch buffers do waste memory by allocating 512 bytes for the
message-id (the average was around 40 last time I checked) - the dbz
library used by the reference implementations only handles keys up to
255 bytes by default so message-ids longer than 254 characters may not
propagate too well.  (I don't know if this restriction is still in
place on the major news peering sites at the moment).  I've reduced
the buffer to 256 and any longer ids are dumped to the log file.

>And another thing!  The 'createkilllists()' func takes no
>special action for empty or comment lines - they get their
>own 500+ byte buffers just like proper entries.  At least the
>smtp code checks for and skips these lines with its lists.
>All such lines now removed from my killfile :-)

The header matching is also pretty dire since each header line is
lower-cased and whitespace-reduced before attempting a match - at the
expense of 200 bytes of memory, I've expanded the kill and keep lists
to 26 lists each, indexed on the first letter of the header (a-z).
Checking only the first character of each incoming header and a much
shorter list to check on a hit is hopefully an performance improvement
for larger kill files.  (No real figures to show it does but I have
broken the 3k/s mark at 14.4k on the TNT lines - this may have
something to do with fixing the byte counts to account for \r\n -> \n
translation).

The downside is not allowing wildcards to match the first character of
a header (which IMHO would be grossly inefficient in the present
implementation) but using * to kill all bodies by default is still
supported - it sets an initial state variable for each header so that
a scan of the kill lists isn't needed, only the keep ones.

Cheers, Graham.
-- 
Graham E. Kinns
Graham@grempc.demon.co.uk

In article <870891657snx@warren.demon.co.uk>
           leveret@warren.demon.co.uk "Nick Leverton" writes:

>In article <870774566snz@grempc.demon.co.uk>
>   newsmaster@grempc.demon.co.uk (Graham E Kinns) writes:
>>
>>The downside is not allowing wildcards to match the first character of
>>a header (which IMHO would be grossly inefficient in the present
>>implementation)

>                                      Ummm, you do have 27 lists
>(assuming conversion of tabs-to-spaces), don't you, not 26 ?  Or have
>you made it handle continuations properly ?

Err, no and, um, no :-(  It didn't (and still doesn't) handle any
lines longer than 512 chars properly - the original idea had 27 lists,
the 27th being for anything not beginning A-Z but I dropped it as a
rule starting with a wildcard would need checking against every header
- removing almost all the advantage of rewriting it in the first place
:-(

Adding the 27th for continuation lines and even coping with lines
longer than 512 should be do-able.  I suppose the answer to the 
wildcard problem is to add yet another list for wildcards and [^a-z ]
to be checked against every header.

>Nick, fearful lest asking these questions further delay the release :-)
 
Nah, just gets added to the Known Problems section ;-)

Cheers, Graham.
-- 
Graham E. Kinns
Graham@grempc.demon.co.uk