🚀 go-pugleaf

Stephen Bridges <newsreply@bh01.demon.co.uk> wrote:

[snip]

> If anyone is interested, the whole script scans NewsRog (or Yam)
> postings and extracts the ICQ UINs and asks if you want to add them to
> your contact list in StrICQ. Because my posts contain my UIN twice, and I
> was using my posts to test the script I thought it might be a nice idea
> for the script to identify repeats and ignore them. Thanks again Tez.

No problem, but now I know what you're doing you will definitely
be interested in using the other syntax I suggested (which I 
first encountered in the OS/2 Rexx nntpclient, and have been
using ever since - mainly in awk scripts :)

The concept is known as "associative indexing" and is really
quite a clever piece of implementation (technically it appears
to be using a hash'ing algorithm, but who cares how it works...).
It's only of use if you need exact string matching (so it's
of little use for killfiles as they normally involve pattern matches),
and is mostly of use only with unique strings.

Here's a simple pseudo-code example of the idea as it works for
message-id checking...

/* open a file containing a list of message-ids */
err=open(hist_io,newshistory,'R')
do until eof(hist_io)
/* read each line */
  messageid = readln(hist_io)
/* and use the clever trick */
  messageids.messageid = 1     /* [1] */
end

/* open a file containing a list of newsgroups and timestamps */
err=open(groups_io,newsgroups,'R')
newsgroups.0 = 0
do until eof(groups_io)
/* read each line */
  newsgroup_num = newsgroups.0 + 1
  line = readln(groups_io)
  newsgroup = word(line,1)
  datestamp = word(line,2)
  timestamp = word(line,3)
  /* be slightly clever */
  newsgroups.newsgroup_num.GROUP = newsgroup     /* [2] */
  newsgroups.newsgroup_num.DATES = datestamp
  newsgroups.newsgroup_num.TIMES = timestamp
  newsgroups.0 = newsgroup_num
end
err = close(group_io)

/* NB this won't actually work as I'm missing out some responses */
/* plus it'll do a couple of dodgy things with the '.' */
/* and it has no error checking */

err=open(nntp_io,'TCP:'newsserver'/nntp','R')
/* read and throwaway the welcome */
err=readln(nntp_io)

err=open(batch_io,newsbatch,'W')
/* command that will get a list of message-ids for my.group */
do newsgroup_num = 1 to newsgroups.0
  newsgroup = newsgroups.newsgroup_num.GROUP
  datestamp = newsgroups.newsgroup_num.DATES
  timestamp = newsgroups.newsgroup_num.TIMES
  nntpcommand='NEWNEWS 'newsgroup' 'datestamp' 'timestamp
  err=writeln(nntp_io,nntpcommand)
  /* read and throwaway the response code */
  err=readln(nntp_io)
  
  /* if it's a '.' then end of message-ids */
  do until messageid == '.'    /* use == to keep AP happy :) */
    /* get the next message-id */
    messageid=readln(nntp_io)
    /* use the clever bit :) */
    if messageids.messageid = 1 then dupes = dupes + 1     /* [3] */
    else do
      newarts = newarts + 1
      /* do something cos it's a new article */
      /* may want to add a killfile in here */
      err=open(temp_io,tempfile,'W')
      nntpcommand = 'ARTICLE 'messageid
      err=writeln(nntp_io,nntpcommand)
      /* read and throwaway the response code */
      err=readln(nntp_io)
      /* if it's a '.' then end of article */
      artsize = 0
      do until line == '.'    /* use == to keep AP happy :) */
        line = readln(nntp_io)
        artsize = artsize + length(line)
        err=writeln(temp_io,line)
      end
      err=close(temp_io)
      err=writeln(batch_io,'#!rnews 'artsize)
      err=open(temp_io,tempfile,'R')
      do until eof(temp_io)
        line=readln(temp_io)
        err=writeln(batch_io,line)
      end
      err=close(temp_io)
      /* use of clever bit to ensure we don't download dupe messages */
      messageids.messageid = 1     /* [4] */
      /* and write it to the newshistory for next time */
      err=writeln(hist_io,messageid)
    end
  end
end

/* be tidy */
nntpcommand='QUIT'
err=writeln(nntp_io,nntpcommand)
err=close(nntp_io)
err=close(batch_io)
err=close(hist_io)

/* */

[1] rather than have a number for each message-id and having to later
scan through that list for each message-id in a big do/while loop
with some sort of break when we hit a match, we actually create a
new variable whose name is the message-id
[2] unrelated, but uses a similar feature
[3] here we directly look up the message-id, if the value is 1 then
we know it's been read from our history file
[4] similar to [1] but done on an ongoing basis

NB all usage of these "stem variables" (variablename.) in ARexx
uses a fair bit of RAM, doing it with strings rather than numbers
doesn't use much more though.

You should be able to do a similar thing for your UINs, meaning
you can take out the inelegant loops :)

PS this article started out as a little code-sample...

-- 
<URL:http://www.pierrot.co.uk/>                                  Team AMIGA
Parallel lines never meet, unless you bend one or both of them.