🚀 go-pugleaf

In article <1992Jun2.171620.25653@PA.dec.com> reid@decwrl.DEC.COM (Brian Reid) writes:
|This is the script for the "arbitron" system that is used to produce the
|data for the monthly USENET readership surveys in news.lists. It is not so
|much a "source" as it is a piece of USENET folklore, posted every month so
|that new people will be sure to find it. (CF the "History of USENET" articles
|posted every month). It is posted to these newsgroups rather than to a source
|group so that it will be widely available but will not be archived.
|[...]


The following patches were sent several times to Brian
but were never responded to, so I'm posting this directly....


 --------- 8< --------- 8< --------- 8< --------- 8< --------- 8< ---------


I went to upgrade to arbitron 2.4.4, and found some
problems which I fixed - the diffs are included below.


There were 3 main kinds of difficulties - shell bugs,
SystemV 14-character filename size problems, and C news
active file format (plus some pesky portability stuff).


At line 45, a test was added for SystemV $LOGNAME variable
(used in place of BSD $USER).

At lines 97-98, the $PASSWD temp file name size was reduced
to 14 chars, and a shell bug work-around was reimplemented.
Also awk was given an argument in place of input redirection,
since some awks will not give output without an argument list
(this could have been done by adding "-" as the file argument
and keeping the input redirection).

At line 101, the need for "\\" escapes was eliminated,
thus avoiding a pesky sh difference from ksh.

At line 150, a test for Cnews-style aliases and "x"ed-out
newsgroups was added so they could be skipped over the
same as most newsreaders so.

At lines 179-183, the numbers were made quite large
since the possibility exists for (usually erroneous)
high groupcounts to be recorded.

At line 197, the regular expression was modified to
reflect the fact that rn sometimes does not leave a
space in .newsrc between the colon and the article
numbers.

At lines 206, the $ACTIVE temp file name size was reduced
to 14 chars, ans the "/tmp" directory was changed to
the variable $TMPDIR.

At line 218, awk was given an argument in place of input
redirection due to the above-mentioned bug.

At line 234, awk was given an argument of "-" for the
same reason.

At line 240, "to." and "local." newsgroup deletion was
added, and the larger numbers changed in lines 179-183
were accounted for.


Cheers,
Bruce Becker	News Administration	Toronto, Ont.
Internet: news@becker.gts.org		UUCP: ...!lsuc!becker!news

 --------- 8< --------- 8< --------- 8< --------- 8< --------- 8< ---------

*** arb244	Thu Feb 20 00:32:48 1992
--- arb245	Thu Feb 20 00:54:19 1992
***************
*** 1,5
  #! /bin/sh
! # @(#)arbitron	2.4.4	10/20/89
  # arbitron -- this program produces rating sweeps for USENET.
  #
  # Usage: arbitron

--- 1,5 -----
  #! /bin/sh
! # @(#)arbitron	2.4.5	02/20/92
  # arbitron -- this program produces rating sweeps for USENET.
  #
  # Usage: arbitron
***************
*** 32,37
  # Fixes for YP and NN by
  #	Dan Kolkowitz, Stanford
  #	Brent Chapman, Capital Market Technology
  #
  # Note that the results of this program are dependent on the rate at which
  # you expire news.  If you are a small site that expires news rapidly, the

--- 32,39 -----
  # Fixes for YP and NN by
  #	Dan Kolkowitz, Stanford
  #	Brent Chapman, Capital Market Technology
+ # Fixes for C news & other things by
+ #	Bruce Becker, GTS, Toronto
  #
  # Note that the results of this program are dependent on the rate at which
  # you expire news.  If you are a small site that expires news rapidly, the
***************
*** 43,48
  NEWS=/usr/lib/news
  SPOOL=/usr/spool/news
  
  # Make a crude stab at determining the system type. If your installation has
  # only one type of system, you can edit out the "if" statement and just turn
  # this into an assignment statement of the correct value.

--- 45,53 -----
  NEWS=/usr/lib/news
  SPOOL=/usr/spool/news
  
+ if [ -z "$USER" ]; then USER=$LOGNAME; fi
+ if [ -z "$USER" ]; then USER=usenet; fi
+ 
  # Make a crude stab at determining the system type. If your installation has
  # only one type of system, you can edit out the "if" statement and just turn
  # this into an assignment statement of the correct value.
***************
*** 94,101
  # many people use this machine. Comment out all but 1 of them; pick the one
  # you like best. Initially the most universal but least reliable of them is
  # uncommented.
! PASSWD=$TMPDIR/arb.passwd.$$
! (ypcat passwd || cat /etc/passwd) > $PASSWD
  
  # # ###### Scheme #1: fast but usually returns too big a number
  nusers=`awk -F: "BEGIN {N=0}\\$3>=$lowUID && \\$3<=$highUID{N=N+1}END{print N}" <$PASSWD`

--- 99,106 -----
  # many people use this machine. Comment out all but 1 of them; pick the one
  # you like best. Initially the most universal but least reliable of them is
  # uncommented.
! PASSWD=$TMPDIR/arb.pwds.$$
! (ypcat passwd > $PASSWD) 2>/dev/null || cat /etc/passwd > $PASSWD
  
  # # ###### Scheme #1: fast but usually returns too big a number
  nusers=`awk -F: '$3>='$lowUID' && $3<='$highUID'{N++} END{print N}' $PASSWD`
***************
*** 98,104
  (ypcat passwd || cat /etc/passwd) > $PASSWD
  
  # # ###### Scheme #1: fast but usually returns too big a number
! nusers=`awk -F: "BEGIN {N=0}\\$3>=$lowUID && \\$3<=$highUID{N=N+1}END{print N}" <$PASSWD`
  
  # # ###### Scheme #2 (works with BSD systems)
  #nusers=`last | sed '/^wtmp begins/d; s/ .*//; /^$/d' | sort -u | wc -l`

--- 103,109 -----
  (ypcat passwd > $PASSWD) 2>/dev/null || cat /etc/passwd > $PASSWD
  
  # # ###### Scheme #1: fast but usually returns too big a number
! nusers=`awk -F: '$3>='$lowUID' && $3<='$highUID'{N++} END{print N}' $PASSWD`
  
  # # ###### Scheme #2 (works with BSD systems)
  #nusers=`last | sed '/^wtmp begins/d; s/ .*//; /^$/d' | sort -u | wc -l`
***************
*** 147,153
  #
  # Active file line:  dispose of previous group (if any), record group, and
  # record first and last article numbers.  Set group's reader count to none.
! 	NF == 4 { if (grpname != "") {
  			printf("%d %s\n",grpcount, grpname)
  		  }
  		  grpname = $1

--- 152,160 -----
  #
  # Active file line:  dispose of previous group (if any), record group, and
  # record first and last article numbers.  Set group's reader count to none.
! # Skip over C news aliases and "x"ed-out groups since newsreaders do too.
! 	NF == 4 && $4 != "x" && $4 !~ /^=/ {
! 		  if (grpname != "") {
  			printf("%d %s\n",grpcount, grpname)
  		  }
  		  grpname = $1
***************
*** 176,186
  		}
  #
  # End of file.  Print the report in 2 columns.
! 	END	{ printf("9999 Host\t\t%s\n","HOSTNAME")
! 		  printf("9998 Users\t\t%d\n",NUSERS)
! 		  printf("9997 NetReaders\t%d\n",realusers)
! 		  printf("9996 ReportDate\t%s\n","DATE")
! 		  printf("9995 SystemType\tnews-arbitron-2.4\n")
  # For reorganized network, report a group even if nobody reads it. This will
  # help us keep track of where the groups propagate.
  		  printf("%d %s\n",grpcount, grpname)

--- 183,193 -----
  		}
  #
  # End of file.  Print the report in 2 columns.
! 	END	{ printf("999999999 Host\t\t%s\n","HOSTNAME")
! 		  printf("999999998 Users\t\t%d\n",NUSERS)
! 		  printf("999999997 NetReaders\t%d\n",realusers)
! 		  printf("999999996 ReportDate\t%s\n","DATE")
! 		  printf("999999995 SystemType\tnews-arbitron-2.4\n")
  # For reorganized network, report a group even if nobody reads it. This will
  # help us keep track of where the groups propagate.
  		  printf("%d %s\n",grpcount, grpname)
***************
*** 194,200
  		printf("sed -n '/^+/s/^. \([0-9]*\) \(.*\)/\2 %s \1/p'",$1)
  		printf(" <%s/.nn/rc;\n",$6)
  		printf("elif [ -r %s/.newsrc ] ; then ", $6)
! 		printf("sed -n '/: [0-9]/s/:/ %s/p' <%s/.newsrc; fi\n",$1,$6)
  		seen[$6]=1;
  	  }
  }

--- 201,207 -----
  		printf("sed -n '/^+/s/^. \([0-9]*\) \(.*\)/\2 %s \1/p'",$1)
  		printf(" <%s/.nn/rc;\n",$6)
  		printf("elif [ -r %s/.newsrc ] ; then ", $6)
! 		printf("sed -n '/: *[0-9]/s/:/ %s /p' <%s/.newsrc; fi\n",$1,$6)
  		seen[$6]=1;
  	  }
  }
***************
*** 203,209
  # First, make sure we have an active file
  if [ -z "$NEWSHOST" ]
  then ACTIVE=$NEWS/active
! else ACTIVE=/tmp/arb.active.$$
       rcp $NEWSHOST:$NEWS/active $ACTIVE
  fi
  

--- 210,216 -----
  # First, make sure we have an active file
  if [ -z "$NEWSHOST" ]
  then ACTIVE=$NEWS/active
! else ACTIVE=$TMPDIR/arb.act.$$
       rcp $NEWSHOST:$NEWS/active $ACTIVE
  fi
  
***************
*** 215,221
  
  # Next, get the list of .newsrc files with duplicates and unreadable files
  # removed.
! awk -F: -f $TMPDIR/arb.pwd.$$ <$PASSWD | sh >$TMPDIR/arb.tmp.$$
  
  # Check to make sure that we found some
  if [ -s $TMPDIR/arb.tmp.$$ ]

--- 222,228 -----
  
  # Next, get the list of .newsrc files with duplicates and unreadable files
  # removed.
! awk -F: -f $TMPDIR/arb.pwd.$$ $PASSWD | sh >$TMPDIR/arb.tmp.$$
  
  # Check to make sure that we found some
  if [ -s $TMPDIR/arb.tmp.$$ ]
***************
*** 234,240
       else egrep '^[a-z][-0-9_a-z]*\.' $ACTIVE
       fi |
       sort - $TMPDIR/arb.tmp.$$ |
!      awk -f $TMPDIR/arb.fmt.$$ |
       sort -nr |
       sed '/^$/d
  	  s/^999[0-9] //' |

--- 241,247 -----
       else egrep '^[a-z][-0-9_a-z]*\.' $ACTIVE
       fi |
       sort - $TMPDIR/arb.tmp.$$ |
!      awk -f $TMPDIR/arb.fmt.$$ - |
       sort -nr |
       sed '/^$/d
  	  /^[0-9]* to\./d
***************
*** 237,243
       awk -f $TMPDIR/arb.fmt.$$ |
       sort -nr |
       sed '/^$/d
! 	  s/^999[0-9] //' |
       $destination
  else echo arbitron: Unable to find any readable .newsrc files >&2
       (exit 1); exit

--- 244,253 -----
       awk -f $TMPDIR/arb.fmt.$$ - |
       sort -nr |
       sed '/^$/d
! 	  /^[0-9]* to\./d
! 	  /^[0-9]* local\./d
! 	  /^[0-9]* clari\./d
! 	  s/^99999999[0-9] //' |
       $destination
  else echo arbitron: Unable to find any readable .newsrc files >&2
       (exit 1); exit


-- 
  ,u,	 Bruce Becker			Toronto, Ontario
a /i/	 Internet: bdb@becker.gts.org	Uucp: ...!lsuc!becker!bdb
 `\o\-e	 "Not only am I the President of the Unfair Club for Men,
 _< /_	 	I'm also a client"  -  G. Bush