Article View: comp.sources.d
Article #1255Fixes for arbitron
From: The Society for Reviling ATnT
Date: Mon, 05 Oct 1992 04:29
Date: Mon, 05 Oct 1992 04:29
294 lines
10080 bytes
10080 bytes
I went to upgrade to arbitron 2.4.4, and found some
problems which I fixed - the diffs are included below.
There were 3 main kinds of difficulties - shell bugs,
SystemV 14-character filename size problems, and C news
active file format (plus some pesky portability stuff).
At line 45, a test was added for SystemV $LOGNAME variable
(used in place of BSD $USER).
At lines 97-98, the $PASSWD temp file name size was reduced
to 14 chars, and a shell bug work-around was reimplemented.
Also awk was given an argument in place of input redirection,
since some awks will not give output without an argument list
(this could have been done by adding "-" as the file argument
and keeping the input redirection).
At line 101, the need for "\\" escapes was eliminated,
thus avoiding a pesky sh difference from ksh.
At line 150, a test for Cnews-style aliases and "x"ed-out
newsgroups was added so they could be skipped over the
same as most newsreaders so.
At lines 179-183, the numbers were made quite large
since the possibility exists for (usually erroneous)
high groupcounts to be recorded.
At line 197, the regular expression was modified to
reflect the fact that rn sometimes does not leave a
space in .newsrc between the colon and the article
numbers.
At lines 206, the $ACTIVE temp file name size was reduced
to 14 chars, ans the "/tmp" directory was changed to
the variable $TMPDIR.
At line 218, awk was given an argument in place of input
redirection due to the above-mentioned bug.
At line 234, awk was given an argument of "-" for the
same reason.
At line 240, "to." and "local." newsgroup deletion was
added, and the larger numbers changed in lines 179-183
were accounted for.
Cheers,
Bruce Becker News Administration Toronto, Ont.
Internet: news@becker.gts.org UUCP: ...!lsuc!becker!news
"'Comefrom' considered useful" - the Anti-Dijkstra
--------- 8< --------- 8< --------- 8< --------- 8< --------- 8< ---------
*** arb244 Thu Feb 20 00:32:48 1992
--- arb245 Thu Feb 20 00:54:19 1992
***************
*** 1,5
#! /bin/sh
! # @(#)arbitron 2.4.4 10/20/89
# arbitron -- this program produces rating sweeps for USENET.
#
# Usage: arbitron
--- 1,5 -----
#! /bin/sh
! # @(#)arbitron 2.4.5 02/20/92
# arbitron -- this program produces rating sweeps for USENET.
#
# Usage: arbitron
***************
*** 32,37
# Fixes for YP and NN by
# Dan Kolkowitz, Stanford
# Brent Chapman, Capital Market Technology
#
# Note that the results of this program are dependent on the rate at which
# you expire news. If you are a small site that expires news rapidly, the
--- 32,39 -----
# Fixes for YP and NN by
# Dan Kolkowitz, Stanford
# Brent Chapman, Capital Market Technology
+ # Fixes for C news & other things by
+ # Bruce Becker, GTS, Toronto
#
# Note that the results of this program are dependent on the rate at which
# you expire news. If you are a small site that expires news rapidly, the
***************
*** 43,48
NEWS=/usr/lib/news
SPOOL=/usr/spool/news
# Make a crude stab at determining the system type. If your installation has
# only one type of system, you can edit out the "if" statement and just turn
# this into an assignment statement of the correct value.
--- 45,53 -----
NEWS=/usr/lib/news
SPOOL=/usr/spool/news
+ if [ -z "$USER" ]; then USER=$LOGNAME; fi
+ if [ -z "$USER" ]; then USER=usenet; fi
+
# Make a crude stab at determining the system type. If your installation has
# only one type of system, you can edit out the "if" statement and just turn
# this into an assignment statement of the correct value.
***************
*** 94,101
# many people use this machine. Comment out all but 1 of them; pick the one
# you like best. Initially the most universal but least reliable of them is
# uncommented.
! PASSWD=$TMPDIR/arb.passwd.$$
! (ypcat passwd || cat /etc/passwd) > $PASSWD
# # ###### Scheme #1: fast but usually returns too big a number
nusers=`awk -F: "BEGIN {N=0}\\$3>=$lowUID && \\$3<=$highUID{N=N+1}END{print N}" <$PASSWD`
--- 99,106 -----
# many people use this machine. Comment out all but 1 of them; pick the one
# you like best. Initially the most universal but least reliable of them is
# uncommented.
! PASSWD=$TMPDIR/arb.pwds.$$
! (ypcat passwd > $PASSWD) 2>/dev/null || cat /etc/passwd > $PASSWD
# # ###### Scheme #1: fast but usually returns too big a number
nusers=`awk -F: '$3>='$lowUID' && $3<='$highUID'{N++} END{print N}' $PASSWD`
***************
*** 98,104
(ypcat passwd || cat /etc/passwd) > $PASSWD
# # ###### Scheme #1: fast but usually returns too big a number
! nusers=`awk -F: "BEGIN {N=0}\\$3>=$lowUID && \\$3<=$highUID{N=N+1}END{print N}" <$PASSWD`
# # ###### Scheme #2 (works with BSD systems)
#nusers=`last | sed '/^wtmp begins/d; s/ .*//; /^$/d' | sort -u | wc -l`
--- 103,109 -----
(ypcat passwd > $PASSWD) 2>/dev/null || cat /etc/passwd > $PASSWD
# # ###### Scheme #1: fast but usually returns too big a number
! nusers=`awk -F: '$3>='$lowUID' && $3<='$highUID'{N++} END{print N}' $PASSWD`
# # ###### Scheme #2 (works with BSD systems)
#nusers=`last | sed '/^wtmp begins/d; s/ .*//; /^$/d' | sort -u | wc -l`
***************
*** 147,153
#
# Active file line: dispose of previous group (if any), record group, and
# record first and last article numbers. Set group's reader count to none.
! NF == 4 { if (grpname != "") {
printf("%d %s\n",grpcount, grpname)
}
grpname = $1
--- 152,160 -----
#
# Active file line: dispose of previous group (if any), record group, and
# record first and last article numbers. Set group's reader count to none.
! # Skip over C news aliases and "x"ed-out groups since newsreaders do too.
! NF == 4 && $4 != "x" && $4 !~ /^=/ {
! if (grpname != "") {
printf("%d %s\n",grpcount, grpname)
}
grpname = $1
***************
*** 176,186
}
#
# End of file. Print the report in 2 columns.
! END { printf("9999 Host\t\t%s\n","HOSTNAME")
! printf("9998 Users\t\t%d\n",NUSERS)
! printf("9997 NetReaders\t%d\n",realusers)
! printf("9996 ReportDate\t%s\n","DATE")
! printf("9995 SystemType\tnews-arbitron-2.4\n")
# For reorganized network, report a group even if nobody reads it. This will
# help us keep track of where the groups propagate.
printf("%d %s\n",grpcount, grpname)
--- 183,193 -----
}
#
# End of file. Print the report in 2 columns.
! END { printf("999999999 Host\t\t%s\n","HOSTNAME")
! printf("999999998 Users\t\t%d\n",NUSERS)
! printf("999999997 NetReaders\t%d\n",realusers)
! printf("999999996 ReportDate\t%s\n","DATE")
! printf("999999995 SystemType\tnews-arbitron-2.4\n")
# For reorganized network, report a group even if nobody reads it. This will
# help us keep track of where the groups propagate.
printf("%d %s\n",grpcount, grpname)
***************
*** 194,200
printf("sed -n '/^+/s/^. \([0-9]*\) \(.*\)/\2 %s \1/p'",$1)
printf(" <%s/.nn/rc;\n",$6)
printf("elif [ -r %s/.newsrc ] ; then ", $6)
! printf("sed -n '/: [0-9]/s/:/ %s/p' <%s/.newsrc; fi\n",$1,$6)
seen[$6]=1;
}
}
--- 201,207 -----
printf("sed -n '/^+/s/^. \([0-9]*\) \(.*\)/\2 %s \1/p'",$1)
printf(" <%s/.nn/rc;\n",$6)
printf("elif [ -r %s/.newsrc ] ; then ", $6)
! printf("sed -n '/: *[0-9]/s/:/ %s /p' <%s/.newsrc; fi\n",$1,$6)
seen[$6]=1;
}
}
***************
*** 203,209
# First, make sure we have an active file
if [ -z "$NEWSHOST" ]
then ACTIVE=$NEWS/active
! else ACTIVE=/tmp/arb.active.$$
rcp $NEWSHOST:$NEWS/active $ACTIVE
fi
--- 210,216 -----
# First, make sure we have an active file
if [ -z "$NEWSHOST" ]
then ACTIVE=$NEWS/active
! else ACTIVE=$TMPDIR/arb.act.$$
rcp $NEWSHOST:$NEWS/active $ACTIVE
fi
***************
*** 215,221
# Next, get the list of .newsrc files with duplicates and unreadable files
# removed.
! awk -F: -f $TMPDIR/arb.pwd.$$ <$PASSWD | sh >$TMPDIR/arb.tmp.$$
# Check to make sure that we found some
if [ -s $TMPDIR/arb.tmp.$$ ]
--- 222,228 -----
# Next, get the list of .newsrc files with duplicates and unreadable files
# removed.
! awk -F: -f $TMPDIR/arb.pwd.$$ $PASSWD | sh >$TMPDIR/arb.tmp.$$
# Check to make sure that we found some
if [ -s $TMPDIR/arb.tmp.$$ ]
***************
*** 234,240
else egrep '^[a-z][-0-9_a-z]*\.' $ACTIVE
fi |
sort - $TMPDIR/arb.tmp.$$ |
! awk -f $TMPDIR/arb.fmt.$$ |
sort -nr |
sed '/^$/d
s/^999[0-9] //' |
--- 241,247 -----
else egrep '^[a-z][-0-9_a-z]*\.' $ACTIVE
fi |
sort - $TMPDIR/arb.tmp.$$ |
! awk -f $TMPDIR/arb.fmt.$$ - |
sort -nr |
sed '/^$/d
/^[0-9]* to\./d
***************
*** 237,243
awk -f $TMPDIR/arb.fmt.$$ |
sort -nr |
sed '/^$/d
! s/^999[0-9] //' |
$destination
else echo arbitron: Unable to find any readable .newsrc files >&2
(exit 1); exit
--- 244,253 -----
awk -f $TMPDIR/arb.fmt.$$ - |
sort -nr |
sed '/^$/d
! /^[0-9]* to\./d
! /^[0-9]* local\./d
! /^[0-9]* clari\./d
! s/^99999999[0-9] //' |
$destination
else echo arbitron: Unable to find any readable .newsrc files >&2
(exit 1); exit
--
Bruce Becker News Administration Toronto, Ont.
Internet: news@becker.gts.org UUCP: ...!web!becker!news
"ATnT blows dead bears" - Just About Everybody
Technical Details
Message-ID:
<1992Oct5.042946.16266@becker.GTS.ORG>Path:
arc1-ca.pugleaf.net!sparky!uunet!utcsri!geac!herboid!cattnts!ncrcan!becker!newsShow Full Headers
Complete Article Headers
Path: arc1-ca.pugleaf.net!sparky!uunet!utcsri!geac!herboid!cattnts!ncrcan!becker!news From: news@becker.GTS.ORG (The Society for Reviling ATnT) Newsgroups: comp.sources.d,news.admin.misc Subject: Fixes for arbitron Message-ID: <1992Oct5.042946.16266@becker.GTS.ORG> Date: 5 Oct 92 04:29:46 GMT Organization: G. T. S., Toronto, Ontario Lines: 294