Thread View: news.software.b
1 messages
1 total messages
Started by Brian Gregory
Wed, 07 Sep 1994 01:09
NewsFeed analyzer script (1/1)
Author: Brian Gregory
Date: Wed, 07 Sep 1994 01:09
Date: Wed, 07 Sep 1994 01:09
215 lines
9152 bytes
9152 bytes
This little script works with the C news log file to analyze the
relative contributions of multiple feeds. If you have a single
feed for news, the script will still work, but its output will not
be as exciting :-) It may work with other log file formats with
or without a little modification... let me know. It expects the
log on stdin, and writes the report to stdout.
Just about every amount and percentage available is printed. You
will find that some stats are more useful than others. Sometime,
if I get a chance, I may post a man page and a guide to what the
numbers mean from a qualitative point of view, but I am doing well
to just get this posted, for now!
Also included is sample output, and a context diff to apply to the
C news "newsdaily" script to automatically call the script once a day,
mailing the results to the news administrator.
Brian Gregory
bgregory@megatest.com
---------------------------------8<--cut here--8<-----------------------------
#! /bin/sh
# This is a shell archive. Remove anything before this line, then feed it
# into a shell via "sh file" or similar. To overwrite existing files,
# type "sh file -c".
# The tool that generated this appeared in the comp.sources.unix newsgroup;
# send mail to comp-sources-unix@uunet.uu.net if you want that tool.
# Contents: nlogstats newsdaily.cdiff sample.output
# Wrapped by bgregory@kax on Tue Sep 6 17:45:28 1994
PATH=/bin:/usr/bin:/usr/ucb ; export PATH
echo If this archive is complete, you will see the following message:
echo ' "shar: End of archive."'
if test -f 'nlogstats' -a "${1}" != "-c" ; then
echo shar: Will not clobber existing file \"'nlogstats'\"
else
echo shar: Extracting \"'nlogstats'\" \(3161 characters\)
sed "s/^X//" >'nlogstats' <<'END_OF_FILE'
X#!/bin/sh
X
X#
X# compute logstats from news log
X#
X# looks at field 4 for name of site
X# field 5 for + (added), j (junked), - (skipped)
X#
X# this is the format that CNews uses - I can't remember if INN and B
X# logs are the same. have a look, and modify to taste.
X#
X# ratio numerators ratio denominators
X# a added from site a total added
X# j junked from site j total junked
X# s skipped from site s total skipped
X# n new from site (a+j) n total new (a+j)
X# b bagged from site (j+s) b total bagged (j+s)
X# f total articles from feed
X# t total articles
X#
X# Brian Gregory
X# bgregory@megatest.com, or bgregory@rahul.net
X#
X
Xecho ''
Xecho "News Feeds Report -- `date`"
Xecho ''
Xawk '
X $5 == "+" { totarts++; arts[$4]++; totadded++; added[$4]++; }
X $5 == "j" { totarts++; arts[$4]++; totjunked++; junked[$4]++; }
X $5 == "-" { totarts++; arts[$4]++; totskipped++; skipped[$4]++; }
X END {
X print "added:";
X for ( feed in added )
X {
X totnew[feed] += added[feed];
X printf (" %12s %6d %7.3g%% (a/a) %7.3g%% (a/f) %7.3g%% (a/t)\n", feed, added[feed], (added[feed]/totadded)*100.0, (added[feed]/arts[feed])*100.0, (added[feed]/totarts)*100.0);
X }
X printf ("\n %12s %6d %7.3g%% (of all)\n\n", "TOTAL", totadded, (totadded/totarts)*100.0);
X
X print "junked:";
X for ( feed in junked )
X {
X totnew[feed] += junked[feed];
X totbag[feed] += junked[feed];
X printf (" %12s %6d %7.3g%% (j/j) %7.3g%% (j/f) %7.3g%% (j/t)\n", feed, junked[feed], (junked[feed]/totjunked)*100.0, (junked[feed]/arts[feed])*100.0, (junked[feed]/totarts)*100.0);
X }
X printf ("\n %12s %6d %7.3g%% (of all)\n\n", "TOTAL", totjunked, (totjunked/totarts)*100.0);
X
X print "skipped:";
X for ( feed in skipped )
X {
X totbag[feed] += skipped[feed];
X printf (" %12s %6d %7.3g%% (s/s) %7.3g%% (s/f) %7.3g%% (s/t)\n", feed, skipped[feed], (skipped[feed]/totskipped)*100.0, (skipped[feed]/arts[feed])*100.0, (skipped[feed]/totarts)*100.0);
X }
X printf ("\n %12s %6d %7.3g%% (of all)\n\n", "TOTAL", totskipped, (totskipped/totarts)*100.0);
X
X print "added+junked (new):";
X allnew = totadded + totjunked ;
X for ( feed in totnew )
X {
X printf (" %12s %6d %7.3g%% (n/n) %7.3g%% (n/f) %7.3g%% (n/t)\n", feed, totnew[feed], (totnew[feed]/allnew)*100.0, (totnew[feed]/arts[feed])*100.0, (totnew[feed]/totarts)*100.0);
X }
X printf ("\n %12s %6d %7.3g%% (of all)\n\n", "TOTAL", allnew, (allnew/totarts)*100.0);
X
X print "junked+skipped (bag):"
X allbag = totjunked + totskipped ;
X for ( feed in totbag )
X {
X printf (" %12s %6d %7.3g%% (b/b) %7.3g%% (b/f) %7.3g%% (b/t)\n", feed, totbag[feed], (totbag[feed]/allbag)*100.0, (totbag[feed]/arts[feed])*100.0, (totbag[feed]/totarts)*100.0);
X }
X printf ("\n %12s %6d %7.3g%% (of all)\n\n", "TOTAL", allbag, (allbag/totarts)*100.0);
X
X print "total:"
X for ( feed in arts )
X {
X printf (" %12s %6d %7.3g%%\n", feed, arts[feed], (arts[feed]/totarts)*100.0);
X }
X printf ("\n %12s %6d\n\n", "TOTAL", totarts);
X }
X'
END_OF_FILE
if test 3161 -ne `wc -c <'nlogstats'`; then
echo shar: \"'nlogstats'\" unpacked with wrong size!
fi
chmod +x 'nlogstats'
# end of 'nlogstats'
fi
if test -f 'newsdaily.cdiff' -a "${1}" != "-c" ; then
echo shar: Will not clobber existing file \"'newsdaily.cdiff'\"
else
echo shar: Extracting \"'newsdaily.cdiff'\" \(542 characters\)
sed "s/^X//" >'newsdaily.cdiff' <<'END_OF_FILE'
XThis context diff, applied with patch, will update the C-News "newsdaily"
Xprogram to add a call to nlogstats. cd to where newsdaily is first! You
Xmay also have to change the path to nlogstats.
X
X*** newsdaily.old Wed Aug 10 13:36:07 1994
X--- newsdaily Wed Aug 10 13:35:22 1994
X***************
X*** 31,36 ****
X--- 31,37 ----
X # keep one generation of log -- it's big
X rm -f log.o
X mv log log.o && >log
X+ /usr/lib/newsbin/maint/nlogstats < log.o | mail $gurus
X
X # keep several generations of batchlog for statistics
X if test -f batchlog
END_OF_FILE
if test 542 -ne `wc -c <'newsdaily.cdiff'`; then
echo shar: \"'newsdaily.cdiff'\" unpacked with wrong size!
fi
# end of 'newsdaily.cdiff'
fi
if test -f 'sample.output' -a "${1}" != "-c" ; then
echo shar: Will not clobber existing file \"'sample.output'\"
else
echo shar: Extracting \"'sample.output'\" \(1882 characters\)
sed "s/^X//" >'sample.output' <<'END_OF_FILE'
XNews Feeds Report -- Wed Aug 10 08:20:03 PDT 1994
X
Xadded:
X dlb 19 0.0339% (a/a) 95% (a/f) 0.0285% (a/t)
X megatest 599 1.07% (a/a) 99.8% (a/f) 0.899% (a/t)
X pagesat.net 35982 64.2% (a/a) 91.6% (a/f) 54% (a/t)
X uupsi 19460 34.7% (a/a) 72.8% (a/f) 29.2% (a/t)
X mcws 2 0.00357% (a/a) 100% (a/f) 0.003% (a/t)
X
X TOTAL 56062 84.1% (of all)
X
Xjunked:
X pagesat.net 2894 49.1% (j/j) 7.37% (j/f) 4.34% (j/t)
X uupsi 3005 50.9% (j/j) 11.2% (j/f) 4.51% (j/t)
X
X TOTAL 5899 8.85% (of all)
X
Xskipped:
X dlb 1 0.0214% (s/s) 5% (s/f) 0.0015% (s/t)
X megatest 1 0.0214% (s/s) 0.167% (s/f) 0.0015% (s/t)
X pagesat.net 398 8.54% (s/s) 1.01% (s/f) 0.597% (s/t)
X uupsi 4263 91.4% (s/s) 15.9% (s/f) 6.4% (s/t)
X
X TOTAL 4663 7% (of all)
X
Xadded+junked (new):
X dlb 19 0.0307% (n/n) 95% (n/f) 0.0285% (n/t)
X megatest 599 0.967% (n/n) 99.8% (n/f) 0.899% (n/t)
X pagesat.net 38876 62.7% (n/n) 99% (n/f) 58.4% (n/t)
X uupsi 22465 36.3% (n/n) 84.1% (n/f) 33.7% (n/t)
X mcws 2 0.00323% (n/n) 100% (n/f) 0.003% (n/t)
X
X TOTAL 61961 93% (of all)
X
Xjunked+skipped (bag):
X dlb 1 0.00947% (b/b) 5% (b/f) 0.0015% (b/t)
X megatest 1 0.00947% (b/b) 0.167% (b/f) 0.0015% (b/t)
X pagesat.net 3292 31.2% (b/b) 8.38% (b/f) 4.94% (b/t)
X uupsi 7268 68.8% (b/b) 27.2% (b/f) 10.9% (b/t)
X
X TOTAL 10562 15.9% (of all)
X
Xtotal:
X dlb 20 0.03%
X megatest 600 0.901%
X pagesat.net 39274 58.9%
X uupsi 26728 40.1%
X mcws 2 0.003%
X
X TOTAL 66624
END_OF_FILE
if test 1882 -ne `wc -c <'sample.output'`; then
echo shar: \"'sample.output'\" unpacked with wrong size!
fi
# end of 'sample.output'
fi
echo shar: End of archive.
exit 0
Thread Navigation
This is a paginated view of messages in the thread with full content displayed inline.
Messages are displayed in chronological order, with the original post highlighted in green.
Use pagination controls to navigate through all messages in large threads.
Back to All Threads