Combining lots of indexes in WAIS

Thread View: comp.infosystems.wais

2 messages

[T] Back to Threads [V] Tree View [G] Group Articles

2 total messages Started by Nigel Metheringham Tue, 26 Jan 1993 18:07

#362

Author: Nigel Metheringham
Date: Tue, 26 Jan 1993 18:07

33 lines
2026 bytes

If I have a set of gopher subdirectories with WAIS indexing, is it
possible for me to make some form of index entry which would combine
all of the little indexes to give effectively one huge index of the
whole thing (at least everything that is indexed).

Now obviously I could do this by just hitting the whole tree with
the indexer, but that seems like an excessively crude way of doing
things - particularly as I already have indexed subsections and
taking yet another large chunk of disk space redundantly offends the
miser in me :-).

Also re-running the indexer on the whole lot when you actually only
need to reindex a small part and link that index into a bigger overall
index seems like a poor use of resources.  Having separate indexes for
say Unix systems questions and cookery recipes looks like a good idea,
but it would also seem good to be able to just say tell me about all
occurences of "chocolate" in this system!

To me it looks as though WAIS is designed to take a list of database
names (or files) in the .src file, but I have been unable to make this work.
The documentation (?) in source.txt states that database-name is a
list, however it appears that this part of the spec was still under
construction!

Any clues as to how (or even if) I can do this?

	Nigel.


-- 
#   Nigel Metheringham  -- (NeXT) EMail: nigelm@ohm.york.ac.uk    #
#   System Administrator, Electronics Dept, University of York    #
#   York YO1 5DD. Phone: +44 904 432374, Fax: +44 904 432335      #

Re: Combining lots of indexes in WAIS

#368

Author: Eric Lease Morgan
Date: Wed, 27 Jan 1993 15:27

203 lines
8822 bytes

In article <1993Jan26.180729.4139@ohm.york.ac.uk> Nigel Metheringham,
nigelm@ohm.york.ac.uk writes:
>If I have a set of gopher subdirectories with WAIS indexing, is it
>possible for me to make some form of index entry which would combine
>all of the little indexes to give effectively one huge index of the
>whole thing (at least everything that is indexed).
>
>Now obviously I could do this by just hitting the whole tree with
>the indexer, but that seems like an excessively crude way of doing
>things - particularly as I already have indexed subsections and
>taking yet another large chunk of disk space redundantly offends the
>miser in me :-).
>
>Also re-running the indexer on the whole lot when you actually only
>need to reindex a small part and link that index into a bigger overall
>index seems like a poor use of resources.  Having separate indexes for
>say Unix systems questions and cookery recipes looks like a good idea,
>but it would also seem good to be able to just say tell me about all
>occurences of "chocolate" in this system!

I have the same problem. I have a number of electronic journals I
wanted to index. So I did. But I also wanted to search the entire
collection electronic journals. Consequently I have had to reindex
the entire pile. Yes, this is a redundant use of computer resouces,
but hard disk space is cheap for me, AND I wrote a shell script to
make it easier. The shell script follows.

Eric Lease Morgan

=== shell script ===

#
# this file creates a wais index for the specified command
# line input
#
# usage: index <leaf name>
#
# first, IR List Digest
if "$1" == "irld" then
        set theIndex=/usr/users/temp/gopher/data/.wais/$1
        set theDirectory=/usr/users/temp/ftp/pub/stacks/$1
        waisindex -d $theIndex -r -t text -export $theDirectory
        exit
endif
#
#
# Senate bibliographies
if "$1" == "senate" then
        set theIndex=/usr/users/temp/gopher/data/.wais/$1
        set theDirectory=/usr/users/temp/ftp/pub/stacks/$1/.index-data
        waisindex -d $theIndex -r -t para -export $theDirectory
        exit
endif
#
# ALCTS Network News
if "$1" == "ann" then
        set theIndex=/usr/users/temp/gopher/data/.wais/$1
        set theDirectory=/usr/users/temp/ftp/pub/stacks/$1
        waisindex -d $theIndex -r -t text -export $theDirectory
        exit
endif
#
#
# next, Public Access Computer Systems Review
if "$1" == "pacsr" then
        set theIndex=/usr/users/temp/gopher/data/.wais/$1
        set theDirectory=/usr/users/temp/ftp/pub/stacks/$1
        waisindex -d $theIndex -r -t text -export $theDirectory
        exit
endif
#
#
# Citation for Serial Literature 
if "$1" == "csl" then
        set theIndex=/usr/users/temp/gopher/data/.wais/$1
        set theDirectory=/usr/users/temp/ftp/pub/stacks/$1
        waisindex -d $theIndex -r -t text -export $theDirectory
        exit
endif
#
#
# the Newsletter for Serials Pricing Issues
if "$1" == "prices" then
        set theIndex=/usr/users/temp/gopher/data/.wais/$1
        set theDirectory=/usr/users/temp/ftp/pub/stacks/$1
        waisindex -d $theIndex -r -t text -export $theDirectory
        exit
endif
#
#
# Publics Access Computer Systems News   
if "$1" == "pacsn" then
        set theIndex=/usr/users/temp/gopher/data/.wais/$1
        set theDirectory=/usr/users/temp/ftp/pub/stacks/$1
        waisindex -d $theIndex -r -t text -export $theDirectory
        exit
endif
#
#
# how about ALAWON?
if "$1" == "alawon" then
        set theIndex=/usr/users/temp/gopher/data/.wais/$1
        set theDirectory=/usr/users/temp/ftp/pub/stacks/$1
        waisindex -d $theIndex -r -t text -export $theDirectory
        exit
endif
#
#
# and now Current Cites
if "$1" == "currentc" then
        set theIndex=/usr/users/temp/gopher/data/.wais/$1
        set theDirectory=/usr/users/temp/ftp/pub/stacks/$1
        waisindex -d $theIndex -r -t text -export $theDirectory
        exit
endif
#
#
#
# index all the titles in the their own indexes
if "$1" == "all" then
#
# pacs news
        set theIndex=/usr/users/temp/gopher/data/.wais/pacsn
        set theDirectory=/usr/users/temp/ftp/pub/stacks/pacsn
        waisindex -d $theIndex -r -t text -export $theDirectory
# pacs review
        set theIndex=/usr/users/temp/gopher/data/.wais/pacsr
        set theDirectory=/usr/users/temp/ftp/pub/stacks/pacsr
        waisindex -d $theIndex -r -t text -export $theDirectory
# alawon
        set theIndex=/usr/users/temp/gopher/data/.wais/alawon
        set theDirectory=/usr/users/temp/ftp/pub/stacks/alawon
        waisindex -d $theIndex -r -t text -export $theDirectory
# current cites
        set theIndex=/usr/users/temp/gopher/data/.wais/currentc
        set theDirectory=/usr/users/temp/ftp/pub/stacks/currentc
        waisindex -d $theIndex -r -t text -export $theDirectory
# newsletter on serials pricing issues
        set theIndex=/usr/users/temp/gopher/data/.wais/prices
        set theDirectory=/usr/users/temp/ftp/pub/stacks/prices
        waisindex -d $theIndex -r -t text -export $theDirectory
# irl digest
        set theIndex=/usr/users/temp/gopher/data/.wais/irld
        set theDirectory=/usr/users/temp/ftp/pub/stacks/irld
        waisindex -d $theIndex -r -t text -export $theDirectory
# aclts netork news 
        set theIndex=/usr/users/temp/gopher/data/.wais/ann
        set theDirectory=/usr/users/temp/ftp/pub/stacks/ann
        waisindex -d $theIndex -r -t text -export $theDirectory
# citations of serial literature
        set theIndex=/usr/users/temp/gopher/data/.wais/csl
        set theDirectory=/usr/users/temp/ftp/pub/stacks/csl
        waisindex -d $theIndex -r -t text -export $theDirectory
# 
# ell
        set theIndex=/usr/users/temp/gopher/data/.wais/ell
        set theDirectory=/usr/users/temp/ftp/pub/stacks/alawon
        waisindex -d $theIndex -r -t text -export $theDirectory
        set theDirectory=/usr/users/temp/ftp/pub/stacks/currentc
        waisindex -d $theIndex -r -a -t text -export $theDirectory
        set theDirectory=/usr/users/temp/ftp/pub/stacks/irld
        waisindex -d $theIndex -r -a -t text -export $theDirectory
        set theDirectory=/usr/users/temp/ftp/pub/stacks/pacsr
        waisindex -d $theIndex -r -a -t text -export $theDirectory
    set theDirectory=/usr/users/temp/ftp/pub/stacks/prices
    waisindex -d $theIndex -r -a -t text -export $theDirectory
    set theDirectory=/usr/users/temp/ftp/pub/stacks/pacsn
    waisindex -d $theIndex -r -a -t text -export $theDirectory
    set theDirectory=/usr/users/temp/ftp/pub/stacks/csl
    waisindex -d $theIndex -r -a -t text -export $theDirectory
        set theDirectory=/usr/users/temp/ftp/pub/stacks/ann
    waisindex -d $theIndex -r -a -t text -export $theDirectory
        exit
endif

#
#
# now the moment you've all been waiting for... create
# the "Electronic Library Literature Index"
if "$1" == "ell" then
        set theIndex=/usr/users/temp/gopher/data/.wais/ell
        set theDirectory=/usr/users/temp/ftp/pub/stacks/alawon
        waisindex -d $theIndex -r -t text -export $theDirectory
        set theDirectory=/usr/users/temp/ftp/pub/stacks/currentc
        waisindex -d $theIndex -r -a -t text -export $theDirectory
        set theDirectory=/usr/users/temp/ftp/pub/stacks/irld
        waisindex -d $theIndex -r -a -t text -export $theDirectory
        set theDirectory=/usr/users/temp/ftp/pub/stacks/pacsr
        waisindex -d $theIndex -r -a -t text -export $theDirectory
        set theDirectory=/usr/users/temp/ftp/pub/stacks/prices
        waisindex -d $theIndex -r -a -t text -export $theDirectory
        set theDirectory=/usr/users/temp/ftp/pub/stacks/csl
        waisindex -d $theIndex -r -a -t text -export $theDirectory
        set theDirectory=/usr/users/temp/ftp/pub/stacks/pacsn
        waisindex -d $theIndex -r -a -t text -export $theDirectory
        set theDirectory=/usr/users/temp/ftp/pub/stacks/ann
        waisindex -d $theIndex -r -a -t text -export $theDirectory
        exit
endif
#
#
# echo the input error
echo "Usage: index <nspi | irld | pacsr | alawon | currentc | ell>"

Thread Navigation

This is a paginated view of messages in the thread with full content displayed inline.

Messages are displayed in chronological order, with the original post highlighted in green.

Use pagination controls to navigate through all messages in large threads.

Back to All Threads