🚀 go-pugleaf

Forwarded message:
>From lexical@crl.nmsu.edu Wed Nov 24 20:25:47 1993
Date: Wed, 24 Nov 93 12:15:13 MST
Message-Id: <9311241915.AA28668@NMSU.Edu>
From: lexical@crl.nmsu.edu (Consortium for Lexical Research)
To: jbowyer@cis.vutbr.cz
Subject: Consortium for Lexical Research Information
Cc: lexical@crl.nmsu.edu


		The Consortium for Lexical Research (CLR)


		      Rio Grande Research Corridor
		      Computing Research Laboratory
		      New Mexico State University
 	      	    Box 30001, Las Cruces, NM 88003




			  lexical@nmsu.edu
			   (505) 646-5466
			Fax: (505) 646-6218


Work in computational linguistics has reached the point where the
performance of many natural language processing systems is limited by
a ``lexical bottleneck''.  That is, such systems could handle much more
text and produce much more impressive application results were it not
for the fact that their lexicons are too small.

The Association for Computational Linguistics proposed that a
Consortium for Lexical Research (CLR) be established, with funding
from DARPA.  The CLR was set up in July of 1991 and is sited at the
Computing Research Laboratory, New Mexico, USA, under its Director,
Yorick Wilks, Associate Director Louise Guthrie, and an ACL committee
consisting of Roy Byrd, Ralph Grishman, Mark Liberman and Don Walker

Any individual or organization wishing to make initial contact with
the CLR and review its procedures, holdings, agreements etc. should
send email to the address above (or write to the mailing address or
fax).


OBJECTIVE

The objective of the Consortium for Lexical Research is to act as a
clearing house, in the US and internationally, for lexical data and
software.  It shares lexical data and tools used to perform research
on machine-readable dictionaries and lexicons, as well as
communicating the results of that research, thus accelerating the
scale and speed of the development of natural language understanding
programs via standard lexicons and software.

A basic premise of the proposal for cooperation on lexical research is
that the research must be ``precompetitive''.  That is, the CLR does not
have as its goal the creation of commercial products.  The goal of
precompetitive research is to augment our understanding of what
lexicons contain and, specifically, to build computational lexicons
having those contents. Members of the Consortium contribute to a
repository and withdraw resources from it in order to perform their
research.  There is no requirement that withdrawals be compensated by
contributions in kind. Members are charged an annual fee to help
support the cost of running the CLR.

The task of the CLR is primarily to facilitate research, making
available to the whole natural language processing community certain
resources now held only by a few groups that have special
relationships with companies or dictionary publishers. There is also
an underlying theoretical assumption or hope: that the contents of
major lexicons are very similar, and that some neutral, or
``polytheoretic,'' form of the information they contain can be at
least a research goal, and would be a great boon if it could be
achieved.  The CLR as far as is practically possible accepts
contributions from any source, regardless of theoretical orientation,
and makes them available as widely as possible for research.

APPROACH

We have set up publicity networks to attract interested donors of
materials and members. From there we have defined agreements for
donors and members, a fee structure and set up computer networking
facilities to carry out donations and withdrawls of materials.

A major activity of the CLR is to negotiate agreements with
``providers'' on reassuring and advantageous terms to both suppliers
and researchers. Major funders of work in this area in the US have
indicated interest in making participation in the CLR a condition for
financial support of research.

CLR RESOURCES AND SERVICES

The Computing Research Lab (CRL) has a range of machines appropriate
for advanced computing on dictionaries (including the construction of
large-scale matrices): DARPA-supported access to a Connection Machine,
and a Sequent Symmetry, and an IBM-ACE parallel machine, as well as
network of UNIX workstations. The Consortium has access to an
appropriate range of large-scale storage machines, and capacities for
accepting and providing materials by network, tape and CD.

The CLR archives include two main areas: a public area and one for
members only.  These are the repository for such lexical items as word
lists; published dictionaries; specialized terminology; statistical
data; synonyms, antonyms, hypernym, pertainyms,etc.; and phrase lists.
They also include tools for lexical data base management, lexical
query, text analysis, dictionary encoding, and dictionary definition
sense taggers.

Repository management involves cataloging and storing material in
disparate formats, and providing for their retransmission (with
conversion, where appropriate tools exist).  In addition, a library of
documentation describing the repository's contents and containing
research papers resulting from projects that use the material is
maintained.  A brief description of the services provided is as
follows:

a.  CLR provides a catalog of, and acts as a clearinghouse for,
    utilities programs which have been written for existing online
    lexical data.

b.  It also provides information for access to repositories of
    corpus-manipulation tools held elsewhere.

c. CLR compiles a list of known mistakes, misprints, etc. that
   occur in each of the major published sources (dictionaries etc.).

d. CRL distributes a monthly newsletter highlighting materials available in
   the consortium's archives.


PROGRESS

Progress during the year has been achieved in the three areas that
correspond to these data bases:


	. Building the contact base

	. Finalizing, membership  provider agreements

	. Setting up the online database archives of the consortium


Building the contact base has entailed publicizing the Consortium and
its purposes to the research community.  A list of addresses was
compiled.  Printed and email announcements were composed and
distributed, with an email address for responses.  The announcement
was posted in relevant newsletters and journals.  To date there have
been three large-scale mailouts.

Response has been enthusiastic and continuous.  Conference
presentations and personal contacts concerning the Consortium have
included ACH/ALLC and others, in the US, Europe, Japan, and elsewhere.
Particular attention has been directed to reaching core researchers,
building the current mailing list to over 500.

The mail directed to lexical@nmsu.edu or to the Consortium staff
themselves has been answered individually, with queries about what
the people are interested in and what they might like to contribute
to the archives.  The responses indicate that there is a great
variety of lexically related software needed and available.

Setting up procedures for the receiving of materials and their legal
protections has led to formulation of drafts and membership and
provider agreements.   The agreements have been finalized and legally
approved at NMSU and memberships have been accepted (see below).  The
major problem which has meant an enormous amount of negotiation with
major publishers, has been creating a general form of provider
agreement that captures the interests of the major dictionary
publishers in a  general  way, not tailored to each.  As we report
below, there has been substantial progress.

Facilities for receiving and providing archival materials have also
been set up.  The directories and file transfer procedures are in
place.  Besides online access and deposit of materials, tape and
diskette have been anticipated, all in a number of formats.
Heavy security has been set up for heavily encumbered
materials.

Software for handling and classifying correspondence has been written
which will permit cross-classification and sorting of member entries
to match user needs and user offerings.  Written in-house reports of
CLR operations have been made regularly.  Other software which will
enable handling of materials in varied scripts is also under
development, so that materials with a variety of orthographies can be
transmitted, etc. (Scripts incl. Japanese, Chinese, Korean, Cyrillic,
some Indic, and other scripts.  This is an item of current interest in
the lexical research community.)

One of the goals of the Consortium is to make electronic versions of
dictionaries and thesauri available within the research community, and
discussions and visits have continued with Oxford University Press,
Collins Publishers, and Longman Group Limited.  In brief, we now have
arrangements with Harper-Collins which facilitate the purchase of
their machine readable dictionaries by members. We expect to reach
that stage soon with Longmans and Oxford.  All this has been slower
than we hoped.  As these negotiations are in final stages, we are
turning to the major US publishers.

Materials in the CLR archives are secured by a ``protection in depth''
scheme.  On the most public level, freely distributable materials are
available via anonymous ftp.  We maintain only a log of recent
contacts for these materials.  For the protection of lightly
encumbered materials, we provide members of the Consortium with
individual ftp accounts by which they can access archived material.
These materials are kept separate from the publicly accessible
materials, and are protected by standard ftp accounting and permission
software.  These accounts are only valid for ftp transfer, and their
passwords are changed regularly by the Consortium.

On the highest level of security, members who have received permission
from the supplier of heavily encumbered material are given a special
temporary ftp account which allows them access to encrypted versions
of the heavily encumbered material.  To obtain the material illegally,
not only would the normal file permissions scheme have to be
subverted, but a highly secure cryptographic system must be defeated.
As an additional security measure, the files are periodically
re-encrypted with freshly generated random passwords.  At no time is
an unencrypted version of the material stored in the ftp accessible
archives.

CLR is now the center of distribution of the data required by
participants in the Fifth Message Understanding Conference (MUC-5).
These research groups are all working on Information Extraction from
actual texts.  The performance of their systems will be evaluated in
August 1993.   CLR's facilities provided a secure and carefully
monitored means of distributing the large volumes of data, such as
gazetteers, rules, training texts, etc., required to build the IE
systems.

The Consortium for Lexical Research currently has in its public
archives contributions of 145 different packages for lexical use.  Of
these 145, eighty are restricted to MUC-5, eleven are restricted to
``members-only'' (lightly encumbered) access and the remainder are
available to anyone (unencumbered). We have placed in the heavily
encumbered category of the archive a recent version of the Alvey
tools.  JUMAN, the segmenter and part-of-speech tagger for Japanese
and the Xerox part-of-speech tagger have both been placed in the
lightly encumbered category, for members-only access.  The materials
in the MUC-5 area include a database and tools for assessing the
message understanding software designed and used by MUC members.

Contributions include thesauri (Roget's Thesaurus and WordNet),
dictionaries (Collins English-Spanish Bilingual Dictionary and Longman's
Dictionary of Contemporary English), wordlists (Gazetteer, Proper
Names, and the Standard Industrial Classification Manual), technical
reports (all software and lexica include technical reports relevant to
the materials being investigated), morphological analyzers (ENGLEX),
lexical parsers (SGML parser for text processing), typesetting
software (Indian, Arabic, Korean, Vietnamese, Japanese, Chinese and
Tibetan fonts), dictionary interface tools (BYU's Morphogen), text
analyzers (Interlinear text processor from SIL), and a phonological
programming language.

Approximately 50 other contributions are in various stages of
negotiations at this time, including the text of German, Greek, and
Latin Vulgate Bibles, the American Heritage Dictionary, POPX, a
Russian-English dictionary of political terms, and a large Chinese-English
dictionary.

The CLR currently has 48 member organizations which include 22
domestic universities, 4 foreign universities, 8 government agencies
and 18 commercial companies(including Apple, Microsoft, Xerox).
MUC-5 participants have all joined the CLR, providing valuable
software and materials. Currently 4 more memberships are pending
signature by NMSU and 8 organizations have indicated that they would
like to join, but we have not yet received membership agreements from
them.

Our most recent and major events are two workshops.  The first was a
CLR workshop in January of 1992 which brought publishers, researchers,
funders and users of lexical materials for three days to New Mexico,
with the support of the ACL and the NSF.  The major issues of
lexicon-reusability and the problems of copyright/ownership of
materials were discussed extensively and full notes of the discussions
and presentations at the workshop will be distributed very soon as a
technical memorandum (The First Workshop at the Consortium for Lexical
Research. MCCS-92-243 from Computing Research Laboratory, Box 3CRL,
NMSU, Las Cruces, NM 88001, price $7). The second workshop entitled
U.S./European Cooperation took place in January 1993.  The workshop
was sponsored jointly by NSF and the European Commission to discuss
international cooperation of lexical computation.  Twenty-five
researchers participated in the workshop.  (MCCS-93-254 from Computing
Research Laboratory, Box 3CRL, NMSU, Las Cruces, NM 88001, price $5).

FUTURE PLANS

Our plans for the coming year are to expand membership and holdings
steadily over the year, and progress toward our long-term goal of
establishing the Consortium as a self-supporting entity.  We hope to
do this by signing agreements with other dictionary publishers to make
their products available through the CLR and by more actively seeking
contributions of software or data from researchers.  Our membership
drive will focus on obtaining more international members, and members
from the community of researchers and language specialists who may not
have everyday access to the internet.

--------------------------------------------------------------------------
Here is our Catalog:

.nf




                       CONSORTIUM FOR LEXICAL RESEARCH


This is the CLR paragraph description catalog as of: Mon., Nov. 22,
1993.

This is the CLR materials catalog. These entries are short paragraph
length descriptions of the holdings available in the CLR archives.

This catalog is available through our ftp site. FTP to clr.nmsu.edu,
login as "anonymous", and when asked for a password use your complete
email address. The file you need to get is "catalog" for a simple
ascii text version of this document; or "catalog.ps" for the
postscript version that you are now reading.

 If you need additional information on a specific program in this
catalog, please use the "More Info" file that you will find in the
info directory. Also, the "catalog-short" file contains pointers to
each programs documentation or expanded information files.

Please feel free to contact us if you have any questions or comments
related to this catalog. And please keep in mind that CLR is always
looking to include new materials that are of use to the lexical
research community. If you have a piece of software that you would
like to make available, please contact us.

------------------------------------------------------------------------------
  CONSORTIUM FOR LEXICAL RESEARCH
------------------------------------------------------------------------------

  Computing Research Lab
  New Mexico State University
  Las Cruces, NM 88003

   phone: (505) 646-5466
     fax: (505) 646-6218
   email: lexical@nmsu.edu
  bitnet: lexical at nmsu

---------------------------------------------------------------

----------------------------------------------------------------

AFGREP

Ftp Directory:     pub/tools/general/

Afgrep is a variant of the mgrep algorithm from the agrep package.  It
provides a high speed multi-string search of a file in the same manner
as the fgrep program.  It also has fewer arbitrary limitations. More
Info: info/0068.

----------------------------------------------------------------

AGREP

Ftp Directory:     pub/tools/general/

Agrep is a tool for high speed text searching, allowing for errors.
Agrep is similar to fgrep, egrep, grep, but is more general and
usually much faster.  The three most significant features of agrep

1. The ability to search for approximate patterns
2. Agrep is record oriented, not just line oriented
3. Multiple patterns can be specified with logical operators (AND and
   OR) for queries. More Info: info/0067.

----------------------------------------------------------------

The Alvey Natural Language Tools

Ftp Directory:     members-only/tools/ling-analysis/syntax/alvey/
Ftp Directory:     members-only/tools/ling-analysis/syntax/alvey-4.0/

The Alvey Natural Language Tools (ANLT) is an initiative to develop a
general-purpose wide-coverage morphological and syntactic analyzer for
English. The ANLT system comprises a morphology and lexicon system
(Pulman et al. 1988), a chart parser for unification-based formalisms
(Phillips & Thompson 1987), word and sentence grammars of English, and
a substantial English lexicon (Carroll & Grover 1989). The ANLT system
has been integrated with a software environment designed to facilitate
rapid development of substantial, wide-coverage grammars (hereafter
the Grammar Development Environment (GDE), Carroll et al. 1988).
Software is available for 500 ecu.

Alvey Tools is now available in an updated version alvey-4.0.
More Info:info/0050 and info/0084

----------------------------------------------------------------

ARABTEX

Ftp Directory:     pub/typesetting/tex/arabic/

ArabTeX is a package extending the capabilities of TeX/LaTeX to generate
the arabic writing from an ASCII transliteration for texts in several
languages using the arabic script.

It consists of a TeX macro package and an arabic font in several sizes,
presently only available in the Naskhi style. ArabTeX will run with Plain
TeX and also with LaTeX; other additions to TeX have not been tried.

ArabTeX is primarily intended for generating the arabic writing, but the
scientific transcription can be also easily generated. For other languages
using the arabic script limited support is available.

This package also has the option of typesetting fully vocalized text
(with vowel diacritics as in the Q'uran) and/or transcriptions as
well. More Info: info/0009.

----------------------------------------------------------------

ARCSGML 1.0

Ftp Directory:     pub/tools/sgml/arcsgml/

ARCSGML is a set of tools for setting up and working with text that
is tagged with your own specialized tags in SGML format.  The tags
permit you to label text structures (such as Part-of-speech, syntactic,
morphological, semantic, or discourse structures).  The parser is
for selectively pulling out corresponding tagged pieces of text.
The ARCSGML toolkit is for use in developing conforming SGML parsers,
systems, and applications.  A validator (for checking your tagged text)
is supplied.  It supports the standard SGML reference concrete syntax
(beginning from 1983) in all features except LINK, CONCUR, and SUBDOC
(although some hooks are in place to get you started on these).  [The
package was originally written to validate the 1983 working draft of
the SGML standard, and was subsequently maintained to track the standard
through its final phases of development, culminating in the amendment.]

Executable sourcecode programs for versions for PC and Unix C (MSDOS REXX,
MSDOS C, and Unix C) are provided. More Info: info/0005.

----------------------------------------------------------------

ATeX

Ftp Directory:     pub/typesetting/tex/arabic/

ATeX is a simple extension to LaTeX allowing the user to typeset, edit
and print documents in Arabic, or in a combination of English and
Arabic.  Simple updates to one of the files will allow one to get ATeX
to do Farsi, Urdu, etc., since the extra-Arabic characters for these
are present in the font used.  Also supplied with this package is the
MSDOS executable for using it on PCs. More Info: info/0010.

----------------------------------------------------------------

CGPARSER

Ftp Directory:     pub/tools/ling-analysis/semantic/cgparser/

CGParser is an implementation of a linear parser of Conceptual Graphs
as described in John Sowa's book _Conceptual Graphs_.  It was
written using the YACC general-purpose language utility. Some simple
and more interesting examples are provided for testing purposes.
More Info: info/0002.

----------------------------------------------------------------
Chemical Abstracts

Ftp Directory:	members-only/corpora/Chem_Abs/

The Chemical Abstracts Service, a Division of the American Chemical
Society, has donated a series of chemical abstracts to the CLR to promote
the use of chemical text in the Natural Language Processing research
community. These are a collection of 32 files containing 100,000
chemical abstracts pertaining to recent research in
Chemistry. The total word count for these files amounts to 11,039,434
words.
These files are classified as lightly encumbered materials.  Please
write lexical@nsmu.edu to request permission for their use.
More Info: info/0087.

----------------------------------------------------------------
CMU-DICT

Ftp Directory:     members-only/lexica/CMU-DICT.0.1/

This pronunciation dictionary was generated at Carnegie Mellon
University for the purpose of tuning and developing speech
understanding systems.  The dictionary contains approximately 100k
words and their transcriptions.  Several independent sources were used
in its development, such as , the UCLA Shoup dictionary, a subset of
the Dragon dictionary, and various other dictionaries, that were
hand-built, syntehsizer-generated, or generated with Orator and
Mitalk. Robert Weide and Peter Jansen from CMU developed this
dictionary for any purpose, commercial or otherwise.
More Info: info/0091.

----------------------------------------------------------------

COGNATE

Ftp Directory:     pub/tools/ling-analysis/cognate/

COGNATE is the implementation of a prototype algorithm for identifying
related words across languages.  Given the same list of words in two
different languages, COGNATE will determine which words are likely to
be regularly derivable from each other, and which are not.  COGNATE is
only available as an MSDOS executable and comes with Dutch, English,
and German word lists. More Info: info/0055.

----------------------------------------------------------------

Conc

Ftp Directory:     pub/tools/concordances/

Conc produces concordances of texts. A concordance consists
of a list of the words in the text with a short section of the
context that precedes and follows each word. Conc also produces
an index, consisting of a list of the distinct words in the text,
each with the number of times it occurs and a list of the places
where it occurs. Conc displays the original text, the
concordance, and the index each in its own window. Clicking on a
word in any one of the three windows causes the other two windows
to display the entries for the same word. More Info: info/0047.

----------------------------------------------------------------

Countries and Currencies

Ftp Directory:	   members-only/lexica/country.currency.codes/

This is a collection of text file lists of countries, country codes,
currencies, and currency codes.  Included are the SWIFT currency codes
with files arranged alphabetically by country, and alphabetically by
currency code.  There is also a file of single and plural currency
names which was extracted from the CIA World Factbook. In addition
there are files of ISO and FIPS country listings with codes.  This
collection was provided by the Department of Defense in connection
with the MUC-5 conference. More Info: info/0099.

---------------------------------------------------------------

CRL-Proper Name Tagging  (Computing Resarch Labs, New Mexico State
			  University)

Ftp Directory: members-only/tools/text analysis/hand-tagged/

Collection of hand-tagged English and Japanese texts and an
accompanying evaluation program for comparison of machine tagging.
Texts were hand tagged for human names and organizations names.
Scoring software allows evaluation of machine proper name tagging of
same texts.  Scoring report gives a value for recall and precision.
More Info: info/0061.


________________________________________________________________

CRL-TOOLS (Computing Resarch Labs, New Mexico State University)

Ftp Directory: members-only/tools/text-analysis/word-in-context.tools/

Collection of tools to segment text and count the pieces.  Programs
are included that segment English text into sentences, words and
word-level n-grams.  In addition programs to count strings are
provided, as well as programs which can statistically analyse the
results of these counts.  These utilities can be used with other
natural languages if a word extraction program is available for use as
a pre-processor.  These programs were used to produce the data
described in the paper ``Accurate methods for the statistics of
surprise and coincidence'' which appeared in the March 1993 issue of
Computational Linguistics.  All programs are written in C and have
been run under SunOS 4.1.1, but should be portable to other
environments which support long integers as default and large arrays.
More Info: info/0046.

----------------------------------------------------------------

CRL- Word Lists (Computing Resarch Labs, New Mexico State University)

Ftp Directory:     members-only/lexica/wordlists/
Ftp Directory:	   members-only/lexica/wordlists/corporations/

These word lists were produced from various sources including the
combined student drectories of Cornell, UNC Chapel Hill, and NMSU, for
the TIPSTER project. Lists of first names, last names,
and personal titles are provided. In the corporations directory is a
list of over 50,000 corporations and their countries of origin, along
with a glossary of corporate abbreviations and designators.
More Info: info/0061.

----------------------------------------------------------------

The DIMAP-2 package and demo

Ftp Directory:     pub/tools/dict-maintenance/dimap-2.pc/
Ftp Directory:     pub/tools/dict-maintenance/dimap-2.sun-4/

DIMAP-2 is a set of PC dictionary creation and maintenance utilities
that are modeled loosely and more flexibly on the utilities developed
by Tom Ahlswede and described at ACL 85.  DIMAP-2 also comes with a
linked machine-readable dictionary (Merriam-Webster Concise Electronic
Dictionary, with 80,000 entries).  The design of the software is
intended to facilitate the development of NLP lexicons in any
formalism (following Allen, HPSG, lexical conceptual structures, ECD;
using Lisp, Prolog, ASCII), which then belong to the developer.
The software itself is obtainable for single users at $125 and for
academic institutions at $500.  Commercial copies are available for
$2,400 for one copy and $6,000 for a site license.  A demonstration
version is also available.  The software is currently being ported to
the UNIX environment.

An MSDOS executable demo is provided and some of the DIMAP-2 features
are restricted for the demo.

Dimap-2 is also available for the Sun-4. More Info: info/0032.
----------------------------------------------------------------

EDICTJ

Ftp Directory:     pub/lexica/edictj/

EDICTJ is a small public-domain Japanese/English Dictionary
(dual-language glossary) in machine-readable form authored by Jim
Breen.  Having the neat form of a dual-language, dual-script glossary,
it can readily be used in any number of applications.  [It was
initially intended for use with MOKE (Mark's Own Kanji Editor) and
related software such as JDIC.] More Info: info/0015.

----------------------------------------------------------------

ENGLEX

Ftp Directory:     pub/tools/ling_analysis/morphology/englex_pc-kimmo/

Englex is a basic lexicon for morphological analysis of English text.
It uses the standard orthography for English. It is intended for use
with PC-KIMMO (or programs that use the PC-KIMMO parser, such as
KTEXT).  With such software and Englex, you can produce sets of
records of the morphological constituents in English texts.  Practical
applications include morphological preprocessing of text for a
syntactic parser and producing morphologically tagged text.  Englex
can also be used to explore English morphological structure.
More Info: info/0045.

----------------------------------------------------------------
FRENCH

Ftp Directory:  pub/language_tutorial/french/

French Plus! (authored by Gene Hayworth) is a tutorial and testing
program, divided into three sections: Vocabulary Review, Vocabulary
Exercises, and Verb Conjugation Exercises.  The demo version,
available for evaluation purposes only, contains approximately 35
words; the full version includes a combination of more than 800 nouns,
adjectives, and commonly used verbs, with their conjugations in four
tenses.  Accent marks have not been included in order to make this
program compatible with as many systems as possible. The program will
run from either a floppy disk or a hard drive when installed in a
single directory. More Info: info/0075.


----------------------------------------------------------------

FONOL

Ftp Directory:     pub/tools/ling-analysis/fonol/

Fonol is a programming language for writing out and applying TG-style
phonological rules (modelled on Chomsky and Halle 'Sound Pattern of
English' and Schane 'Generative Phonology') to see their effect.  It
also incorporates the input and output filters (conditions) which came
into common use about the same time.  It is intended to aid students
of phonology to grasp the ideas behind phonological rules and to help
phonologists manage large complex bodies of rules in the theory of
their choice.  (Notation style modified for writing in IBM PCs.)
More Info: info/0019.

----------------------------------------------------------------

Gazeteer

Ftp Directory:     members-only/lexica/wordlists/gazatteer/

The TIPSTER Gazetteer is a compilation and reformulation of gazetteer
information from a number of sources, primarily CIA's RWDB2, the US
Geological Survey, the CIA World Fact- book (version 11), and the
Board of Geographic Names namelists. Coverage varies drastically,
based on the degree of completion for the region or attribute. Version
4.0 has over 240,000 place names. Depending on the source, multiple
entries may exist for the same geographic entity, under various
spellings. More Info: info/0062.

----------------------------------------------------------------

German Stemmer

Ftp Directory: members-only/tools/ling-analysis/morphology/German.Stemmer/

This German Stemmer, designed by Daniel Stieger from Institut fuer
Informationssysteme, Zuerich, is available in Modula-2.  Its design is
based on the "Porter Algorithm". It includes only one semester's work
and is, therefore, unfinished. The program uses an automatically
generated dictionary of 215,000 German words for the decomposition
task. The dictionary is available along with the software.  A report
written in German is available in hard copy. More Info: info/0097.

---------------------------------------------------------------

GRAMPT411

Ftp Directory:     pub/tools/ling-analysis/grammar/

MSDOS executables and data for Grammar Tranformation System GRAMTSY
interpret the  more or less liguistically familiar notation of
transformational grammars and applies the grammars so linguists may
analyze and interpret the implications of intricate TG-rule or
TG-grammar networks.  More Info: info/0056.

----------------------------------------------------------------

GREEK

Ftp Directory:     pub/typesetting/troff/

"Greek" is a (di)troff filter that takes Greek text written with Latin
characters using the typewriter letter correspondence, and converts it
into the corresponding character sequences for the Greek letters.
"Greek" follows the ``monotoniko'' (single-accent) system.  Text
may have intermixed Greek and Roman characters. More Info: info/0008.

----------------------------------------------------------------

HALF-UNCIAL fonts

Ftp Directory:     pub/typesetting/tex/fonts/

These are the half-Uncial (Irish typeface) fonts pre-generated at
300dpi for use with (La)TeX. More Info: info/0044.

----------------------------------------------------------------

HOMOPHONES

Ftp Directory:     pub/lexica/homophones/

This is a list of homophones in "General American English", based on
the book HANDBOOK OF HOMOPHONES by William Cameron Townsend, 1975.
The list contains words that sound the same (or very nearly the same)
but are spelled differently. It occasionally includes spelling
variants of the same word when there  is another word in the same
entry; the only difference between "homophones" and "spelling variant"
is whether or not the words are lexically "the same". The list also
contains a few common proper names.

This list of homophones was provided by Evan Antworth from the Summer
Institute of Linguistics. More Info: info/0083.

----------------------------------------------------------------

Hum

Ftp Directory:     pub/tools/concordances/

This is the "hum" concordance and textual analysis package done by
Bill Tuthill when he was at Berkeley (1981).  A package of programs
for literary and linguistic computing, emphasizing the preparation of
concordances and supporting documents.  Both keyword in context and
keyword and line generators are provided, as well as exclusion
routines, a reverse concordance module, formatting programs, a
dictionary maker, and lemmatization facilities.  There are also word,
character, and digraph frequency counting programs, word length
tabulation routines, a cross reference generator, and other related
utilities.  The programs are written in the C programming language.
More Info: info/0069.

----------------------------------------------------------------

InterBASE

Ftp Directory: pub/tools/database/interbas/

Interbas is a natural language front end for relational databases in
DBase compatible format. The program was a finalist in the "Software
in Europe" contest at CeBIT 93 in Hanover, Germany.  This demo
contains both an English and a Russian version of the InterBase
system; additionally there are several demo linguistic processors for
various applied software systems. More Info: info/0102.

----------------------------------------------------------------

The Interlinear Text Processor

Ftp Directory:     pub/tools/ling-analysis/itp/

The IT ('eye-tee' for Interlinear Text) software is a set of tools
from SIL [in executable MSDOS binary code] that are for developing a
corpus of annotated interlinear text -- for what linguists, literary
scholars, translators, and anthropologists call the "glossing" of text.
Primary among these tools is `itp'.  The interlinear text file produced
using itp is a clean ASCII file which is accessible by other text
processing software for purposes such as concordancing, indexing, or
display formatting. In addition to itp, the IT package includes a
collection of other software tools which support the conversion of
conventional texts to interlinear text format and which support the
maintenance of the auxiliary lexical database files.  IT views text as
a sequence of text units, each of which contains a text line plus a
multidimensional set of annotations entered according to a model
provided by the analyst.  In addition to word and morpheme level
annotations, the IT system supports freeform annotations of the whole
text unit, such as translations. More Info: info/0020.

----------------------------------------------------------------

ITALIAN PLUS

Ftp Directory:     	pub/language_tutorials/italian/

Italian Plus! (authored by Gene Hayworth) is a tutorial and testing
program, divided into three sections: Vocabulary Review, Vocabulary
Exercises, and Verb Conjugation Exercises.  The demo version,
available for evaluation purposes only, contains approximately 35
words; the full version includes a combination of more than 800 nouns,
adjectives, and commonly used verbs, with their conjugations in four
tenses.  Accent marks have not been included in order to make this
program compatible with as many systems as possible. The program will
run from either a floppy disk or a hard drive when installed in a
single directory. More Info: info/0077.

--------------------------------------------------------------

ITRANS3.20

Ftp Directory:     pub/typesetting/tex/indian

This is a package for printing Indian language script text. It only
does the transliteration mapping--each letter in an Indian language is
assigned as English equivalent, but the actual printout is in an
Indian language script. Font support is available for two versions of
the Devanagari script (one of which was developed by Frans Velthuis),
as well as for Tamil, Telugu and Bengali. The preferred input
interface is TeX, though a dumb textual interface is available for the
PostScript Devanagari font. More Info: info/0076.

----------------------------------------------------------------

Japanese/German Dictionary

Ftp Directory:     pub/lexica/jddict/

This is a Japanese/German dictionary entered by Helmut Goldenstein
from the book "Langenscheidts Lehrbuch und Lexikon der Jap. Schrft",
author Wolfgang Hadamitzky.  It contains 11627 Japanese words and
22000 German translations.  This dictionary may not be available after
December of 1992, so please make SURE you read the documentation file
that comes with it. More Info: info/0051.

----------------------------------------------------------------

Japanese Morphological Dictionary

Ftp Directory:     pub/lexica/jmorphdict/

This is a Japanese morphological dictionary with a search program
included for accessing it.  The documentation is all in Japanese.
More Info: info/0052.

----------------------------------------------------------------

JKWIC

Ftp Directory:     members-only/tools/text-analysis/word-in-context.tools

Jkwic is a simple Key Word In Context program that has the capability
of working with EUC Japanese.  It allows both simple keyword searches
and limited regular expression specification of the keyword.
More Info: info/0066.

----------------------------------------------------------------

JUMAN-MCC

Ftp Directory:  members-only/lexica/JUMAN-MCC

Juman is a program which segments Japanese into words and tags
these words with parts of speech.  It was produced at Kyoto
University and then heavily modified by researchers at MCC.
The tables used for tagging are generated by a prolog program,
but the program which actually does the tokenizing and tagging
is written in C, so that users do not need to have a working
prolog implementation if they just want to use Juman.
More Info: info/0073.

----------------------------------------------------------------
JUMAN-1.0

Ftp Directory:     members-only/lexica/JUMAN-1.0/

Juman-1.0 is the newest version of the segmenter-tagger for Japanese.
It includes the JUMAN-MCC version.  In addition, it provides a 120K
dictionary.  Other minor changes have been made so that the software
is more convenient to use. More Info: info/0084.

----------------------------------------------------------------

The KGEN Program

Ftp Directory:     pub/tools/ling-analysis/kgen/

KGEN is an auxiliary program for PC-KIMMO. PC-KIMMO is a program for
doing computational phonology and morphology. KGEN is typically used
to build morphological parsers for natural language processing
systems.  The KGEN program which this document describes will be of
very little use to you without the PC-KIMMO program and book.  The
PC-KIMMO software is available for MS-DOS (IBM PCs and compatibles),
Macintosh, and UNIX.  ["PC-KIMMO: a two-level processor for
morphological analysis" by Evan L. Antworth, published by the Summer
Institute of Linguistics (1990). The book (including software) is

   International Academic Bookstore
   7500 W. Camp Wisdom Road
   Dallas TX, 75236
   U.S.A.

   phone 214/709-2404
   fax   214/709-2433] More Info: info/0017.


----------------------------------------------------------------
KN-PARSER

Ftp Directory:  members-only/tools/ling-analysis/syntax/KN-parser/

This is a Japanese parser which detects either a dependency or a case
structure.  In the case of the dependency structure, a set of
heuristic rules are used to detect the unique structure of a sentence.
In the case of a case structure, the analysis is performed on a sample
set of sentences found in the case-frame dictionary. The current
case-frame dictionary contains less than 1,000 verbs.  This parser
makes use of gmake, gcc, and juman.

The authors of KN-parser are Profs. Sadao Kurohashi and Makato Nagao,
Kyoto University. More Info: info/0086.

----------------------------------------------------------------
KOREAN

Ftp Directory:     	pub/tools/word_processing/korean

21word is a Korean word processor running on IBM PClones. According to
the original poster, it supports VGA, SVGA(Trident and ET3000? only)
and Herculis mono graphics. Warning never specify swap directory as
one with some files in it since the program wipes out things in the
directory specified as swap directory. It should be comparable to
those commercially available.  More Info: info/0078.

----------------------------------------------------------------
The KTEXT program

Ftp Directory:     pub/tools/ling-analysis/morphology/ktext/

KTEXT is a text processing program that uses the PC-KIMMO parser (see
info about PC-KIMMO). KTEXT reads a text from a disk file, parses
each word, and writes the results to a new disk file. This new file is
in the form of a structured text file where each word of the original
text is represented as a database record composed of several fields.
More Info: info/0018.

----------------------------------------------------------------

The LDB demo program

Ftp Directory:     pub/tools/ling-analysis/ldbdemo/

The Linguistic DataBase (LDB) is a database program created by the
TOSCA corpus linguistics group at Nijmegen University for the storage
and exploration of syntactically analysed texts. It features a tree
viewer and an extensive query language.  It was designed on the basis
of the Nijmegen 130,000 word English corpus, also available.  This
MSDOS LDB demo program demonstrates some of the features of the
Linguistic DataBase program. More Info: info/0004.

----------------------------------------------------------------
LINK

Ftp Directory:	   pub/tools/ling-analysis/grammar/LINK/

This directory contains a system for parsing English. The parser is
based on Link Grammar, a context-free formalism for the description of
natural language also designed by LINK's authors (Daniel Sleator,
Carnegie Mellon University, and Davy Temperley, Columbia University).
It is a lexical system, where each word has a combinatorial formula
representing all the ways in which that word can be correctly used in
a sentence.  A sentence is grammatical if links can be drawn above the
words in such a way that (1) each word's combinatorial requirements
are satisfied, (2) the links do not cross, and (3) the graph of links
and words is connected.  The system is comprised of a parser which
reads in a link grammar (words and their corresponding formulas), and
parses sentences according to the given grammar.  This system also
includes a Link Grammar for English.  This grammar has roughly 700
definitions and 25000 words, and captures many phenomena of English
grammar, such as noun-verb agreement, questions, imperatives, complex
and irregular verbs, different types of nouns, past or present
participles in noun phrases, commas, a variety of adjective types,
prepositions, adverbs relative clauses, possessives, etc.
More Info: info/0072.

----------------------------------------------------------------

The LQ-TEXT package

Ftp Directory:     pub/tools/lq-text/

LQ-TEXT searches text for phrases in it that you previously indexed.
A browser and a program to generate keyword-in-context style lists are
also included.  The software is primarily designed for Unix systems.
The necessary indexing program (lqaddfile) is enclosed.  Indexes are
usually less than the size of the data, and sometimes half that.
There is a browser (lqtext) for System V, and a shell script (lq) for
any Unix system.  There is also a program (lqkwik) that turns the
output of lqphrase or "lqword -l" into a keyword in context-style
list. More Info: info/0021.

________________________________________________________________

Male and Female Names List

Ftp Directory:     pub/lexica/wordlists/

A file of almost 3000 male names and 4967 female names compiled by
Mark Kantrowitz.  Copyright is contained in the names.README.Z file.
More Info: info/

----------------------------------------------------------------
MACLEX

Ftp Directory: pub/tools/dict-maintenance/maclex/

MacLex is a program for field linguists that manages lexicon/dictionary
files of a specified format. It supports editing, find/change, user-
defined sorting order, and reversals. It is written and supported by
Bruce Waters of SIL (Summer Institute of Linguistics).
More Info: info/0074.

----------------------------------------------------------------
MAP3.1

Ftp Directory:     members-only/tools/ling-analysis/morphology/

This is a dictionary system suitable for use in a natural language
parsing system.  In addition to the basic dictionary, a morphological
analysis system can be used to cope with inflections and derivations
of words in the lexicon.  This system allows the user to write
dictionaries and analyzers in the language of their choice, but the
system includes an example of British English.

This system was designed by Graeme Ritchie, Steve Pulman, Graham
Russell and Alan Black More Info: info/0095.
----------------------------------------------------------------

The MORFOGEN demo program

Ftp Directory:     pub/tools/ling-analysis/morphology/morphogen/

MORFOGEN is a morphological rule compiler and dictionary interface
tool which consists of a finite state compiler (that converts inflectional
and derivational paradigms into a finite state machine) and a
recognizer, which accepts inflected forms as input and returns base
forms (constrained by inflection class information in the lexicon) as
well as any morphemes that matched during analysis.  MORFOGEN can
handle concatenative as well as non-concatenative morphology, and can
be customized, for use on languages of inflecting as well as agglutinating
types.

The demo program provides executables for Sun4 OS 4.1.1.
More Info: info/0001.

----------------------------------------------------------------

MTRAN
Ftp Directory:     pub/tools/machine-translation/

MTran is an MS project by Douglas Witmer (University of Texas at
Arlington), that analyzes 5 different linguistic theories with regard
to machine translation.  The MTran program was written to implement a
proposed a fixed text machine translation technique, arguing two
primary points: much of machine translation work can be done by a
person fluent in only one language, and analysis of text only needs to
be done once during the translation of text into multiple languages.

MTran is written in the ICON language and ICON interpreters for 8088
and 80386 architectures are provided. More Info: info/0052.

----------------------------------------------------------------

MUC-5 Word Lists (Fifth Message Understanding Conference)

Ftp Directory:	  members-only/lexica/MUC5.wordlists/

These files were gathered for the Fifth Message Understanding
Conference participants, and later released to all CLR members. Files
include a nationalities file of 216 countries with noun and adjective
forms of the nationalities, and two files on organization names, one
listing UN organizations and the other listing 187 international
organizations.  The BBN corporation provided corporation names in English
and Japanese, Japanese human names and place names, and a lexica of
Japanese words in the business domain. More Info: info/0104.

_______________________________________________________________

MY Russian Translation program

Ftp Directory:	   members-only/tools/translation/Russian/

MY Russian translation program translates in a fully automatic mode
or a semi-automatic mode which allows interactive correction for new
words. It allows updating of the dictionaries; "teaching" the system
new words and context sensitive translation; and editing of text while
in the process of translating.  The program was originally designed
for college students.  The My Russian translation program was written by
Yuri Yulaev. More Info: info/0103.

---------------------------------------------------------------

NIST SGML parser/validator program

Ftp Directory:     pub/tools/sgml/

The NIST NCSL OSE SGML package is an SGML parser and validation suite
for text into which tags in SGML format have been inserted.
Currently no other documentation other than comments in the code is
available.  The C source code is supplied in this package, which is
designed primarily for Unix systems. More Info: info/0006.

----------------------------------------------------------------

OED2 package

Ftp Directory:     pub/tools/oed2/

Oed2/Ox2 is a package of utilities used to manage network access to
the Oxford English Dictionary at the Waterloo Center for OED research
for online lookup of words, definitions, examples, or other patterns.
[Oed2/Ox2 is a front-end to the Pat pattern searching applied to the
Oxford English Dictionary Version 2.  Version 2 of the OED comprises
the merged Version 1 dictionary and the Supplement.]  Pat combines
very fast search capabilities over a very large text file with an
awkward user interface.  The oed2/ox2 program knows about the
structure of the dictionary file.  At the request of Oxford University
Press, this program should be installed as ox2 (not oed2) at non-UIUC
sites.  Because oed2/ox2 is a network resource, the oed2/ox2 program
can be compiled to use a remote Pat server.  In addition, a modified
telnet server is supplied for remote network access. More Info:
info/0012.

----------------------------------------------------------------
OLDC

The On-line Dictionary of Computing is a dictionary of programming
languages, architectures, domain theory, mathematics, networking, etc.
It was compiled by Dennis Howe, from the Theory and Formal Methods section
of the Department of Computing at Imperial College of Science,
Technology and Medicine, London. More Info: info/0090.

----------------------------------------------------------------

The PC-KIMMO program

Ftp Directory:     pub/tools/ling-analysis/morphology/pc-kimmo/

PC-KIMMO is a new implementation for microcomputers of a program
dubbed KIMMO after its inventor Kimmo Koskenniemi (see Koskenniemi
1983). It is of interest to computational linguists, descriptive
linguists, and those developing natural language processing systems.
The program is designed to generate (produce) and/or recognize (parse)
words using a two-level model of word structure, in which a word is
represented as a correspondence between its lexical-level form
(components) and its surface-level form (the way it is written). More
Info: info/0016.

----------------------------------------------------------------

PLEUK grammar development system

Ftp Directory:	   members-only/tools/ling-analysis/syntax/pleuk/

The Pleuk grammar, published by the University of Edinburgh's Centre
for Cognitive Science, is a shell for grammar
development.  Many differnt grammatical formalisms can be embedded
within it, including Cfg, HPSG-PL, Mike, SLE, and Term.  Sample
grammars are provided for these formalisms.  Pleuk is being made
available by its authors in the hope that it will provide a set of
facilities for the production of new grammar formalisms. Pleuk
provides both an operating system in which to develop grammars as well
as a uniform environment in which grammar writers and testers can
situate implementations of a variety of grammar formalisms.  The
system offers a way of describing the  modules of a grammar
formalism and defining operations over files and the objects described
in them, such as compilation, editing and display. Pleuk uses a
standard Prolog environment and takes advantage of functionalities
like high quality graphs or menu-based input when these are available.
More Info: info/0101.

_______________________________________________________________

The Poor Man's TeX system

Ftp Directory:     pub/typesetting/tex/

PM-TeX provides a simple macro-based approach to typeset Chinese,
Japanese, and Korean text using either LaTeX or TeX.  It comes with
programs that generate MetaFont fonts for these scripts from existing
bitmap fonts.  A number of conversion programs to convert the Chinese,
Japanese, or Korean text into a form that PM-TeX can use are included.
PM-TeX works with almost any (La)TeX system on almost any platform.
More Info: info/0011.
----------------------------------------------------------------

The POPX Prolog code archive

Ftp Directory:     pub/tools/programming/prolog/popx/

These are some of the libraries and programs from the DEC-10 Prolog
Library as well as some other interesting Prolog programs.  The list
is long, so please consult the info page. More Info: info/0064.

----------------------------------------------------------------
PREPCOB.AWK

Ftp Directory:	 members-only/tools/prepcob

This is a program (designed by Archibald Michiels and Jacques Noel,
Universite de Liege) that reads the Collins CoBuild English Language
Dictionary (Harper-Collins and renders it so that its information is
more accessible.  The information that is kept includes: the lexeme
with the reading number, the headword with morphological variants,
grammar information, the definition, and examples. The result is a
list of awk records, each separated by a blank line. More Info:
info/0088.

----------------------------------------------------------------

The Roget 1911 thesaurus

Ftp Directory:     pub/lexica/roget-1911/

This thesaurus is an electronic version of the edition of Roget's
classic Thesaurus published in 1911 by the Crowell company.  The large
number of English words is subdivided by sense into a series of large
antonym groups and into subgroups, indexed by subjects whose tags
appear in the outline.  (Alphabetical indexing is not provided as it
remains under copyright; most of the original printer's effects such
as italics for borrowings appear.) More Info: info/0013.

----------------------------------------------------------------
ROOK

Ftp Directory:     pub/tools/ling-analysis/grammar/ROOK/


Rook is a system for authoring descriptive grammars in HyperCard for
the Macintosh.  It is a tool for interactively and incrementally
developing a grammar description based on an interlinear text corpus
(as produced by SIL's IT program). The resulting on-line descriptive
grammar exploits the capacity of the computer to provide instant
access to cross-referenced topics, text examples, explanations of
morpheme glosses, and so on.  It was designed by J. Randolph
Valentine. More Info: info/0080.

----------------------------------------------------------------

SAX SYNTACTIC ANALYZER (synana)

Ftp Directory:     pub/tools/ling-analysis/syntax/

SAX (Sequential Analyzer for syntaX and semantics) is a syntactic
analyzer based on logic programming. SAX employs a bottom-up and
breadth-first parsing algorithm. The SAX grammar rules are basically
written in Definite Clause Grammar (DCG). The SAX grammar rules are
translated into a parsing program written in Prolog. SAX is
implemented in SICStus Prolog Ver 0.7.

Included with this system is a Japanese grammar and some sample
Japanese data. More Info: info/0058.

----------------------------------------------------------------

SGML2LATEX

Ftp Directory:     pub/tools/sgml/

The sgml2latex system is a set of SGML document type definitions for
the LaTeX document styles (articles, books, reports, letters, slides),
for BibTeX bibliographies and for Unix manual pages, a set of programs
for doing the translation from SGML to LaTeX or troff/nroff, and a
program for extracting source code from documentation, providing a
simple "literate programming" facility.

To use the 'qwertz' documentation system, the 'sgmls' SGML parser is
required (also available from the CLR). More Info: info/0054.

----------------------------------------------------------------

SGMLS

Ftp Directory:     pub/tools/sgml/sgmls/

Sgmls is an SGML parser derived from the ARCSGML parser materials
which were written by Charles Goldfarb.  It works on Unix, MS-DOS and
VAX/VMS.  It should be straightforward to port to most systems that
provide ANSI C and use an ASCII-based character set.  It outputs a
simple, easily parsed, line oriented, ASCII representation of an SGML
document's Element Structure Information Set (see pp 588-593 of ``The
SGML Handbook'').  It is intended to be used as the front end for
structure-controlled SGML applications.  For compatibility with the
Amsterdam SGML Parser (ASP), there is also a filter that translates
the output of sgmls using an ASP replacement file. More Info:
info/0053.

----------------------------------------------------------------
SHAKE AND BAKE Demo

Ftp Directory:     members-only/tools/translation/shake_and_bake/

This is a Spanish-English translation system demo written by John L.
Beaven, at the University of Edinburgh.  Other contributors are Guy
Barry, Robin Cooper, Mark Johnson, and Chris Mellish.  The system
exploits recent advances in lexicalist unification-based grammar
theories.  The system provides greater modularity of the monolingual
components.  The approach is demonstrated by presenting very different
Unification Categorial Grammars for small fragments of English and
Spanish; the grammars contain linguistically interesting phenomena
such as word order variation and clitic placement.  The monolingual
grammars are put into correspondence by means of a bilingual lexicon.
More Info: info/0098.

----------------------------------------------------------------

The SHOEBOX program

Ftp Directory:     pub/tools/ling-analysis/database/

SHOEBOX is a database management program, designed expressly to meet
the needs of the field linguist. Using SHOEBOX, the linguist can
easily enter, edit, and analyze lexical, textual, anthropological and
other types of data in multiple datafiles. For example, with SHOEBOX,

+ Maintain a simple dictionary, or a more complex lexicon,
+ Interlinearize text, where new words are automatically entered
  into the dictionary,
+ Do grammatical filing and analysis of text data,
+ Enter and file cultural notes,
+ Maintain nonlinguistic types of databases, such as address lists
  or library catalogs. More Info: info/0022.

----------------------------------------------------------------

The Standard Industrial Classification Manual (SICM)

Ftp Directory:     members-only/lexica/

The SICM is a manual which defines the classification of economic
activities for the production of Federal economic statistics. Economic
activities are classed under 99 major headings, each with a potential 99
subheadings. Thus manufacture of bulletproof vests is in class
3842 -- Orthopedic, Prosthetic, and Surgical Appliances and Supplies
which is in group 384 -- Surgical, Medical, and Dental Instruments and
Supplies which, in turn, is in major group 38 -- Measuring, Analyzing, and
Controlling Instruments; Photography, Medical and Optical Goods; Watches
and Clocks. The manual is used to provide classifications for a
variety of computer applications. For example companies maintaining
mailing lists may classify the organizations on the list by SIC code and
use this  to target or specialize the type of mail sent to the
organization.

The manual is available in machine readable form and is an interesting
lexical resource in its own right. More Info: info/0070.

----------------------------------------------------------------
SUSANNE

Ftp Directory:     pub/lexica/Susanne/

The SUSANNE Corpus comprises an approximately 128000-word subset of
the Brown Corpus of American English, annotated in accordance with the
SUSANNE scheme.

The SUSANNE scheme attempts to provide a method of representing all
aspects of English grammar which are sufficiently definite to be
susceptible of formal annotation, with the categories and boundaries
between categories specified in sufficient detail that, ideally, two
analysts independently annotating the same text and referring to the
same scheme must produce the same structural analysis. More Info:
info/0063.

----------------------------------------------------------------

SYNONAME

Ftp Directory:     pub/tools/ling-analysis/personal-names/

Names are a key entry point for researching and indexing historical
information.  However, the format and spelling of personal names vary
greatly from one institution to the next, reflecting traditional
differences in practice.  If historical information is to be compiled
or shared, matching different versions of personal names is a
necessity.  The computer program SYNONAME automatically matches many
possible forms of a single personal name by using an ordered sequence
of twelve algorithms for pattern matching that include both character-
and word-matching techniques.  The matched pairs of names are
considered to be "candidate matches" until confirmed by a human
name-authority editor.  Run against a merged file of artists' names
from museum collections data, the program performed with an accuracy
rate of 97.4% and an optimum efficiency rate of 90.8%.  Accuracy can
increase to nearly 99% at the expense of some efficiency.  The
concepts behind the algorithms and their imlpementation may be useful
to others merging data in different contexts. More Info: info/0060.

----------------------------------------------------------------
TACT

Ftp Directory:     members-only/tools/text-analysis/TACT.tar

TACT is an interactive full-text retrieval system for MS-DOS with
a number of analytical tools.  Like others of its kind, TACT
retrieves segments of text according to specified word forms. In
addition, it can find words or character-strings that match
criteria the user specifies. TACT generates simple graphs to show
the distribution of forms throughout an entire text, or within
various structural divisions determined by the user. TACT also
allows retrieval by metatextual `categories'.

TACT was designed by John Bradley and Lidio Presutti at the University
of Toronto. More Info: info/0093.

----------------------------------------------------------------
TAGGER.v0.92

Ftp Directory:     members-only/tools/ling-analysis/syntax/

This is a part-of-speech tagger designed by Eric Brill at MIT.  The
tagger can be trained to tag, or an already trained tagger for English
can be used. The trainer uses a two-stage process: in the first stage
the tagger learns rules for tagging unknown words; in the second, the
rules learned involve the use of contextual cues to improve tagging
accuracy. The trained tagger also uses a two-stage process.  It
assigns most likely tags to every word in isolation. In the second
stage, contextual transformations are used to improve accuracy. More
Info: info/0096.

----------------------------------------------------------------

TIMIT

Ftp Directory:     pub/lexica/TIMIT/

TIMIT is a database of 6,100 English words with their most likely
pronunciation. Each entry is made up of two lines: the first line has
the word number followed by the spelling of the word, and the second
contains the transcription of the word using the set of 61 TIMIT
phones. This word list was provided by Chuck Wooters from ICSI at
Berkeley.  More Info: info/0081.

----------------------------------------------------------------
TULIP

Ftp Directory:     members-only/tools/ling-analysis/phonology/

This is a two-level phonological analyzer based on the system
described in S.G. Pulman and M.R. Hepple, "A Feature Based Formalism
for Two Level Phonology: a description and implementation."  More
Info: info/0094.

----------------------------------------------------------------

TURKLEX

Ftp Directory:     pub/tools/ling-analysis/morphology/turklex/

Turklex is a set of files containing specification for Turkish
morphology: morphotactics, nominal roots, compound nouns, verbs,
pronouns, proper nouns, connectives, exclamations, questions and some
adverbs, special cases, some acronyms and some technical jargon. It is
intended for use with PC-KIMMO (or programs that use the PC-KIMMO
parser, such as KTEXT). It produces a morphological breakdown of a
Turkish word into all possible morphological parses.

The specification is provided by Bilkent University, Ankara, Turkey,
via Summer Institute of Linguistics, and is in the Public Domain, but
may not be used for commercial purposes.  More Info: info/0071.

----------------------------------------------------------------

UNIVERSITY OF PENNSYLVANIA Morphological Analyzer

Ftp Directory:    members-only/tools/ling-analysis/morphology/UPenn/

The morphology package contains a large morphological database, an
X-window based maintenance program, and C and Lisp hooks for
interfacing the database to other software programs.  The database
itself consists of approximately 316,000 inflected items, along with
their root forms and inflectional information (such as case, number,
mode).  There are 13 parts of speech - Noun, Proper Noun, Pronoun,
Verb, Verb Particle, Adverb, Adjective, Preposition, Complementizer,
Determiner, Conjunction and Interjection, and Noun/Verb Contraction.
Nouns and Verbs are the largest categories, with approximately 213,000
and 46,500 inflected forms, respectively.  The access time for a given
inflected entry is .6 msec.  The maintenance program runs under the
X-window interface, and allows the user to customize the database to
their needs.  A inexperienced user can easily add, delete, or modify
entries to the existing database, and a person passingly familiar with
X windows and C array structures can customize the package for a
different language, building their own database with different parts
of speech and/or inflectional information.  There are C and Lisp
functions that provide hooks to allow developers to incorporate the
database into existing research projects.  The entire package requires
about 25-30 M of space. More Info: info/0100.

__________________________________________________________________

VIETNAMESE

Ftp Directory: pub/tools/vietnamese/

This directory includes a variety of tools and information for
processing Vietnamese text. The tools are fonts, a text converter, and a
text printer.  A proposal for the Vietnamese Standard for Information
Interchange is also included. More Info: info/0079.

----------------------------------------------------------------
The word lists

Ftp Directory:     pub/lexica/wordlists/

These word lists are copies of a number of word lists that are freely
available from a number of sites in Europe and North America.  The
origins of some of them are currently unknown, but are being checked.
Current languages are English, Dutch, English (shorter list), German,
Norwegian, Italian, and Swedish.  Several word lists containing names
are also available. More Info: info/0014.

----------------------------------------------------------------

The OTA word lists

Ftp Directory:     pub/lexica/wordlists/ota/

The Oxford Text Archive word lists contains word lists from the
following. More Info: info/0049.
Australian
Chinese (only a list of the HanYu PinYin)
Computer (various stuff including common passwords, domains, etc.)
Danish
Dutch
Finnish
French
German
Italian
Japanese (List of words in Romaji - see edictj reference in info/15)
Literature (including various authors and genre)
Movies&TV (including Monty Python and Star Trek word lists)
Names (includes names in a number of languages and others)
Norwegian
Place Names (including colleges, wordl factbook, zip codes, etc.)
Random (includes various random sorts of word lists)
Religion (includes Q'ran and King James Bible word lists)
Science (includes asteroids and biology lists)
Spanish
Swedish
Yiddish

----------------------------------------------------------------

WordNet

Ftp Directory:     pub/lexica/wordnet/wordnet1.2/
Ftp Directory:     pub/lexica/wordnet/wordnet1.3/
Ftp Directory:     pub/lexica/wordnet/wordnet1.4/

An English thesaurus of modern design, WordNet is an on-line lexical
reference system organized as a semantic net.  Its design
is inspired by current psycholinguistic theories of human lexical
memory.  English nouns, verbs, and adjectives are organized into
synonym sets, each representing one underlying lexical concept.
Different relations link the synonym sets: synonymy, antonymy,
meronomy (part-whole), and hyponymy (hierarchical inclusion).  The
data files are in ASCII format, the demo programs run on a variety of
machines.

The WordNet 1.4 system includes a database that is close to 13.5
megabytes, exclusive of the search code.  The entire package is
approximately 17.5 megabytes.  It also has a semantic concordance: a
textual corpus linked to a lexicon with semantic tags.  The
concordance consists of 103 files from the Brown Corpus annotated with
pointers to word senses in the WordNet 1.4 database.  An X Windows
application, Escort, is provided for searching the concordance files
for occurrences and co-occurrences of semantic tags.  Escort has been
ported to the Sun-4, NeXT and DECstation platforms.  You must install
WordNet 1.4 before installing and using the semantic concordance
package.  The semantic concordance package is approximately 20
megabytes. More Info: info/0003.

----------------------------------------------------------------

WORDSURV

Ftp Directory:     pub/tools/ling-analysis/ling-survey/

A typical language survey may involve activities like determining
linguistic relationships through the comparison of word lists, testing
dialect intelligibility by playing back tape-recorded texts, and
studying sociolinguistic aspects of language use and language
attitudes in multilingual situations. WORDSURV is designed to aid the
first of these areas--the collection and analysis of word lists. It
functions in three main areas: (1) data entry and maintenance, (2)
data analysis, and (3) data output. WORDSURV also supports specialized
kinds of analysis, including lexicostatistics, phonostatistics, and
comparative reconstruction. More Info: info/0059.

----------------------------------------------------------------
XEROX Part-of-speech Tagger

Ftp Directory:     members-only/tools/ling-analysis/syntax/xerox-tagger/

This part-of-speech tagger, designed by Doug Cutting and Jan Pederson
at Xerox, was written in ANSI Common Lisp. Its development was done
in Franz Allegro Common Lisp version 4.1 on SunOS4.x and MacIntosh
Common Lisp 2.0p2. The following code is provided:  source code, a
tokenizer for plain ASCII English, an English lexicon enduced from the
Brown corpus, a table of mappings for word suffixes to likely
ambiguity classes, and an HMM trained on the odd numbered sentences in
the Brown corpus. More Info: info/0082.

----------------------------------------------------------------
YUGOSLAV CORPUS

Ftp Directory:     members-only/corpora/Yugoslav.Corpus.tar

This is a Serbo-Croatian corpus consisting of approximately 700, 000
words.  The texts are taken from modern Yugoslav fiction and all
Serbo-Croatian-speaking areas-- Serbia, Croatia, Montenegro, and
Bosnia-Hercegovina-- are represented.  The texts are in ASCII format
and the Latin alphabet is used. A list of ASCII values of the special
Serbo-Croatian characters is provided in info file 0092. More Info:
info/0092.
----------------------------------------------------------------