🚀 go-pugleaf

From: "Stefan Wermter" <wermter@cs.umass.edu>
To: mod-ki@unido.uucp
Date: Wed, 30 Aug 89 00:11 EST
Subject: Natural Language Processing and Connectionism at CogSci 89
Keywords: Berichte

mod-ki message 0285
************************************************************************
              Natural Language Processing and Connectionism
   A Survey from the 11th Conference of the Cognitive Science Society

                             Stefan Wermter

This survey gives an overview about technical sessions and trends at the
11th Annual Conference of the Cognitive Science Society (CogSci 89) with
a specific focus on connectionism and natural language processing.

The CogSci 89 took place  at  the  University  of  Michigan,  Ann Arbor,
Michigan, USA from August 16 to August 19. 125 papers had been  accepted
in  the  areas   of   connectionism,   learning,   language,   knowledge
representation,  reasoning,  and  skill  acquisition.  The  papers  were
presented either as talks (66 papers) or as posters  (59 papers).  Since
it would be impossible to cover all papers in the area of  connectionist
natural language processing in detail, I concentrate on summarizing  the
main issues in the following.

Connectionist Models and Phonology
----------------------------------
In two complementary papers Dave Touretzky  described  a  rule-following
connectionist network for chunking context sensitive rewrite  rules.  He
proposed a backpropagation architecture with  buffers  for  the  current
change and the last remembered change of a rewrite rule. If the  current
change and the remembered change can be composed, chunking  takes  place
by training connections to a "chunked change buffer". For each  step  of
applying a rewrite rule the input pattern is updated with  a  so  called
"String Editing Network". Using several of these networks Touretzky maps
a morphological level to a phonemic and to a phonetic level implementing
Lakoff's  new  theory  of   cognitive   phonology.   Other   interesting
connectionist  models  of  phonology  were   Marchman   and   Plunkett's
association task for present and past tense English verbs  and  Peterson
et al.'s phonological competition model of form related priming effects.

Connectionist Models and Structured Representations
---------------------------------------------------
Structured representations, hierarchies, and issues of  compositionality
continue to be important issues  for  connectionist  systems.  In  three
presentations (paper, poster and  panel discussion) Elman described  the
advantages of "Simple Recurrent Networks". These networks are similar to
Jordan's recurrent networks, but in contrast to  a  Jordan  network  the
hidden layer and not the  output  layer  of  a  3-layer  backpropagation
network is mapped back to the input layer. Using this architecture Elman
demonstrated how lexical  categories  and  syntactic  structure  can  be
learned  in  a  distributed  connectionist  network.  Based   on   these
experiments Elman argued that connectionist networks can  learn  "richly
structured and openended" representations.

Connectionist Models and Syntactic Issues
-----------------------------------------
Several  new  connectionist  models  were  presented  for  deterministic
parsing, unification, and the propagation of  syntactic  head  features.
Kwasny and Faisal proposed a model  for  combining  Marcus'  concept  of
deterministic parsing with advantages  of  distributed  representations.
The described system learns 23 grammar  rules  using  a  backpropagation
architecture.  The  results  showed  that  grammatical   sentences   and
ungrammatical sentences could be differentiated by the network using  an
error measurement based on the Euclidean distance  between  current  and
desired   output.   While   Kwasny   and   Faisal    used    distributed
representations, Stolcke applied localist representations for processing
unification-based grammars. The rules of the grammars are  expressed  as
structure fragments which are represented in  a  connectionist  network.
The process of unification is implemented by  spreading  activation  and
applied to sentence generation. Another approach which  used  constraint
satisfaction techniques as well was proposed by Tomabechi and Levin.  In
their model of head-driven  massively  parallel  constraint  propagation
syntactic head features are propagated through a 4-layer marker  passing
model. This approach attacks linguistic problems like  case,  agreement,
and complement order which - according  to  Tomabechi  and  Levin - have
been somewhat neglected in earlier marker passing models.

Connectionist Models and Semantic Issues
----------------------------------------
Connectionist   models   for   semantics   concentrated   on    semantic
relationships within noun phrases. Hollbach Weber described  a  localist
structured connectionist  network  for  interpreting  the  relationships
between adjectives and nouns. Specifically focussing on  the  figurative
meaning of adjectives she proposed a spreading  activation  model  which
captures inferences which are invoked if an adjective is not used in its
literal  meaning.  While  Hollbach  Weber  concentrated   on   adjective
relationships in a localist framework, Wermter  proposed  a  distributed
model for understanding restricted semantic  relationships  in  compound
nouns. A modular backpropagation  architecture  learned  basic  semantic
relationships between nouns and generalized  learned  relationships  for
new compound nouns. This  model  can  be  used  to  detect  similarities
between   compound   nouns   and    noun    phrases    in    a    hybrid
connectionist/symbolic system.

Concluding and evaluating commments
-----------------------------------
In addition to the main areas summarized above two  research  areas  got
quite   a   bit   of   attention:   variable    binding    and    hybrid
connectionist/symbolic methods. Several papers attacked the  problem  of
variable binding in connectionist  systems:  Anandan  et  al.  translate
frame definitions into objective functions. Ajjanagadde and Shastri  use
a phased clock to create and propagate  variable  bindings.  Harris  and
Elman use simple recurrent networks to represent the binding  between  a
target item and a global state. Although some  progress  has  been  made
variable  binding  will  continue  to  be   an   important   issue   for
connectionist systems.

Since current connectionist systems have - apart from their advantages -
some inherent disadvantages  (e.g.  variable  binding,  compositionality
and  capacity)   there   is   an   important   trend   to   use   hybrid
symbolic/connectionist  representations.  Lange  et  al.   described   a
simulation environment for integrating  representations  from  different
symbolic  and  connectionist  paradigms.   Lee  et  al.  demonstrated  a
symbolic/connectionist script applier mechanism. Rose and Belew showed a
hybrid  system  for  tasks  in  information  retrieval.   These   hybrid
connectionist/symbolic systems seem to be particularly promising because
they allow to combine advantages from both paradigms.

In general, the conference of the Cognitive Science Society succeeded in
bringing together researchers from  artificial  intelligence,  cognitive
science  and  linguistics.  A  comparison  with  the  last  few   CogSci
conferences shows that connectionist  natural  language  processing  has
taken an important position at this conference (A  strong  tendency  for
connectionist or hybrid  connectionist/symbolic  models  could  also  be
noticed at IJCAI 89). The poster sessions,  the relatively small size of
the CogSci conference (compared to IJCAI) and the  conference  rooms  at
the University of Michigan provided  good  opportunities  for  intensive
discussions. Concluding,  I  think,  CogSci 89  was  a  very  successful
conference in terms of its organization and technical quality, and I  am
looking forward to CogSci 90.

     Stefan Wermter

     Department of Computer and Information Science
     University of Massachusetts at Amherst
     Amherst, MA 01003
     USA

     wermter@cs.umass.edu

------------------------------------------------------------------------
The proceedings can be ordered at Lawrence Erlbaum Associates, Inc.,
365 Broadway, Hillsdale, New Jersey 07642, USA.
I was told the price is $35.
------------------------------------------------------------------------
************************************************************************