🚀 go-pugleaf

Neuron Digest   Monday, 17 Feb 1997                Volume 97 : Issue 7

Today's Topics:
                         New Book in Neurocontrol
            Paper on leave-one-out cross validation available
            MSc in Information Processing and Neural Networks
ICML-97 workshop on Automata Induction, Grammatical Inference, and Language Acq
            Overview paper and theses on the RF-LISSOM project


Send submissions to "neuron@psych.upenn.edu". Send administrative
questions and [un]subscribe requests to "neuron-request@psych.upenn.edu".
The ftp archives (including back issues) are available from
psych.upenn.edu in pub/Neuron-Digest or by sending a message to
"archive-server@psych.upenn.edu".

----------------------------------------------------------------------

Subject: New Book in Neurocontrol
From:    oomidvar@udc.edu
Date:    Thu, 06 Feb 1997 14:32:46 -0500


This is to announce a new book of potential interest to students,
faculty and practitioners in the fields of  Artificial Neural Networks 
and Automatic Control.

"Neural Systems for Control,"  edited by Omid Omidvar and  David Elliott. 
Academic Press, Boston; ISBN:  012-526430-5; list price:  $49.95.
The twelve chapters, by active researchers in the U.S., Germany, Italy,
and Korea, are

Introduction: Neural Networks and Automatic Control  (D. L. Elliott)

Reinforcement Learning  (A. G. Barto)

Neurocontrol in Sequence Recognition (W. J. Byrne and S. A. Shamma)

A Learning Sensorimotor Map of Arm Movements: 
     a Step Toward Biological Arm Control (S. Cho, J. A. Reggia and M. Jang)

Neuronal Modeling of the Baroreceptor Reflex with Applications in 
   Process Modeling and Control (F. J. Doyle III, M. A. Henson, 
   B. A. Ogunnaike, J. S. Schwaber, and I. Rybak)
 
Identification of Nonlinear Dynamical Systems Using Neural Networks
  (A. U. Levin and K. S. Narendra)

Neural Network Control of Robot Arms and Nonlinear Systems
  (F. L. Lewis, S. Jagannathan, and A. Yesildirek)

Neural Networks for Intelligent Sensors and Control--- 
   Practical Issues and Some Solutions (S. J. Qin)

Approximation of Time-Optimal Control for an Industrial Production 
  Plant with General Regression Neural Network (C. Schaeffner and D. Schroeder)

Neuro-Control Design: Optimization Aspects (H. T. Su and T. Samad)
 
Reconfigurable Neural Control in Precision Space Structural Platforms
  (G. G. Yen)
 
Neural Approximations for Finite- and Infinite-Horizon Optimal Control
  (R. Zoppoli and T. Parisini)


------------------------------

Subject: Paper on leave-one-out cross validation available
From:    "Michael J. Kearns" <mkearns@research.att.com>
Date:    Thu, 06 Feb 1997 15:25:58 -0500



The following paper is now available in compressed postscript
format at http://www.research.att.com/~mkearns under the
"Model Selection/Complexity Regularization" section of my
publications.

Algorithmic Stability and Sanity-Check Bounds
for Leave-One-Out Cross-Validation

Michael Kearns (AT&T Labs) and Dana Ron (MIT)

In this paper we prove "sanity-check" bounds for the error of the 
leave-one-out cross-validation estimate of the generalization error:
that is, bounds showing that the worst-case error of this estimate
is not much worse than that of the training error estimate. The name
sanity-check refers to the fact that although we often expect the
leave-one-out estimate to perform considerably better than the training 
error estimate, we are here only seeking assurance that its performance 
will not be considerably worse. Perhaps surprisingly, such assurance
has been given only for rather limited cases in the prior literature on 
cross-validation.

Any nontrivial bound on the error of leave-one-out must rely on some
notion of algorithmic stability. Previous bounds relied on the rather 
strong notion of hypothesis stability, whose application was primarily 
limited to nearest-neighbor and other local algorithms. Here we introduce 
the new and weaker notion of error stability, and apply it to obtain
sanity-check bounds for leave-one-out for other classes of learning 
algorithms, including training error minimization procedures and Bayesian 
algorithms. We also provide lower bounds demonstrating the necessity of 
error stability for proving bounds on the error of the leave-one-out 
estimate, and the fact that for training error minimization algorithms,
in the worst case such bounds must still depend on the Vapnik-Chervonenkis 
dimension of the hypothesis class.

------------------------------

Subject: MSc in Information Processing and Neural Networks
From:    Ton Coolen <tcoolen@mth.kcl.ac.uk>
Date:    Fri, 14 Feb 1997 13:53:03 +0000

M.SC. IN INFORMATION PROCESSING AND NEURAL NETWORKS

Centre for Neural Networks
King's College, University of London
Programme 1997-1998


1. Introduction

1.1 Aims and objectives
1.2 King's College London
1.3 Centre for Neural Networks

2. Details of the M.Sc. degree

2.1 Structure of the programme
2.2 Entrance requirements and fees
2.3 Programme of lecture courses

3. Further information

3.1 Facilities
3.2 Scholarships and funding
3.3 Applications


1. Introduction

1.1 Aims and objectives

This text describes the University of London M.Sc. degree in Information 
Processing and Neural Networks at King's College London. There have been 
numerous advances in the last few years in our understanding of information 
processing in living and artificial systems. These developments have been 
both at a theoretical level and at a software and hardware level. Students 
wishing to enter this rapidly expanding field after a first degree will not  
be adequately prepared by their undergraduate training. The purpose of the 
interdisciplinary M.Sc. degree in Information Processing and Neural Networks 
is to teach students 

(i)   the definitions and properties of the most commonly used neural 
      information processing systems and algorithms, addressing both their 
      potential and their restrictions. 

(ii)  the mathematical techniques required to analyse the operation of these
      systems and algorithms (e.g. statistics, information theory, non-linear 
      systems theory, statistical mechanics).  

(iii) how to use these techniques for quantifying the operation of neural 
      information processing systems, for improving upon present ones and for  
      designing new ones, and for making quantitative predictions on their 
      performance and reliability. 

(iv)  where and how neural information processing systems are used in 
      information technology and engineering, and how they compare to more 
      standard approaches.

(v)   how to obtain and maintain access to the most recent research results 
      in the academic community. 

On successfully completing this degree a student should be able to contribute 
to research and development in an academic or industrial environment. 

The course was accepted as an M.Sc. course of the University of London by the 
separate Boards of Physics, Engineering and Mathematics in 1989, and was 
formally approved in 1990 by the Senate of King's College London as a course 
leading to the award of an M.Sc. degree in the Faculty of Science (presently: 
the School of Physical Sciences and Engineering, which includes the 
Departments of Mathematics, Physics, Engineering and Computer Science).  

1.2 King's College London

King's College London, founded in 1829, is one of London University's largest 
and most prestigious Colleges. It is a university within a University, with 
nearly 9,000 undergraduate and some 2,900 postgraduate students, of whom about 
one in five is from outside the UK, and almost equal numbers of men and women. 

King's has a long tradition of excellence in both teaching and research, and 
in the most recent national research assessment seven of its subject areas 
received the top rating (5*), an achievement exceeded by only three other 
British universities. Postgraduate students at the College therefore often 
study with staff who are the leaders in their field. 

King's Strand Campus, where the Centre for Neural Networks is based, is the 
most central of all the London Colleges, overlooking the River Thames, midway 
between the Houses of Parliament and St Paul's and at about five minutes' walk
from Trafalgar Square. 

1.3 Centre for Neural Networks 

Within the School of Physical Sciences and Engineering of King's, the Centre
for Neural Networks coordinates teaching and research in the subject of Neural
Networks, including the running of the University of London M.Sc. in 
Information Processing and Neural Networks and a Ph.D. programme. The Centre 
has developed into an internationally renowned focus of research in this new, 
exciting and rapidly expanding field, and constitutes the central node of 
`NeuroNet', the EU-funded Network of Excellence in Neural Networks.  

The Centre acts as a forum for interaction and collaboration between members 
of staff from different departments. It consequently operates across a broad 
and interdisciplinary range. Research covers the theory and mathematical and 
numerical analysis of Biological and Artificial Neural Networks, with 
applications to speech, vision, system control, medical diagnosis, time series
tasks, and hardware development of probabilistic RAM chips. The Centre runs a 
regular term-time seminar on Neural Networks and a newsletter, and is involved
in the organisation of various national and international conferences. 

The four key departments involved are Mathematics, Engineering, Physics and 
Computer Science, with additional M.Sc. lecture courses and/or seminars being 
provided by the Departments of Physiology and Philosophy and the Department of
Mathematics of the London School of Economics. 

Further information on the Centre's activities can be obtained through
Internet on the site:  http://www.mth.kcl.ac.uk/cnn


2. Details of the M.Sc. degree

2.1 Structure of the programme 

The M.Sc. programme in Information Processing and Neural Networks is based on 
course work and a project and requires either one year of full-time study, or 
may be taken part-time over two years. Candidates choose their lecture courses
in consultation with their tutor, and usually offer seven or eight courses for 
written examinations. Passes in a minimum of six of these courses are required
The present programme of available courses, which may be updated to reflect 
new developments in the field, is described in detail in the next section.  

Each M.Sc. candidate must complete a research project in some area of 
Information Processing and Neural Networks at the postgraduate level after 
passing the written examinations (i.e. in the three months June-August). Where
appropriate this project can also be carried out and supervised in academic or
industrial institutions outside KCL.  

Students are in addition expected to participate in the Centre for Neural 
Networks seminar series.  This allows them to become familiar with neural 
network research in a broad sense, and to  know which is the state of the art
and where are the present foci of academic and industrial research. 

All M.Sc. candidates are assigned tutors on arrival at College. Their role is
to guide and advise them in their studies throughout the year and to oversee 
their general progress. Any tutor will be happy to see his/her students at any
time to discuss problems, technical or otherwise, with them. The tutor will 
help in the choices of courses and the project topic, and will ensure that
students find suitable supervisors for their projects. An overview of the 
M.Sc. project subjects of the last few years can be found via the Centre for
Neural Networks internet site http://www.mth.kcl.ac.uk/cnn.  

2.2 Entrance requirements and fees

The normal entrance requirements for both programmes are a first degree or its
equivalent in any one (or jointly) of Mathematics, Physics, Computing,
Engineering or other suitable scientific disciplines,with distinction first or
second class. A sound background in basic mathematics, in particular 
familiarity with standard concepts of calculus, linear algebra, differential 
equations and elementary probability theory, will be assumed. 
Overseas students are welcome to make applications. 

The fees for the courses for the academic year 1997-1998 are now under review,
but for 1996-1997 they were 2,490 pounds for UK and EU students, and 7,250 
pounds for overseas students. The fees cover registration, tuition and 
examinations, and membership of the College Union of Students. The fees for 
part-time M.Sc. students are one-half the full-time fees per annum. 

2.3 Programme of lecture courses 

The following postgraduate courses, contributed by the various KCL in the 
Centre for Neural Networks, as well as the Department of Mathematics of the 
London School of Economics (LSE) and the Department of Physiology, are open to 
candidates for the M.Sc. in Information Processing and Neural Networks:

Mathematics:            Neural networks 
                        Advanced neural networks
                        Information theory in neural networks 
                        Statistical mechanics of neural networks
                        Dynamical systems theory
                        Control theory

Mathematics (LSE):      Computational learning theory 
                        Complexity theory

Engineering:            Communications
                        Digital signal processing
                        
Computer Science:       Sequential numerical methods
                        Parallel numerical methods

Physics:                Theory of information processing
                        Digital image analysis

Physiology:             Neurobiology 


3. Further Information

3.1 Facilities

The Strand campus has an excellent journals and science library. As students 
of the University of London, as well as of King's, all M.Sc. candidates may 
have full access to these, as well as to those in the more than 40 other 
University of London institutions spread across the capital. A photocopying 
service is available on a do-it-yourself basis.   

The College mainframe computer is a cluster comprising a Digital VAX4000-100A,
a VAX-Server 3600 and an Alpha AXP 2100 (to be replaced shortly by a new UNIX
cluster, which can be accessed via a college-wide ethernet network. In 
addition, the College provides extensive PC and Macintosh facilities. M.Sc. 
candidates in Information Processing and Neural Networks also have access to 
the terminal/PC room for postgraduate students in the Mathematics Department, 
with a variety of PC software including TeX, LaTeX, and several neural network
simulation packages (like Cortex-Pro and Neurosolution Professional), running 
on Pentium machines, but also SUN and NeXT Unix workstations (running software
including Mathematica and Maple, with Fortran, Pascal and C compilers) and 
with access to the Internet. Access to more powerful machines can be arranged,
depending on the nature of the M.Sc. project. 

3.2 Scholarships and Funding

Eligible applicants (usually holders of First or Second Class degrees) will be
considered for research council grants. For the M.Sc. in Information 
Processing and Neural Networks the relevant UK research council is the EPSRC, 
offering a small number of Advance Course Studentships. The ODASS Scheme 
offers awards for scholars from developing Commonwealth countries embarking on
M.Sc. degrees. All tuition and maintenance fees, return air fares and an 
allowance for thesis costs will be met. The funding information here is 
incomplete and intended only as a guide; further details concerning funding 
and awards can be obtained via the postgraduate secretary (see below). 

3.3 Applications

A more detailed information booklet can be download via the Centre for Neural 
Networks internet site http://www.mth.kcl.ac.uk/cnn. For further details and 
application forms please contact: 

Mrs. Darakshan Khan
Postgraduate Secretary
Department of Mathematics
King's College London
Strand, London WC2R 2LS
England

Tel:    +44(0)171-873-2107
Fax:    +44(0)171-873-2017
E-mail: darakshan.khan@kcl.ac.uk







------------------------------

Subject: ICML-97 workshop on Automata Induction, Grammatical Inference, and Language Acquisition
From:    Vasant Honavar <honavar@cs.iastate.edu>
Date:    Fri, 14 Feb 1997 13:55:54 -0600

 
 Call for Papers:  Workshop on 
 Automata Induction, Grammatical Inference, and Language Acquisition
 http://www.cs.iastate.edu/~honavar/cfp-mlworkshop.html
 
 The Fourteenth International Conference on Machine Learning (ICML-97)
 July 8-12, 1997, Nashville, Tennessee
 
 
 The Automata Induction, Grammatical Inference, and Language Acquisition 
 Workshop will be held on Saturday, July 12, 1997 during the Fourteenth 
 International Conference on Machine Learning (ICML-97) which will be
 co-located with the Tenth Annual Conference on Computational Learning Theory
 (COLT-97) at Nashville, Tennessee from July 8 through July 12, 1997. 
 Additional information on ICML-97 and COLT-97 can be found at:
 http://cswww.vuse.vanderbilt.edu/~mlccolt/ 
 
 
 Objectives
 
 Machine learning of grammars, variously referred to as automata induction,
 grammatical inference, grammar induction, and automatic language acquisition, 
 finds a variety of applications in syntactic pattern recognition, 
 adaptive intelligent agents, diagnosis, computational biology,
 systems modelling, prediction, natural language acquisition,
 data mining and knowledge discovery.
 
 The workshop seeks to bring together researchers working on
 different aspects of machine learning of grammars in a number
 of different (and until now, relatively isolated) areas including
 neural networks, pattern recognition, computational linguistics,
 computational learning theory, automata theory, and language acquisition 
 for fruitful exchange of the relevant recent research results.
 
 
 Workshop Format
 
 The workshop will consist of 3--5 invited talks offering different
 perspectives on machine learning of grammars, interspersed with
 short (10--15 minute) presentations of accepted papers. The workshop
 schedule will allow ample time for informal discussion.
 
 
 Topics of Interest
 
 Topics of interest include, but are not limited to:
 
 Different models of grammar induction:
        e.g., learning from examples, 
              learning using examples and queries, 
              incremental versus non-incremental learning,
              distribution-free models of learning,
              learning under various distributional assumptions
              (e.g., simple distributions).
 
 Theoretical results in grammar induction: 
        e.g., impossibility results,
              complexity results, 
              characterizations of representational and search
              biases of grammar induction algorithms.
 
 Algorithms for induction of different classes of languages and 
 automata:  
        e.g., regular, 
              context-free, and  
              context-sensitive languages,
              interesting subsets of the above under additional
              syntactic constraints, tree and graph grammars,
              picture grammars, multi-dimensional grammars,
              attributed grammars, etc.
 
 Empirical comparison of different approaches to grammar induction.
 
 
 Demonstrated or potential applications of grammar induction in
              natural language acquisition,
              computational biology,
              structural pattern recognition,
              adaptive intelligent agents,
              systems modelling, 
              and other domains.
 
 
 
 Submission Guidelines 
 
 Full paper submissions are highly recommended although
 extended abstracts will also be considered.
 The manuscript should be no more than 10 pages
 long when formatted for generic 8-1/2 x 11 inch pages using the 
 formatting macros and templates available at:
 http://www.aaai.org/Publications/Templates/macros-link.html
 
 Postscript versions of the manuscripts should be emailed 
 so as to arrive by March 15, 1997 at:  
 honavar@cs.iastate.edu, pdupont@cs.cmu.edu, giles@research.nj.nec.com.
 
 
 Deadlines
 
 
 Deadline for submission of manuscripts: March 15, 1997
 
 Decisions regarding acceptance or rejection emailed to authors: April 1, 1997 
 
 Final versions of the papers due: April 15, 1997
 
 
 Selection Criteria
 
 Selection of submitted papers will be on the basis of review
 by at least two referees. Review criteria include: originality, technical
 soundness, clarity of presentation, relevance of the results and 
 potential appeal to the workshop audience.
 
 
 Workshop Proceedings
 
 Workshop proceedings will be published in electronic form on the world-wide
 web. Authors of a selected subset of accepted workshop papers might also be
 invited to submit revised and expanded versions of their papers for possible
 publication in a special issue of a journal or an edited collection of papers
 to be published after the conference.
 
 
 Workshop Organizers:
 
 
 Dr. Vasant Honavar
 Department of Computer Science
 226 Atanasoff Hall
 Iowa State University
 Ames, IA 50011
 honavar@cs.iastate.edu
 
 
 Dr. Pierre Dupont
 Department of Computer Science
 Carnegie Mellon University
 5000 Forbes Ave
 Pittsburgh, PA 15213
 pdupont@cs.cmu.edu
 
 
 Dr. Lee Giles
 NEC Research Institute
 4 Independence Way
 Princeton, NJ 08540
 giles@research.nj.nec.com

------------------------------

Subject: Overview paper and theses on the RF-LISSOM project
From:    jbednar@cs.utexas.edu
Date:    Fri, 14 Feb 1997 16:11:31 -0600



The following paper (to appear in Psychology of Learning and Motivation)
gives an overview of the RF-LISSOM project on modeling the primary
visual cortex, ongoing at the UTCS Neural Networks Research Group.

For those seeking details, please refer to the dissertation and the
theses also announced below. All these publications (and others) are
available from our web page at http://www.cs.utexas.edu/users/nn (under
publications, self-organization; the direct ftp addresses are included
below as well).  Public domain LISSOM code, for self-organization in
laterally connected maps, will be announced in the near future.

- -- The Authors

- -------------------------------------------------------------------------
SELF-ORGANIZATION, PLASTICITY, AND LOW-LEVEL VISUAL PHENOMENA 
IN A LATERALLY CONNECTED MAP MODEL OF THE PRIMARY VISUAL CORTEX

Risto Miikkulainen, James A. Bednar, Yoonsuck Choe, and Joseph Sirosh
Department of Computer Sciences, The University of Texas at Austin.
In R. L. Goldstone, P. G. Schyns, and D. L. Medin (eds.), Psychology
of Learning and Motivation, vol. 36, 1997 in press (36 pages).
ftp://ftp.cs.utexas.edu/pub/neural-nets/papers/miikkulainen.visual-cortex.ps.Z

Based on a Hebbian adaptation process, the afferent and lateral
connections in the RF-LISSOM model organize simultaneously and
cooperatively, and form structures such as those observed in the primary
visual cortex. The neurons in the model develop local receptive fields
that are organized into orientation, ocular dominance, and size
selectivity columns. At the same time, patterned lateral connections
form between neurons that follow the receptive field organization. This
structure is in a continuously-adapting dynamic equilibrium with the
external and intrinsic input, and can account for reorganization of the
adult cortex following retinal and cortical lesions. The same learning
processes may be responsible for a number of low-level functional
phenomena such as tilt aftereffects, and combined with the leaky
integrator model of the spiking neuron, for segmentation and
binding. The model can also be used to verify quantitatively the
hypothesis that the visual cortex forms a sparse, redundancy-reduced
encoding of the input, which allows it to process massive amounts of
visual information efficiently.

- -------------------------------------------------------------------------
A SELF-ORGANIZING NEURAL NETWORK MODEL OF THE PRIMARY VISUAL CORTEX

Joseph Sirosh
Department of Computer Sciences, The University of Texas at Austin.
PhD Dissertation and Technical Report AI95-237, August 1995 (137 pages).
ftp://ftp.cs.utexas.edu/pub/neural-nets/papers/sirosh.diss.tar

This work is aimed at modeling and analyzing the computational processes
by which sensory information is learned and represented in the
brain. First, a general self-organizing neural network architecture that
forms efficient representations of visual inputs is presented.  Two
kinds of visual knowledge are stored in the cortical network:
information about the principal feature dimensions of the visual world
(such as line orientation and ocularity) is stored in the afferent
connections, and correlations between these features in the lateral
connections. During visual processing, the cortical network filters out
these correlations, generating a redundancy-reduced sparse coding of the
visual input.  Through massively parallel computational simulations,
this architecture is shown to give rise to structures similar to those
in the primary visual cortex, such as (1) receptive fields, (2)
topographic maps, (3) ocular dominance, orientation and size preference
columns, and (4) patterned lateral connections between neurons.  The
same computational process is shown to account for many of the dynamic
processes in the visual cortex, such as reorganization following retinal
and cortical lesions, and perceptual shifts following dynamic receptive
field changes.  These results suggest that a single self-organizing
process underlies development, plasticity and visual functions in the
primary visual cortex.

- -------------------------------------------------------------------------
TILT AFTEREFFECTS IN A SELF-ORGANIZING MODEL OF THE PRIMARY VISUAL CORTEX

James A. Bednar
Department of Computer Sciences, The University of Texas at Austin.

Masters Thesis and Technical Report AI97-259, January 1997 (104 pages).
ftp://ftp.cs.utexas.edu/pub/neural-nets/papers/bednar.thesis.tar

The psychological phenomenon known as the tilt aftereffect was used to
demonstrate the functional properties of RF-LISSOM, a self-organizing
model of laterally connected orientation maps in the primary visual
cortex.  The same self-organizing processes that are responsible for
the development of the map and its lateral connections are shown to
result in tilt aftereffects as well.  The model allows analysis of
data that are difficult to measure in humans, thus providing a view of
the cortex that is otherwise not available.  The results give
computational support for the idea that tilt aftereffects arise from
lateral interactions between adapting feature detectors, as has long
been surmised.  They also suggest that indirect tilt aftereffects
could result from the conservation of synaptic resources.  The model
thus provides a unified computational explanation of self-organization
and both direct and indirect tilt aftereffects in the primary visual
cortex.

- -------------------------------------------------------------------------
LATERALLY INTERCONNECTED SELF-ORGANIZING FEATURE MAP IN HANDWRITTEN 
DIGIT RECOGNITION

Yoonsuck Choe
Department of Computer Sciences, The University of Texas at Austin.

Masters Thesis and Technical Report AI95-236, August 1995 (65 pages).
ftp://ftp.cs.utexas.edu/pub/neural-nets/papers/choe.thesis.tar.Z 

An application of biologically motivated laterally interconnected 
synergetically self-organizing maps (LISSOM) to off-line recognition of 
handwritten digit is presented. The lateral connections of the LISSOM map 
learn the correlations between units through Hebbian learning. 
As a result, the excitatory connections focus the activity in local 
patches and lateral connections decorrelate redundant activity on the map.
This process forms internal representations for the input that are 
easier to recognize than the input bitmaps themselves or the activation 
patterns on a standard Self-Organizing Map (SOM). The recognition rate 
on a publically available subset of NIST special database 3 with LISSOM 
is 4.0% higher than that based on SOM, and 15.8% higher than that based 
on raw input bitmaps. These results form a promising starting point for 
building pattern recognition systems with a LISSOM map as a front end. 

------------------------------

End of Neuron Digest [Volume 97 Issue 7]
****************************************