🚀 go-pugleaf

Archive-name: auto/comp.ai.neural-nets/Technical-Report-available

The following technical report is now available via ftp. It is placed in the 
Cambridge University ftp archive svr-ftp.eng.cam.ac.uk. The ftp instructions 
for retrieval follow the abstract. 

------------------------------------------------------------------------------
	A function estimation approach to sequential learning with
			neural networks


		Visakan Kadirkamanathan and Mahesan Niranjan


      		    Technical Report CUED/F-INFENG/TR.111

		 Cambridge University Engineering Department
			Cambridge CB2 1PZ, England



				ABSTRACT

In this paper, we investigate the problem of optimal sequential learning, 
viewed as a problem of estimating an underlying function sequentially rather 
than estimating a set of parameters of the neural network. 

Firstly, we arrive at a sub-optimal solution to the sequential estimate which 
can be mapped by a growing Gaussian radial basis function (GaRBF) network. 
This network adds hidden units for each observation. The function space 
approach in which the estimates are represented as vectors in a function 
space, is used in developing a growth criteria to limit its growth. 
A simplification of the criterion leads to two joint criteria on the distance 
of the present pattern and the existing unit centres in the input space and 
on the approximation error of the network for the given observation to be 
satisfied together. This network is similar to the resource allocating network 
(RAN) and hence RAN can be interpreted from a function space approach to 
sequential learning. 

Secondly, we present an enhancement to the RAN. The RAN either allocates a 
new unit based on the novelty of an observation or adapt the network 
parameters by the LMS algorithm. The function space interpretation of the RAN 
lends itself to an enhancement of the RAN in which the extended Kalman filter 
(EKF) algorithm is used in place of the LMS algorithm. 
The performance of the RAN and the enhanced network are compared in the 
experimental tasks of function approximation and time-series prediction 
demonstrating the superior performance of the enhanced network with fewer 
number of hidden units. The approach adopted here has led us towards the 
minimal network required for a sequential learning problem. 

------------------------------------------------------------------------------

FTP INSTRUCTIONS:

unix> ftp svr-ftp.eng.cam.ac.uk
Name: anonymous
Password: (send userid@site)
ftp> cd reports
ftp> binary
ftp> get kadirkamanathan_tr111.ps.Z
ftp> quit
unix> uncompress kadirkamanathan_tr111.ps.Z
unix> 

--
Dr.Visakan Kadirkamanathan                    email: visakan@uk.ac.cam.eng
Cambridge University Engineering Department
Trumpington Street, Cambridge CB2 1PZ, UK.    Tel  : +44 223 332754
------------------------------------------------------------------------------

===========================================================================
Co-moderator:  Richard Golding, Computer & Information Sciences, UC Santa Cruz
	    compdoc-techreports-request@ftp.cse.ucsc.edu