🚀 go-pugleaf

>[Please note follow-ups]

In article <343C022B.2087@dial.pipex.com>, Howard Snart
<aac88@dial.pipex.com> wrote:

>Thanks for your reply. 
>
>Louis Savain wrote:
>> 
>> 
>> >1-firing a neuron strengthens its links
>> 
>>   I have to disagree with this......
>I understood this to be true, but it is not critical to the scheme. 
>
  Actually I think this is very important.  This is the method that
gets rid of uncorrelated links.  Of course, if an input link can also
be weakened if it repeatedly fails to fire before the target neurons
fires.  What causes the target neuron to fire in the first place is a
little something I'm saving for later.  I'll just say it's related to
causal order.  Like everything else.

>> >4-once fired, the link is temperorerily boosted(its weighting is
>> >increased), this decays quite fast. I term this increase 'boost', for
>> >want of a better term.
>> 
>>   I'm not sure what you mean by the fast decay part.  Again here, I
>> would disagree that the simple firing of a neuron is enough to
>> strengthen its synapses.  The neurophysiological evidence says
>> otherwise.  We interpret this as part of the natural method used by
>> the nervous system to weed out non-associated synapses.
>Let me clarify. 
>A link has a strength, or weight (as per usual). In this scheme, the
>links strength, can be thought of as the summation of 2 parts:
>-the weight: This changes slowly, in response to use, as per Hebbian
>learning. The weight degrades slowly.
>-the 'boost': This changes quickly w.r.t. the weight. Once fired, a
>synapses 'boost' increases, this degrades fast w.r.t. the weight. This
>could be thought of in terms of increasing the neurotransmitter levels
>in respose to excitation.

  I think I vaguely see what you are saying.  You mean potentiation or
something similar.  Correct me if I'm wrong.

>[...]
>>   Signal propagation speed is important but not crucial.  What is
>> crucial is the temporal relationship between signals.
>
>The reason I include signal propogation speed is because, I believe, it
>important for convergense in this scheme. I include it such that signal
>propogation in my net will progress in a wave. I do not want a disparate
>neurone ( in terms of its position in 3d space.. or nD space) to fire
>instataniously, just because it is topographically 1 away.

  Well, let me say that if you have a system based on temporal order
and assuming the speed characteristics of the various neurons do not
change much w.r.t. each other, you will find a significant degree of
self-adjustment and error tolerance, even if the neurons are not
synchronized or have different processing speeds.

>>   This is the straight Hebbian adaptive mechanism.  Unless a temporal
>> factor is added, not much will be learned by this.
>> 
>.....
>>   Sure.  But you need a strong causal ordering mechanism for
>> correlations to be useful or even meaningful.  Actions (output effects
>> or behavior) will depend on conditions.  Conditions are the sufficient
>> causes of effects.  Unless the network is designed to deal with causes
>> and effects from the start, not much will be learned.
>> 
>yes it is. The temporal factor is supplied by my sinchronised inputs, my
>boosted paths, and my basic nervous system. A great deal of work will
>have to be done to establish the correct basic nervous system, and the
>brain topology to promote the desired behaviour.

  There is more to temporal relationship than contiguity.  There are
also predecessors and successors.  This is important if you are
looking for conditions or causes.  These things are essential to the
acquisition of common sense.

>[...]
>>   This is getting into even more uncharted territory.
>One of the purposes of this posting is to see if I am stating the
>blatently obvious.... or to see if there is something here worth
>investigating.

  It's definitely worth pursuing but if I were you I would check out
work already done in this field or wait for us to publish our package.

>[...]
>>   This is what David Longley would call "intensional" stuff and I
>> would agree.  The best thing to do in situations where speculation is
>> beginning to get out of control is to run a set of experiments.
>> 
>That is what I intend to do... I just wanted to see if I was
>re-inventing the wheel...

  Maybe but more heads are better than one.  The more the better.
 
>> >16- from 4: Using a thought, gears up the mind for thinking that way.
>> >So, for example, the brain can gear itself up to spotting patterns, like
>> >seeing a computer game when your eyes are closed.
>> 
>>   I have no idea what this means.
>
>This is quite important. 
>If my clarification of 'boosted' links in this response makes sense
>then:
>-Input will cause a certian cascade firing pattern.
>-Thus, that will 'boost' those pathways.
>-Thus, on recieving the next input , the neural firing is more likely to
>progress along the 'boosted' pathways than it would otherwise. 
>-Thus I can corralate between temorally distinct inputs.
>-This is what I ment by an Idea Mesh, and the 'current thought'(agreed
>its a fuzzy term, but I hope you understand what I mean by it) 

  This maybe important but until you have a working model to tinker
with you'll never know for sure.

>[...]
>
>What I have been describing is a method for teaching a very deep neural
>net using Hebbian learning. My introduction of the 'boost' is something
>I believe happens (this has been confirmed to me after discussions with
>various research neuro-scientists), and I believe it has important
>concequences for learning, some of which I have outlined here. 
>In this scheme, the basic nervous system filters the inputs and
>feedback, and this teaches the net. They effectively modify the inputs
>to provide the training sets. They need not be static, and can therefore
>change with time to affectively alter the training data when the desired
>initial behaviour has been established. This, again, has biological
>precident.
>
>From what I have gathered from the morsels of information you have been
>releasing about your scheme, your description of the neurone and synapse
>should apply well to this scheme. I would be inturested to hear your
>oppinion on this.

  What you are calling 'boost' we may be calling something else
although I can't think of an exact analogy offhand.

>If you follow my reasoning so far, then the obvious problems to be
>solved are:
>-Producing the basic nervous system for a given scenario. (this I see as
>the hardest problem.)

  Why "for a given scenario?"  If by "scenario" you mean "domain" or
"environment", I would think that your network should be designed in
an object-oriented fashion and with generality in mind.  IOW, it
should be ready to be plugged into any sort of environment.  Just add
the proper sensors and effectors and fire it up.  Of course, it is
never this easy.

>-Designing the brain topology, to promote the desired structures,
>behaviours and to take advantage of the inputs.

  I'm not sure what this means.  If you mean creating sub-systems or
sub-networks to handle different domain such as one for touch, one for
vision, etc., I agree.  Then all you would need to do is find a way to
integrate the outputs of the sub-networks.  I readily admit that it is
easier said than done.  It will require extensive experimentation.

>As I have previously stated here; My primary purpose in posting this is
>to try and establish whether this approach is new, or worth
>investigating.

  I don't know how new this is because I have only a cursory
understanding of it.  All I can tell you is that any idea is worth
pursuing in this business.

> I have read most of the Neuro-science, Neural-net, and AI
>journals(more specifically for the past 3 years), and not found this
>approach anywhere.

  The Hebbian part of it is not new.  I'm not sure about the 'boost'
part.

> But this does not mean that it is not well
>established, and had been previously discounted.

  So what if it is discounted.  If you think it's worth it even after
searching the litterature, you should pursue it.  You should make up
your own mind.  For instance, David Longley has posted lots of quotes
from various authorities in an attempt to discount induction as a
viable method of learning.  I can't tell you that this will not deter
either me or most of the other people in this field from continuing
our work.  If we fail, oh well, at least we tried and we had some fun.
We'll give him a call when we are old and gray and one or the other
will say "I told you so".  :-)

>I do make ambitious claims for this approach, and yes, I am excited by
>the possibilites it presents (both in terms of Neural-nets, and in terms
>of explaining biological mechanisms).

  Biological plausible neural nets that are based on causality are
good things to be excited about.  It's the way to go IMO.

>I need to properly research the scheme, both in terms of further
>research and in terms of experimentation and demonstration. BUT, I
>wanted to make sure I wasnt re- inventing the wheel before commiting
>myself on such an undertaking.
>
> Many thanks for your constructive criticisms,

  You are welcome.  Even if you are reinventing the wheel, so what?
You gain experience this way and you can always use your experimental
framework or test-bed for further research.

Regards,

Louis Savain

President, Marengo Media, Inc.