🚀 go-pugleaf

Let me begin by saying that I am new to this news group, and so have not
had the opportunity to follow this discussion as much as i'd have liked.
I agree with much (perhaps all) of what I have understood of Louis
Savain's approach.
Some months ago I came up with the following idea, which seems to
explain, to me, much of  how the brain works, and is, as far as I can
tell, origional. (please tell me if I am mistaken).

This approach stems from several misgivings I have over the existing
methods of teaching neural nets... most significantly
backtracking.(which I see as un-scalable, and fundementally wrong... you
force the outputs to conform, whereas the problem may lie far from the
outputs)
I also think that we know all we need to know to produce a good
description of how the brain functions.

I base this scheme on the following (established) points:
1-firing a neuron strengthens its links
2-a neuron firing causing another one to fire strengthens the link even
more.
2a-a neuron firing, without an input firing, weakens that input link.
3-the links strength slowly degrades through lack of use, and can
eventually be destroyed.
4-once fired, the link is temperorerily boosted(its weighting is
increased), this decays quite fast. I term this increase 'boost', for
want of a better term.
5-a neuron has a 'recharge' time, thus they cannot fire continusouly.
They can only fire, typically, once every millisecond

ok,
now, given a weakly connected, deep, network... my equivalent to a new
brain..
These neurones are connected like those in a biological brain, and
signal propogation along the synapses is not instantaneous. i.e. it
takes time for a signal to pass down the synapse.
...and a source of input... say an eye.. i.e.

        |---------------|
eye --> |    brain      | --> outputs
        |---------------|

Now, when I stimulate my 'eye':
 6-from 1: The input neural firing will not initially propogate, because
of the weak synapses. But, this firing strenghtens the input synapses,
and, eventually, the next layer will fire. Thus it progresses, so
eventually, given enough stimulous, we will end up with outputs as a
result of inputs... or be they random.
 7-from 3: If it does not get input, then a neurons strength will
degrade, and could eventually be destroyed. 

This next point could be demonstrated with just my 'eye', but, for
clarification, i'll include another input source... an 'ear'

i.e.
ear --> |---------------|
eye --> |    brain      | --> outputs
        |---------------|

 8- from 2,6: Now, because I am being stimulated by the 'real world', my
inputs will be corralated. e.g. if I sit watching someone clap their
hands. Thus i will have waves of neural firings propogate from my input
neurones. Thus I will have constructive interference where those waves
intersect. i.e. The synchronous inputs will cause those neurons
stimulated as a result of both inputs to be reinforced. 
 9- from 8: Thus, I should see structures emerge that correspond to the
corralation of the inputs. 
 10- from 9: Now, given corralated input, these corralation points
should be adequately stimulated, and be strong centres of neural firing.
 11- from 10: Thus, I should end up with outputs that correspond to
corralated inputs. e.g. by looking at the outputs I might be able to
find a neurone that fires when I am looking at a red bell. 

Note, now we can corralate inputs and produce some outputs based on that
corralation. This is still not very usefull as the outputs are random.
If I add a further 'sense'. This equates to 'right' and 'wrong'. It
fires more when the current situation is 'good'. 

i.e.
ear --> |---------------|
eye --> |    brain      | --> outputs
good--> |---------------|

 12- from 11: I should now produce outputs that tend to corralate, and
maximise my inputs. Outputs that cause my good sense to fire more will
increase the good sense, and therefore become prevalent. 'bad' behaviour
is not removed, it is superceeded by 'good' behaviour.

But, what is 'good'?, and how does it apply?
Now, if I remove the good sense, and replace it with a basic set of
drives. This equates to a very basic, low level, behaviour. This equates
to our basic nervious system. 
e.g.

ear --> |---|---------------|---|
eye --> |-A-|    brain      |-B-| --> outputs
        |---|---------------|---|

A = input basic nervous system. 
B = output basic nervous system.

The job of A is to simplify the environement, to amplify 'good' input(
or reduce 'bad' input). 
The job of B is to simplify the brain outputs, and to provide multiple
paths to the actual outputs.

The initial basic drive could be 'closeness to parent = good'. And, in
my simple creature, this could be represented by my 2 sense organs as
'red' and 'noisy'.
The basic nervous system, upon encountering 'red' or 'noisy' will
amplify that data, or reduce input that is not 'red' or 'noisy'.

 13- from 12: 'good' imputs, i.e. the strong inputs, are likely to
produce the initial structures, and will stimulate the brain more. 
 14- from 13,6,8: Thus 'good', corralated behaviour is likely to result.
Outputs that enhance 'good' behavoiur will be prevalent, the net will
have learned to solve the situation encoded in A and B.
 

Further:
15- from 3: Old memories will gradually degrade, if not used. The
destructuion of unused, or redundant, neurones could be seen as an
optimisation process.
16- from 4: Using a thought, gears up the mind for thinking that way.
So, for example, the brain can gear itself up to spotting patterns, like
seeing a computer game when your eyes are closed. 
17- from 4: This is equavalent to the current 'thought' that the brain
is having. And is equivalant to an idea 'mesh'.
19- from 17,4: If the neurone becomes 'stressed' it could be that its
'recharge' time, and boost duration become large. This would make the
brain stagnate, and reduce its capacity for new thought ( the current
thought links would be too established). Thus sleep is needed, to
replenish the neurones and reverse this process. 
20- from 19: It is possible that this process is important, so that an
important idea can dominate, so a solution can be found. 
21- from 19,20: 'stressed' neurones = overloaded boosted links =
physical(chemical) exhaustion. The increased long term link between
neurones means gets stressed less, under the same load.
22- from 8,9,10,11: If there is a structure that is regularly
stimulated, like 'red'. Such structures could be regarded as inputs to
the rest of the brain. Thus corralation between these can prevail. Thus
complex, derrived, behavoiur can be produced.
23- from 5: This prevents cyclic loops on the small scale, and so helps
prevent paralysis.

NB:
-this scheme will not work well with small numbers of neurones.
-there are a lot of variables to this scheme, and finding values that
promote convergence might be hard.

Sorry to go on, but i'd be very inturested to know what you think.
This represents a condensed, and simplified version of the scheme. There
are further concequences which I havnt mentioned here.

Howard Snart