🚀 go-pugleaf

herwin@mason1.gmu.edu (HARRY R. ERWIN) writes:

>It has been suggested by a number of workers (Hebb, Marr, Carpenter, and
>Grossberg) that the brain supports autoassociative systems. It has been
>pointed out to me that given one way the neuron functions (typically by
>detecting synchronization), an autoassociative network involves the
>creation of short-term coherent states. Those can be easily read by
>sampling techniques. Hence it is an easy evolutionary step from a system
>of cooperating autoassociators to a system of 'antennae', with known
>objects corresponding to coherent states in each antenna. If the
>coherent states can be generated fast enough, such a system requires
>much less connectivity than the autoassociator system. Who has been
>working in this direction?

Sounds similar to the resonances for copying spatiotemporal patterns that 
I emphasize in CONVERSASTIONS WITH NEIL'S BRAIN, last chapter.  There's a 
slightly different take on the subject in my Scientific American article
that'll be out in October.
    William H. Calvin   WCalvin@U.Washington.edu
    University of Washington  NJ-15
    Seattle WA 98195 USA     fax +1(206)720-1989

The basic act of perception is Thing from non-Thing: light from darkness,
finding the particions and the dimensionality which allows the heap to be 
sorted. Systems have to learn the programme (etc) by which they do this.
The prime issue is that of the objective function: how does the system
"know" when it is doing better or worse? In control theory terms, what
constitutes the error feed back; not least when what is being detected is
discontinuous and non-linear, in the sense of being part of both the 
detection and sensorium sides of the equation.

I have suggested elsewhere that an important element is a "!" detector: that
which detects conditions in some way and says "! Note this! Learn this!" such
that certain states become re-inforced. If the system which is being 
strengthened itself plays a part in the "!" process, then we can see (vaguely)
how this might bootstrap itself into a state from which higher-order patterns
might merge.
_________________________________________________

  Oliver Sparrow
  ohgs@chatham.demon.co.uk

In article <769860827snz@chatham.demon.co.uk>,
Oliver Sparrow <ohgs@chatham.demon.co.uk> wrote:
>I have suggested elsewhere that an important element is a "!" detector: that
>which detects conditions in some way and says "! Note this! Learn this!" such
>that certain states become re-inforced. If the system which is being 
>strengthened itself plays a part in the "!" process, then we can see (vaguely)
>how this might bootstrap itself into a state from which higher-order patterns
>might merge.

You're talking about value systems. It is my understanding that these
have been found in the brain and central nervous system, corresponding
to (for instance) the ability of a person to hold something in their
hand.

I don't know whether the "!" value system itself has value systems,
but it is possible.

Apparently, they work by averaging (in some sense). In that, when the
'note this' occurs, all the inputs weights are changed to reflect the
current inputs further.

Any inputs that have nothing to do with the 'note this' average out,
but the ones that have a real link, don't.

Some of the things that I expect to be important:

- stress
- holding things in hands
- sex
- talking
- listening
- eating
- physical contact

This is probably why holding hands and talking with your (new)
girlfriend, and eating chocolate is so popular. ;-)

>_________________________________________________
>
>  Oliver Sparrow
>  ohgs@chatham.demon.co.uk

-Ian

William Calvin (wcalvin@u.washington.edu) wrote:
: herwin@mason1.gmu.edu (HARRY R. ERWIN) writes about coherence
:: detection systems as a natural evolutionary step beyond
:: autoassociative systems and asks who is working in this area.

: Sounds similar to the resonances for copying spatiotemporal patterns that 
: I emphasize in CONVERSATIONS WITH NEIL'S BRAIN, last chapter.  There's a 
: slightly different take on the subject in my Scientific American article
: that'll be out in October.

I suspect this goes a bit further, although there's a related article in
Neural Networks, Vol 7, Nr 3--Matsugu and Yuille, 1994, "Spatiotemporal
information storage in a content addressable memory using realistic
neurons." pages 419-440. Once you have a coherence detector, the
selective pressure will be on reducing the information needed to declare
a coherence is forming and doing the declaration as early as possible.
That will evolve towards a system where the downstream elements will cue
the upstream elements as soon as they've declared the coherence so that
the upstream elements can move to the next coherence without finishing
the previous one. Freeman seems to be pushing me to think in this
direction, but my foundations aren't firm yet. Something of this nature
is needed to address the evidence for nonstationary dimensionality in
EEG dynamics. 

--
Harry Erwin
Internet: herwin@gmu.edu 
Working on Katchalsky nets....

Oliver Sparrow (ohgs@chatham.demon.co.uk) wrote:
: ...
: The prime issue is that of the objective function: how does the system
: "know" when it is doing better or worse? In control theory terms, what
: constitutes the error feed back; not least when what is being detected is
: discontinuous and non-linear, in the sense of being part of both the 
: detection and sensorium sides of the equation.

For olfactory processing, this seems to involve the norepi system. I
haven't addressed that here, since it's about four stages down the road:
1. model the connectivity of the OB and PC in greater detail than
   has so far been done. Freeman's models of the OB are relatively simple,
   and my extension has not been towards the biology. I have to remedy
   that.
2. study the coherent states that develop. What is required to
   full-blown coherence? Are most such states transients?
3. examine the process by which the components of the olfactory
   system communicate. Bandwidth is key. 
4. And then address training. (Skinner has done work on the biology
   here). 

: I have suggested elsewhere that an important element is a "!" detector: that
: which detects conditions in some way and says "! Note this! Learn this!" such
: that certain states become re-inforced. If the system which is being 
: strengthened itself plays a part in the "!" process, then we can see (vaguely)
: how this might bootstrap itself into a state from which higher-order patterns
: might merge.

It's the norepinephrinergic system for the olfactory sense. It reverses
the habituation of excitatory cortico-cortical connections. That causes
stp and ltp for cc connections involved in the patterns present. Here,
we're dealing with the reticular formation of the pons and medulla (locus
coeruleus and lateral tegmental cell system). Each NE neuron in those
areas projects to diverse areas of the brain, so you're basically dealing
with a 'fire alarm' system. It's not quite as bad as that, since it gets
triggered quite frequently (at least once every time you have an orientation
reflex). It seems to react nonspecifically to novelty.

--
Harry Erwin
Internet: herwin@gmu.edu 
Working on Katchalsky nets....

Some comments from Leslie Kay (lmk2@garnet.berkeley.edu) and my
responses (edited heavily and posted with permission):

>In article <2s8ihf$nbo@portal.gmu.edu> you write:
>>
>>For olfactory processing, this seems to involve the norepi system. I
>>haven't addressed that here, since it's about four stages down the road:
>>1. model the connectivity of the OB and PC in greater detail than
>>   has so far been done. Freeman's models of the OB are relatively simple,
>>   and my extension has not been towards the biology. I have to remedy
>>   that.
>
>I'd be interested to know what you would add.  The drive over here 
>has been to try to simplify the model so that the topological features
>can be derived. 

I think it's too simple. The interesting differences between an
autoassociator and a coherence detector should emerge when the
connectivity required for an autoassociator is too big. I'll be
increasing the number of activation networks to around 10 (Freeman and
Yao uses one and I worked with two) and the number of glomeruli to 20.
I'll also rewire things a bit in the PC and AON and sample the
activation networks a bit more lightly, too. (An activation network
is a collection of excitatory KI networks connected by strongly
potentiated cortico-corticals. In other words, it's the network that
wins or loses when a known pattern is seen. I'd like to see the
circumstances under which you get a 'symmetry-breaking' when known
patterns are presented. Once the symmetry is broken, the next
question is whether the system can attain a globally coherent state.
coherence==synchronization of KI networks in this context.)

>Adding more detail often doesn't work, because it
>is a macroscopic model.  THe things that happen at the mass action
>level are the result of hundreds of thousands of neurons all doing
>approximately what they are supposed to do.  So, to try to put more
>pieces into the OB may make the model less physiological.  Unless,
>of course, you are speaking of adding the noradrenergic "bathing"
>that the neurons get from time to time. (BTW noradrenergic is the
>proper word for norepinephrinergic, or at least the more commonly
>used word.)

I'm still not 'mass-action' oriented enough for Dr. Freeman's
liking. There's got to be some way of getting to the mass action
level without abandoning the middle level (between neuronal models and
the 'thermodynamic' models). Huberman and Glance have done some elegant
work in this direction, but to push it suitably will take more time
than I have right now.

Thanks for the correction. That's one of the advantages of grad school.
I'm starting in the fall, but not full time yet. 8( The norepi
'bathing' that the neurons get is step 4 in my plan.
>
>>2. study the coherent states that develop. What is required to
>>   full-blown coherence? Are most such states transients?
>
>I'm not sure there ever is "full-blown" coherence.  Again, this
>is an approximate thing at the mass action level.  And Yes, I
>believe that all states in the brain are transients.  In fact,
>perhaps it is a misnomer to say "states" at all, since really 
>there are only processes.  It is we, with our awareness or
>consciousness or whatever, who say that there are states of
>perception or awareness.

States are transients--yes, but if there's enough settling time, they
can be considered as separate things in themselves. 
>
>>3. examine the process by which the components of the olfactory
>>   system communicate. Bandwidth is key. 
>
>Absolutely.  I think that it is not the bandwidth, per se, but rather
>some other process, which we measure as bandwidth.  Really it is
>a particular part of the brain doing a particular thing at a particular
>time and its relationship with the part to which it is connected, which
>we read as bandwidth.  But it seems that the same structure may exhibit
>different bands for different functions (I think the EC is doing this).

The bandwidth measure will be the key parameter that will force one or
another model to be accepted. If the bandwidth supported by the OB/PC
interface is too low, direct sampling will not be supportable. Rather
the OB will have to sample itself and report the results to the PC.
Instead of the OB's being characterized by the cells in the PC as
'seeing a semantic object' by direct measurement, the OB will recognize
it is 'seeing' something it knows and enter a coherent state that can
be recognized by the PC. There's a lot more here, and it also relates to
some nasty unsolved problems in chaos theory and evolution. Whoever
_really_ solves it will deserve a Nobel Prize. I know some very good
physicists who think they're not really up to it, and Prigogine got his
prize in this area. It's a nasty one. Having my nose pushed into it
last week resulted in four rewritten presentations.
>
>>4. And then address training. (Skinner has done work on the biology
>>   here). 

HRE Clarification: understanding how the OB is trained via that
norepi interaction. Skinner has some interesting results here.
>
>
	
Cheers,
   harry erwin

The Human state of aparia is so intense that if it didn't create God, it 
certainly has been working quite hard to get Its Almight Attention. i'd 
wager most animals have a similar relationship with the enviornment: they 
may not dance to please the gods(?), but when it doesn't rain for too 
long something in them must scream "This is unnacceptable!". the trick, 
in my arrogant opinion, is to get an artificial mind to scream like that. 
Not simply to come up with the silicon version of chocolate.

It should also be considered just what our reaction should be to an 
existential cry for help from an ai. Should we be any more or less 
helpfull to them than our gods have been to us?

Sean Thomas McGill