🚀 go-pugleaf

In <BvAyE7.7F3@ens-lyon.fr>, Jean-Yves Briend writes:
> 
>    Well, in the name of the news group, there is the word "theory". But I 
> don't see much theory round here... Could'nt it be possible to post some 
> TeX article or some general references?

hmm, I agree only partially. It seems that \frac{signal}{noise} \ge 20dB.
It might be worse; take a look at comp.theory :-(


Well here is a very serious question:

"How can I compute directed information flow rates in dynamical systems ?"


* Suppose we have an a priori high dimensional dynamical system such as a 
  model of a chemical oscillator (e.g. the famous FKN-model or the 
  "Oregonator" or "Montanator").
  The question is how the various variables (i.e. chemical species) are 
  coupled to each other.  In a dumb approach, one could compute the 
  correlation functions, but this is a linear approach and thus supposed to 
  be inadequate.

* An adequate approach is to compute the amount of information flowing 
  between these variables.
  
  The mutual information K(i;j) between two variables i and j is a better 
  solution, but there is no direction implied. 
  
  There exists a paper from K.Matsumoto and I.Tsuda, "Calculation of 
  information flow rate from mutual information", J.Phys. 21A, 1405-1414 
  (1988), in which this is done for maps; The basic definitions are:

      K(i;j) = K(i --> j) + K(j --> i) + K(i and j)                         (1) 
 
  K(i;j) is the mutual information as defined by, e.g. Fraser (see IEEE 
  Trans.Inform.Theory 35, 245 (1989).)
  
  K(i --> j) is the "information, that first appears in i and then moves to 
  j"; the definition is:

      K(i --> j) = lim_{n --> \infty} I(i_0, ... ,i_n ; j_n) - K(i and j)   (2)

  I(i_0, ... ,i_n ; j_n) is the mutual information between j_n and the 
  elements { i_0, ... , i_n }, which lie "before" j_n in the sense of 
  time. In other words, it is equal to the number of bits with which j_n 
  can be predicted if the sequence { i_0, ... , i_n } is known.

  K(i and j) is defined as:

     K(i and j) =
        lim_{n --> \infty} I_{i_0,j_0, ... ,i_{n-1},j_{n-1}} (i_n;j_n)      (3)

  which means the conditional mutual information from (i_n,j_n) independent 
  of the predecessors.

  K(j --> i) can be defined in the same way.

* K(i;j) is accessible via Fraser's standard algorithm, so the question is 
  how to compute the desired quantities of directed information flow,
  K(i --> j) and K(j --> i) ?
  (The authors of the paper cited compute K(i;j), the "cross information flow 
  rate", only.)

  Is the term K(i and j) simply equal to K(i,j), the common information from 
  both time series, as defined in the basic relation:

      K(i;j) = K(i) + K(j) - K(i,j)                                         (4)

  If so, how to realize the computation of (2) ? In terms of time delayed 
  discrete measurements of the continuous system one would approach the series 
  {i_0, ... ,i_n ; j_n} as:

      {i_0, ... ,i_n ; j_n} := {x(t),x(t+T), ... ,x(t+nT)}                  (5)

  where T is the delay time, equal to a multiple of the measurement interval,
  \tau, i.e.:
  
         T = N \Delta \tau                                                  (6)

  A computation of I(i_0,...,i_n;j_n) = I[x(t),x(t+T),...,x(t+nT); y(t+nT)]
  is then straightforward.

* Thus the basic questions are:
  (a) How to choose a proper delay time T for eq. (5) ? Any criteria ?
  (b) Is K(i and j) really equivalent to the definition of K(i,j), the 
      information from both time series ?
  (c) Is I(i_0,...,i_n;j_n) (from eq. (2)) really a marginal redundancy or 
      simply the common information. I doubt that this is really the marginal 
      redundancy, since the term K(i and j) is subtracted. I suspect that
      K(i --> j) is the marginal redundancy itself. 
  (d) How to choose n ? From the definition of the limes it should converge 
      for n --> \infty, but it is well known that the data requirements are 
      astronomically for n > 4. So can I just take the maximum in a plot 
      of I() vs. n ?


I'm sorry for my questions relating to the definitions of the K's, but the 
paper itself is somewhat confusing, and I'm a mathematically poor chemist only.

May be there is some expert out there who can enlighten me.

Otherwise, I would appreciate any pointer to monographies and/or articles on 
that topic.

If I find out the solution, I'll post here and will supply the algorithm as 
usual via anonymous ftp on 'ftp.phys-chemie.uni-wuerzburg.de'. 
 

Thanks for the help,
                    Thomas.
-- 
Thomas-Martin Kruel  Institut f. Physikalische Chemie / Universitaet Wuerzburg
                     Marcusstr. 9-11, D-8700 Wuerzburg, Tel. +49 931 31579