Home
Contact
Short CV
Research
Publications
Software
Links
Impressum

University of Plymouth | Centre for Theoret. and Comput. Neuroscience | Home   

Stochastic Interaction and Complexity in Neural Systems

In some neural systems it make sense to code information into independent channels as for instance in the retina where information from the outside world has to be sqeezed into a relatively small number of optical nerve fibres. Linsker's Infomax principle, for example, claims that neurons code as much information as possible in their outputs about the stimuli at their inputs. This has been convincingly demonstrated in the retina, and in a modified version also explains the structure of simple cells in the primary visual cortex.

On the other hand, in order to "process" information from various channels in any given situation the information has to be brought together in the brain, that is, the neurons coding for some entities or events have to interact with each other. This is believed to be expressed in various types of spatio-temporal correlation patterns described by physiologists, e.g., synchronised oscillations, synfire chains, or more general "functional couplings". This idea suggests an ever on-going re-organisation process of activity in the brain perhaps in a way comparable to THIS abstract simulation of a swarm of rats or THIS simulation of a network of coupled oscillators.

Stochastic Interaction is an information-theoretic measure that quantifies dependencies in systems comprising of n sub-elements (like neurons,rats,or oscillators). Is is formally defined as the Kullback-Leibler divergence of the joint-distribution over the system states from independence, ie, the product of marginal distributions. This quantity has also been termed "multi-information" in generalisation of "mutual information" for just two coupled systems. However, the concept can be generalised from distributions over states to the probabilistic properties of state transitions, too. In this case the generalised measure is formally based on the mathematical concept of "Markov chains" instead of probability distributions, and it is called spatio-temporal stochastic interaction. In contrast to multi-information, it does not measure the common information in the states of a neural system, but the information shared in their temporal behaviour; in a sense it states how strongly the units cooperate.

We believe that stochastic interaction plays a role in the real nervous system and have therefore studied it in that context. The papers below show that the interaction in associative spiking neuron systems is indeed high (Wennekers and Ay, 2004). Ay and Wennekers (2003) demonstrates properties of systems with particularly high -- that means, maximised-- stochastic interaction. Those turn out to be almost deterministic if looked at from a global perspective, but the single unit activity is nonetheless almost random. A similar situation seems to pe present in cortex according to the synfire chain idea, where single neuron spike trains are almost random and do not carry much information, but global activation patterns are virtually deterministic by following well-defined synaptic pathways. Wennekers and Ay (2005) generalises the result to the case with external input described by its spatio-temporal (stochastic) dependencies. Strongly interacting systems in that case turn out to be almost deterministic finite state automata. They can possibly emerge in neural systems by employing temporal Hebbian learning rules.

Selected References

• Wennekers, T., Ay, N.:
  Finite state automata resulting from temporal information maximization and a temporal learning rule.
  Neural Computation, in press.
  PS File (208kBytes)


• Ay, N.; Wennekers, T.:
  Dynamical Properties of Strongly Interacting Markov Chains.
  Neural Networks 16, 1483-1497, 2003.
  PDF File (343kBytes)


• Wennekers, T.; Ay, N.:
  Spatial and Temporal Stochastic Interaction in Neuronal Assemblies.
  Theory in Biosciences 122, 5-18, 2003.
  Gnuzipped PostScript File (113kBytes)

   © 2004 by -thowe-