Neural network models with higher order neural interactions

NEURAL NETWORK MODELS WITH HIGHER ORDER NEURAL INTERACTIONS. A.E. Busch and L.E.H. Trainor. D e p a r t m e n t of Physics, University of Toronto, 60 St. George St., Toronto, Ontario, C a n a d a M5S 1A7. Structurally, real neural networks are highly interconnected a n d neurons are very extensively arborized. Even with a fully connected network, it is doubtful t h a t neural models with pairwise interactions between neurons can a d e q u a t e l y simulate the interactions t h a t occur in real neural networks as a consequence of their a r b o r i z a t i o n a n d interconnection. Thus, for example, the axo-axonal interactions c o m m o n l y found in the neocortex cannot be included in such models. In our view it is essential to include higher order interactions in n e u r a l network models to achieve a reasonable degree of physiological realism. By higher o r d e r interactions we m e a n interactions in which a given neuron interacts with a cluster of neurons a c t i n g t o g e t h e r in consort. We have investigated the effect of including higher o r d e r n e u r o n interactions in a Hopfield t y p e network model. N u m e r i c a l simulations show t h a t t h e storage c a p a c i t y increases d r a m a t i c a l l y when the o r d e r of the interactions is increased. F u r t h e r m o r e , a n a l y t i c investigation by means of a m e a n field t h e o r y indicate there is a discontinuous j u m p in the order p a r a m e t e r from the trivial solution to a solution which has an overlap with only one m e m o r y state in c o n t r a s t to tile continuous change in the pairwise case. This makes the m e m o r y states of the model very stable against the s i m u l t a n e o u s flipping of a large n u m b e r of neurons. Our results accord with observations on the a r c h i t e c t u r e of a c t u a l neural systems in achieving functional efficiency, viz. the 'cost' to m a k e a n e u r o n is high, whereas the cost to make a connection is much lower. P r e s u m a b l y these results are also in agreement with an e v o l u t i o n a r y s t r a t e g y in which the neural s y s t e m o p t s for a high degree of a r b o r i z a t i o n and s y n a p t i c interactions between neural processes r a t h e r t h a n for a higher density of neurons.

79