A programmable interface to neuromolecular computing networks

ELSEVIER

BioSystems 35 (1995) 223-227

A programmable

interface to neuromolecular networks Kiumi

Department of Computer and Information Science,

computing

Akingbehin’ University of Michigan-Dearborn,

Dearborn, MI 48128, USA

Abstract A programmable interface is provided to a simulated network of reaction-diffusion neurons. The interface allows special ‘learn’ and ‘decide’ syntactic constructs to be intermixed with conventional programming constructs. This hybrid combination allows the power of programmability to be combined with the power of adaptability to provide innovative solutions to complex problems. The network uses reaction-diffusion neurons instead of adaline neurons. A mesh topology is used instead of a feedforward topology. The performance of the mesh reaction- diffusion network compares favorably with that of conventional feedforward adaline networks. Enhancements to incorporate short- and long-term memory are described. Keywords:

Biocomputing;

Molecular;

Neural;

Neuromolecular;

1. Introduction The computational style of supervised artificial neural networks and other adaptable computing networks is inherently non-programmable. Instead of executing a pre-written program (set of instructions), such networks go through a learning phase and a subsequent performing phase. During the learning phase, the networks evolve so as to seek out the best possible method for solving a problem with the information available. A logical way of utilizing the non-programmability of such networks is to augment the non-programmability with a programmable user interface. Such a hybrid configuration essentially takes advantage of ’Current address, Towson

State

University,

Towson,

MD

21204, USA. 0303-2647/95/$09.50

0 1995 Elsevier

SSDI 0303-2647(94)01519-D

Science

Ireland

Ltd. All rights

Computing

the proven and well-tested power of programmability without sacrificing the emerging power of non-programmability. A non-feedforward artificial neural network is simulated with concurrent processes in a two-dimensional mesh topology. A small language and interpreter is developed to provide a programmable interface to the network. The language consists of a mixture of conventional programming constructs and a selection of special constructs needed for the unique computational style of adaptable networks. A special ‘learn’ construct, used during the learning phase, informs a network to evolve appropriately so as to associate a labeled training sample with a known classification. A corresponding ‘decide’ construct, used during the performing phase, requests a network, reserved

224

K. Akingbehin

0 I

/ BioSystems

35 (1995) 223-227

1 ~1

Merent

Neuron Node

I

= excitase enzyme

(a) neuron model

(b) mesh topology Fig. 1. Reaction-diffusion

based on its current evolved classification for an unlabeled 2. Network of reaction-diffusion

state, to produce sample.

a

neurons

A reaction-diffusion neuron is formally defined as an abstract membrane consisting of a finite number of compartments (Conrad et al., 1987). This neuron model is shown in Fig. la. As indicated in the figure, each compartment exists in a definite state at any instant. Associated with each compartment is a binary input and an excitase count. The entire neuron fires if the state of a compartment is within a specified threshold/ceiling at any given time. To permit gradual change in a neuron, the excitase distribution can be varied. This model exhibits strong correlation to neurophysiological evidence involving the role of the cyclic nucleotide in the nervous system. An extension of the reaction-diffusion neuron, the Lorenz-Turing neuron, has been proposed (Kirby, 1989). The Lorenz-Turing neuron encompasses chaotic endogeneous dynamics and has been successfully simulated using first-order diffusion dynamics. When used for computational purposes, each neuron maps an input bit ‘pattern’ to a corresponding output bit ‘class’. In Fig. la, the input bit pattern is shown as the ones and zeros (vertical lines) incident on the membrane. The

neuromolecular network.

mapping is described in more detail in a separate paper (Akingbehin and Conrad, 1989). The network topology (a mesh) is illustrated in Fig. lb. The network consists of a control node and a network of reaction-diffusion-type neurons. The control node has bidirectional communication with all the neuron nodes. Each neuron node in turn only communicates with its immediate neighbour. The control node typically broadcasts a request to all the neurons and then waits for the responses. The neurons process the request and then send the results back to the control node. A decentralized copy-thy-neighbour learning algorithm (Akingbehin, 1987) is used. Fig. 2 shows the typical convergence obtained with this learning algorithm. In the figure, the initial rapid convergence can be attributed to the ‘copy-thyneighbour’ phase. After the best neighbours have propagated through the network, further improvements are obtained through random mutations of excitase levels within the neurons. This results in slower convergence as shown in the latter part of the curve. During the performing phase, the control node builds a histogram of all the responses and then selects the most frequently occurring response as the aggregate response. The neighbour-only communication mode greatly reduces the number of neuron-to-neuron connections needed in a mesh topology. The num-

K. Akingbehin

/ BioSystems

35 (1995) 223-227

225

12 _ hamming distance from desired pattern

8-

40 0

I

I

2

I

I

4

I I I I I 6 8 10 number of “learn” cycles I

Fig. 2. Learning

ber of connections C in a mesh topology rows and m columns is given by:

with n

C=2[n+(n-l)m+(m-l)n+n] =4nm-2m+2n which, in O-notation, is O(N), a low-order polynomial where N = nm. In comparison, the number of connections in a feedforward topology with n rows and m columns is given by: C=n+(n2m)+n which is O(nN), a higher-order connection complexity, where N = nm. An autoassociative feedforward network with full connectivity (Maren, 1990) has a high-order connection complexity of 0(N2). The large number of connections required makes such networks prohibitive and impractical for a large number of neurons (Akingbehin, 1991). 3. The programmable interface Two syntactic constructs were initially provided for programming the mesh topology. A ‘LEARN’ construct, used during the training phase, provides facilities for training the network. A ‘DECIDE’ construct, used during the performing phase, elicits information from a trained network. In contrast to conventional STORE and RETRIEVE operations, these constructs incorporate gradualism and generalization, essential charac-

I

I

12

I

I

14

I

convergence.

teristics of adaptable networks. It should be noted that the solution to a large class of problems can be reduced to a mapping of one bit string to another. In fact, any object can be encoded as a bit string if limitations to the length of such a bit string are ignored (Akingbehin and Conrad, 1989). The LEARN and DECIDE statements are formally defined by the following context-free BNF productions: :: = LEARN < pattern1 > IS < pattern2 > < decide-statement > :: = DECIDE < pattern < pattern1 > :: = sequence of 1s and OS < pattern2 > :: = sequence of 1s and OS

1>

The LEARN statement essentially supplies one or more labeled samples to the network. By means of the learning algorithm described above, the network is gradually modified until the input pattern is correctly mapped to the corresponding output pattern. The DECIDE statement supplies an unlabeled sample to the network and request a classification for the pattern. In response to the DECIDE request, the network performs a mapping operation and responds with a resulting output pattern. As an example, one may combine the LEARN and DECIDE constructs with conventional programming constructs as shown in the following pseudo-code routine:

K. Akingbehin / BioSystems 35 (1995) 223-227

226

DO N TIMES LEARN LINE1 LEARN LINE2 END DO DECIDE LINE3 DECIDE LINE4 DECIDE LINE5

IS STRAIGHT IS STRAIGHT

In this example, the network is trained with two lines which are known to be straight. After training, the network is used to decide whether other lines are straight. All lines are encoded as bit strings. The pattern ‘STRAIGHT’ is also encoded as a bit string. 4. Further applications interface

of the programmable

The programmable interface has been used with both reaction-diffusion mesh networks and adaline feedforward networks. Supervised learning is conveniently handled by the interface as already described. Unsupervised learning is also handled by the interface. In the case of unsupervised learning, the network is trained by means of input patterns only. Similar or same inputs produce the same output grouping. The training phase in such a case is handled by the following type of pseudocode: DECIDE LEARN

input * which produces new-input IS new-output

‘new-output’

Associative learning, ‘including various extensions like autoassociative, heteroassociative, bidirectional associative memory (BAM) and other recurrent variations can still be handled. The basic approach is to map alternate inputs to corresponding outputs as depicted below: input

* output * input * input * output

The programmable as follows: DECIDE DECIDE

input 1 output 1

power

Adaptive resonance theory (ART) is subject to the problem of multiple associations, where new learning wipes out previous learning (Akingbehin et al., 1990). The basic functionality needed to deal with this problem involves storing a set of ‘known patterns’. When a new pattern is encountered, a search is made for a match (within a specified tolerance or ‘vigilance’) from among the stored known patterns. If a match is found, then the new pattern is grouped with the stored patterns. If no match is found, then a new group is created and the new pattern is added to the new group. Note that in an actual feedforward ART network, one neuron suppresses firing by all others in the same layer. Implementation techniques that have been used include parallel processors, operational amplifiers and so on. To handle ART functionality by the programmable interface, the following additional features are needed: (a) short-term memory (STM); (b) long-term memory (LTM); and (c) syntactic constructs for the STM and LTM. The basic algorithm is as follows: (1) Search LTM. (2) If a match is found, stored configuration becomes the active STM. (3) If a match is not found, train as the new STM and then add to LTM. This scheme is depicted in Fig. 3. To accommodate short- and long-term memory, long-term (LT) and short-term (ST) options are added to the LEARN and DECIDE constructs. The complete BNF is now: < statement > :: = < learn > ( < decide > 1 < query > I< configure > :: = LEARN < bit-pattern > IS < bit-pattern >

* output = .. . of the interface

=- which produces * which produces

Compute difference between output1 Repeat until difference is constant

is used

‘output1 ’ ‘outputa’

and output2 Fig. 3. Enhanced programmable interface.

K. Akingbehin

ILEARN ST < bit-pattern > IS < bit-pattern ILEARN LT < bit-pattern > IS < bit-pattern < decide > :: = DECIDE < bit-pattern > IDECIDE ST < bit-pattern > IDECIDE LT < bit-pattern > < bit-pattern > :: = sequence of 1s and OS

/ BioSystems

> >

Note that the LEARN ST has the same semantics as the previous LEARN and the DECIDE ST has the same semantics as the previous DECIDE. The ST option is only provided for symmetry. This expanded syntax is now being implemented and the results will be published in a future paper. 5. Conclusion Performance of the reaction-diffusion mesh network has been comparable to the performance of the conventional adaline feedforward network (Akingbehin, 1987, 1990, 1994; Akingbehin and Conrad, 1989). Such non-feedforward networks can complement the more conventional adaline feedforward networks in hybrid topologies. Such hybrid networks may provide the key to the next generation of very large artificial neural networks. The performance of such hybrid networks can further be enhanced by incorporating programmability into the user interface. Such hybrid combinations can take advantage of the proven and well-tested power of programmability without sacrificing the emerging power of non-programmability. References Akingbehin, K., 1994, Programmable intelligence in a nonfeedforward reaction-diffusion neural network (Symposium on Intelligent Systems in Communication and Power, SISCAP ‘94). Akingbehin, K., 1992, On the topological complexity of parallel simulations (Annual Pittsburgh Conference on Modeling and Simulation).

35 (1995) 223-227

227

Akingbehin, K., 1991, On the interconnection of molecular processing devices (IEEE EMBS 13th International Conference). Akingbehin, K., 1990, A mesh topology for programmable neural computing (IEEE International Conference on Systems, Man and Cybernetics). Akingbehin, K., Khorasani, K., Shaout, A. and Smari, W., Models for neuromolecular computing (Collection Book from MNEMO ‘90). Akingbehin, K., 1987, A decentralized algorithm for learning in adaptable networks (IEEE International Conference on Neural Networks, ICNN ‘88). Akingbehin, K. and Conrad, M., 1989, A hybrid architecture for programmable computing and evolutionary learning. J. Parallel Distributed Comput. 6, 245-263. Conrad, M., 1988, The price of programmability, in: The Universal Turing Machine A Fifty Year Survey, R. Herken (ed.) (Oxford University Press, New York) pp. 2855307. Conrad, M., 1986, The lure of molecular computing. IEEE Spectrum October, 55-60. Conrad, M., Kampfner, R. and Kirby, K., 1987, Simulation of a reaction-diffusion neuron which learns to recognize events, in: Rapprochement of artificial intelligence and dynamics (appendix). Eur. J. Oper. Res. 280-290. Kirby, K., 1989, Information processing in the lorenz-turing neuron (IEEE EMBS International Conference). Maren, A., 1990, Neural networks bookshelf: heteroassociative and resonating networks. J. Neural Network Comput.

Further suggested eading Domany, E., Van Hemmen, J.L. and Schulten, K. (eds.), 1991, Models of Neural Networks (Springer-Verlag). Pao, Y., 1989, Adaptive Pattern Recognition and Neural Networks (Addison-Wesley, Reading, MA). Wasserman, P., 1989, Neural Computing: Theory and Practice (Van Nostrand Reinhold, New York). Schultz, A., 1993, Collective recall via the brain-state-in-a-box network. IEEE Transactions Neural Networks 4(4), 580587. Shawe-Taylor, J., 1993, Symmetries and discriminability in feedforward network architectures. IEEE Transactions Neural Networks 4(5), 816-826. Widrow, B. and Winter, R., Neural nets for adaptive filtering and adaptive pattern recognition. IEEE Spectrum March, 25-39.