Application of artificial neural networks to the real-time operation of conducting polymer sensors: a pattern recognition approach

ELSEVIER

SyntheticMetals 82 (1996) 27-33

Application of artificial neural networks to the real-time operation of conducting polymer sensors: a pattqn recognition approach Afshad Talaie *, Jose A. Romagnoli ICI Laboratory

of Process System Engineerirq,

Chemical

Engineering

Department,

University

of Sidney, Sidney, NSW 2006, Australia

Received23 April 1996;accepted9 May 1996

Abstract

An artificial neuralnetwork (ANN) -basedpatternrecognitionmethodis adoptedfor conductingpolymer (CP) sensors. The methodis capableof creatingdifferent patternsandmodelsbasedon an on-linedatacollectionfrom a multichannelanalog/digital(AD) device.The flow of informationis directedfrom the surfaceof theCP electrodeinto an AD devicewhichis connectedto anANN-trainedcomputer.The ANN software(Turbo Neuron) usedin thisstudy acceptsthe dataasits inputsandcreatesthe bestpossiblepatterns,basedon pre-selected parameters, to classifythe type of ionsexistingin the operationalenvironment.The methodis recommended to be usedin the field of CPs wherepassiveanalyticalmethodshavenot beensuccessful in addressing reusabilityof the CPelectrodes. Keywords:

Artificialneuralnetwork;Sensors; Chemical sensors; Patternrecognition

1. Introduction The scope of electrochemical sensing technology has improved in recent years, particularly with the advent of chemically modified electrodes [l] and polymer electrodes [ 2-41. However, the practical utilization of thesesensorshas been hindered by their poor selectivity, repeatability and/or reusability. Also of concern is the lack of compatibility between the dynamic nature of thesesurfacesand the usually adopted passive analytical approach. Even in caseswhere selectivity is improved by incorporating chemical reagents, the optimum performance is often not realized dueto the lack of adequaterepeatability and reusability. Owing to the instability of the sensorresponse,quantification can often only be accomplishedby useof either calibration curves or standardaddition approaches[ 5,6]. While theseapproachesmay yield useful quantitative data, they do not fully utilize the capability of the sensors.More significantly, the adoption of thesequantitative approachesdefeats someof the purposesof modern sensingtechnology in terms of speed, repeatability, reusability and ease of use. These problems resulted in discontinuation of financial support in someconducting polymer (CP) researchlaboratoriesaround the world [7].

Despite theseproblems, CPs have unique physical/themical properties which cannot be simply ignored. Therefore, we have tried recently to addressboth the problems and the solutions for these novel materials [8,9]. It has been discussedthat the problemsrequire the adoption of novel strategy for the fabrication of sensorswith artificial intelligence that will enable the identification, characterization and classification of the responsepattern [ lo]. Such pattern recognition, if practically and commercially feasible, will enable reliable determination of the concentration of analytes and, thus, reduce the emphasisor concern on the variation of sensorresponsewith repeateduse. As part of our on-going interest in the application of artificial intelligence (AI) methods in CPs, in this study we introduce an artificial neural network (ANN) method with flexibility in its mathematical operations to control the dynamic nature of CP electrodes.In this paper we alsoexamine the performance of ANN to solve the existing problems in CP sensorsand its advantagesover other AI methods [ 8lo] is discussed.Since the application of ANN in CP-based sensorsis of our interest, the paper containsgeneraland basic information about ANNs and is more focused on its application in the areaof CPs. 2. Experimental

2.1. Reagents and standard solutions * Corresponding author.

Fax:

f612

351 2854;

e-mail:

afshadachem.

eng.usyd.edu.au. 0379-6779/96/$15.00 PIISO379-6779(

96)

0 1996 Elsevier 03725-3

Science S.A. All rights reserved

The polymer synthesissolution consistedof 0.1 M pyrrole (Sigma), freshly redistilled prior to use,and 0.5 M of sodium

28

A. Talaie, J.A. RomagnoEi/Synthetic Metals 82 (1996) 27-33

chloride (Aldrich). These components were dissolved in deionized (milli-Q) water. Nitrogen was used for deoxygenation of the solutions before the polymerization process. Input3

2.2. Instrumentation

weights

Scheme 2. Artificial neuron.

Voltammetric data were obtained using a BAS CV27 voltammograph (Bio Analytical System.s,Lafayette, PA, USA), Data were collected using a Maclab (Analog Digital Instruments, Sydney, Australia) interface and a Macintosh computer. A resistometer (developed by CSIRO, Division of Mineral Products, Melbourne, Australia) was employed to collect the polymer’s resistance. The quartz crystal microbalance was used to log the data for the changes in the polymer’s mass, based on the design previously published [ 1l131. During the experiments, the current, mass and resistance data were logged in simultaneously to related files so that later they could be used and processed for pattern recognition. Since data processing was implemented in the MATLABTM environment the presented modelling was carried out on a personal computer. Turbo Neuron is a commercialized software provided by NEXSYS Co. 2.3. Procedures The polymer was galvanostatically (current density 2 mA/ cm2 for 2 min) deposited onto a gold crystal. A Ag/AgCl reference electrode and a platinum counter electrode were employed. Electrochemical characterization was carried out using the SMAC (simultaneous multidimensional analysis of conductors) technique [ 141 in three different salts (sodium chloride, Na salt, lithium chloride, Li salt, and calcium chloride, Ca salt).

3. Results and discussion

An ANN is a biologically inspired computational structure composed of many simple, highly interconnected processing elements, as shown in Scheme 1. As can be seen from the scheme, a neural network consists of a number of distinct layers each with various numbers of mathematical neurons. These are input, output and .hidde:n layers. The network accepts a series of input data which can be directed into the processing elements within the hidden layer, usually called Input

ddenLayer Nodes

outpllt Scheme 1. An artificial neural network.

neuron or nodes. These elements execute in parallel and exchange information very similar to the neurons and synapses within the brain [ 151. Each neuron, as shown in Scheme 2, receives signals from several neurons, processes the information, and passes a signal on to several more neurons in a manner analogous to biological neurons, There are three phases in neural network development. In the first phase the neural network is trained in a process called ‘supervised learning’ [ 161. A training set of data is presented to the neural network causing the weights in each processing element, initially set to small random numbers, to be modified to minimize the difference between the actual outputs and the desired outputs. When an individual pattern is presented to the neural network this is called a ‘training cycle’. When the network adjusts itself to minimize error this is termed an ‘epoch’. An epoch may occur after many training cycles. In the second phase, the training set is taken through the trained network, but without the weights being adjusted. This recall phase compares the output values to the correct values of the training data, allowing one to determine how well the neural network learned the training set. With sufficient classification accuracy achieved on recall of the training set, the third phase of neural network development can then occur by evaluating a test set of data. This results in creating patterns or models based on the input data to be used in classification of unknown inputs. Neural networks have been applied to such diverse fields as predicting the stock market [ 151, medical diagnosis [ 171, water industry [ 181, biomedical technologies [ 19], etc. However, it has been rarely used to address multi-dimensional problems in CP-based sensors. This paper focuses on the classification problem of some chemicals while a conductive polypyrrole-based sensor and Turbo Neuron software are used. 3.1. ANN modelling experiments for CP sensor The collected data, saved in a digital format on the disk, was transferred to the MATLAB environment to be processed for ANN modelling. The data included four variables (electrical potential E, current I, mass &i, and resistance R) collected versus time. The typical plots of these collected data versus the data number, for NaCl (polypyrrole (PPY) /Cl), are presented in Figs. 14. The profiles of simultaneous changes in current (Fig. 2), mass (Fig. 3) and resistance (Fig. 4) of CPs, while a pulse potential (Fig. 1) is applied, have been already discussed [ 291. According to the figures, 200 data for each variable were collected in an on-linelrealtime manner for one single experiment in 1 M NaCl. In the

A. Talaie, J.A. Romagnoli/Synthetic

Metals

.o,4t

4.6

-.

. . j

. . . . . . . . . . . i . . . ...!

i .nn

. . . . . :j

i

,

,

40

60

a0

. . . . . . . .

,

. i

. . . /

(

,

, .

100 Datapolnt#

120

140

160

potential

vs. data number

:

.V.V

0

20

Fig. 1. The profile 1 M NaCI.

81

!

of the applied

!

!

!

!

. . . j.

(

t

:I :

-0.6'

200

. . . . . j . . . . ..I . . . . . . ;..\.j

I.....?

. .

0

u

20

40

120

140

j

. . if . . . . 4

..1

160

180

200

Data point #

!

:

for PPY/Cl

:

!

6I,......: .. ..iy......iii........I...:......i......j............. .... 4\ . . .

. . q.....;

29

.o.6k..f ...f..i .i ..j;..w 6080100

. . ..-

180

82 (1996) 27-33

:

;I/ . . . . + . . . . . . . . . . . .iT . . . . . . . . . . . . . . . . ..I . . . . . . . .

in

Fig. 3. The profile 1 M NaCl.

of the mass changes vs. data number

for PPY/Cl

in

160.

I

il 41

. J....

.~~:::.:::::::::~::::~:::::::::~:::::I:::::::::.::::1 0

-20

Fig. 2. The profile 1 M N&l.

40

60

60

100 Data

of the current

paint #

120

140

160

changes vs. data number

160

for PPYlCl

200

in

sameway, data were collected in other environments (CaCI, and LiCI) . A set of 10 experimental works was carried out for each solution, resulting in 30 experiments for three salts with four variables. According to the figures almost 20 data at the beginning and 20 data at the end of the collected data were redundant. Therefore such data were omitted from the original data, resulting in a matrix comprising of 120 (30 experimentsand four variables) rows and 160 columns as a final product of the data processingstagewithin the MATLAB environment. Since the Turbo Neuron software can be installed within Microsoftexel, the processeddata were submitted from the MATLAB into the Microsoftexel. A part of data presented in Microsoftexel is shown in Table 1. The table consistsof data numbersfrom 156 to 160 for the current variable in three different salts used in this study. The first row of the table indicates the number of data presented to the network as inputs and outputs. The other rows are part of real-time data collected for current during the experiment. As the table shows,for the selected160inputs

0

20

40

Fig. 4. The profile 1 M NaCl.

60

60

of the resistance

100 120 Data point #

140

160

changes vs. data number

160

200

for PPYlCl

in

there are three outputs considered as Data Nos. 161-163. Since the network cannot deal with letters, the outputs are classifiedin onesand zeros. In this study outputs 10 0,O 10 and 0 0 1 representcalcium, lithium and sodiumsalts,respectively. It is important to note that, since the scale of the collected data for the four variables in this task is different, the network is not ableto deal with all simultaneously.Therefore, the input data were presentedto the network separately resulting in three different modelsand patterns for our three main variables (I, R and M) . Electrical potential (E) was only applied to create oxidation/reduction switches for the polymer electrode to create physical and chemical changes at the solution/electrode interface [ 14,201. Table 1 Part of typical

... . .. . .

156 -0.09 -0.11 -0.10

data presented 157 - 0.09 - 0.09 -0.10

in Microsoftexel 158 - 0.09 -0.10 -0.10

159 -0.09 -0.10 -0.11

160 -0.10 -0.11 -0.10

Ca 1 0 0

Li 0 : 0

Na 0 0 1

30

A. Talaie, J.A. Romagnoli/Synthetic Metals 82 (I9961 27-33

As explained before, there are three main stages involved in the ANN-based pattern recognition applications. The first stage was followed for the processed data by definition of inputs (DataNos. l-160) and outputs (DataNos. 161-163) to the Turbo Neuron network. In the second step, the test and train data were randomly (30% as test and 70% as training data) selected. In the final stage of pattern recognition, different conditions were imposed to train the network. The Turbo Neuron consists of different parameters for training. The main parameters are: degree of fidelity (which is related to the number of nodes in the hidden layer and selected in

advance to represent the intrinsic dimensionality of the data set), excitation function (which is applied to the neuron’s activation value to generate the neuron’s response and has sigmoid or Gaussian functions) and the iteration number (which is associated with the maximum number of training cycles to reach the minimum error in the predicted model). These parameters give flexibility to the network for pattern recognition and classification applications. While one particular condition may not create a reliable pattern, the other conditions may result in a reliable predicted model as the same data are used.

3.274

Epoch No. Fig. 5. The profiles of training and test during modelling vs. percentage of epoch and error for I data with the following conditions: degree of fidelity: 80; maximum iteration number: 60; excitation function: sigmoid; learning rule: conjugated gradient.

24

35

48.

Epoch No. Fig. 6. The profiles of training and test during modelling vs. percentage of epoch and error for M data. Conditions are as in Fig. 5.

A. Talaie, .i.A. Romagnoli/Synfhefic Metals 82 (1996) 27-33

Following these three steps, different patterns were created for I, M and R data for all three salts, All three variables were modelIed under the same conditions (degree of fidelity: 80, maximum iteration number: 60, excitation function: sigmoid and conjugated gradient as learning rule). Although the same conditions were applied the results are different (Figs. 5-7). The model created based on the mass variable resulted in a faster model and less epoch (Fig. 6). It takes less than a minute (55 s) for the system to be trained while it takes 63 and 67 s for training resistance (Fig. 7) and current (Fig. 5) data, respectively. The training of data requires 28 epoch (data cycling) for mass data (Fig. 6)) 50 for resistance data

31

(Fig. 7) and 60 for current data (Fig. 5), for 60 selected maximum iteration numbers. The errors for the training (0.009 76 for M, 0.009 80 for A and 0.030 20 for I) also suggest better training for the mass variable. The errors in the training of current data indicate that the system would be better trained if the maximum iteration number was more than 60. This can be confirmed by considering only a 91% improvement in training errors (at the last epoch) compared with 97% for M and R data (Figs. 5-7). Examining the classification results (Figs. X-IO), it can be said that sodium and calcium can be classified in a reliable manner using the pattern created with R data, whereas the

100-

.-O/338

i\ il 80 -i \ \ :

'0.268

Training

i \iJ 6n-

-u.z01

\\

5 z a T =

m 2

v 40 _

?I

\\

-0.w

\\ i I ! 20‘

0.067

f, t

Test $-\%. -1 . ,I__ ------------12'

---.

-N

24

36'

-‘--

-- 48

-f 80

Epoch No. Fig. 7. The profiles of training and test during rnodelling vs. percentage of epoch and error for R data. Conditions are as in Fig. 5.

7.07

Li

Fig. 8. Salt classification with the percentage of error at the last epoch (60) for I data. The classification was carried out using the training presented in Fig. 5.

A. Talaie, J.A. Romagnoli/Synthetic

Metals 82 (1996) 27-33

Na

Li

Fig. 9. Salt classification with the percentage of error at the last epoch (28) for M data. The classificationwas carried out using the training presented in Fig. 6.

I.27

3.13

0.07

Ca

Li

----Lme-

Na -xv------ -Lft%#l

Fig. 10. Salt classification with the percentage of error at the last epoch (SO) for R data. The classification was carried out using the training presented in Fig. 7.

model using M data classifies the 1ithGum in a more accurate way. This is more evidence of benefitting from using multidimensional techniques such as SMAC [ 141 with the capability of simultaneous on-line multi-data collections. This method gives the possibility of choosing the data based on the fastest and the most reliable computerized modelling obtained. To evaluate the accuracy of each model three sets of R, I and M data (unknown for the trained system) were selected randomly from different experimental work; these data were not included in the modelling. The models were run with

these inputs and the output results are shown in Table 2. The results prove that the system predicted correct output. The table reveals three outputs for each set of 160 inputs which can be associated with the type of the salt used in this work. 3.2. ComparisonbetweenTurbo Neuron and ASMOD

The ASMOD (adaptive spline modelling of observation data) algorithm automatically identifies nonlinear multi-variate models from empirical data, using B splines as the internal representation. The capability of ASMOD to find model

A. Talaie, J.A. Romagnoli/Synthetic Table 2 Output results for running three random sets of unknown created patterns:(a) I data, (b) M data and (c) R data

Metals

82 (19963 27-33

33

output 2

output

1.001218 1.000623 0.006376

- 0.0209 - 0.01974 0.999193

0.007818 0.006928 - 0.02523

(b) Mdata Na Ca Li

- 0.01768 1 .011793 - 0.00987

-0.11035 0.073661 0.98108

0.95254 - 0.10175 0.00729

out. Turbo Neuron is an ANN-based software with flexibility in the variety of parameters for better learning and training. It accepts the data as input and, with pre-selected conditions, trains the network for the most reliable modelling. Since the modelling can be done during experimental work in an ‘online/real-time’ manner, Turbo Neuron can be recommended for pattern recognition and classification applications in the field of CP sensors. The integrated polymer electrode/ANNtrained computer system, if feasible, will solve most of the existing problems such as selectivity, sensitivity, reusability and short shelf time in the area of CPs.

(c) R data Li Ca Li

- 0.00354 1.014853 - 0.00987

0.999641 - 0.02195 0.98108

- 0.00605 - 0.00072 0.00729

References

Salt (a) I data Ca Ca Li

output

1

inputs with the

3

structures which are relevant for the problem have earlier been demonstrated for many real and simulated problems [ 2 1] , Using ASMOD the models can be trained on-line, and a method for step-wise model refinement is applied during model training for gradually increasing the modelling capability until the desired or best possible accuracy is obtained 19,221. Since Turbo Neuron and ASMOD are two different softwares based on different rules, it is very difficult to compare them. However, as far as their performances and applications are concerned, both are useful to be applied in the field of CPs to control their dynamic attitude. Both can be utilized in characterization, modelling and classification applications. Nevertheless, Turbo Neuron has more flexibility in training by varying different parameters, while ASMOD relies only on a few parameters such as spline degree. ASMOD accepts the entire data which include all R, M and I data as input variables and it is capable of creating reliable models. It also has the ability to ignore the irrelevant variable. On the other hand Turbo Neuron is capable of detecting important data, using only one of the variables as its input, to create a faster model with less data. The detailed comparison of these two recommended methods has been carried out in CP-based biosensors where the data have more complexity and the comparison would highlight the advantages for each method [ 231. Both techniques are recommended to be applied simultaneously to model and to control the real-time operation of CP sensors. 4. Conclusions The modelling and characterization of the dynamics of a CP-based sensor using an ANN have been carefully carried

[ 11 J. Janata, Principles of Chemical Sensors, Plenum, New York, 1989. [2] R.W. Murray, in R. Kalvoda and R. Parsons (eds.), Electrochemistry Research and Development, Plenum, New York, 1984. [3] A. Talaie and Wallace, Synth. Met., 63 (1994) 83. [4] C.B. Duke, Synth. Met., 21 (1987) 5. [5] S.B. Adeloju, S.J. Shaw and G.G. Wallace, Anal. Chim. Acta, 281 (1993) 621. [6] S.B. Adeloju, S.J. Shaw and G.G. Wallace, Electroanalysis, 6 (1994) 865. [7] B. Wessling, Adv. Ma&r, 3 (1991) 507. [Sl A. Talaie,N. Esmaili and F. Talaie, Anal. Proc. Incl. Anal. Commun., 32 (1995) 405. [9] A. Talaie, A. Shahri and F. Talaie, Synth. Met, 79 (1996) 63. [lo] A. Talaie, M. Esmaili, F. Talaie and J.A. Romagnoli, Proc. ht. Symp. Micro Systems, Intelligent Materials and Robots, Sendai, Japan, 1995, p. 289. [ 111 S. Bruckenstein and S. Swathirajan, Electrochim. Acta, 30 (1985) 851. [ 121 J.R. Reynolds, N.S. Sundaresaan, M. Pomerantz, S. Basak and C.K. Baker, J. Electroanal. Chem., 250 (1988) 355. [ 131 K. Naoi, M. Lien and W.H.J. Smyrl, Electrochem. Sot., I38 (1991) 440. [ 141 A. Talaie, Solid State Ionics, 74 (1994) 219. [ 151 J. &pan and J. Gasteiger, Neural Networks for Chemists, VCH, Weinheim, 1993. [ 161 R. Young, Neural Network, Internet, 1995. [ 171 H.B. Burke, P.H. Goodman and D.B. Rosen, ht. Neural Network Society Ann. Meet., San Diego, CA, USA, 1994, p, I-53. [ 181 S.J. Van Deventer, C. Aldrich and D.W. Moolman, ZEE Proc., Perth, Australia, 1995, p. 3068. [ 191 E.M. Tzanakou, Int. Neural Network Society Ann. Meet., San Diego, CA, USA, 1994, p. I-69. [20] A. Talaie, in B.V.R. Chowdari (ed.), Solid State Ionic Materials, World Scientific, Singapore, 1994, p. 335. [21] T. Kavli, Int. J. Control, 58 (1993) 947. [22] T. Kavli, Learning principles in dynamic control, Ph.D. Thesis, University of Oslo, Norway, 1992. [23] A. Talaie, Z. Boger, J.A. Romagnoli, S. Adeloju and A. Yuan, Synth. Met., to be submitted.