ZSM-5 zeolite

~C~. ,;-~ ::

~~' ~ ~"

.:~,

I~

'

APPLIED CATALYSIS A: GENERAL

ELSEVIER

Applied Catalysis A: General 132 (1995) 261-270

Application of a neural network to the analysis of catalytic reactions Analysis of NO decomposition over Cu/ZSM-5 zeolite Motoi Sasaki *, Hideaki Hamada, Yoshiaki Kintaichi, Takehiko Ito National Institute of Materials and Chemical Research 1-1 Higashi, Tsukuba, Ibaraki 305, Japan

Received 9 January 1995; revised 26 April 1995; accepted 27 June 1995

Abstract An artificial neural-network analysis was applied to the prediction of conversions and yields in catalytic reactions: decomposition of NO into N2 and 02 over Cu/ZSM-5. With a proper network structure and training sets, the evolved neural-network system provides plausible activities of catalysts from a discrete and finite number of experimental data. A two- or three-dimensional graphical expression of the calculated data seems to be useful to reveal the relationship between the catalytic activity and reaction conditions. Therefore, they can be utilized in finding optimum reaction conditions or in examining effects of various factors on catalytic reaction. Keywords: Neural network; Back-propagation; Design of catalysts; Nitric oxide decomposition; Graphical explanation

1. Introduction The purpose of catalyst design is to find more efficient catalysts suitable for a particular reaction. Some researchers attempted systematic catalyst design [ 1-4], but they were not fully successful. Trial and error and repetition of experiments are the still main strategy in catalyst development. With the development of computer technology, computer aided data processing seems suitable for treatment of vast amounts of accumulated knowledge on catalysts [ 5-8]. Recently, neural networks, which are parallel information networks, are * Corresponding author. E-mail [email protected], Tel. ( + 81-298) 544648, fax. ( + 81-298) 544709. 0926-860X/95 / $09.50 © 1995 Elsevier Science B.V. All rights reserved S S D I 0 9 2 6 - 8 6 0 X ( 9 5 ) O 0 17 I-9

M. Sasaki et al. /Applied Catalysis A: General 132 (1995) 261-270

262

being watched with keen interest as a kind of data processing technique. The feature of the technique is its simple algorithm and learning ability. Particularly, the back-propagation method, which is a type of supervised neural network, has been successfully applied to many fields [9-16] of chemical design. The method requires no theoretical and structural knowledge about catalysts to learn reaction data. Experimental data can be used directly as supervising data to teach the network. The network learns the data automatically. Some catalyst researchers [ 13-16] applied a neural network to estimation of the relationship between various properties of catalysts and the selectivity of products. In this study, we examined the applicability of a back-propagation type of neural network to catalytic reaction analysis. Decomposition of NO into oxygen and nitrogen, which is known to be the most ideal method for elimination of NO in diesel exhaust streams, was adopted as an example of catalytic reactions. We have studied various catalytic NO reductions and have accumulated many experimental data using various catalysts and have chosen the Cu/ZSM-5 catalyst that is known as the most effective catalyst for the reaction [ 17]. This decomposition reaction is expressed as 2NO

~

N2+ 0 2

(

1)

This catalytic reaction produces no byproducts other than the major products N 2 and 02. Although, the reaction seemed very simple, the optimum conditions changed in a complicated way with the composition of feed and reaction conditions. Experimentally obtained conversions of NO were used as training sets of neuralnetwork analysis. Conversions or yields predicted in the analysis as functions of various factors were presented as multi-dimensional graphs and were compared with experimental values.

2. Calculation A neural network is a parallel and distributed information processing structure consisting of processing units (which possess a local memory and carry out localized information processing operations) interconnected through unidirectional signal channels called connections. Each connection behaves as a transfer function that affects the data flow. The diagram of neural networks and its relationship with the catalytic reaction are shown in Fig. 1. Coefficients in the function are evolved through the learning process. Optimization of the transfer function was performed with the back propagation (BP) method. BP is a method of supervised learning to adjust the coefficients of transfer functions of each connection by minimizing the square summation of the differences (SSD) between calculated values and training sets of values (experimental data). Learning is repeated until the values of the SSD become sufficiently small. After sufficient learning, the network holds knowledge on the pattern of the

M. Sasaki et al. /Applied Catalysis A: General 132 (1995) 261-270

Start Materials

~

:- Products

~omposition'~ Quantity. )

I ~

/TemperatureN~ L Pressure )

I

i /.~

i

i

y

~

I ~-~/'~

Bypr°ducts)f ~x Yield \ .

i ~ l

,

I ~ I

//

,

~

I ~

I

I

t

,,rrans~ r, ,-,~, 'Function, '

Training sets (Condition)

I

,~,Transfer, , Function, II A

i i

Input Layer

263

: ;

I]Iidden Layer

/

.,,

.

'

h

~::~" " -. q)

I

q9

, i r

, ,

Output Layer

"~ Training > Sets v (Results)

Fig. 1. The relationship between catalytic reaction and the architecture of three-layered neural network. The network is a hierarchical design of fully interconnected layers. Each layer consists of some units. There is no connection between units in the same layer. The connection hands a value from a unit to a unit of the next layer by the transfer function. Training is performed by adjusting the coefficients of the transfer function in relation to the difference of the SSD. The result of the training is memorized on each connection as the coefficient.

training sets as an optimized function (summation of some optimized transfer functions). The catalytic performance of the target reaction can be estimated by the use of the trained network. The outline of the data processing is explained in the caption of Fig. 1. This learning process is not confirmed to be focused on the minimum. However, it is proved that the transfer function can be trained in the steepest gradient to descend to a local minimum. Thus, it is not insured that a limited number of training steps gives an effective result. In our calculation, the order of the training set to teach the network had very little effect on the results after enough training; however, an effect of the order was observed at the initial step of training. Details of the BP algorithm are described in books [ 18,19]. The main routine program used in this study was originally published in ref, [20]. We modified it to control some parameters. The BP type neural network had three layers, named input layer, hidden layer and output layer. Each layer consisted of some units as shown in Fig. 1. In this neural-network analysis, experimental conditions were used as input units and conversions or yields were used as output units.

M. Sasaki et al. /Applied Catalysis A: General 132 (1995) 261-270

264

Table 1 Learning conditions of each case

Case 1

Input units

Number of hidden units

Output unit

Number of training sets

Calculation time (cpu)

Z (exp. data calc. data) 2

SiO2/AlzO~

16

Conversion into N2

32

3h

I%

32

Conversion into N2

100

1 day

1%

32

Conversion into N2

60

1 day

5%

Cu loading Reaction temp

Case 2

SiOz/Al203 Cu loading Reaction temp 02 conc. NO conc.

Case 3

SiO2/AI20~ Cu loading Reaction temp 02 cone. a

Either 0% or 5%.

A FACOM M-780/20 (UNIX system) was used as host computer. Networks were trained under three different training conditions shown later. Training was repeated until the SSD became less than a certain value (Table 1 ). In most cases, the number of repetitions goes up to over 100 000. The number of training steps necessary for appropriate learning was very sensitive to the balance between learning and unchanging efficiency of the training functions. Of course, the structure of the program and the number of units strongly affect the number of repetition. Prediction obtained by the learned network was shown as a graph or a table in this system. In this study, the authors encountered a problem. The problem is selection of an appropriate transition function for the treatment of continuous data. It is important to find a function suitable for continuous data treatment, because discrete data are employed in most reports. Some researchers have tried to use a linear function for the treatment of monotonous data [ 12 ]. The authors also tested such a function but learning did not progress sufficiently. Some other functions gave also insufficient results. Finally, sigmoid was selected as the transfer function and it gave fairly well results, though it is inherently suitable for discrete data.

3. Experimental The Na/ZSM-5 was synthesized using the usual hydrothermal method. The catalysts were prepared by ion exchange of Na/ZSM-5 in copper acetate aqueous

M. Sasaki et al. /Applied Catalysis A: General 132 (1995) 261-270

265

solution. Details of the procedure are described in our previous paper [ 21 ]. Catalytic reactions were carried out under various conditions using a fixed bed flow reactor charged with Cu/ZSM-5. The catalytic activity was evaluated by NO conversion to N2 using a gas chromatograph with a Molecular Sieve 5A column [22,23]. Experimental conditions of all supervised data is in the following range; SiO2/ A1203:34.6-216 molar ratio, Cu loading: 0-3.86 wt.-%, reaction temperature: 300600°C, oxygen concentration: 0-10 vol.-%, NO concentration: 525-20 000 ppm. Fixed conditions; flow-rate 62 cm 3 m i n - 1, catalyst 1.0 g.

4. Results and discussion First, we tried BP learning under a simple training condition (case 1 ). In this case, there were only three input units, one output unit and 16 hidden units as shown in Table 1. Fig. 2 shows the estimated conversions of nitrogen monoxide as a function of temperature and copper loading; conversions are expressed as a curved c,l

~0

///I

...-4

/ t!

."

..".

0

2:

.

0

.

.

.

.

~

/

"~';

.. X". .......... '.. ." ,~ \,~ -...." ~.

•

..

.,-./

~ ,_..,,.

!

...,"

i

....."

q.. .............."

# ~-

, "T ~. ~. ~ ~ • ! ..,.,. . ',~ -'" ~ " ~ ,_,. \ ,-,,., %~......

-

/ ~.,,

"":.:-

~,. • .........

,.'

~

.,

.

• t

.i

. ./

)4

.f

'., -

"..

/

~c~:",

"., [~., . . , \

\---..¢..

':'...-,, ...~ \ ......'\ .."

"'"' ""

'':i

\

,;,.. .. ~

',, \"',-.-\

.'"

o,...%

",0¢~

~.

:., ",,

"x,t %.. ...... ." ..... ,= -

"~,, ', ........ ~',......\ "~

,

%;,- .°.~. .

t

,

\ , -"¢',-.-. ". ,, .......

" .t..

""

-.

\

"

"

'"

.

".

""

""

...:"-: ..... 11......:--:....... ....... L: ....... ::.... ....."

0 573

873 Reacn.

TemD.

/

K

Fig. 2. Estimated conversion of nitrogen m o n o x i d e in case 1. ( S i m p l e situation). C o m m o n reaction conditions: c a t a l y s t = 1.0 g; flow-rate = 62 c m 3 m i n - ~ ; N O = 2.0%; 0 2 = 0 % . C o m p u t a t i o n a l conditions are s u m m a r i z e d in Table 1. Fixed value in this figure: SiO2/AlzOs ratio ~ 70.

M. Sasaki et al. /Applied Catalysis A: General 132 (1995) 261-270

266

~a

c o .,-I ..,~.~t~_~.

: ..... "~-.

.,. ;.,/ ~,.,., :

,-

~.

~ ~ - ...... : ' , . , ~,.. \ t,,

~,- . . . .

~,.

...

'~ .,. ~ '....

".~ ~ "-.. ; -...,,_.. . ...--~ ~ ~ ~..,.,.. , 4.',~~ -.... '...,~ ...... .. . . . . . . . . . . . . . . . . . . \ ", ".~ "~. . .~. . . . _ _ ~_~. . . . ~ ~. ~'k~. ~,.., \ : -. • ~,~" • , "~.., ~-'., Vs'... ~ " ' < L ".. "".

0 Z

o

~

~-J

0

~ \

~

\

:

~,

~

-,--

~

~ ".,,~. " ~ - - ~",

,

-___.,_ r -°"

;

~. \

!

\

:

~.

,

\

\ -.,. '-, ",.,. ...,~ ~ ....... ......... ~. ....... ; -~., ~,., ~....... \ " - - ~ . . .. -.. -..

',

~

.~. -. . .

\

,~

,

,

~

\ t~

•

~"

,

~.

', -,.:~.,.::, ~

-'".-.,

~

~ ~.

! ~ ....

............

\

......; ........:........ :

~',,

......\

::

~...

~

~ ~

~" ~. --,

~ ....

~

~

\

~

......~

".'.

.

:

'.

.:......... ~........ :.. ........ -.

"",, ~ ' ' ~ . " ~ -,..~

,

~:

' - ......... .:.........

V"-~--..;

,~ '-.,:~,.,:,...~!,

L---~a .i.~-

"....~

":~.-, "* ~... "~.t~-,. ',~

..

----.

: : :. .... ,.: . . . . . . . . .. . . . . . . . . :

•

.

~ ....... ;:. ....... ...

~'-<~--_~__,

• :.

.,

........... .. . . . . . . . . - . . . . . . . . - . . . . . . . . . :

673

873 Reacn.

Temp.

/ K

Fig. 3. Estimated yield of nitrogen in case 2 ( M o r e complicated situation) Common reaction conditions: catalyst = 1 . 0 g ; f l o w - r a t e = 6 2 cm 3 min - ~. Computational conditions are summarized in Table 1. Fixed values in this figure: Cu loading = 2 . 5 % ; 0 2 = 0 % ; N O = 1 0 5 5 ppm.

surface. Estimated conversions are in good agreement with experimental values (shown as ( + ) symbol in Fig. 2) and interpolate discrete experimental data satisfactorily. Next, BP learning was conducted under a more complicated situation (case 2). The estimated yields are shown in Fig. 3. It provided adequate conversions as a three-dimensional graph. In this case, conversions are presented as a function of temperature and SiO2/AI203 mole ratio, and Cu loading and 02 concentrations are fixed at a certain value. Both experimental (training sets) and calculated values are listed in Table 2. Predicted conversions were in good agreement with experimental values throughout all the experiments shown. The bottom data of Table 1 compare a predicted conversion with an experimental conversion that was not used in this training. Although these experimental data were not used in the training, the learned network anticipated this conversion well. Fig. 4 shows predicted conversions of nitrogen monoxide as a function of oxygen concentration in case 2. This figure suggests that oxygen strongly retards the rate of NO decomposition at lower temperatures. Although no theoretical assumption was employed in this process, this tendency agrees with the experimental results.

M. Sasaki et al./ Applied Catalysis A: General 132 (1995) 261-270

267

Table 2 Performance of neural-networkprediction Cu loading (%)

SiO2/ A1203

NO conc, (ppm)

02 conc. (%)

Reaction Temp. (°C)

Experimental conversion (%)

Calculated conversion ( % )

2.81 2.81 3.25 2.30 1.15 3.08 3.86

34.6 34.6 54.0 72.1 216 72.1 34.6

1055 1055 5040 1055 1075 20000 1055

0.0 1.04 0.05 10.25 0.0 0.0 0.0

400 400 400 500 500 500 600

17 2.3 46 1.98 16 78 7.1

15.8 1.45 46.0 2.21 18.3 77.5 5.54

3.08"

72.1 ~

20 000 ~

5.0 ~

500 a

50 ~

48

~' Experimental value, not used in this training.

These findings indicate good credibility of anticipation given by the learned network and applicability of the neural network to complicated situations like catalytic reactions. In addition, these processes can be used to get some information on the relationship between factors used as input and output, i.e., the correlation between temperature and the yield of N2 in NO decomposition or between oxygen concentration and N: yield. All of these analyses can be carried out on the base of nonsystematic, scattered experimental data without any data processing. o o

!

t

i

0.i

i

i0

c~ Z 0 ~n o

0 4n > 0 O 0 Z o

0.01

0 2 concentration

/ %

Fig. 4. Calculated conversion of nitrogen monoxide as a function of oxygen concentration in case 2. Fixed values in this figure: Cu loading = 3.08%; N O = 1055 ppm; reaction temp.; ( ) 400°C, (- - -) 500°C, ( . . . . . ) 600°C.

268

M. Sasaki et al. / Applied Catalysis A." General 132 (1995) 261-270

o,o

0 0 r--t

I

I

I

¢q Z 0 4~ C, .,q C 0 ~4

> c o o o x |

0

0.01

0.i

02 concentration

1

--

10

/ %

Fig. 5. Calculated nitrogen yield as a function of oxygen concentration in case 2 (Using localized data). Fixed values in this figure: Cu loading = 3.08%; NO = 1055 ppm; reaction temp = 400°C.

However, there were some cases where the results of BP analysis were inadequate. We tested the analyzing ability of BP based on the training sets in which data were localized within a narrow range of experimental conditions. The effect of localized training sets was tested in case 3. Data of one unit (oxygen concentration) out of four units is localized; experimental results under an oxygen concentration of 0% or 5% were used as the training set. Thus the training sets have no information about the effect of oxygen concentration except at 0% and 5%. Fig. 5 shows one example of NO yields predicted by a learned network for case 3 (5% SSD). The shape of the curve is sigmoid-like although experimental results were not different from Fig. 4. This inadequate estimation can be ascribed to insufficient information on the effects of oxygen in the training sets. Therefore, distribution of data used as training sets should be set over a wide range of calculation conditions, otherwise there is a probability of anticipating incorrect results. When the learning was imposed under very strict SSD ( 1% SSD) conditions, the prediction (Fig. 6) became different from that obtained under less strict SSD conditions (Fig. 3). The shape of the graph became winding, although each experimental value exactly agreed with the estimated curves. This may be caused by excessive learning under strict SSD conditions. Empirically, over-training is caused by the localized data and excess numbers of hidden units. However, at the moment, there is no general prescription to prevent over-training. This is one of the important problems in application of a neural network. In spite of these defects, the BP method seems to be useful for reaction analysis. The most important feature is that the BP method can deal with raw experimental

M. Sasaki et a l . / Applied Catalysis A: General 132 (1995) 261-270

269

~Ao

C Q .,-t

.

h

w "J

",1

- -

-,....... -

',

..

•

•

t

........y'S ~. ,; )~,~' ~---", .......,,~-----4,, " ,' I l \',

C 0 0

\

:,..

t', 0

','

"

,

,

,

,

,

,

,

i

l

": ....

'

'

,

4

'

~

',

',

\,

",. i

~'

~:

t" ~

i,"

\

':'..

".

~ i

, ,

i

~

,

t,

t

,

I

....

t

!

. . . . ~ ........ " ....... ~ :

i

',

'

',

:,,~ :'

I

'

,, i

i

- ' ~ .

I"

~

.

•

•.~

•

I ,

I

I~

0

,'l

", '

t '

. ..

....... ',.

"~" ~ ~ "

: 'l .q

.."~.

-.....-.... ....... ;,........ ',:,....... ?-.......... .......... -........

20

200

SiO2/AI203 Calculated conversion of nitrogen monoxide in case 3 (Using localized data and strict SSD value) Fixed values in this figure: 0 2 = 0 % ; N O = 2.0%. Fig. 6.

data without any theoretical approaches and provide anticipation without hypotheses of the researcher. These seem to be the merits of neural networks when applied to data processing of catalytic reactions because the control factors of catalysis are not well defined as described above.

5. Conclusion An artificial neural network was applied to analysis of catalytic reactions. Activities of Cu/ZSM-5 for NO decomposition were well estimated by a back-propagation type neural network when reaction conditions were reasonably dispersed. These predictions are very helpful when we want to analyze the correlation between variables which express reactions. In these processes any theoretical knowledge on catalysis was not required. In conclusion, back-propagation was confirmed to be applicable to analysis of catalytic reactions. However, there are still some problems to be overcome such as over-training.

270

M. Sasaki et al. /Applied Catalysis A: General 132 (1995) 261-270

Acknowledgements The authors are indebted to Prof. T. Hattori and Prof. S. Kito for drawing their attention to the application of neural networks to the analysis of catalytic reactions.

References [ 1] D.L. Trimm, Design of industrial catalysts, Elsevier, Amsterdam, 1980. [2] D.A. Dawden, Chem. Eng. Prog., Symp. Ser., 63 (1967) 90. [ 3 ] Y. Murakami, Syokubai Sekkei (Catalyst Design), Kodansya, Tokyo, 1985. [4] J.T. Richardson, Principles of catalyst development, Plenum Pless, New York, 1989. [5] S. Kito, T, Hattori and Y. Murakami, Appl. Catal., 48 (1989) 107. [6] W.T. Wipke and H.W. Jeffery, Am. Chem. Soc. Symp. Ser., 61 (1977) 97. [7] S. Kito, T. Hattori and Y. Murakami, Chem. Eng. Sci., 45 (1990) 2661. [8] T. Hattori and S. Kito, Catal. Today, 10 ( 1991 ) 213. [9] T. Aoyama, Y. Suzuki and H. Ichikawa, J. Med. Chem., 33 (1990) 583. [ 10] D.W. Elrod, G.M. Maggiora and R.G. Trenary, J. Chem. Inf. Comput. Sci., 30 (1990) 477. [ 11 ] J. Gasteiger and J. Zuhan, CICSJ Bull., 9 ( 1991 ) 14. [ 12] T. Aoyama and H. Ichikawa, Chem. Pharm. Bull., 39 (1992) 372. [ 131 S. Kito, T. Hattori and Y. Murakami, Anal. Sci., 7 (1991) 761, [ 14] S. Kito, T. Hattori and Y. Murakami, Ind. Eng. Chem. Res., 31 (1992) 979. [ 15 ] S. Kito, T. Hattori and Y. Murakami, Computer Aided Innovation of New Materials I1, Elsevier, Amsterdam, 1993, p. 901. [ 16] S. Kito, T. Hattori and Y. Murakami, Appl. Catal. A, 114 (1994) L173. [ 17] M. Iwamoto, H. Yahiro, Y. Mine and S. Kagawa, Chem. Lett., 1989 (1989) 218. [ 18] R. Heicht-Nielsen, Neurocomputing, Addison-Wesley, 1990. [ 19] D.E. Rummerhart, G.E. Hinton and R.J. Williams, Nature (London), 323 (1986) 533. [ 20] Y. Anzai, Ninsiki to Gakusyu (Recognition and Learning), Iwanami, Tokyo, 1989. [21 ] H. Hamada, N. Matsubayashi, H. Shimada, Y. Kintaichi, T. Ito and A. Nishijima, Catal. Lett., 5 (1990) 189. [22] Y. Kintaichi, H. Hamada, M. Sasaki and T. Ito, J. Nat. Inst. Matl. Chem. Res., 2 (1994) 209. [23] H. Hamada, Y. Kintaichi, M. Sasaki and T. Ito, Chem. Lett., 1990 (1990) 1069.