Qualitative Interpretation of Sensor Patterns using a Similarity-Based Approach

Copyright © IFAC On-line Fault Detection and Supervision in the Chemical Process Industries, Delaware, USA, 1992

QUALITATIVE INTERPRETATION OF SENSOR PATTERNS USING A SIMILARITY -BASED APPROACH J,R. Whiteiey· and J.F. Davis" ·School a/Chemical Engineering, 423 Engineering North, Oklahoma State University , Stil/water, OK 74078-0537, USA "Department a/Chemical Engineering, 140 West 19th Avenue , Ohio State University , Columbus , OH 43210-/ 180, USA

Abstract. A machine methodology with ability to generate qualitative interpretations (Qls) of two-dimensional sensor patterns is described. The approach provides the capability to interpret multi-sensor patterns under transient conditions at a level comparable to that of an experienced plant operator. The system provides a more robust and adaptable means to interface numeric plant operating systems with symbolic knowledge-based system technology. Exemplar-based, supervised learning is utilized to construct a high dimensional QI-Map on the surface of a unit hypersphere. During run-time, qualitative interpretations are generated for input patterns based on their location on this QI-Map. Adaptive resonance theory introduced by Grossberg (1976a 1976b) is utilized with modification to provide the system with human-like memory attributes. The system is demonstrated for a dynamically simulated chemical process. Keywords. Pattern recognition, neural nets, expert systems, artificial intelligence, adaptive systems, knowledge engineering

INTRODUCTION

generation of state descriptions, primarily normality identification.

Knowledge-based system (KBS) technology represents an important means to improve control, monitoring, and optimization of modem chemical plants. A key to the successful utilization of this technology is the ability to create useful symbolic abstractions from sensor and product quality data. We have previously defined this numeric to symbolic transformation as Qualitative Interpretation (Whiteley and Davis, 1992a).

Normality identification is typically context dependent. That is, it is necessary to consider information beyond that provided by a single process variable. Plant operators routinely perform context-dependent QI based on observations of multi-sensor trend patterns l . The nature of the problem suggests use of an inductive learning approach. The process expert can easily specify windows in time during which data patterns exhibit behavior of a specific type. These labeled pattern exemplars can then be used in a supervised, inductive learning manner.

KBS researchers recognize the need for more robust QI (Bhatnagar and co-workers, 1988; Oyeleye, Finch and Kramer, 1989). The quality of abstractions generated using existing methods compromises the power of KBS. Consequently, the lack of robust QI capability represents a key bottleneck in the widespread application of KBS technology within the chemical process industries (CPl).

This paper describes our work on providing contextdependent normality identification at a level comparable to that of an experienced plant operator. Our machine methodology has the ability to generate Qls from multi-sensor trend patterns and is founded on the principles of similarity-based pattern

KBS heavily utilize three types of QI abstractions as input: trend, landmark, and state descriptions. Each involves consideration of the temporal evolution of one or more process variables as recorded by plant sensors. Our interest lies in the automatic

lControl panels and video displays are typically arranged with sensor groupings which provide context. 115

recogmtIOn. Although illustrated for normality identification, the methodology is general purpose and applicable to any context-dependent QI problem.

Problem Characterization

discriminants makes a commitment to unambiguously classify all patterns. While the orientation of the linear discriminants in Fig. 1 may be correct for the limited number of patterns used for learning, it is inappropriate to assume that they correctly partition the entire space. As described in (Kramer and Leonard, 1990; Whiteley and Davis, 1992a), such an approach leads to the potential for extrapolation e"ors.

We view QI in its most general form as a knowledge-based, pattern recognition problem. This characterization is motivated by the nearly universal use of trend recorders and graphical displays in control rooms. We view this as strong evidence of the importance of pattern recognition in the cognitive processes associated with the operation of modern chemical plants.

The inability to provide representational adequacy for the QI problem dictates use of a pattern recognition approach which focuses on local rather than global structure. Similarity or clustering-based pattern recognition methods provide this capability. These methcxls identify local pattern structure and can be applied to problems with limited and poorly distributed pattern data. '

The dynamic nature of the CPI demands that the pattern recognition process associated with QI be adaptive. Sensor patterns are affected by production rates, quality targets, feed compositions, equipment condition, etc; all of which are subject to change. Nevertheless, operators exhibit remarkable ability to correctly interpret sensor patterns under constantly changing conditions. In order to generate QIs as rich as those generated by human operators, adaptive or learning capability must be incorporated in any QI methcxl.

Utilization of a similarity-based approach for QI has several key benefits. One of the most important is the ability to identify situations which are outside the range of those used for learning purposes. This recognition avoids the potential for 'extrapolation errors' and their potentially catastrophic consequences. By focusing on local structure, similarity-based methods implicitly provide a 'don't know' capability which is comparable to that which exists in real world situations. Finally, similaritybased methods minimize the demands on the expert relative to the other traditional pattern recognition approaches.

lRADITIONAL PATIERN RECOGNITION AND QI

There is an additional attribute of the QI problem which must be acknowledged to successfully automate the QI process. Chemical and refining processes are highly controlled. Consequently, most sensor patterns correspond to 'normal' or pseudo steady-state behavior. Furthermore, the process itself constrains the number of physically realizable sensor patterns. The net result is that the patterns available for learning represent only a very small fraction of the total number possible. The inability to provide representational adequacy for the QI problem has profound implications on potential QI problem-solving approaches.

THE SIMILARITY-BASED QI-MAP APPROACH The objective of our approach is to create a 'QIMap' of known pattern classes. As shown in Fig. 2, a QI-Map consists of spatially distinguishable regions of patterns as they exist in an n-dimensional representation space. The map is generated from a set of labeled pattern exemplars using an externally supervised, clustering-based methcxl. In operation, the QI for an input pattern is generated based on its position on the QI-Map.

Incompatibility of Linear Discriminant Methods A fundamental prerequisite to this approach is the existence of adequate clustering characteristics by the QI pattern data. Our studies using a time domain pattern representation for a number of examples confirm that this requirement is satisfied for the normality identification problem.

Use of linear discriminant pattern recognition as employed by the back-propagation neural network (BPN) is fundamentally inappropriate when representational adequacy cannot be provided. We have previously illustrated this point in (Whiteley and Davis, 1992a) where we considered application of the supervised BPN for QI.

Conceptually, such a result is expected. Control systems for continuous processes are typically designed to maintain steady-state behavior. Furthermore, 'normal' behavior is typically associated with a whole range of steady-state

Figure 1 illustrates the problem associated with use of linear discriminant methods such as BPN for normality identification. The use of linear

116

conditions which reflect the possible variations in production rate and quality.

ARTI NEURAL NETWORK IMPLEMENTATION

Each set of steady-state conditions represents a stationary point which can be viewed as one of many cluster centers. The shape of each cluster is determined by the response of the control system to normal perturbations experienced by the process. The existence of a continuum of steady-state operating points as well as the ability to make smooth transitions between them suggests that 'normal' clusters are connected in the representation space forming regions of 'normal' behavior2. The existence of one or more of these regions provides the local structure required for a clustering approach.

Implementation of the QI-Map approach required specification of a clustering method. The QI problem is best addressed using an algorithm providing the following key attributes. • • •

Variable rather than fixed number of clusters Incremental rather than batch pattern processing Stable cluster formation which provides evolving memory capability

While numerous clustering algorithms satisfy the first two properties, the ARTI neural network (Carpenter and Grossberg, 1987) is unique in its ability to also provide the third.

An overview of our QI-Map methodology is presented in Fig. 3. It consists of five linked modules which collectively provide the ability to acquire and apply QI knowledge. The modular nature provides flexibility to investigate different types of input pattern representations, clustering methodologies, etc.

The ARTI neural network utilizes Grossberg's (1976a 1976b) Adaptive Resonance Theory (ART). The ART2 network represents one in an evolving series of autonomous learning models which incorporate adaptive resonance theory. Our use represents a novel application of this technology.

Creation of a QI-Map is a two step process; unsupervised map generation followed by a supervised labeling step. The basic constructs of the map are the pattern clusters identified during unsupervised analysis of the learning pattern set. The subsequent labeling step associates regions of the map with symbolic QI abstractions.

The ARTI neural network is a self-organizing system which employs competitive learning. From a QI perspective, the network performs a sophisticated form of vector quantization. The prototypes formed in this process are represented by the weights associated with the top layer of nodes in the ART2 network.

In order to be useful, a map must be created at a scale which prevents patterns from different QI classes being included in the same cluster3. Analytically, map scale is determined by the clustering criterion (defined in terms of pattern similarity) used to form the pattern clusters. Hence, the scale of a QI-Map is determined by the most similar sensor patterns belonging to different pattern classes.

Use of an internal representation space is a key feature of the ART2 network. Input patterns to the network are normalized to unit length. This transformation is necessary as similarity in an ARTI network is based on pattern vector directionS. Consequently, the clusters associated with ARTI prototypes can be viewed as hypercones originating from the origin in the pattern representation space. The QI map is formed from the intersection of these hypercone clusters with the surface of a unit hypersphere.

Classification of input patterns is performed by calculating the similarity between the pattern and its nearest neighbors on the QI-Map. A rule-based output interpretation module generates the symbolic abstraction from the quantitative similarity results. The knowledge in a QI-Map system is represented by the clustering criterion, the labeled pattern prototypes4, and the rules contained in the output interpretation module.

Numerous modifications were necessary to incorporate the ARTI network into our QI-Map system. The most significant are described in (Whiteley and Davis, 1992b). The interested reader is referred to (Carpenter and Grossberg, 1987) for a complete description of the ART2 model.

2This argument does not imply the existence of a single 'normal' region. Multiple regions may be formed due to the existence of widely different operating targets. 3Excluding 'don't know' 4Referred to as weights in Fig. 3.

SSimilarity is analytically measured by the vigilance parameter p as the length of a two pattern vector sum. 117

DEVELOPMENT OF A QI-MAP SYSTEM FOR A SIMULATED RECYCLE REACTOR

at t=1O minutes (Fig. 5). Both 'nonnal' and 'abnonnal' simulations were run starting from 25 different steady-state operating points8. Simulations leading to 'nonnal' coolant flow response utilized a feed temperature disturbance only; 'abnonna!' utilized a simultaneous feed temperature and composition disturbance.

We have applied the ARTI-based QI-Map approach to nonnality identification of a simulated CSTR recycle reactor system (Fig. 4). Results are briefly described in this section. A more complete description is presented in (Whiteley and Davis, 1992c).

Expert analysis of the patterns resulted in specification of an 8.1 minute window9 and identification of the QI transition points in time. Individual simulations were subsequently processed10 to create a series of 8.1 minute labeled pattern slices which represent the evolving view of the process in time. Each 30 minute simulation contributed 75 labeled pattern slices; the complete pattern database included 3,750 pattern slices.

Coolant Flow Nonnality Problem The recycle reactor system shown in Fig. 4 is used to carry out a highly exothennic, second order reaction 2A ~ B. The simulated system utilizes PI control with cascade loops for reactor temperature and composition control.

OI-Map Creation (Learning) Coolant flow provides an indirect measure of the reactor heat duty and is useful for monitoring reactor operations. The problem of interest involves identifying the state, 'nonna!' or 'abnonnal', of the coolant flow to the external heat exchanger. The heat release in the reactor is detennined by the rate, temperature and composition of the feed 6. Consequently, operators consider reactor feed conditions when establishing 'nonnality' of the coolant flow.

Creation of a QI-Map requires specification of the clustering criterion which sets the scale of the map. A quantitative analysis of the similarity between th( 'nonna!' and 'abnonnal' pattern slices in the pattern database was used to establish the maximum allowable scale (clustering criterion)ll. In one experiment, all 3,750 pattern slices in the pattern database were included in the learning set in random order. The clustering analysis was perfonned at the maximum allowable map scale. The structure of the map was fonned after one pass through the learning set 12 . Subsequent passes refined the initial structure. After five passes through the learning set, only 0.5% of the pattern slices were located within different clusters (of the same class) from the previous pass.

The reactor feed stream is prone to measured feed temperature disturbances (sudden increases of 5°C). In some situations, a simultaneous unmeasured feed composition also occurs. Although the reactor control scheme can compensate for feed temperature disturbances, unmeasured composition disturbances cause production to go off-spec. Operators recognize and react to composition disturbances based on the 'abnonnal7 ' coolant response to a feed temperature disturbance (Fig. 5). The objective for this problem is to provide comparable machine capability to distinguish between 'nonnal' and 'abnonnal' coolant flow using feed rate, feed temperature, and coolant flow sensor patterns as input.

The resulting QI-Map consisted of 303 clusters. Based on the clustering method employed by the ARTI network, it was expected that many of the clusters would overlap as shown in Fig 2. This structure was confirmed by noting that most 8Range of feed rates = 0.36 - 0040 m3/min, range of feed temps = 66 - 68°C. 9This window defines the pattern input to the QIMap system. Process characteristics set the minimum window size. l~aw sensor data was smoothed before sampling. Individual pattern slices were converted to directionally unique pattern vectors to be compatible with the ART2 network. 11 In ARTI tenns, this corresponded to a vigilance value of p = 0.99996. This represents an extreme in the existing literature. Two patterns are parallel (i.e. identical) when p = 1.0. 12Required 0.27 CPU sec/pattern on a VAX 8550 mainframe. Pattern dimension =162.

Pattern Database Learning and test sets were drawn from a pattern database covering 25 hours of simulated process behaviors. Individual simulations were generated for 30 minutes of operation with disturbances injected 6Fixed reactor outlet composition 7'Nonnal' coolant response implies a feed temperature disturbance only. 'Abnonnal' implies simultaneous feed temperature and composition disturbances. 118

patterns were located within more than one cluster. The overlapping structure is desirable as it minimizes the potential for uncovered 'holes' between clusters.

Four key conceptual issues emerged from our initial work with the system; pattern representation, window size, map scale and learning set construction. The latter three issues are tightly interwoven (Whiteley and Davis, 1992c). We are currently extending our investigation into each of these issues within the framework of an ART -based QI-Map system.

QI PERFORMANCE Experiments were performed to measure classification performance using different subsets of the pattern database for learning and test purposes. In all cases, the system correctly recognized 'normal' and 'abnormal' patterns that were within the range of the learning set. Equally as important, the system correctly identified novel sensor patterns, i.e., those patterns not within the range of the learning set.

ACKNOWLEDGEMENTS The fmancial support of BP America, Mobil Research and Development, and the Du Pont PhD Fellowship program is gratefully acknowledged.

REFERENCES When tested using sequential patterns evolving in time, the system correctly mimicked the expert's ability to identify the transition from 'normal' ~ 'don't know' ~ 'normal / abnormal'. Timing of transition recognition also mimicked that of the expert.

Bhatnagar, R., M.S. Gandikota, J.F. Davis, B.K. Hajek, D.W. Miller, and J.E. Stasenko (1988). An intelligent database for process plant expert systems. Proceedings of the ISA, Houston, TX 374-385. Carpenter, G.A., and S. Grossberg (1987). ART 2: self-organization of stable category recognition codes for analog input patterns. Appl. Opt., 22.4919-4930. Grossberg, S. (1976a). Adaptive pattern classification and universal recoding: I. Parallel development and coding of neural feature detectors. Bioi. Cybern .. 23.,121-134. Grossberg, S. (1976b). Adaptive pattern classification and universal recoding: II. Feedback, expectation, olfaction, illusions. Bioi. Cybern., 23., 187-202. Kramer, M.A., and J.A. Leonard (1990). Diagnosis using backpropagation neural networks - analysis and criticism. Computers chem. Engng, 14, 1323-1338. Oyeleye, 0.0., F.E. Finch, and M.A. Kramer (1989). Qualitative modeling and fault diagnosis of dynamic processes by Midas. Presented at A1ChE 1989 Annl. Mtg, San Francisco. Whiteley, J.R., and J.F. Davis (1992a). Knowledge-based interpretation of sensor patterns. In press, Computers chem. Engng. Whiteley, J.R., and J.F. Davis (1992b). Adaptation of "fast learn" adaptive resonance theory for knowledge-based interpretation of sensor patterns. Technical report. Whiteley, J.R., and J.F. Davis (1992c). The QIMap system: a framework for knowledgebased interpretation of sensor patterns. Technical report.

Classification performance of the system was consistent and correct under all test conditions. This success is attributable to the focus on local pattern structure and the use of interpolation within the boundaries defined by a QI-Map.

CONCLUSIONS The ARTI-based QI-Map methodology provides the capability to interpret multi-sensor patterns under transient conditions at a level comparable to that of an experienced plant operator. This capability is required to eliminate a key bottleneck in the development of KBS for monitoring, control, and operator advisement during periods of startup, shutdown, and implementation of emergency procedures. The QI-Map approach is very robust with respect to not providing an incorrect classification. When confronted with novel situations, patterns are classified as 'don't know'. Breadth of knowledge is defined by the amount of pattern data used for learning. In application, one can anticipate supplementing a learning set with new pattern data on an ongoing basis. The system was specifically designed to accommodate incremental development. By founding the methodology on the principles of adaptive resonance theory, human-like memory attributes are imparted to the system. Significant information management and system maintenance benefits are realized as a result.

119

.

1 1 I' I···· · ·

Representation Space

., .,

."

Normal

:.', \~~:: .. .. ""~/v. .. . . . . ..

,

~ ..... . ~ I.. .. .. . . . " . . Abnormal

..

,. .,... .

..

.,,

~" . . .. .

. f · ./

Fig. 4. Recycle reactor system

Fig. 1. Commitment by linear discriminants to partition entire representation space

"Nannal' cool"'" now response to

1.0

I

..,..._..:.~feed="=mpcra====lWe::::distur= · =bona:::;:=::::::::!.._~

I

.i.........l,..~.d~~.~.~~+.~.~~.~.~f.~.~~.":. J ......... ·····r·····~ i r1 ; ···+··· .. ···-1-····· ~.:.:~ta!~~owl ·· F=I Row

t z

:

:

;

.~,

.......... , ...... ... .,. ....:JI"'f' ....... : """'·'tT·"";_~_...l

0.0 -'-_~~_-'-_--'-_ _'--_-'-_....J

'Abnormal' ne

Jc:J'.......'1 /

'Nonnal' patterns

f j

'Don't know' patterns

:

:

-,··t····-r

1

-.-..-~

. . . . . ;.~~.~.~.~:~ ~.~ ~ ~ +~ ~~.~:":~.:~~.~~~ . ~

j

r

··········;····J.;l:········t·········j··········

z

0.0 10

• S.nsor dala • Product quality data • OWr r.l••alll into

2S

30

Fig. 5. Sensor patterns associated with 'normal' and 'abnormal' coolant flow

Fig. 2. QI-Map defined by local pattern structure in representation space

Process Pattern{s)

IS 20 Timc, awu.ll.ea

LEARN Motbd. (ItU1Itify local palt.rn structur.)

~

L....-----I

.-----, lABEL Motbd. (A1Ialysis and V.rifICatio1l)

_ _ __ ..

W.igllls /

'--------J -,

I

CLASSIFY Modul. ( QI g.1Ieratio1l)

Output 11It.rpretatio1l Modul.

Fig. 3. QI-Map framework

120

Labd.d (QI pall.rn L...-map""':"":")----I