Fuzzy Clustering and Classification for the Identification of Process Types in Technical and Biomedical Systems

FUZZY CLUSTERING AND CLASSIFICATION FOR THE IDENTIFICATION OF PROCESS TYPES IN TECHNICAL AND BIOMEDICAL SYSTEMS St. F. Bocklisch*, W. Meyer** and B. Straube*** *Technical Highschool of Karl-Marx-Stadt, Automation Section **AdW GDR, Central Institute of Heart & Bloodcircuit Regulation ***AdW GDR, Central Institute of Cybernetics and Information Processes

Abstract. Process types are sets of process trajectories with similarities. For the identification of process types clustering techniques are applied considering a set of samples as a vector in a high-dimensional parameter-space. In the paper 3 different approaches are discussed in the context of concrete applications: - graph-theoretic approach with nonparametrie cluster analysis - hierarchically organised parametric clustering concept - fuzzy variant of the Isodata method from Dunn/Bezdek /8/ Keywords. Adaptive systems; artificial intelligence; biomedical; computer applications; data reduction and analysis; electrocardiography; hierarchical systems; pattern recognition; probabilistic logic; INTRODUCTION primary information. The identification of process types should be performed in an interactive mode with the specialist. The methods are presented together with 3 applications: - differentiation of blood pressure curves of hypertonic rats - Fuzzy adaptive control for a positioning device in machinetools - Screening test of diabetesmellitus patients as risk group for heart & blood-curcuit deseases in a part of the city of Karl-Marx-Stadt

Classification is a structurefree model-building approach where because of complexity a theoretical process analysis is impossible. The corresponding aggregation concept should take into account the fuzziness on the given level and the reduction effect of the fuzziness in passing to the next higher level. Basic informations are fuzzy elementary measurements. From these primary data we get over the step offorming clusters to a fuzzy description of classes. In the paper 3 different methods of fuzzy classification are discussed. They complete each other in the sense, that we can choose that approach fitting best to the given data structure and to the aim of the parameter space after reduction of 679

680

St. F. Bocklisch, W. Me yer and B. Straube

HIERARCHICAL FUZZY CLASSIFICATION Forming of Clusters The hierarchical clustering starts with elementary fuzziness and uses local density as criterion for fusion. The similarity of classes is here a consequence of similarity of adjoining pairs of objects. Demands for central properties of object configuration cannot be introduced. Reduction of Information Sharply given elementary information (measurements) are made more and more fuzzy in a multistage process with stepwise increasing value of a threshold for the local density and afterwards fusioning adjoining objects in a corresponding configuration of clusters. Known or estimated relations between the main signal and a noise influence can be taken into account. The result of the reduction of information is a hierarchically ordered sequence of possible class configurations. Description of Clusters For the description of elementary membership-functions of measurements and global membership-functions of clusters a unique concept was used with parameters u OI a,b,c and d fittable to the corresponding information ~(U)

= ______~a~______

1 +(1/b-1) «u-u o )/c)d successfully applied already in /3/ For constructing higher-dimensional membership-functions we use this concept on the gravity lines of the cloud of points and apply an ellip-

tic interpolation concept. ~•.t As elementary membership-functions serves the following special case with a=l , b=O.5 and d=2. The elementary membership functions are aggregated with the help of the fuzzy disjunction operation (max, sum etc). Afterwards the result of the aggregation again is approximated within the parametric concept above described. ~i. 1~ Steps of the Classification Algorithm 1. Normalisation and Reduction of primary data 2. Forming of Clusters with account of informations about the credibility of measurements; with increasing degree of fuzziness different configurations of classes will be offered. 3. After choosing of corresponding clusters for the classes they will be described by membershipfunctions. 4. Classification of new measurements with the help of socalled sympathyvectors evaluating the similarities to the classes. NON HIERARCHIC FUZZY CLASSIFICATION Forming of Clusters This is a fuzzy variant of /5/. As a nonparametric method it can explore clusters with an irregular structure. In seeking the successors of a point we get disjoint directed subgraphs. In every point the n next neighbours and their mean distance are computed. The fuzzy set

Fu z zy clustering a nd classific ation

681

Fuzzy Clustering and Classification "distant" corresponding to Fig. 3 is established with u as largest and 1 as smallest distance. The fuzzy expression "Points lie dense around Pi" (Fig. ". ) on the normalised distance d is defined. The steepness of the straight line will be smaller, if the points around P.l. are very dense and has the significance of membership of the mean distance to the fuzzy set "distant". The heights h of the different j fuzzy intersections of the densities of the possible successors are determined. We conclude a certain compromise between the densities of two points and their distance. The neighbour belonging to the maximum height is a successor, if the "largeness" of its mean distance is smaller than that of Pi. For example in Fig. ~ the point belonging to r 2 is successor. Otherwise the path ends with Pi. Because this decision rule relatively sensitive reacts on local density variations many clusters arise. The attraction force of a point is the larger the smaller the distance between the neighbour points is. Opposite to it we can argue that the attraction force of less dense clusters is more farreaching than that of dense clusters. To take this aspect into account the fuzzy expression "Points around Pi are not dense lying" is introduced using the fuzzy negation of "distant".

1/(1 + bklr 2 ). Here 1 is the index of the gravity point of the class number k and b kl is determined in such a way, that the mean distance of all points fixing the gravity point is the turning point of the membership-function. The class Ck is then the fuzzy disjunction of all its local classes Ckl • Every local class Ckl can be understood also as a fuzzy statement /6/. Ooing this the classificator is a fuzzy algorithm. Fig. f~ shows the isolines of equal membership.

Reduction of Information

with u ok - class and density centre of the class k d - slope parameter, power of fuzziness

By local parametrisation we save the advantage of the cluster method to explore irregular clusters. We form a certain number of local gravity points fuzzily described by Ckl(r)~

ITERATIVE FUZZY CL<\SSIFICATION In one step the forming of classes and the description of classes are united. The proposed method is a fuzzy modification of the minimum distance partitioning /7/ and corresponds to the ISODATA concept /8/. The Forming of Clusters with a Global Fuzzy Concept The model of fuzzy classes is determined by a fuzzy partitioning of the data. In comparison with a cluster oriented envelope concept considered as a bottom up method for constructing classes this concept is a global top-down concept. A parametric class concept of the following form is used A(u) :; 1

The continuous normalisation factor

St. F. Bocklis ch, W. Me ye r and B. Straub e

68 2

suboptimum partition with the smallest value of the minimum. adjusts the class membership in such a way on the influence of the other classes, that the whole binding force will be 1 (Fig. ~ ). The centres of classes and fuzzy descriptions in the form of membership-functions for the classes are found by minimization of the following criterion, called fuzzy class heterogenity: Jd{.I', uo )

=

L N I'E{,.ki,k)2/d+1I1xi-uok

z::~.,

2

U

It is assumed that we are to find L classes, x.~ are the attribute vectors of the measurements. The method is especially promisable for data with adjoining or intersecting object clouds with global properties and singular measurements. The Iteration Process The iteration process starts with an arbitrary distribution of elementary fuzziness ~ik on the classes. In the first step of correction strategy the density centres uok are localised in such way that the fuzzy class heterogenity will assume a minimum value. In the next step the centres u ok will be fixed aronow new values of ~ik are determined demanding again minimum value of Jd{P ,u o ). This sequence will be repeated sometimes, the class centres are moving into the direction to the density centres in the attributes space (Fig. ;r ). Because we have to take into account the existence of some local extrema of the heterogenity J d we have to proceed with some different start distributions and then to choose that

Reduction of Information Beginning with a relatively large number L of classes we reduce L step by step always performing the minimisation process above described. The elementary fuzziness will be increased by the value 1/{L{L-1». In the result of the whole reduction process we get a sequence of fuzzy partitions. The quality of these partitions can be quantified in a different way by for example the normalised fuzzy class heterogenity, the normed increase of heterogenity, the mean value of the class fuzziness, the minimum distance of classes etc. Remarkable changes of monotony of these criteria signalize suitable levels of the fuzzy structure consideration. The fuzzy classification is performed with the help of the optimum fuzzy partition (Fig. ~~ ). The Algorithm for the Forming of Classes 1. Fixing a number L of classes 2. Fixing of a start partition (or class centres positions) 3. Computation of the gravity points of the fuzzy classes

/Ill .r. Ll'ikxi /

H.t

I..,

~ ik / 4. Checking the stopping rule !l(JA.

If

=

,.~.,

,,#~~w _~~!d '~t ~o-3

continue, else go back to 3 5. Computation of the fuzzy variance criterion A J2{~ ,m) Go back to 1,2 or Stop.

Fuzzy c lus t er i ng and c l a s s i f i cat i on

683

Fuzzy Clustering and Classification APPLICATIONS For an adaptive positioning feedback control consisting of a power amplifier, motor, gear, spindle, support, measuring system and a parameter adjustable controller (Fig. " ) on the base of 40 runs of the process in the learning phase with classification 3 typical process regimes were found (compare Figrs.'a-e). We deduced attributes from the error signal e(t) between the nonlinear dynamic, timevariant process (S and R) and a roughly adjusted reference model (M) with fixed parameter afterwards. With the values of these attributes the primary data for clustering were given. The semantic classes (I, 11 and Ill) correspond to significantly different process regimes. For the purpose of classification we determine on-line the actual value of the comparison criterion Q between object and model, with the identificator I we get by a fuzzy carespondence of the process state the relationship to the fuzzy classes. On the base of this information the subsystem ST generates then an optimum control (changing of the controllerparameters). With the same principle we applied the polymodel-method (See /2/) and performed thus the identification with some parallel processing models. For a biomedical task in a screening test examination of diabetis mellitus patients with oscillography (unbloody registration of the arterial blood pressure timefunctions at the extremities) 5 typical classes of blood curcuit disturbances wer~ found. The attributes in this case were samples of the blood pressure curves. In the identification

phase a given patient corresponds to the fuzzy classes with different memberships. Especially of interest isthe trend in time of the process of the des ease respective the recovery. These trend considerations together with questions of trend forecasting are possible with the help of the variations of the sympathy vectors over time. ACKNOWLEDGEMENT The authors thank very much all members of the interdisciplinary seminar "Fuzzy models for decisionmaking" for the very helpful discussions, especially the leader of the seminar Prof.Dr.M. Peschel for his continuous advises and useful comments. REFERENCES /1/ Bocklisch, St.F. and F. Bilz (1977). Systemidentifikation mit unscharfem Klassenkonzept. In M. Peschel (Ed.), Kennwertermittlung und Modellbildung. TH Karl-Marx-Stadt. /2/ Bocklisch, St.F. (1978). Grobmodellierung des arteriellen Systems. In M. Zwiener, U. Tiedt (Edrs.), Modellierung von Herz-Kreislauf-Funktionen in Experiment und Klinik, VEB Gustav Fischer Verlag p.ll. /3/ Bilz, F., St.F. Bocklisch, W. Meyer, M. Peschel and B. Straube (1978). Fuzzy Concepts for Systems Identification of Biomedical Systems. In proceedings of the VII. World Congress of IFAC Helsinki 1978, Vol. I, p.499.

684

St. F. Bocklis c h, W. Meyer and B. Straube

/4/ Straube, B. (1979). Unscharfe Clusterbildung und Klassifikation (In preparation)

~(u)

/5/ Koontz, W.L.G., P. Narenda and K. Fukunaga (1976). A graphtheoretic approach to nonparametric cluster analysis. IEEE Trans. on Comp. C 25, No 9, pp 936-944. /6/ Straube, B. and F. Arendt (1979). Prediction of waterflows in a riversystem by fuzzy algorithms (in preparation).

Fig. 2.Elementary membershipfunction

1

/7/ Spath, H. (1975). Cluster-Analyse-Algorithmen zur Objektklassifizierung und Datenreduktion. Oldenbourg-Verlag, Munchen/Wien.

/8/ Dunn, I.C. (1974). A Fuzzy Relative of the ISODATA Process and its Use in Detecting Compact and Well-separated Clusters. J.Cyb. 3,3. /9/ Peschel, M. (1978). Modellbildung fur Signale und Systeme. VEB Verlag Technik Berlin.

0,5 0,1

L

Fig. 3.

u

r

Fuzzy set "distant"

0,5

o +-L-______ o

~~------~~

2

cl

Fig. 4. Fuzzy set "dense"

../

. .- .-.'"

.

.

\)1"

\

\

O+-------~-+--~--~-­

o

Fig. 6.

Normalized class membershipfunctions for method 3.

2 d

Fig. 5. Determination of possible successors.

Fuzzy clustering and classification

//----

Mz.

[mmHgJ J/

18

/

/

",-

/

/

/

/ 18

I

15

",

---

..... '\

).. . ..

// ,...J

.

\

\

"

A

_ ..... 0,6/

1*

/

\

\

\ :

I

(

I

I

,

'\ \

I

J

+

"

'\

/ - - ........

I

,....1

/

..........

I

685

I

<

//0,2

d,lt (

13

/

//

12

4-

/'

11

10 9

10

11

12

13

14

15

16

17

18

M1

[mmHg1

110 Q,.

-f{)O

Fig. 1. Clusters for the different approaches: a Hierarch.fuzzy Classif. b Nonhieracch.fuzzy Classification c Iterative fuzzy Classification belonging to two aggregated attributes from blood pressure curves of hypertonic rats.

1bO

1~O

"'LtJ

"'/fotJ

M., (mmJl.fJ

~l !~rf!l.,

~

. 0

'1. 1).0

11.()

Fig. 7. The moving of class centres in the iteration process

St. F. Bocklisch, W. Meyer and B. Straube

686

to Fig. 8. Structure of the adaptive positioning curcuit with a controller parameteradjusted depending on

Gl. L------T~~~~~::::::~==~~~~+_----~~--

o

'" fJ.,

2.

1,0

1\ J V\

,I

I \ I ' \ / \/ \/~

V

\

/V

I

\-

,", ."

n

\

\/

/

\

\;

0,5

\

,.,.

/"

,..

\" ,

i V \

V

/

\I

Q

,

ill'

\

.

"

.\

./

0

~z.

.,0

/'" /',.."

,.,.

:\

G

-- ..---- .....-

./

.I \

./

"- "-

"- .......

""--

5

0

-

.

~

--- --

10

Q

)I (a) 1.D

/ - ...... /,)1L

.,

./ I'm ,..-'-'_.,,...-

I~'

1\./

.,

I

I

\.1 \\

. '1

./

.I

! ',

./,

·2

o

1

Fig. 9. The class membership-functions for the 3 identified process regimes determined by a Hierarchical fuzzy classification b Nonhierarchic fuzzy classification c Iterative fuzzy classification

--

10 u0!P

c. 11.

Q

classified process regimes.

Fuzzy cluster i ng and cl a s sification

fix

68 7

@

~

.lS,OJ

g~

(()

(,11,6')

,loo ~/(i ,~

)~ ~

.b .i.llt

tW ~

Fi g .

~ .

Fig. 4.

~

.~

A?

r5P

6P

X

Tr .:: jectories for two efficient solutions to measurements of alloy 2S 960

Efficiency -diagramm for alloy 2 S 960

ESl

Abstract proposed model an( casting I a genera : cesses i~ The meth( of the gl Keywords, tion and nonlineal INTRODUCl

Growth processe~ in biology, eco] dustry and agric chanisms are oft determined by e~ found more appro and parabolic gr motivated by wea most independent these types of g be simultaneousl generalized logi equation. For fi concept to concr identify non line meters. In the fuzzy mod fication method ments to the out zed model-concep sponding samples