Self Organizing Systems and the Problem of Pattern Categorization

SELF

ORGA~IZI~G

SYSTEMS AXD THE

PROBLE~

OF PATTERN CATEGOR I= .\ T!O\

L. GERARDIN Chief Engineer, Electronics Group J. PLAMENT Radar and Elec tronic Systems Division Cie Frse THOMSON HOUSTON-HOTCHKISS BRANDT BAGNEUX (F rance)

SUM~ I ARY

:

1.

The Position of the Problem.

2.

The Perception: A Machine for Recognition Without Categorization.

3.

A Ge ner al Categoriza ti on Method.

4.

An Example of Categorization of Unidimensional Patt e rns.

1.

THE POSITION OF THE PROBLEM

the patterns that th ese numbers repr e :sent . In the particu l ar case of number~ the c lassifi catio n prohlem is simple because the number of different classes is perfectly defined, ten, neither more nor less. It might be that the number of possible classes is unknown a priori. If one wishes that the control system will be able to organize itself when confronted with the environment, then it is abso lutely necessary that the system be ab l e to evaluate the proximity of two patterns. In effect, in this case, if the machine is confronted with an environment of an unknown c lass Ci it is first of all necessarY to determine which is the closest class C j amongst the patterns that it had recognized previously, the system will react first in the same way that it had previously reacted when con fronted with the pattern Cj' finally it will ev olve in order to be tter adapt itself to Ci anc so on.

Con trol systems have a tendency to become more and more comp licat ed, due to this fact they ofte n use a pattern recognition "Ubsystem in order to he ahle to e l ahora t e in an adeq uat e manner the actions that th e control l ed system is ca lled upon to exec ute in order to act u pon it s en vi r 0 n men t . ( "E'e Fig . 1)

A self organizing system having this generalitv impli es a deeper st udy of pattern is, in orde r to be able to eva luate the distance between pattern which, a priori one does not know if they are part of the classes that ha ve previous To be ahle to de I v heen recognized. fine this proximity in a quantitative "'ay , it is necessarv to r educe the original patterns to a minimum pattern; in other words, it is necessary to have defined a categorization procedu re.

To h e gin ~ith, it is necessary to kno~ h o ~ to determine what the par ticul a r pat t e rn o f th e envi ronment is at a giv e n time. Pattern r eco gniti o n p r ocesses ca n be quite simple, fo r instance it can be reduced to stating the absence or pres e n c e 0 f d a t a ". h i ch, b \. the i r ins tan t e o use n erg \' 0 r b \' the e v 0 I uti 0 n 0 f t hat energy will furnish the data r eq uired by the control process , But. pattern r e o ogn iti on might be more comp li cated. For example, by an automatic postal sorting machin e which, bv means of a numerica l code of a given length \, hi ch is pa rt of the add ress, for instance in France the departm e nt numher o r the =IP :edc in the USA. In this case. th e con trol sy stem directs a lctte r car rving a given code to a cor r esponding case , So that the machine can perform thi s task, it is necessary that th e control system kno~s ho~ to identifv th e successive numbers that make up the cod e . Therefore. it must know how to classifv

~8S

Very often, we say that

a patter n can be summari=ed by a finite number of coefficients which one can consider as being th e components of a vector. A prototvpe vec tor of this tvnc can he associated with each possibi~ cla ss of patterns. These prototvpes vectors will be stored in th e memorv of classcs.

The problem of classification of a partic ular pattern bv comparing the correspo nding pattern with a ll the prototype vec tors is a s ubiect on which many studics havc hcen p~hlished. somethi~g

Data sensor

categorization

Association network

cl as s i fi cat i

Ol

Memory of classes Weighting (memory of classes)

Deci sion Discrimination

Decisions

FIG . 1. PATTERN RECOGNITION PROCESS

FIG. 2 . GENERAL SCHEMATIC OF A PERCEPTRON

786

which we do not propose to deal with in thi s paper. But, this classification pro cess i s va lu e l ess unles s one has previously determined the components of the vector. This preliminary stage of categorization is m uch more diffi cu lt than the later class liication phase . 2.

THf PERCEPTION:

A MACHIKE WITHOUT

lAIEGORIZAI ION

Sometimes it is possible that the pr e liminary step of ca tegorization is not strictly n ecessa ry . Thi s is particularly the cas~ if th e pattern classes making up the e nvironment are wholly predetermined; in other words, if th e control system is a conditioned reflex type wher e, having determin ed a priori the specific r eac tion to eac h of the possible classes of e nvironment, i t wi 11 in effect s uffice to recognize which is the class actually p r ese nted so that the contro l system can react. The s uhs ystem of pattern r ecognl tion to use in thi s case might b e s imple to e nsure on l y the separabi lit y of the c la sses to be recognized. The perceptron is a good examp le of this type of operating. The general sc h e ma of a perceptron is well known and is summa riz ed in Figure 2. This machine mainl y has a network called th e association network which is link e d with the data sensor hy man y randomly defi ned co nnections. The elements of the association network feed a discriminator with the suitable weighting coefficients which permits d ec is io n of which class such a given pattern belongs. Can the association network be considered as a catego rizer ? We sha ll see that it cannot , and thi s is because random organization of the con n ections hetween sensing ce ll s and association n e twork requir es a much greater number of association cells than se nsing cel ls. Why must we have a much greater number ~ ~f association ce lls than the number P of cells? On ce a set of random co nnections at the input has been defined, there are ~! P! (~ -P ) ! different vecto r s possible at the output o f the assoc i ation net,,' ork. lIe restri ct to the case of simole dichotom v , (c l asses A and B), i n practice the numb er of differen t patterns in each class is always finite and for the eviJent reasons of economy in experiment, it i s even a lw ays r e latively small. Suppose a and b the numb e rs o f these diffe r ent p0ssible patterns, cor res p 0 n cl i n g res re e t i \' e 1 y t 0 cl ass e 5 A and B, If the total a+b is clea rl y much sma ller than the tot a l numb er of differe nt vectors possible, that is if:

b)

(a -I-

«

P!

(~

- P'

( 1)

there are g ood c hanc es that all the (a + b) vectors actually used are indepeendent; therefore, the classification will be possible without error. More ex· actly, if we give the weight +1 to each element of a class A pattern we will almost certainly have P as a sum and if \,e give the weight - I to each element of a Class B pattern we will almost surely have - P as the sum, The separation of the two classes is then possible. The learning process of a perceptron returns by the action over the weight s, to the search for combinations that are effectively all different. Bu designating th e probability of confusion of the perceptron by Pc, the values of Nand P must be such that the following un eq uality will be verified: N

P'

CN

P)

>

a

+

b

- -P-c-

The second member of that unequality is always greater than 1. So that the unequality will be ve rified, it is necessary that N > P (this is much more so where th e number of da ta sensors is smaller) . The input of a perce Dtron (rando m connections and association network) is not at all a categori:er because if we have considered all this as a catego ri ze r, we will end up with the result that the number of compone nts of the vector summarizing a pattern will be greater than the numb er of th e raw data elements at the input, thu s, is manifestly non se n se: by def initi o n a summa r y must be much smaller than the summarized subject and not larger. Also, with a perceptron it is impossible to define the more or less great proximity between a ny two patterns. :;,

A

GE~E

RAL C.HEGOR 1: ,- \1'10\ ,IETIIOD

In a very general way a pa tt ern can be represented by a certain characteristic function s defined ove r a set of points E. The problem of categorization is to s ummar ize in some ~3Y the pattern s with the a i d of a finite set as small as possible of numhers gi, For thi s purpose, being given the characteristic space 5 of the patterns t o be recogn l: ed the simplest way i s to associate a group Gi of operators such as a particular pattern s, one can make a set of numbers gi of the abstract space G:

G·1

1, 2, ... , n , .. . )

(i

The orthonormal functions have a certain number of interesting properties which are useful for a better specification of the categorization performed with their aid. In particular, one can show that the infinite series of the squares of the coefficients gi is convergent. In a still more precise way, one shows that for certain groups of orthonormal functions, called totals, one has the Parseval equality:

(3)

The numbers gi can be considered as the components of the vector envisaged above which can summarize a pattern. For a long time physicists have known procedures of this type: Fourier's analysis, for example, where for a given class of signals there is a discreet and finite set of spectral lines. Fourier's analysis uses the properties of the orthogonality of a group of circular functions.

L i

Thanks to this, one can compare several signals by calculating the sum of the d i f fer e n c e s ( 0 r the s urn 0 f s q u are s 0 f the differences) between the successive frequency lines. The greater this quantity becomes, the more the functions (signals) will be distinct. The Fourier analysis method has a very solid theo retical foundation which permits envisaging its generalization as a basis of a general categorization process. In effect, every structure s has a finite energy so the integral:

J

s2 (E) dE E

(7 )

This last property is very interesting because it allows evaluation of the quality of the categorization made with the aid of the group of operators (S). The straight part of the relation (6) in effect measures the total energy of the structure s. One can know in function to the rank it what is the percentage of energy of the pattern summarized in the series of the i first coefficients by calculating the sum of their squares and decide to stop the analysis, for exampl~ when one has reached 80 or 90% of the total value of the energy. The distance between two patterns s (E) and a(E) is the sum of the squares of the differences of the corresponding coefficients gi and Yi:

exists and has a finite value. The set of the characteristic functions constitute a Hilbert space L2. The integral (4) is the norm of the characteristic s and its square root the generalized length of s. One can give L2 a scalar product which will permit the definition over L2 of a family of characteristics If i which will be orthogonals between themselves, that is to say, they will be such that:

Ifi (E)

If~

(E)

J

dE

o

00

D (s, a)

-1

i

If i

(E)

'f~

(E)

dE

'i~ J

(E)

dE

J

'PI

(E)

E

o

J

E

s (E)lfi

(E) dE

1

1

s (x) x

( 8)

1

i

dx

these moments we can center (m l = 0) and normalize (in relatiDn to mo )' But the increase in the value of successive moments is without any law. There is no relation analog to (8) of distance between structures and no means of knowing where to stop the analysis with the certainty of having a good pattern summarization.

i-1 (S)

i

4.

The group of op erators Gi whose role w~ have seen in a possible general categorlzation process can be the group of the scalar products: gi =

L(g._y.)2

Sometimes one has proposed categorizing the patterns by their moments, that is to say, in the case where E is unidimensional the coefficients:

One can with If' * complex conjugate of If. more accurately define a family" i which is orthonormal, i.e., which will be such that:

J

f

(4)

E

~

i

AN EXAMPLE OF CATEGORIZATION OF UNIDIMENSIONAL PATTERNS

The categorization process defined by the group (6) is very general. In order to see what the possibilities of practical application are, we have considered

(6)

788

a very simple concrete case by ing the unidimensional pattern Morse type signals: dot, dash equals two dots), and blank (a equal to one dot).

considerformed by (a dash length

Several s ets of total orthonormal functions exist on a linear space, for a finite and limited space; the Fourier series, Tschebycheff's polynomials (which in fact are another way of writing the sinusoidal analysis of Fourier) and Legendre's polvnomials. If the space is limited to a single side: Laguerre's po~ ynomials, and finally Hermite's polynomials if there is no limit .

The definition of pattern classes is always a very imp ortan t point. It is natural to characterize each class by the same energy value, but this is not sufficient. One may have signals each of the same duration (i n this case, the characteristic function s (x) has the value 1 on a dot and a dash, and zero over a blank), or a constant total length of the pattern (in which case s (x) has a value inversely proportional to the total energy of the pattern).

We have chosen the latter in order to have every generality. More exactly, the total orthonormal base Herm (x) chosen is connected with the cl~ssic Hermite's polynomials P n (x) by the relat i on : 2

Numeric calculations have been made in the first case, but the second case is valuable as well. Certainly the final results are not the same. Thus, we see that when one speaks of categorization, it is very important to have well defined beforehand the rules for forming pattern classes.

Herm

+

(x) . e

_ x 2/2

n

(x)

then we have calculated the coefficients

(x)

Herm.

1

(x) dx

(9 )

by using the functions (8) in the generaal relation of categorization (6).

We then obtain all the patterns of class n by adding a do t to all the class n patterns or a dash to all the class n - 2 patterns, thus the recurrence relation between the numbers S of patterns forming the successive cl~sses of energy n:

= Sn _ 1

P

(8)

In the particular case envisaged, the class of energy I only contains a single possible pattern; the dot. In the class of energy 1, two patterns are possible; two dots with a blank between or a single dash.

Sn

m

x/2

Sn _ 2

which is nothing other than the classical relation of Fibonacci. In order not to unduly prolong the time of numeric computations we have studied classes energy 3 and 6, that is, class of energy 3:

*** *

We have calculated the gi up to the order of SO. The detailed numerical results are given in the annex. Figure 3 summarizes the results graphically without normalizing by the value of the energy. The first coefficients are close enough, but the difference appears later for the much higher rank coefficients. Figure 4 is analogous but the coefficients hav~ been normalized by the value of the energy. This time there is a separation from the beginning. Categorization made along this way may then very usefully serve as the basis for a process of classification. CONCLUSION The example discussed above is the simplest possible because it only concerns unidimensional patterns. To extend to multidimensional patterns it is necessary to have adequate orthonormal functions available. The studies described here are actually followed in the case of bidimensional geometric figures. Evidently the general case is of a degree of extending complexity.

Class of energy 6:

* * * * * *

*

* * * * * * * * * * * * * * * * * * * * * * * * * * * * *

In all cases a deeper study of pattern meaning, especially in view of pattern categorization process can only benefit the development of more and more complex selforganizing systems. The method

789

,,<

:.:c

. ..!l..

'"'.. '"

-" "

~I

d

:,-

X

0

U

~

t

:.l

::>

U

'"

:.;

'"

~

.-'l

-<

~

C

:?

rn

<5

C;j f-< E-<

-<

""'

::>

~Z '

""

/.

0 U;

~

S

S

" )(

"" )(

)(

C

~

)(

O·

0

)(

,"-

00

.. ...

'"' '"

0

....d

-'"

x" 0"

0

cm

0

OOO
CCO 00

aD

000 CO 00

0 >'XX

o:xmro

X

(JllIID

CD



o a:o:o

000 00000000


0



x

Cl 0 ~

QI)

l'JaJl

0 0 OD X


X


"

CD>
" 00

am

0

X


011)

X


0

~

><

0

0000

lI1)




())



a>

0000


0

~

00

~!ll

OIIOCXJ)

0 0 O
0

0

O(I)O

cmxo


m

-

cc

0

0

~

CJ

;()

())

~

0

0

>0 §(>

0

a(>

~

Cl')

Clna:. ~

790


of categorization by projecting on an orthonorma1 total set is a pathway rich in promise but it will be necessary to go deeper into the definition of what a pattern is lnd what is a class of patterns.

ANNEX: Below we give the exact values of the co efficients gi for the various patterns studied. *

gl g2 g3 g4 g5 g6 g7 g8 g9 g10 gll g12

BI BLI OG RAPHY G.

E.

Lowitz

"La reconnaissance des structures par la theorie descriptive de l'information" These de Doctorat Jouve 1964 - Paris 62 p.

G.

E.

Lowitz

"The Problem of Shape Extraction in Pattern Recognition", PRC Seminar - janvier, 1966.

g13

Kenneth M. Sayre

g14 g15 g16 g17

Recognition: A Study in the philosophy of artificial intelligence, University of NotreDame Press - 1965.

g18 g19 g20 g21 g22 g23

g24 g25 g26 g27

g28 g29 g30 g31 g32 g33 g34 g35 g36 g37 g38 g39 g40 g41 g42 g43 g44 g45 g46 g47 g48 g49 g50 g51

791

.484690 -.561368 .126497 -.167982 .698607 -.957416 .602527 -.150114 .089840 -.226868 .170133 .048328 -.167964 .141949 -.077598 -.005600 .105468 - .122940 .018620 .078437 -.064365 .007099 .008290 -.013670 .040043 -.032383 -.025087 .051767 -.013858 -.020553 .018485 -.021651 .022382 .028246 -.073567 .009781 .094879 -.064656 -.074361 .101457 .030409 -.104548 .008492 .082968 -.025102 -.057802 .021248 .045389 -.011285 -.048240 .00 9519

* * * .482977 -.550276 .092213 -.085967 .539219 -.701674 .266286 .196312 -.146706 -.228207 .476909 -.513512 .478904 -.371371 .145719 .074356 -.152181 .134376 -.113525 .066335 .029818 -.090410 .062152 -.016208 .006818 .011543 -.052972 .047786 .012875 -.040756 .011884 -.001433 .020915 .002219 -.053025 .031294 .044719 -.045513 -.026937 .031217 .039730 -.030193 -.071015 .071571 .072626 -.131206 -.019245 .156473 -.062273 -.119267 .118178

- - * .637135 - . 929936 .383591 .007813 .058560 -.243329 .311265 -.408593 .544538 -.431022 .022054 .293075 -.248592 .049762 .029090 -.025182 .064232 -.084855 . 009163 .065728 -.045049 . 000072 -.002855 -.004398 .049096 -.040125 -.041980 .068722 . 008901 -.058704 .004958 .038419 .008066 -.039826 -.020873 .063112 .007855 -.081807 .029327 .071117 -.065341 -.029476 .073414 - .021614 -.045027 . 054155 -.006016 -.051778 . 054222 .017734 -.077981

* * gl g2 g3 g4 g5 g6 g7 g8 g9 g10

gl1 g12 g13 g14

g15 g16 g17

g18 g19 g20 g21 g22 g23 g24 g25 g26 g27 g28

g29' g30 g 31 g32 g33 g34 g35 g 36 g37 g38 g39 g40 g41 g42 g43 g44 g45 g46 g47 g48 g49 g50 g51

.482977 -.550280 .092233 -.()86409 . 5395 03 -.702536 .26862 4 .190578 -.133879 -.254531 .526671 -.600349 .618877 -.579595 .430797 -.282989 .254 315 -.278972 .252011 -.198939 .163262 -.096814 -.013157 .066623 - .0 05246 -.101863 .199573 -.317910 .444578 -.494117 .460354 -.435083 .434137 -.379147 .281623 -.251459 .29080 1 -.285745 .233378 -.249047 .310217 -.250588 .075222 -.004910 .100145 -.123716 -.064815 .246022 -.180647 -.014037 .070248

* * * * .482977 -.550276 .09 22 13 -.085968 .539221 -.701681 .266311 .196237 -.146498 -.228744 .478201 -.516423 .485054 - . 383596 .168610 .033942 -.084890 .028751 .042600 -.150535 .312033 -.432829 .446774 -.411751 .372457 -.282077 . 136559 -.026711 -.010999 .038905 -.065747 .017749 .098851 -.184490 .228769 -.317658 .432786 -.453789 .391734 -.389722 .448987 -.407775 .257708 -.205879 .316325 -.370776 .240300 -.125823 .221958 -.374701 .323815

* .637177 -.930289 .38 5038 .003094 .071480 -.273923 .374994 -.526513 .739268 -.718137 .398436 -.14101 7 .182584 -.302951 .240794 -.077571 .000900 -.001425 .015594 -.106708 .303853 -.473395 .5]6086 -.504984 .504471 -.451592 .331101 -.262215 .293000 -.307002 .245360 -.216630 .268645 -.263201 .118352 .023366 -.026898 -.012398 -.051514 .160979 -.158541 .039229 .063811 -.075443 .031667 .007685 - .0 05335 -.040569 .070377 -.017535 .076893

*

.48 4691 -.561371 .126517 -.168063 .698891 -.958279 .604865 -.155848 .102666 -.253192 .219896 - .038508 -.027991 -.066275 .207480 -.362945 .511963 -.536288 .384156 -.186837 .069078 .000695 -.067020 .069160 .027978 -.145789 .227458 -.313929 .417845 -.473914 .466955 -.455300 .435604 -.353120 .261080 -.272973 .340960 -.304887 .185954 -.178808 .300897 -.324943 .154730 .0064 88 .002418 -.050312 -.024322 . 134938 -.129658 .056989 -.038410

792

* * .637135 -.929940 .383611 .007731 .058844 -.244191 .313603 -.414327 .557364 -.457346 .071816 .206238 -.108618 -.158462 .314168 -.382527 .470728 -.498202 .374699 -.199547 .088394 -.006331 -.078165 .078433 .037032 -.153530 .210566 -.2969 75 .440604 -.512065 .4 53429 -.395230 .421287 -.421193 .313774 -.219642 .253937 -.322039 .289642 -.209147 .205147 -.249871 .219651 -.098094 -.017508 .061644 -.051586 .037771 -.064151 .122964 -.125911

*

*-* *

******

.482977 -.550280 .092233 - .086049 .539503 -.702536 . 268624 .190578 -.133879 -.2545 3 1 .526671 - .600349 .618877 -.579595 .430796 -.282986 .254306 -.278951 .251962 -.198831 .163029 -.096333 -.014117 .068568 - .008672 -.095728 . 1889 7 1 -.300235 .416163 -.45009 3 .394676 -.340844 .30429 1 -.207716 .065373 .008113 -.003947 .028026 -.075113 .023283 .107948 -.150 032 .099646 -.163580 .384797 -.508424 .380121 -.212418 .246196 -.373507 .340670

.482977 -.550276 .092213 -.085968 .539221 -.701681 .266311 .196237 -.146498 -.228744 .478201 -.516423 .485054 - .383596 .168610 .033942 -.084890 .028751 .042600 - . 150534 .3 12032 -.432827 . 446771 -.411743 .372440 -.282039 .136479 -.026551 -.011314 .039506 -.066861 .019756 .095338 -.178515 .218898 -.301827 .408141 - .4 16 576 .337270 -.312531 .343168 -. 267679 .078933 .0 1345 2 .058523 - .08 1870 - .066 172 .178445 -.055184 -.152373 .183201

* * g1 g2 g3 g4 g5 g6 g7 g8 g9 g 10 gl1 g12 g 13 g14 g15 g 1 :; g17 g18 g19 g20 g21 g22 g23 g24 1:25 g26 g27 g28 g29 g30 g 31 g32 g33 g34 g35 g36 g37 g38 g39 g40 g41 g42 g43 g44 g45 g46 g47 g48 g49 g50 g51

6371 77 -.930289 .3850 38 .003094 .071480 -.273923 .374994 -.526513 .739267 -.718134 .398427 -.140993 . 182523 -.302801 .240447 -.076805 -.000711 .001808 .009395 -.095351 .283974 -.440147 .462984 -.424072 .387029 -.289570 .119331 -.001219 -.008134 .014517 -.065856 .045133 .098 355 -.221124 .226916 - .236533 .360l42 -.480344 .437890 -.289834 .205675 -.211118 . 196006 -.104117 -.017389 .105212 -.123456 .073328 - .0 18074 .029792 -.077088

*

* * *

.484691 -.561371 .126517 -.168063 .698891 -.958279 .604865 - . 155848 .102666 -.253192 .219896 -.038508 -.027991 -.066275 .207478 -.362941 .. 511955 -.536267 .384107 -.186729 .068846 .001176 -.067979 .071005 .024552 -.139653 . 216856 -.296254 .389430 -.429890 .401277 -.361062 .305759 -.181689 . 044831 -.013400 .046213 .008884 - . 122536 .093523 .098628 -.2243 87 .179 154 -.152182 .287070 -.435020 .420614 -.323502 .297185 -.302480 .232012

* * * *

* * *

.637135 -.929940 .383611 .007731 .058844 -.244191 .313603 -.414327 .557364 -.45 7346 .0718 16 .206238 -.108619 -.158462 .314167 -.382523 .470719 -.498181 .374650 -.199438 .088162 -.005850 -.079124 .0802 78 .033606 -.147395 .199963 -.279299 .412189 -.468040 .387751 -.300992 .291442 -.249761 .097525 .039931 -.040811 -.008268 -.018849 .063183 .0028 78 -.149315 .244075 -.256764 .267144 -.323064 .393350 -.420668 .362692 -. 236506 .144511

.637135 -.929940 .383611 .007731 .058846 -.244198 .313628 -.414402 .557571 -.457880 .073100 .203352 -.102530 -.170536 .336710 -. 422168 .536390 -.600552 .524527 -.404843 .350263 -.3 14535 .251426 -.232483 .278324 -.272743 .166884 -.074643 .057821 -.021030 -.070125 .080233 .058658 -.204948 .241845 -.261514 .364975 -.457906 .415233 -.300959 .250275 -.245772 .175728 -.035884 -.070028 .082233 -.036337 .004557 -.041530 .129716 -.155589

793

.482977 -.550276 .092213 - .085968 .539221 -.701681 .266311 .196237 - . 146497 - .228747 .478210 -.516446 .485115 -.383746 .168957 . 033177 -.083280 . 025518 .048799 -.161891 .331912 -.466076 .499876 -.492663 .489899 -.444099 .348328 -.287707 .290134 -.282613 .245468 -.244013 .269142 -.226567 .120205 - .057759 .045746 .014157 -.097670 .06 1091 .084771 -.157428 .125513 -.177205 .365381 -.468303 .358421 -.239720 .310409 -.422028 .324009

* .484691 -.561371 .126517 -.168064 .698893 - .958286 .604890 - .155923 .102873 -.253726 .221180 - .041395 -.021902 - .078349 .230021 -.40258 6 .577626 -.638637 .533984 -.392133 .330947 -.307509 .262571 -.241756 .269270 -.265001 .183776 -.091598 .035062 .017120 -.056599 .020163 .072975 -.136875 .189151 -.314845 .45 1999 - .440754 .311545 - .2706 19 .34602 5 -.320844 .110806 .068698 -.050102 - . 029723 -.009073 .101723 -.107037 .063742 -.068088