A program for non-orthogonal rotation in factor analysis

226

trends in

A program for non-orthogonal factor analysis L. Sarabia* and M.C. Ortiz

analytical chemistry, vol. 72, no. 6, 1993

rotation in

Orthogonal and non-orthogonal

rotations

Burgos, Spain

R. Leardi and G. Drava Genoa, Italy Rotation of initial factors is a very important step in factor analysis. In this article the program OBLIQUE, which performs the nonorthogonal “Oblimin” rotations, ranging from “Quartimin” to “Covarimin”, with all the possible intermediate solutions, is described.

After an orthogonal rotation performed on N objects, V variables, and F factors, the final result is expressed [l] by a matrix SNF (scores) and a matrix of orthogonal rows LFV (loadings), plus an orthogonal rotation matrix RF,. A non-orthogonal rotation produces a much more complex result, since it is expressed by a matrix SNF (scores), a matrix V,, (structure loadings), a matrix W FV (pattern loadings), a matrix YFF (correlation between factors) and a rotation matrix AFF. If AFV is an orthogonal initial solution, the following relationships are fulfilled:

Introduction A very important step in factor analysis is the rotation of the initial factors. The aim is to obtain other factors, witch have a real meaning. This goal is attained when every rotated factor is strongly correlated with some of the variables and weakly correlated with other variables. The most commonly applied group of rotations is known under the name “Varimax” [ 11. The program, OBLIQUE, which is described here, performs the non-orthogonal “Oblimin” rotations, ranging from “Quartimin” to “Covarimin”, with all the possible intermediate solutions. Non-orthogonal rotations are more flexible than orthogonal rotations for the determination of the factors since the constraint of orthogonality is not required. Note that orthogonal rotation can be considered to be a particular case of the rotation methods, in which orthogonality has been imposed. More details about the methods of factor rotation can be found in texts about factor analysis and principal components.

:cTo whom correspondence

016%9936/93/$06SN

should be addressed.

V:F = &F&F

(1)

Each element Vijof the matrix V is the orthogonal projection of thej-th variable on the i-th rotated factor. Since autoscaling of factors and variables is a normal procedure in factor analysis, vij in this case is the correlation coefficient between the variable and the factor, and its absolute value is I 1. Pattern loadings

These are the projections parallel to the axes, n’4 being the contribution of the i-th factor to thej-th variable. They measure the linear dependence of the variable on the factor, and in this sense they can be interpreted as a regression coefficient (they can be I, also, when the factors are autoscaled). When the rotation matrix is orthogonal, then eqns. 1 and 2 coincide, since in a system of orthogonal axes both projections are the same. When there is no constraint of orthogonality a greater simplicity will be obtained.

0

1993 Elsevier Science Publishers B.V. All rights reserved

227

trends in analytical chemistry, vol. 72, no. 6, 1993

For a non-orthogonal rotation, one must decide which of the matrices (V or W) has to be simple. The factor analysis of a set of variables can be considered as an analysis of the off-diagonal terms of its correlation (covariance) matrix. Frequently, matrix V is a correlation matrix, and it then seems logical to impose the simplicity criteria and to compute the adequate rotation matrix A, from which W can be obtained. W is very useful in the interpretation of the resulting factors: the values Wij, which differ significantly from zero, specify the variables having weight in the i-th factor; also, their relative magnitudes allow one to give them a qualitative chemical meaning.

Analysis of rotated factors Correlation between factors

YFF

= &F

AFF

(3)

Since factors are non-orthogonal, part of the variance explained by them (which is the same variance explained by the retained initial factors), is collected by the internal correlations between factors, so that the sum of the variance of the rotated factors will be lower as the mutual correlation of the factors becomes greater. It is then useful to examine the matrix Y, whose generic element YU is the correlation between factors i and j. In the orthogonal case this matrix is the identity matrix. Eqn. 4 shows the relationship between the three matrices which are essential in the analysis of the rotated factors: VbF

= WbflFF

(4)

The result of an oblique rotation can be expressed on two systems of coordinates - primary axes and reference axes. These originated from Thurstone’s graphical methods [5], by which he qualitatively formalized the criteria of simplicity. Primary axes. These are defined as the (normalized) factors obtained by applying an objective simplicity criterion to the initial factorial matrix in order to obtain a simpler model. Reference axes. These are the system of rotated axes which are biorthogonal to the primary

axes. Each reference axis is orthogonal to a hyperplane formed by (F-l) primary axes. If PF, is the matrix of pattern loadings on the primary axes, and VFV is the matrix of structure loadings on the reference axes, then

(5) where D is a diagonal matrix containing the correlations between the primary axes and reference axes. This shows that the structure loadings of each reference axis are proportional to the pattern loadings on the related primary axis. As a consequence, the rotation which simplifies the matrix V is the same as that which simplifies the matrix P. In other words, the simple model based on the primary axes is equivalent to the simple structure on the reference axes. Once PFV is obtained (analogous to W) the rotation matrix TFF(analogous to A) is calculated. These two matrices are associated through eqn. 2 _ and then, from it, the structure matrix SF” in the primary axes (analogous to V) is obtained using eqn. 1. Finallly, the correlations between the primary axes
Optimizing criteria As previously shown, the methods of oblique rotation described here produce a rotation A such that the matrix V (the structure loadings on the reference axes) is as simple as possible. By this we mean that its elements are very different. Depending on the function vii, by which the idea of simplicity is formalized, several methlods are available. For each of them the solution is unique and independent of the initial factorial solution (matrix A). Quartimin

This minimizes the sum of the inner products of the square of the loadings

(6) j=l

i
Covarimin

This minimizes the covariances of the squares of the loadings, and is the extension of the Varimax

228

trends in analytical chemistry, vol. 12, no. 6, 7993

TABLE 1. Loadings on the primary axes after Quartimin (01, Q2) and Covarimin (Cl, C2) rotations Data from Fig. 1. Var. Index

Covarimin

Initial factors

Quartimin

At*.2

Structure S’s,*

Pattern P$,2 Ql

Fl

F2

Ql

0.66 0.73 0.49 0.53 0.77 0.64 0.72 0.81 Correlation matrix 1 .oo

0.20 0.31 0.43 0.38 -0.35 -0.41 -0.31 -0.30

0.67 0.52 0.79 0.54 0.64 0.27 0.65 0.33 0.50 0.85 0.36 0.75 0.48 0.78 0.56 0.86 Correlation matrix, CD22 1.00 0.60 0.60 1 .oo

0.00

0.00 1 .oo

criterion to the non-orthogonal to be minimized is then:

F

c=c i
1

Q2

0.57 -0.18 0.73 0.18 0.74 -0.17 0.70 -0.09 -0.00 0.85 -0.14 0.83 0.02 0.77 0.08 0.82 Rotation matrix, T22 0.87 0.91 0.48 -0.41

Pattern P’s,2

Cl

Cl

c2

0.37 0.36 0.10 0.16 0.81 0.75 0.75 0.81 Rotation matrix, T22 0.79 -0.62

0.56 0.69 0.64 0.63 0.20 0.07 0.20 0.26

c2

0.40 0.58 0.38 0.71 0.12 0.64 0.18 0.63 0.82 0.23 0.76 0.10 0.76 0.23 0.82 0.29 Correlation matrix, @22 1 .oo 0.04 0.04 1 .oo

0.65 0.76

case. The function V

”

V

VC

L

V$V!j- C

V$ C

j=l

j=l

j=l

/

Q2

Structure !&,2

\ V$jj

(7)

I

F2

While Quartimin produces a solution with very correlated factors, Covarimin qives a structure very similar to that obtained by an orthogonal rotation. Biquartimin

This produces a solution intermediate between Quartimin and Covarimin. The function to be minimized is B=Q+$

(8)

where Q is the Quartimin function, C is the Covarimin function and V is the number of variables. The factors obtained by this criterion are generally placed between those obtained by the two parent methods. Oblimin

This is a whole class of criteria, of which the previous three are particular cases. It uses the Biquartimin function, B, by giving suitable weights to the functions Q and C:

B=PlQ+f+

(9)

Fig. 1. Primary axes after an oblique rotation: Ql and Q2 with the Quattimin method, Cl and C2 with the Covarimin method. Fl and F2 are the initial factors. The index of the variables is shown.

229

trends in analytical chemistry, vol. 12, no. 6, 7993

If y = &/(pr + pz), then eqn. 9 becomes

the representation of complex objects such as food products, environmental samples, or mixtures. In these cases it is necessary to relate the mathematical factors to the global characteristics of the food, pollution sources, components, etc. To show such an application the experimental data obtained in ref. 6 have been used. The initial matrix has 120 rows, each row being the chromatogram of a sample of “orujo” (mart) of different origin, and nine columns (the percentage of the area of chromatographic peaks of nine volatile compounds). After autoscaling, the first three components explain 85.8% of the variance. Table 2 shows the loadings of the first three components (PCl, PC2 and PC3). Each component has high loadings of several variables and, apart from ethanol, each variable has high loadings on at least two axes. As a consequence, it is difficult to interpret the meaning of these axes. The components obtained by the Quartimin rotation are easier to interpret, on the basis of the volatility. The first one (RPCl) is basically connected with the four most volatile compounds, while the two least volatile are explained by the second one, and propanol and 2-butanol (intermediate volatility) by the third one. Ethanol was very important on PCl; after rotation its weight was almost the same on the three components, i.e. it does not characterize any of them, despite being the major component. The identification of amyl alcohols with the second rotated component allows us to interpret this axis as the quality of the “orujo” so that the score for each sample on RPC2 can be used as a quality index.

(10) i
j=]

j=l

j=l

I

If y = 1 the solution is that of Covarimin (minimum obliquity), if y = 0 the solution is Quartimin (maximum obliquity) and if y = 0.5 the solution is Biquartimin (intermediate obliquity). If, in the initial solution, the communality is low, then the values of the loadings are almost nil, thereby hiding the effect of a rotation. To overcome this, the loadings can be normalized by dividing them by the communalities of the variables. The results of Quartimin and Covarimin rotation applied to the data of Fig. 1 are shown in Table 1. The primary axes obtained by the Quartimin criterion form an angle of 53.13” (the arc sin of the correlation coefficient), while the axes obtained by the Covarimin rotation are almost orthogonal (87.71”). The Covarimin rotation produces a greater simplicity of the pattern loadings, corresponding to the simplest structure of the reference axes. In other words, each cluster of variables is strictly identified with each Quartimin factor in a much clearer way. If these groups of variables have a specific meaning, this is the same as the meaning of the associated axis.

A chemical application The need for interpreting the factorial axes is important when the chemical measurements are

TABLE 2. Gas chromatographic See text for further explanation.

data of “orujos”. Quartimin rotation of principal components,

Principal components loadings

Acetaldehyde Methyl acetate Ethyl acetate Methanol Ethanol Propanol 2-Butanol lsobutyl alcohol Amy1 alcohols

pattern loadings on primary axes

Quartimin rotation pattern loadings

PC1

PC2

PC3

FiPCl

RPC2

RPC3

-0.34 -0.38 -0.37 -0.39 0.44 -0.22 -0.16 -0.31 0.30

-0.38 -0.28 -0.36 -0.10 -0.07 0.49 0.42 0.36 0.30

-0.08 0.15 0.09 0.16 -0.05 0.27 0.60 -0.45 -0.54

0.47 0.49 0.51 0.40 -0.33 -0.04 0.01 -0.01 -0.00

0.03 -0.06 -0.06

-0.19 0.05 -0.05

0.04

0.17

-0.23 0.19 -0.11 0.66 0.68

-0.23 0.58 0.74 0.04 -0.06

trends in analytical chemistry

230

Program details From an initial orthogonal solution, the program OBLIQUE computes structure and pattern loadings, the rotation matrix, and the correlation between the rotated factors for both systems of axes (primary and reference), for every y value between zero and unity. The algorithm implemented [7] is iterative and at each cycle the lowest eigenvalue of an “ad hoc” matrix is extracted. If one has the scores of the initial factorial solution, then OBLIQUE can also compute the scores of the N objects on the new rotated factors. OBLIQUE is a program for IBM personal computers and compatibles. It is written in Microsoft QuickBASIC (v. 4.0). Th e source code is available from the authors upon request.

References 1 M. Forina, S. Lanteri and R. Leardi, Trends Anul. Chem., 6 (1987) 250.

vol. 72, no. 6, 1993

2 R.J. Rummel, Applied Factor Analysis, Northwestern University Press, Evanston, IL, 1970. 3 H.H. Harmann, Modern Fuctor Analysis, University of Chicago Press, Chicago, IL, 3rd ed. revised, 1977. 4 J.E. Jackson, A User? Guide to Principul Components, Wiley, New York, NY, 1991, Ch. 8. 5 L.L. Thurstone, Multiple Factor Analysis, University of Chicago Press, Chicago, IL, 1947, Ch. XV. 6 J.A. Saez, M.C. Ortiz and J. L6pez Palacios, “Tipificacidn de destilados alcohdlicos gallegos”. IIth Chemical International Congress of the ANQUE. Food Science and Technology: industry and Distribution, ANQUE, Burgos, 1992, p. 138. 7 J.B. Carroll, Psychometrika, 18 (1953) 23.

L. Sarabia (Department ofMathematica/Analysis) and M. C. Ortiz (Department of Analytical Chemistry) are at the Colegio Universitario de Bufgos, Apdo. 231, 09080 Burgos, Spain. R. Leardi and G. Drava are at the lstituto di Analisi e Tecnologie Farmaceutiche ed Alimentari, Universita di Genova, Via Brigata Salerno (ponte) 16147 Genoa, Italy.

For advertising information please contact our representatives

GREAT BRITAIN T.G. Scott & Son Ltd.

Tim Blake Portland House, 21 Narborough Road COSBY, Leicestershire LE9 5TA Tel.: (0533) 753-333, Fax: (0533) 750-522

JAPAN ESP-Tokvo Branch Mr. S. Onoda 20- 12 Yushima, 3-chrome, Bunkyo-Ku TOKYO 113 Tel: (03) 3836-0810, Fax: (03) 3839-4344 Telex: 02657617

Mr. M. White or Mrs. A. Curtis 30-32 Southampton Street, LONDON WC2E 7HR Tel.: (071) 240-2032, Fax: (071) 379-7155 Telex: 29918 adsale/g U.S.A. AND CANADA Weston Media Associates Mr. Daniel S. Lipner P.O. Box 1110, GREENS FARMS, CT 06436- 1110 Tel.: (203) 261-2500, Fax: (203) 261-0101

REST OF WORLD USEVIER SCIENCE PUBLISHERS Ms. W. van Cattenburch P.O. Box 2 11, 1000 AE AMSTERDAM The Netherlands Tel.: (20) 5153220, Fax: (20) 6833041 Telex: 16479 els vi nl