A computer program for constructing multivariate reference models

Computer Methods and Programs in Biomedicine 53 (1997) 191 – 200

A computer program for constructing multivariate reference models Marcel Hekking a,*, Jan Lindemans b, Edzard S. Gelsema a a b

Department of Medical Informatics, Erasmus Uni6ersity, P.O. Box 1738, 3000 DR Rotterdam, Netherlands Laboratory for Clinical Chemistry, Uni6ersity Hospital Rotterdam, Dijkzigt, 3015 6D Rotterdam, Netherlands Received 5 November 1996; received in revised form 26 February 1997; accepted 4 March 1997

Abstract From a statistical point of view the simultaneous interpretation of multiple variables should be performed with a multivariate reference model rather than with multiple univariate reference intervals. A computer program for constructing and testing multivariate reference models is described. The use of the computer program is illustrated with a data set of total serum calcium concentrations and serum albumin concentrations from 222 2nd year medical students. Using a single univariate reference interval for total serum calcium, 17 students were classified as having an abnormal calcemic status while using a bivariate reference model for total serum calcium and serum albumin, 13 of these 17 students had in fact normal total serum calcium concentrations, taking into account their serum albumin concentrations. © 1997 Elsevier Science Ireland Ltd. Keywords: Multivariate modeling; Reference values; Graphics; Clinical chemistry

1. Introduction To improve the health of their patients, physicians collect empirical data which they interpret by comparing these with specific reference data, using their medical knowledge and clinical experience [1]. The interpretation of medical laboratory

* Corresponding author. Tel.: + 31 10 4088116; fax: + 31 10 4362882; e-mail: [email protected]

data is a special case of decision making by comparison. Typically, a laboratory value of a patient is compared with a reference interval determined on a sample of values that is thought to be representative of a specific, often healthy, population. The concept and use of the univariate 95% reference interval is well established in clinical chemistry [2]. Although the 95% univariate reference interval is generally considered as the standard tool for the interpretation of laboratory data, some well-known limitations exist. In 1969 Schoen and Brooks [3] reported a statistical

0169-2607/97/$17.00 © 1997 Elsevier Science Ireland Ltd. All rights reserved. PII S 0 1 9 - 2 6 0 7 ( 9 7 ) 0 0 0 1 8 - 7

192

M. Hekking et al. / Computer Methods and Programs in Biomedicine 53 (1997) 191–200

dilemma involving the use of multiple 95% univariate reference intervals. With the simultaneous use of more than one 95% univariate reference interval, the percentage of observations classified as normal decreases with the number of reference intervals [4–7]. For instance, a healthy person evaluated with one 95% univariate reference interval has an a priori probability of 95% of being classified as normal. The same person evaluated with 10 independent 95% univariate reference intervals will have an a priori probability of only 0.9510 = 60% of being classified as normal for all intervals. Therefore, the probability of a false positive observation increases with the number of univariate intervals. One way to deal with this dilemma is to automatically adjust the reference intervals in the case of multiple tests in such a way that the probability of falling in all reference intervals will be 95% (Bonferroni adjustment) [8,9]. Another option is the use of a multivariate reference model [4,5,9 – 12]. The multivariate reference model yields a single multivariate index of abnormality which takes into account the interrelationships between the variables. The use of a multivariate model rather than multiple univariate reference intervals not only decreases the probability of false positive observations but also decreases the probability of false negative observations. Such a false negative observation occurs when univariate observations are normal according to their respective univariate reference intervals but quite abnormal if the correlations between the variables are considered. Medical decisions often require the simultaneous interpretation of many, often correlated, pieces of information and the multivariate reference model therefore seems the most appropriate model [7,13]. In this paper we describe a Microsoft® Windows™ based computer program developed specifically for constructing and testing multivariate reference models. Undoubtedly the functionality present in this program can also be achieved by using pieces of existing software. This will, however, involve tedious sessions of starting and closing applications and copying and pasting data. Moreover, an inexperienced user may not be familiar with such software or may not even have

access to it. Our goal was to develop a computer program with all the necessary functionality to interactively construct, test and visualize both uniand multivariate reference models. To our knowledge no such computer program is available today. Recently, Solberg described an MS-DOS® Pascal program called RefVal for the implementation of the statistical treatment of reference values as recommended by the IFCC Expert Panel on Theory of reference values [14]. The RefVal program enables the calculation of univariate reference intervals. Constructing and testing multivariate reference models, however, is not within its scope. The use of the multivariate modeling computer program developed by us is illustrated with an example.

2. Program description

2.1. Functionality and computational methods At startup a window at the top of the screen appears with a menu bar containing the program’s options, including a data import facility. At the present moment the program can only import data from an ASCII file. Variables in the input file must be separated with a single space, comma or a tab. After a successful import, the data can be viewed (but not altered) and the options are enabled. The textual and graphical output can be viewed on screen and sent to a printer. Basic statistical analysis includes the calculation of mean, S.D., skewness, kurtosis and correlation coefficients. In the following sections the more advanced options and associated computational methods are discussed.

2.1.1. The 95% multi6ariate reference model A multivariate reference model is defined as follows. Assuming a theoretical multivariate gaussian distribution with known mean vector and variance-covariance matrix (m, S), the 95% reference region includes all cases x having a squared Mahalanobis distance [15]. D 2 = (x− m)t S − 1(x−m)

(1)

M. Hekking et al. / Computer Methods and Programs in Biomedicine 53 (1997) 191–200

smaller than or equal to the 0.95 fractile of a x 2-distribution with k degrees of freedom, where k is the number of variables included in the multivariate model [16]. The superscript t stands for the transposition of a column-vector to a row-vector, x is the observation vector and S-1 is the inverse of the variance-covariance matrix. When defining a reference region based on a sample, the model parameters m and S are unknown and are replaced by the sample estimates x¯ and S, respectively. To incorporate the uncertainty in estimating the mean vector and the variance-covariance matrix and the uncertainty of a single observation, Chew and Albert propose the following approximation of the 0.95 fractile based on the F-distribution [11,17]; 2

C =k(N −1)F(0.95; k, N −k)/N(N − k)

(2)

where k is the number of variables in the analysis, N is the number of cases and F(0.95; k, N − k) is the 0.95 fractile of the F-distribution for k and N−k degrees of freedom. In geometrical terms these 0.95 fractiles delimit specific regions in k-dimensional space and are known as equal density ellipses (for k =2) or equal density ellipsoids (for k \2). The region delimited by C is also known as the 95% prediction region [11]. The use of a 95% prediction region as a valid reference model is only justified if the multivariate sample distribution is gaussian. Hence, an essential step in constructing a multivariate reference model is the verification of the multivariate gaussian assumption. We implemented the following method. If a multivariate distribution is gaussian then the D 2 values (Eq. (1)) are distributed according to a x 2-distribution with k degrees of freedom where k is the number of variables in the multivariate distribution (i.e. the dimension of the model) [5,11,18,19]. With a 1-dimensional goodness-of-fit test like the Kolmogorov-Smirnov test or the Anderson-Darling test this fit can be quantified [18]. If the test fails to show a significant deviation of the D 2 distribution from the x 2-distribution then the hypothesis of the multivariate distribution being gaussian is very likely.

193

2.1.2. The 95% uni6ariate reference inter6al At least three different definitions of a parametric 95% univariate reference interval exist: the 95% tolerance interval, the 95% prediction interval and the 95% interfractile interval [20]. Although the differences in results are small for sufficiently large sample sizes (N ]100) we chose to implement the 95% prediction interval which is conceptually the same as the 95% prediction region as discussed in Section 2.1.1. The 95% prediction interval takes into account the uncertainty in estimating the mean and the S.D. and the uncertainty of a single observation. It is calculated as follows [11,16]: x¯ 9 (1+ 1/N) t(0.025; N− 1)s where t(0.025; N− 1) is the 0.025 fractile of the Student’s t distribution with N− 1 degrees of freedom, x¯ is the sample mean and s is the sample S.D. If a distribution is non-gaussian and remains non-gaussian even after a gaussian transformation procedure, the program offers the calculation of a non-parametric 95% interfractile interval based on the rank-based method described in [21].

2.1.3. Goodness-of-fit testing Statistical conclusions based on samples often assume that the sample values are distributed according to a specific theoretical distribution. For instance, a parametric 95% reference interval is only valid if the sample distribution is gaussian. Goodness-of-fit tests are used to test the fit of a sample distribution with a specific theoretical distribution. We implemented two goodness-of-fit tests. The Kolmogorov-Smirnov test statistic Dmax is the largest vertical distance in a cumulative probability plot between a sample distribution and a theoretical distribution (the concept of a cumulative probability plot is explained in Section 2.1.6). A significant large Dmax indicates that the sample distribution significantly deviates from the specified theoretical distribution. The size-adjusted Anderson-Darling test statistic A 2 is more complex; it is a weighted of all vertical distances between a sample distribution and a theoretical distribution in a cumulative probabil-

194

M. Hekking et al. / Computer Methods and Programs in Biomedicine 53 (1997) 191–200

ity plot. The Anderson-Darling test is said to be more powerful than the Kolmogorov-Smirnov test [21]. Both tests are discussed in more detail in Appendix A.

2.1.4. Transforming data Specific transformation functions and procedures exist to transform univariate non-gaussian distributions to approximately gaussian. We implemented the two-stage transformation procedure as recommended by the IFCC [21]. In the first stage of this procedure a significant positive or negative skew is iteratively eliminated with the aid of the exponential function of Manly [22]. In the second stage a remaining significant kurtosis of the resulting symmetrical distribution is iteratively eliminated with the modulus function of John [23]. For a full description of the two-stage transformation procedure the reader is referred to Appendix A. Although the fact that all marginal distributions being gaussian in a multivariate reference model does not automatically imply that the multivariate distribution is also gaussian, it is a necessary condition. Therefore, the values of a transformed distribution are saved in a separate variable so that a multivariate model can be constructed using any combination of original and transformed variables. 2.1.5. Principal component analysis As a result of correlations between the variables, a multivariate sample distribution may contain redundant information. This can have a profound impact on a multivariate reference region since redundant information introduces noise into the model. Therefore it is desirable, prior to the construction of a reference model, to try to reduce the dimensionality of a multivariate distribution whilst preserving as much of the original information as possible. This can be done with a principal component analysis or PCA [24]. PCA finds new orthogonal axes (the principal component axes) for a multivariate data set, such that the first axis covers the largest proportion of the total variance, the second axis the second largest proportion, and so on. If a data set contains redundant information then the last principal

component axes explain only a small proportion of the total variance. The program finds these principal component axes with an eigenvalue transformation of the correlation matrix. Moreover, principal component values (PCs) are calculated and saved as separate variables so a 95% multivariate model can be constructed in a principal component subspace.

2.1.6. Scatterplots and cumulati6e probability plots A graphical representation of data can be obtained by making scatterplots and cumulative probability plots. In the case of a bivariate reference model the 95% prediction region is drawn into the scatterplot as an ellipse. Cumulative probability plots graphically depict the fit of a sample cumulative probability distribution with a theoretical cumulative probability distribution. Such a plot is constructed by ranking the input values of a sample distribution in ascending order and plotting the cumulative probability P, which is the rank number divided by the total number of observations, of each observation against the actual value of the observation. Then, in the same plot a theoretical cumulative distribution (e.g. gaussian, x 2 or F) is plotted. 2.2. Hardware and software specifications and de6elopment The program was developed with the use of Microsoft® Visual Basic™ version 3.0, Professional Edition on a 66MHz 80486 Personal Computer with 12 Mb RAM and requires Microsoft® Windows™ version 3.1 or higher. Visual Basic (VB) is a rapid application development tool (RAD) that enables the fast development of event driven programs with a graphical user interface (GUI). We extensively used the technique of dynamic linking which is a key feature of the Windows platform. Dynamic-link libraries (DLL’s) are compiled libraries of functions which can dynamically be linked to applications at run time. We used the DLL technique for the implementation of most of the numerical routines our program. The numerical routines are taken from the book ‘Numerical Recipes in C. The Art of

M. Hekking et al. / Computer Methods and Programs in Biomedicine 53 (1997) 191–200

195

Table 1 Numerical functions and procedures used in the program and their origin Name

Type

Description

invert moment gammp

C-procedure NR-procedure NR-function

erff

NR-function

pearsn

NR-procedure

ksone

NR-procedure

betai

NR-function

jacobi transform sasquare

NR-function VB-procedure VB-function

Calculates the inverse of a matrix. Uses the NR-procedures ‘ludcmp’ and ‘lubkdsb’ Calculates mean, S.D., variance, kurtosis and skewness from an array of input values Incomplete g function for the calculation of the theoretical cumulative probability for a specific x 2-value with a given number of degrees of freedom Error function for the calculation of a theoretical cumulative probability for a specific gaussian z-score Returns Pearson’s correlation coefficient r and corresponding P-value for two arrays of input values Returns the Kolmochorov-Smirnov test statistic Dmax and corresponding P-value for an array of input values and a specific theoretical cumulative probability distribution Incomplete b function for the calculation of a theoretical cumulative probability for a specific F-value with a given number of degrees of freedom Returns the eigen-vectors and -values of a symmetrical matrix Two-stage transformation procedure (see text) Returns the Anderson-Darling statistic A 2 for an array of input values and a specific theoretical cumulative probability distribution

Note: the suffix ‘NR’ indicates a Numerical Recipes procedure or function (Copyright 1987-1992 Numerical Recipes Software). The suffix ‘VB’ indicates a Visual Basic function or procedure. All non-VB functions and procedures are located in a DLL called ‘numrec.dll’.

Scientific Computing’ by Press et al. [25]. The routines were linked into one single DLL using the Borland® C + + compiler version 4.5. From within VB the functions and procedures in the DLL were declared according to the standard declaration syntax of VB which made them available to our VB program. Table 1 summarizes the most important functions and procedures in the program and their origin. Linking these tailor-made compiled numerical routines to our program had some major advantages: (1) the routines are reliable and highly efficient; (2) no extensive programming and debugging was needed; and (3) routines in compiled DLL’s are faster compared to routines written in VB itself.

3. Example Calcium is an electrolyte which plays an important role in many physiological processes, like the excitability of nerve function and neural transmission, blood coagulation and the mineralization of bone [26]. The clinical interpretation of the total serum calcium concentration is complicated by

the binding of calcium to circulating proteins, mainly albumin. About 40% of the calcium is in this bounded form and therefore changes in albumin concentration may affect total serum calcium concentration without changing the biologically important ionized fraction of calcium [27]. Therefore, for a valid interpretation of the calcemic status, the total serum calcium concentration and the serum albumin concentration should be interpreted simultaneously. With the aid of the program we developed a bivariate 95% prediction region for total serum calcium and serum albumin based on the values obtained from 222 2nd year medical students. Blood samples were collected in SST-Vacutainer tubes (Becton Dickinson, Rutherford, NY) and centrifuged within an hour. Total serum calcium and albumin were analyzed on a Chem-1™ routine chemistry analyzer (Bayer-Technicon, Tarrytown, NY) using the standard Chem-1 reagents (bromcresol green for albumin). Table 2 shows the results of the first examination of the data. The univariate distributions of both total serum calcium serum albumin were found to be non-gaussian when tested with the Anderson-Darling test. After the two-stage trans-

196

M. Hekking et al. / Computer Methods and Programs in Biomedicine 53 (1997) 191–200

Table 2 Distribution characteristics and goodness-of-fit test results for total serum calcium concentration and serum albumin concentration in 222 2nd year medical students

Mean S.D. Skewness Kurtosis A2 Dmax

Total serum calcium (mmol/l)

Serum albumin (g/l)

Untransformed

Transformed

Untransformed

Transformed

2.433 0.103 0.006 −0.909a 1.138b 0.066

0.003 1.809 0.016 −0.114 0.494 0.057

49.635 3.006 −0.174 −0.318 1.892b 0.117c

0.033 0.997 −0.007 −0.270 1.603b 0.108c

Significant kurtosis at a= 0.05. Significant large Anderson-Darling’s A 2 at a=0.05; data do not fit a gaussian distribution. c Significant large Kolmochorov-Smirnovs Dmax at a=0.05; data do not fit a gaussian distribution. a

b

formation procedure only the total serum calcium values were successfully transformed to a gaussian distribution. To illustrate the nature of the effect of a marginal distribution on a multivariate reference model, two bivariate reference models were constructed; untransformed serum albumin versus untransformed total serum calcium and untransformed serum albumin versus gaussian transformed total serum calcium. The calculated D 2 values of the model based on untransformed total serum calcium and untransformed serum albumin did not fit a x 2-distribution with 2 degrees of freedom as tested with the Anderson-Darling test (significant Anderson-Darling statistic A 2 of 2.108 at a = 0.05). Using gaussian transformed calcium values rather than the original calcium values resulted in a fit of the D 2 values with the specific x 2-distribution (non-significant Anderson-Darling statistic A 2 of 0.441 at a =0.05). Fig. 1 is the cumulative probability plot of these D 2 values. Fig. 2 shows the resulting bivariate 95% prediction region for untransformed serum albumin and gaussian transformed total serum calcium. This 95% prediction region corresponds to a threshold value of 6.13 as calculated with Eq. (2). Calculated D 2 values 56.13 are located on or inside the ellipse. Note that this threshold of 6.13 is slightly more conservative than the 0.95 percentile of the x 2-distribution with 2 degrees of freedom ( = 5.99). The ellipse is tilted because of the significant positive correlation between untransformed serum albumin and gaussian transformed

total serum calcium (r = 0.461 at a= 0.05). The calculated lower and upper limit of the parametric univariate 95% reference interval for the gaussian transformed total serum calcium are − 3.542 and 3.548, respectively (Fig. 2). By applying the inverse of the transformation functions, these dimensionless numbers were back-transformed to their original dimension yielding a lower- and upper limit for the total serum calcium concentration of 2.25 and 2.62 mmol/l, respectively. Using these limits, 7 of the 222 students would be regarded as hypercalcemic while 10 students would be regarded as hypocalcemic. If we use the 95% prediction region rather than these univariate reference limits, 13 of the 17 students with apparent positive findings appear to have normal total serum calcium concentrations taking into account their serum albumin concentrations. These 13 individuals are typically located outside the univariate 95% reference interval for gaussian transformed total serum calcium but within the 95% ellipse. Note that these numbers cannot be reconstructed from Fig. 2 since several observations are overlapping.

4. Discussion Multivariate modeling of biomedical data still remains outside the domain of biologists and physicians [28]. This is illustrated by the relative small number of papers describing the use

M. Hekking et al. / Computer Methods and Programs in Biomedicine 53 (1997) 191–200

197

Fig. 1. Cumulative probability plot of the untransformed serum albumin versus Gaussian transformed total serum calcium reference model. (’’’’’), cumulative probability distribution of calculated D 2 values; (——), theoretical x 2 cumulative probability distribution with 2 degrees of freedom.

of multivariate reference regions in a medical domain. One of the first multivariate reference regions is that presented by Winkel et al. [10] in 1972. In their article they describe the generation of multivariate reference regions for 5 variates relevant to the metabolism of hemoglobin; blood hemoglobin (HGB), serum Iron (FE), serum transferrin (TRANS), serum cobalamine (B12) and serum bilirubin (BILI). Although the number of sample individuals was rather low, they concluded that the multivariate reference region is useful for screening purposes because of the decrease in false positive results as compared to the use of multiple univariate reference intervals. Also in 1972, Gramms et al. [4] already described a computerized laboratory data analysis system that included a module for the generation of multivariate reference regions. In 1978, Ka˚gedal et al. [29] determined a trivariate reference region for free thyroxine index, free triiodothyronin index and thyrotropin based on the data of 3885 women gathered during a gynaecological health-screening

program. Comparing the performance of this region with univariate intervals, they confirmed the conclusion of Winkel et al. that a significant decrease in false positive observations can be achieved using multivariate reference regions. Naus [6] constructed a bivariate reference region for total protein and albumin on an unselected data set of 3840 laboratory measurements. Gelsema et al. [30,31] developed a multivariate reference region for arterial pH, PCO2 and base excess values coming from an unselected respiratory Intensive Care Unit (ICU) population. Although from a theoretical point of view the multivariate reference region is regarded as superior over the use of univariate reference intervals, there are indeed some pitfalls and drawbacks related to the use of multivariate reference regions. First of all, the calculated multivariate index of abnormality D 2 does not give information about which univariate value too low or too high. Furthermore, marginal distributions must be approximately gaussian in order to yield a

198

M. Hekking et al. / Computer Methods and Programs in Biomedicine 53 (1997) 191–200

Fig. 2. The bivariate 95% prediction region for untransformed serum albumin and gaussian transformed total serum calcium. Note: the vertical lines denote the univariate 95% prediction interval for gaussian transformed total serum calcium.

valid multivariate gaussian distribution. If a nongaussian distribution cannot be transformed to a gaussian one then it is unlikely that this distribution can be included in a multivariate reference model. Finally, in simulation experiments Linnet [16] showed that a multivariate reference region can be very sensitive to analytical imprecision occurring after the region has been established. For a 20-dimensional model, an increase of the total S.D. of each original variable by a factor of 1.25 decreased the proportion included in the 95% ellipsoid from 0.95 to 0.54. Boyd and Lacher [18] showed that the multivariate reference region is rather insensitive for the detection of highly abnormal results for a single test in the multivariate profile. They constructed multivariate reference regions from the results of 20 clinical chemistry analytes that were measured in two successive medical-school classes (118 individuals in 1979 and 143 individuals in 1980). Both multivariate models showed a reduction of false positive observations. On the other hand, with a 20-test model

they showed that for a single test an abnormal observation of 5.5 S.D. from the mean was required to be detected multivariately while all the other tests were held at their mean. For this reason multivariate reference models should not be used in cases with dimensions higher than 3 [7,8]. In conclusion, the multivariate reference model can be helpful in some cases but must be seen as an addition to the univariate reference interval and not a replacement. Moreover, special care must be taken when constructing multivariate reference models with regard to the testing of the underlying multivariate gaussian assumption. The described computer program can be a helpful tool in this process.

5. Availability of the The computer program described in this paper can be downloaded from the world-wide-web site:

M. Hekking et al. / Computer Methods and Programs in Biomedicine 53 (1997) 191–200

http://www.eur.nl/FGG/MI/annrep95/p01.html. For other ways of delivery please contact the corresponding author.

The authors wish to thank Mr B. Schijvenaars for carefully reading the manuscript and for valuable suggestions.

Appendix A A.1. The Kolmochoro6-Smirno6 test The test statistic Dmax is defined as the largest difference between a sample cumulative distribution and a theoretical cumulative distribution. On a ranked data set of N observations, Dmax is calculated as follows [21,25]: D + =max(i/N−Pi ),

to 0 mean and 1 S.D. Then, in the first stage, a significant positive or negative skew in the distribution is eliminated with the aid of the exponential power of Manly [21,22]: yi = (exp(g× xi )− 1)/g,

Acknowledgements

(i = 1,…, N),

D − =max(Pi − (i−1)/N),

(A1)

(i =1,…, N),

yi = xi,

where Pi is the theoretical cumulative probability of index i. A large Dmax means a poor fit. Critical values for the calculated Dmax can be found in statistical tables. A.2. Anderson-Darling test

g"0,

(i= 1,…, N), (A3)

g= 0.

The optimal value of g is estimated in an iterative procedure in which the coefficient of skewness is recalculated after each transformation with a specific value of g. The procedure is stopped when the coefficient of skewness is sufficiently close to 0, or a predefined maximum number of iterations is reached. The resulting distribution yi, i= 1,…, N is then standardized to mean 0 and 1 S.D. and a significant kurtosis of the distribution is corrected with the aid of the modulus function of John and Draper [21,23]. zi = sign{( yi + 1)l − 1/l},

l"0,

(i= 1,…, N), zi = sign{(ln yi + 1)},

Dmax = max(D + , D − ),

199

(A4) l= 0,

where ‘sign’ represents the original (+ or −) sign of yi. The optimal value of l is estimated in an iterative procedure in which the coefficient of kurtosis is recalculated after each transformation with a specific value of l. The procedure is stopped when the coefficient of kurtosis is sufficiently close to 0, or a predefined maximum number of iterations is reached.

The Anderson-Darling test A 2 is defined as [19,21]: References

N

A 2 = − % ([2i− 1] i=1

× {ln(Pi )+ln(1 − Pn + 1 − i )}/N) −N.

(A2)

A large A 2 means a poor fit. Critical values for the size-adjusted test statistic A 2(A 2 ×(1 +4/N − 25/N 2)) can be found in statistical tables. A.3. Two-stage transformation A univariate distribution xi , i= 1,..., N (where N is the number of observations) is standardized

[1] H.E. Solberg, Establishment and use of reference values, in: C.A. Burtis and E.R. Ashwood (Eds.), Tietz Textbook of Clinical Chemistry, Saunders, Philadelphia, PA, 1994, pp. 454-484. [2] The International Federation of Clinical Chemistry (IFCC), Scientific Committee, Clinical Section, Expert Panel on Theory of Reference Values and the International Committee for Standardization in Haematology (ICSH), Standing Committee on Reference Values, H.E. Solberg, Approved Recommendation (1986) on the theory of reference values. 1. The concept of reference values, J. Clin. Chem. Clin. Biochem. 25 (5) (1987) 337 – 342.

M. Hekking et al. / Computer Methods and Programs in Biomedicine 53 (1997) 191–200

200

[3] I. Schoen, S.H. Brooks, Judgment based on 95% confidence limits, Stat. Considerations 53 (1969). [4] R.R. Grams, E.A. Johnson, E.S. Benson, Laboratory data analysis system: section III—multivariate normality, Am. J. Clin. Pathol. 85 (1972) 188–199. [5] E.K. Harris, Statistical aspects of reference values in clinical pathology, Prog. Clin. Pathol. 8 (1981) 45–66. [6] A.J. Naus, A. Borst, P.S. Kuppens, Determination of n-dimensional reference ellipsoids using patient data, J. Clin. Chem. Clin. Biochem. 20 (1982) 75–80. [7] I.H. Stamhuis, P.D. Bezemer, D. Kuik, Evaluation of univariate ranges with a multivariate standard, J. Clin. Epidemiol. 41 (4) (1988) 359–366. [8] H.B. Slotnick, P. Etzell, Multivariate interpretation of laboratory tests used in monitoring patients, Clin. Chem. 36 (5) (1990) 748 – 751. [9] P. Winkel, Multivariate analysis and expert systems. [Review], Scand. J. Clin. Lab. Invest. 219 (1994) 12–24. [10] P. Winkel, J. Lyngbye, K. Jo¨rgensen, The normal region — a multivariate problem, Scand. J. Clin. Lab. Invest. 30 (1972) 339 –344. [11] A. Albert, E.K. Harris, Multivariate interpretation of clinical laboratory data, in: Statistics: Textbooks and Monographs, vol. 75, Marcel Dekker, New York, 1987. [12] R.W. Wulkan, Expert Systems and Multivariate Analysis in Clinical Chemistry, Erasmus University, Rotterdam, 1992, 111 p. [13] E.K. Harris, Comparing multivariate and univariate subject-specific reference regions for blood constituents in healthy persons, Clin. Chem. 28 (3) (1982) 422–426. [14] H.E. Solberg, RefVal: a program implementing the recommendations of the international federation of clinical chemistry on the statistical treatment of reference values, Comput. Methods Programs Biomed. 48 (1995) 247–256. [15] P.C. Mahalanobis, On the generalized distance in statistics, Proc. Natl. Inst. Sci. India 2 (1936) 49–56. [16] K. Linnet, Influence of sampling variation and analytical errors on the performance of the multivariate reference region, Methods Inform. Med. 27 (1) (1988) 37–42. [17] V. Chew, Confidence, prediction, and tolerance regions for the multivariate normal distribution, J. Am. Stat. Assoc. 61 (1966) 605–617. [18] J.C. Boyd, D.A. Lacher, The multivariate reference range: an alternative interpretation of multi-test profiles, Clin. Chem. 28 (2) (1982) 259–265.

.

[19] K.V. Mardia, Tests of univariate and multivariate normality, in: P.R. Krishnaiah (Eds.), Handbook of Statistics, North-Holland, Amsterdam, 1980, pp. 279-320. [20] H.E. Solberg, R. Gra¨sback, Reference values, Adv. Clin. Chem. 27 (1994) 1 – 79. [21] International Federation of Clinical Chemistry (IFCC), Scientific Committee, Clinical Section, Expert Panel on Theory of Reference Values, H.E. Solberg, Approved recommendation (1988) on the theory of reference values. Part 5. Statistical treatment of collected reference values. Determination of reference limits, J. Clin. Chem. Clin. Biochem. 25 (9) (1987) 645 – 656. [22] B.F.J. Manly, Exponential data transformations, Statistician 25 (1976) 37 – 42. [23] J.A. John, N.R. Draper, An alternative family of transformations, Appl. Statist. 29 (1980) 190 – 197. [24] I.T. Jolliffe, B.J.T. Morgan, Principal component analysis and exploratory factor analysis, Stat. Methods Med. Res. 1 (1992) 69 – 95. [25] W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P. Flannery, Numerical Recipes in C. The Art of Scientific Computing, Cambridge University Press, Cambridge, 1992. [26] J.P. Ashby, D.J. Wright, M.G. Rinsler, The adjusted serum calcium concept — a reappraisal, Ann. Clin. Biochem. 23 (5) (1986) 533 – 537. [27] J. Whicher, C. Spence, When is serum albumin worth measuring?, Ann. Clin. Biochem. 24 (6) (1987) 572 – 580. [28] R. Benigni, A. Giuliani, Quantitative modeling and biology: the multivariate approach, Am. J. Physiol. 266 (5 Pt 2) (1994) R1697 – R1704. [29] B. Ka˚gedal, A. Sandstrom, G. Tibbling, Determination of a trivariate reference region for free thyroxine index, free triiodothyronine index, and thyrotropin from results obtained in a health survey of middle-aged women, Clin. Chem. 24 (10) (1978) 1744 – 1750. [30] E.S. Gelsema, B. Leijnse, R.W. Wulkan, Detection of aberrant observations in a background of an unknown multidimensional gaussian distribution, Methods Inform. Med. 29 (1990) 236 – 242. [31] E.S. Gelsema, B. Leijnse, R.W. Wulkan, A multi-dimensional analysis of three chemical in the blood, Med. Inform. 16 (1) (1991) 43 – 54.