- Email: [email protected]
Computerized pigment design based on property hypersurfaces
ARTICLE IN PRESS
Journal of Physics and Chemistry of Solids 68 (2007) 744–746 www.elsevier.com/locate/jpcs
Computerized pigment design based on property hypersurfaces Jaroslava Halovaa, Petra Sulcovab, Karel Kupkac, a
Institute of Inorganic Chemistry of the Academy of Sciences of the Czech Republic, CZ 250 68 Rez-Husinec 1001, Czech Republic Department of Inorganic Technology, Faculty of Chemical Technology, University of Pardubice, Na´meˇstı´ legiı´ 565, CZ 532 10 Pardubice, Czech Republic c TriloByte Statistical Software, Ltd., Jiraskova 21, CZ 530 02 Pardubice, Czech Republic
b
Received 28 September 2006; accepted 31 January 2007
Abstract Competition is tough in the pigment market. Rational pigment design has therefore a competitive advantage, saving time and money. The aim of this work is to provide methods that can assist in designing pigments with defined properties. These methods include partial least squares regression (PLSR), neural network (NN) and generalized regression ANOVA model. Authors show how PLS bi-plot can be used to identify market gaps poorly covered by pigment manufacturers, thus giving an opportunity to develop pigments with potentially profitable properties. r 2007 Elsevier Ltd. All rights reserved. Keywords: A. Inorganic compounds; A. Oxides; B. Chemical synthesis; D. Optical properties
1. Introduction Designing physical, biological, environmental or chemical properties of chemical compounds using mathematical modelling is increasingly attractive due to growing computing power and low time and financial costs compared to lengthy and expensive experimentation [1]. Various methods of predictive statistical models have been used in chemistry, biochemistry, pharmaceutical and food industry and research, material research but also in econometry and finance. This paper shows how use of predictive modelling could save time and funds in designing properties of inorganic pigments. 2. Theoretical 2.1. Partial least squares To find relationship between two groups of variables and predict one from another was used a statistical procedure based on the partial least squares (PLS) approach, first Corresponding author. Tel.: +420 466615725; fax: +420 466615735.
E-mail addresses: [email protected] (J. Halova), [email protected] (P. Sulcova), [email protected] (K. Kupka). 0022-3697/$ - see front matter r 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.jpcs.2007.01.052
published by Herman Wold in the late 1960s [2]. This mathematical methodology has been used in econometrics, chemometrics, biometrics, sensometrics and recently, some applications in technology and qualimetrics also appeared. Assume that we have n rows of the measured process parameters, that form a matrix X(n p), and the same number of measured corresponding quality parameters in matrix Y(n q), where not necessarily n4p, q. In order to extract maximum information (variability) into lowerdimensional space PLS uses an analogy to well known orthogonal principal component approach (PCA) and transforms X and Y into scores and loadings X ¼ TP0 þ E, Y ¼ TQ0 þ F. To ensure maximum relevance of chosen X-components to Y-values, the two transformations are tied together by common scores matrix T. Dimensionality (number of columns) of T is typically smaller than that of X and Y and columns in T are orthogonal. This improves stability of the model and maximizes information gain. Noise and irrelevant useless information concentrates in ‘‘garbage’’ matrices E and F. Writing U ¼ TB (where B is a square diagonal matrix) gives us the tool for prediction
ARTICLE IN PRESS J. Halova et al. / Journal of Physics and Chemistry of Solids 68 (2007) 744–746
745
of Y from X ^ ¼ TQ0 or Y ¼ XB Y with T computed from the new X-data. Unlike classical least squares or principal component regression (PCR), in PLS regression, X and Y are equivalent, or interchangable. This property predetermines PLS for use in biotechnology to predict properties or activity of a product from chemical composition or structure and vice versa. Multivariate statistical modelling is thus a very fertile platform for newly emerging interdisciplinary sciences like chemometrics, qualimetrics, biometrics. 2.2. Neural network Neural network (NN) [3] is used to relate multivariate input variable and multivariate output variable. NN can be thought of as a nonparametric modelling tool with network structure. In network knots, or neurons, each normalised input variable xi is weighted by wji and transformed by activation function sj+1,i (z) ¼ 1/(1+ez), where z is a linear combination of input variables, zi ¼ a0+Saijzi1,j. Weights wji represent links between variables and neurons, or in multilayer network, between neurons in consecutive layers and are sometimes referred to as synapses as an analogy to biological neurons Fig. 1. Output variables are neurons that transform values from the last hidden layer so that resulting sn+1,i (zi) is as close as possible to the measured experimental values or output variables y. This is done by optimization of all weights and coefficients aij in every neuron. Genetic algorithms are usualy employed to find optimal parameters of the network. This process is called training of the network. The trained network can than be used to predict unknown output variables from given values of input variables. Generally, neural networks are equivalent to a non-linear local nonparametric regression model. As such, NN cannot be generally used to predict beyond the range of the input variables used for its training (Fig. 2).
Fig. 2. Resulting NN model for predicting colour properties from input variables, synapses width suggest thar Ln and M are more important than x and T, as expected.
3. Experimental Color properties were studied on synthetised lanthanide oxide-type pigments doped with 5B group metals Cr or V [4]. The general formula of the pigment is Ln2 Zr(2x) Mx O7, where Ln ¼ lanthanoids, M ¼ Cr or V, x ¼ 0.05–0.2. The synthesis was carried out at two temperatures T ¼ 1400 or 1500 1C, respectively. Lanthanoids used in place of Ln were: La, Ce, Pr, Nd, Sm, Eu, Gd, Tb, Dy, Ho, Er, Y. The aim was to model resulting colour coordinates a*, b*, L*, dE* (property hypersurface) [5] using composition and conditions during the synthesis. The input, or independent variables thus were: type of lanthanoid Ln, type of the doping metal M, ammount of the doping metal x, and temperature T. Atomic volumes were used to quantify used lanthanoides. Output variables are a*, b*, L*, dE*. Neural network with one and two hidden layers and a PLS-2 regression were used to model and predict output properties of the pigment for given composition and temperature. Both statistical methods were also used the reverse way to propose composition and temperature to synthesise pigment with required properties. All computations were carried out on the statistical system QC Expert [6]. 4. Results
Fig. 1. The structure of a single-layer neural network.
In our study, multilayer NN were better for predicting the target properties than single-layer NN, or PLS regression. The resulting network is shown at Fig. 1, the width of the synapses correspond to the values wji, the wider the connecting synapsis the more information flow between the respective knots. PLS-biplot was found to be useful in identifying gaps in the property hypersurface, Fig. 3a and b. This feature may be exploited to efficiently design new products defined by their target parameters.
ARTICLE IN PRESS 746
J. Halova et al. / Journal of Physics and Chemistry of Solids 68 (2007) 744–746
sented by dots; pigments of the same brands form clusters. The pigments are distributed over a colour space rather unevenly with substantial gaps. These gaps may indicate unutilized market opportunities. It may be to the advantage of the producer to fill these gaps with appropriate pigments designed utilizing PLS statistical modelling, as mentioned above. The presented method of computerized pigment design based on the mapping of property hypersurfaces is a powerful tool for identifying targets for the synthesis of new pigments. The method we have developed offers the user a powerful competitive advantage on the pigment market. 5. Conclusion
Fig. 3. (a, b) PLS-biplot for all and (a) only output (b) variables, considerable gaps in properties can be seen.
Since it cannot be definitely stated in advance if the targets determined in this way are actually the ‘‘best’’ particular candidates for pigment design from the viewpoint of multiple properties, the ‘‘best’’ candidates form a ‘‘fuzzy set’’, characterized by the grade of membership of the pigments to the ideal targets. The data on optical and other target properties of inorganic pigments are analyzed by the PLS method [2], thus providing a PLS biplot—a clever tool which extracts whatever information can be extracted into a 2D picture, Fig. 3a and b. Hence one can see the distribution of the measured samples from all brands over the space of color and other characteristics. Individual samples are repre-
The set of these targets for computerized pigment design is not definitely the absolutely best one. Because of experimental and computational errors, we cannot be absolutely certain if chosen target input values are definitely the best ones from the viewpoint of multiple evaluation criteria. Therefore, they belong to a ‘‘fuzzy set’’ characterized by a grade of membership of target pigments in the ‘‘best target set’’. The proposed method consists in finding a gap on the tough competitive pigment market as a space of potentially new pigments of unexplored properties. In short, rational pigment design represents a powerful competitive advantage, saving user time and money replacing empirical trials and errors used by competitors. References [1] J. Halova, P. Kyselka, J. Dubsky, J. Org. Reactivity 1 (102) (1995) 107–108. [2] H. Wold, Estimation of principal components and related models by iterative least squares, in: P.R. Krisnah (Ed.), Multivariate Analysis, Academic Press, New York, 1966. [3] P. Wasserman, Neural Network Computing, Van Nostrand Reinhold, New York, 1989. [4] M. Trojan, P. Sˇulcova´, D. Brandova´, Utilisation of derivatograph for study of synthesis of new inorganic pigments, J. Therm. Anal. Calorim. 53 (1998) 2. [5] R.S. Hunter, R.W. Harold, The Measurement of Appearance, second ed., Wiley, New York, 1987. [6] K. Kupka, QCExpert—Advanced Statistical Methods, User Manual, TriloByte, /http://www.trilobyte.czS, 2006.
Journal of Physics and Chemistry of Solids 68 (2007) 744–746 www.elsevier.com/locate/jpcs
Computerized pigment design based on property hypersurfaces Jaroslava Halovaa, Petra Sulcovab, Karel Kupkac, a
Institute of Inorganic Chemistry of the Academy of Sciences of the Czech Republic, CZ 250 68 Rez-Husinec 1001, Czech Republic Department of Inorganic Technology, Faculty of Chemical Technology, University of Pardubice, Na´meˇstı´ legiı´ 565, CZ 532 10 Pardubice, Czech Republic c TriloByte Statistical Software, Ltd., Jiraskova 21, CZ 530 02 Pardubice, Czech Republic
b
Received 28 September 2006; accepted 31 January 2007
Abstract Competition is tough in the pigment market. Rational pigment design has therefore a competitive advantage, saving time and money. The aim of this work is to provide methods that can assist in designing pigments with defined properties. These methods include partial least squares regression (PLSR), neural network (NN) and generalized regression ANOVA model. Authors show how PLS bi-plot can be used to identify market gaps poorly covered by pigment manufacturers, thus giving an opportunity to develop pigments with potentially profitable properties. r 2007 Elsevier Ltd. All rights reserved. Keywords: A. Inorganic compounds; A. Oxides; B. Chemical synthesis; D. Optical properties
1. Introduction Designing physical, biological, environmental or chemical properties of chemical compounds using mathematical modelling is increasingly attractive due to growing computing power and low time and financial costs compared to lengthy and expensive experimentation [1]. Various methods of predictive statistical models have been used in chemistry, biochemistry, pharmaceutical and food industry and research, material research but also in econometry and finance. This paper shows how use of predictive modelling could save time and funds in designing properties of inorganic pigments. 2. Theoretical 2.1. Partial least squares To find relationship between two groups of variables and predict one from another was used a statistical procedure based on the partial least squares (PLS) approach, first Corresponding author. Tel.: +420 466615725; fax: +420 466615735.
E-mail addresses: [email protected] (J. Halova), [email protected] (P. Sulcova), [email protected] (K. Kupka). 0022-3697/$ - see front matter r 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.jpcs.2007.01.052
published by Herman Wold in the late 1960s [2]. This mathematical methodology has been used in econometrics, chemometrics, biometrics, sensometrics and recently, some applications in technology and qualimetrics also appeared. Assume that we have n rows of the measured process parameters, that form a matrix X(n p), and the same number of measured corresponding quality parameters in matrix Y(n q), where not necessarily n4p, q. In order to extract maximum information (variability) into lowerdimensional space PLS uses an analogy to well known orthogonal principal component approach (PCA) and transforms X and Y into scores and loadings X ¼ TP0 þ E, Y ¼ TQ0 þ F. To ensure maximum relevance of chosen X-components to Y-values, the two transformations are tied together by common scores matrix T. Dimensionality (number of columns) of T is typically smaller than that of X and Y and columns in T are orthogonal. This improves stability of the model and maximizes information gain. Noise and irrelevant useless information concentrates in ‘‘garbage’’ matrices E and F. Writing U ¼ TB (where B is a square diagonal matrix) gives us the tool for prediction
ARTICLE IN PRESS J. Halova et al. / Journal of Physics and Chemistry of Solids 68 (2007) 744–746
745
of Y from X ^ ¼ TQ0 or Y ¼ XB Y with T computed from the new X-data. Unlike classical least squares or principal component regression (PCR), in PLS regression, X and Y are equivalent, or interchangable. This property predetermines PLS for use in biotechnology to predict properties or activity of a product from chemical composition or structure and vice versa. Multivariate statistical modelling is thus a very fertile platform for newly emerging interdisciplinary sciences like chemometrics, qualimetrics, biometrics. 2.2. Neural network Neural network (NN) [3] is used to relate multivariate input variable and multivariate output variable. NN can be thought of as a nonparametric modelling tool with network structure. In network knots, or neurons, each normalised input variable xi is weighted by wji and transformed by activation function sj+1,i (z) ¼ 1/(1+ez), where z is a linear combination of input variables, zi ¼ a0+Saijzi1,j. Weights wji represent links between variables and neurons, or in multilayer network, between neurons in consecutive layers and are sometimes referred to as synapses as an analogy to biological neurons Fig. 1. Output variables are neurons that transform values from the last hidden layer so that resulting sn+1,i (zi) is as close as possible to the measured experimental values or output variables y. This is done by optimization of all weights and coefficients aij in every neuron. Genetic algorithms are usualy employed to find optimal parameters of the network. This process is called training of the network. The trained network can than be used to predict unknown output variables from given values of input variables. Generally, neural networks are equivalent to a non-linear local nonparametric regression model. As such, NN cannot be generally used to predict beyond the range of the input variables used for its training (Fig. 2).
Fig. 2. Resulting NN model for predicting colour properties from input variables, synapses width suggest thar Ln and M are more important than x and T, as expected.
3. Experimental Color properties were studied on synthetised lanthanide oxide-type pigments doped with 5B group metals Cr or V [4]. The general formula of the pigment is Ln2 Zr(2x) Mx O7, where Ln ¼ lanthanoids, M ¼ Cr or V, x ¼ 0.05–0.2. The synthesis was carried out at two temperatures T ¼ 1400 or 1500 1C, respectively. Lanthanoids used in place of Ln were: La, Ce, Pr, Nd, Sm, Eu, Gd, Tb, Dy, Ho, Er, Y. The aim was to model resulting colour coordinates a*, b*, L*, dE* (property hypersurface) [5] using composition and conditions during the synthesis. The input, or independent variables thus were: type of lanthanoid Ln, type of the doping metal M, ammount of the doping metal x, and temperature T. Atomic volumes were used to quantify used lanthanoides. Output variables are a*, b*, L*, dE*. Neural network with one and two hidden layers and a PLS-2 regression were used to model and predict output properties of the pigment for given composition and temperature. Both statistical methods were also used the reverse way to propose composition and temperature to synthesise pigment with required properties. All computations were carried out on the statistical system QC Expert [6]. 4. Results
Fig. 1. The structure of a single-layer neural network.
In our study, multilayer NN were better for predicting the target properties than single-layer NN, or PLS regression. The resulting network is shown at Fig. 1, the width of the synapses correspond to the values wji, the wider the connecting synapsis the more information flow between the respective knots. PLS-biplot was found to be useful in identifying gaps in the property hypersurface, Fig. 3a and b. This feature may be exploited to efficiently design new products defined by their target parameters.
ARTICLE IN PRESS 746
J. Halova et al. / Journal of Physics and Chemistry of Solids 68 (2007) 744–746
sented by dots; pigments of the same brands form clusters. The pigments are distributed over a colour space rather unevenly with substantial gaps. These gaps may indicate unutilized market opportunities. It may be to the advantage of the producer to fill these gaps with appropriate pigments designed utilizing PLS statistical modelling, as mentioned above. The presented method of computerized pigment design based on the mapping of property hypersurfaces is a powerful tool for identifying targets for the synthesis of new pigments. The method we have developed offers the user a powerful competitive advantage on the pigment market. 5. Conclusion
Fig. 3. (a, b) PLS-biplot for all and (a) only output (b) variables, considerable gaps in properties can be seen.
Since it cannot be definitely stated in advance if the targets determined in this way are actually the ‘‘best’’ particular candidates for pigment design from the viewpoint of multiple properties, the ‘‘best’’ candidates form a ‘‘fuzzy set’’, characterized by the grade of membership of the pigments to the ideal targets. The data on optical and other target properties of inorganic pigments are analyzed by the PLS method [2], thus providing a PLS biplot—a clever tool which extracts whatever information can be extracted into a 2D picture, Fig. 3a and b. Hence one can see the distribution of the measured samples from all brands over the space of color and other characteristics. Individual samples are repre-
The set of these targets for computerized pigment design is not definitely the absolutely best one. Because of experimental and computational errors, we cannot be absolutely certain if chosen target input values are definitely the best ones from the viewpoint of multiple evaluation criteria. Therefore, they belong to a ‘‘fuzzy set’’ characterized by a grade of membership of target pigments in the ‘‘best target set’’. The proposed method consists in finding a gap on the tough competitive pigment market as a space of potentially new pigments of unexplored properties. In short, rational pigment design represents a powerful competitive advantage, saving user time and money replacing empirical trials and errors used by competitors. References [1] J. Halova, P. Kyselka, J. Dubsky, J. Org. Reactivity 1 (102) (1995) 107–108. [2] H. Wold, Estimation of principal components and related models by iterative least squares, in: P.R. Krisnah (Ed.), Multivariate Analysis, Academic Press, New York, 1966. [3] P. Wasserman, Neural Network Computing, Van Nostrand Reinhold, New York, 1989. [4] M. Trojan, P. Sˇulcova´, D. Brandova´, Utilisation of derivatograph for study of synthesis of new inorganic pigments, J. Therm. Anal. Calorim. 53 (1998) 2. [5] R.S. Hunter, R.W. Harold, The Measurement of Appearance, second ed., Wiley, New York, 1987. [6] K. Kupka, QCExpert—Advanced Statistical Methods, User Manual, TriloByte, /http://www.trilobyte.czS, 2006.