Inter-relating data sets for product development

Food Quality and Preference 11 (2000) 105±119

www.elsevier.com/locate/foodqual

Inter-relating data sets for product development: the reverse engineering approach $

Howard Moskowitz* Moskowitz Jacobs Inc., 1025 Westchester Avenue, White Plains, New York 10604, USA Received 7 September 1998; accepted 12 July 1999

Abstract Product researchers often use a variety of data sets for the same set of products. They include formulations, consumer ratings, expert panel ratings, and instrumental measures. Although there are attempts to inter-relate these measures, for the most part, it is rare to include all sets of data in one general model, and then estimate the pro®le of one subset of variables using the pro®le of another subset of variables. This paper shows how the researcher can integrate formulations, consumer data, expert panel data, and instrumental measures using a set of equations created from the same set of independent variables. These variables may be systematically arrayed by experimental design, or created from principal components factor analysis. In order to inter-relate two pro®les, the researcher sets one pro®le as the target or goal and then systematically explores the combinations of ingredients (or factor scores). The researcher looks for that combination that generates an expected pro®le lying `close' to the goal pro®le. # 1999 Elsevier Science Ltd. All rights reserved.

1. Introduction The task of relating multiple data sets in product and sensory analysis is important, as shown by the large list of papers on the topic. The task of inter-relating the data sets goes back six decades, to work on rheology and ¯avor. The topic maintains its fascination and importance today, perhaps even more than in previous years (Cunningham, Acree, Barnard, Butts & Barell, 1986; Moskowitz, Kapsalis, Cardello, Fishken, Maller & Segars, 1979; Naes & Kowalski, 1989; Piggott, 1990; Powers, 1976; Szczesniak, Brandt & Friedman, 1963; Williams, Rogers & Collins, 1988). The motives to inter-relate data sets arises out of a variety of research and business needs. One need is simply to understand how di€erent data sets correlate with each other (Munoz, Chambers & Hummber, 1996). Another need is to estimate the reactions of a more expensive indicator (e.g. people) using data as a less expensive indicator (e.g. instruments, expert panels; Burton, 1989). A third need is to create a real-time data analysis system that can be used to control a process (e.g. programming a $ Presented at: Sensometrics IV, Copenhagen, Denmark, July, 1998. * Tel.: +1-914-421-7400; fax: +1-914-428-8364. E-mail address: [email protected] (H. Moskowitz).

set of sensors so that their measures can be used to estimate consumer acceptance; Taguchi & Wu, 1980). Recent advances in inter-relating data include partial least squares (Kermadec, Durand & Sabatier, 1997), which use a reduced set of factor scores to relate two data sets. Partial least squares has not, however, been used routinely to specify a goal pro®le from one data set (e.g. a pro®le of sensory attributes from experts) and then estimate the likely pro®le from another data set (e.g. a pro®le of instrumental measures). Moskowitz (1994b) presented a technique, called reverse engineering, which enables the researcher to inter-relate data sets, whether these data sets come from products that have been systematically varied, or from products that are unrelated to each other but have been mapped in a coordinate system. One of the interesting features of the reverse engineering approach is that it can use any subset of data, and relate that subset of data to any other subset of data obtained for the same set of stimuli. 2. Inter-relating data sets using traditional linear regression Before going into reverse engineering it is instructive to consider the traditional strategy of linear regression

0950-3293/99/$ - see front matter # 1999 Elsevier Science Ltd. All rights reserved. PII: S0950-3293(99)00054-3

106

H. Moskowitz / Food Quality and Preference 11 (2000) 105±119

(Schutz, 1983, 1987). The strategy builds a regression equation relating one variable (e.g. consumer liking) to a set of other variables (e.g. ingredients, expert panel sensory ratings). The independent variables are selected with caution, so that they are reasonably uncorrelated with each other. The independent variables can be selected on the basis of a factor analysis (viz. the one actual attribute that correlates most highly with each factor), or they can be the factor scores themselves (which are, by de®nition, uncorrelated). At ®rst glance, this is an attractive approach because it promises a simple way to relate two sets of variables. Regression analysis is straightforward to apply and is widely available. The easiest method to use is stepwise multiple regression. There are three aspects/problems with this strategy, not only when factor scores are used as predictors, but also when statistically designed variables are used. 2.1. Too few predictors to show the richness within the data When the researcher uses sensory attributes as predictors these often correlate with each other. A principal components analysis of the sensory data often accounts for a substantial amount of the variability (80±85%). Thus, so much of the variation is accounted for in the sensory attributes that the regression equation would have only four variables, one for each factor. Which of the attributes that load highly on a factor should the researcher use in the model? Should the researcher use the attribute that loads most highly on a factor? By selecting only one attribute per factor as the predictor, the researcher misses the richness of the sensory experience. 2.2. Selecting the linear predictors often misses relevant curvature The relation between liking and sensory attributes is curvi-linear (or at least not monotonic over a wide range; Conner & Booth, 1992; Moskowitz, 1981). The predictor variables above need to be augmented. Often one needs to force in quadratic terms. These terms would not come in by conventional regression analysis that uses increases in goodness-of-®t statistics to allow entry into the equation. 2.3. Asymmetrical analysis may lead to inconsistent results Traditional regression modeling goes in only one direction Ð viz. from experts to consumers. Given a pro®le of expert panel ratings the researcher can now estimate the pro®le of consumer pro®le ratings. What if the researcher wants to turn the problem around and predict experts from consumers? A new set of equations must be developed, this time with the independent variables

being a limited set of the consumer sensory attributes. The predictions may be inconsistent with each other. The researcher might begin with an expert panel pro®le and use that pro®le to predict the likely consumer pro®le. Using the second set of equations, the researcher could then use the likely consumer panel pro®le to predict the expert pro®le. Does the expert pro®le used as starting inputs agree with the expected expert pro®le emerging from the two stages of pro®le estimation? This is an empirical question to be answered on a case by case basis. 3. Bene®ts from using experimentally designed stimuli When the researcher systematically varies the formulations (or underlying processes) of a product, the ability to interrelate data sets becomes stronger because the researcher can now estimate the likely formulation pro®le (under direct, operational control) corresponding to the goal pro®le. This capability enables the researcher to reverse-engineer products (viz. identify the formulation corresponding to a response pro®le), create instruments which read out product characteristics in `real time production' instantaneously estimating the likelihood of consumer acceptance. Moskowitz (1994a) described a production system which, using this approach, yields substantially reduced numbers of product defects. 4. What if the variables are not arrayed in an experimental design? In many studies that inter-relate databases, the stimuli are not experimentally designed. Rather, the researcher created a set of unrelated products, measured them with instruments, with experts, and with consumers. Often there is a plethora of measures available (especially instrumental measures), so that the researcher must use methods such as stepwise linear regression to identify the best instrumental predictors of product quality. Such an approach makes sense because once the researcher identi®es the relevant predictors it becomes possible to estimate the likely quality rating (from judges) simply by measuring these relevant predictors and substituting them into the equation. The extension of the approach to sensory attributes (instead of instrumental measures) as predictors is natural. Once the researcher creates the model relating sensory attributes to quality/acceptance, then all the researcher needs to do to predict responses to a new product is to obtain these sensory measures, and then estimate liking/quality. Furthermore, for optimization the researcher simply needs to identify the combination of sensory attributes leading to the best product. The same statistical approach can be used with the factor scores as independent variables, but the following problems ensue:

H. Moskowitz / Food Quality and Preference 11 (2000) 105±119

(a) If the researcher wants to identify the likely quality rating for a new product, then what should the levels of the independent variable be in the equation relating factor scores to quality? (b) If the researcher wants to optimize quality, how should this be done? (c) If one can ®nd the combination of factor scores that optimize liking, what does this mean in terms of the attributes? A new issue arises in the use of factor scores, because these factor scores do not necessarily generate a sensory pro®le whose values all lie within the convex hull of levels achieved in the actual study. What happens when the expected optimal level of one or several attributes (corresponding to the optimal factor scores) lies outside the convex hull? Can any set of orthogonal variables be used as predictors (ingredients, factor scores related to the product, factor scores unrelated to the product)? Moskowitz (1994a) reported that the independent factor scores must come from attributes in the data set, rather than comprise an equal number of factor scores obtained by principal components analysis of random numbers. 5. The fundamental approach of reverse engineering The objective of reverse engineering is to identify the levels of one set of variables, given the levels of another set of variables. One speci®c instance is the identi®cation of ingredients given responses. That is, given a pro®le of rating attributes (e.g. from a consumer), what is the likely formulation corresponding to that pro®le of rating attributes? Another speci®c instance is the estimation of a pro®le of attribute ratings assigned by a consumer, if one is provided with a pro®le of attribute ratings assigned by an expert. These examples of reverse engineering are relevant to the development of food products, and to the creation of systems for quality control (e.g. in the decision whether to accept/reject a batch of a product, the use of expert panels to de®ne what a consumer might perceive). Finally, it should be noted that reverse engineering does not ask the panelist directly for the relative importance of attributes as drivers of liking, but simply attempts to estimate one pro®le from another. That is, it is up to the researcher to identify the pro®le of attributes that will be matched in reverse engineering. Reverse engineering works with two sets of attributes. The ®rst set of attributes comprise those that contribute to the basis set (viz. either ingredients that are by their nature systematically varied and thus independent, or a derived set of variables developed from factor analysis to generate an orthogonal array). The second set of attributes comprises those not used to create the basis

107

set. The basis set becomes the set of independent variables. Each attribute in the study (each consumer sensory attribute, each consumer-liking attribute, each expert panel attribute, each physical measure, and each ingredient) generates its own equation. The variables in the basis set appear in a variety of terms in the equations (e.g. linear, square, cross terms). The speci®c form of the equation developed with the basis set is left to the researcher, and may be based on purely statistical considerations (viz. what variables best account for the data), purely theoretical considerations (viz. the assumed true form of the function), or some combination thereof. One approach for reverse engineering appears in Table 1. The approach may be modi®ed in terms of the nature of the equations used, the type of loss function considered for minimization and the like. In the end, however, the approach generates an estimated pro®le in one domain corresponding to a goal pro®le in another domain. 6. Relation of reverse engineering to other methods Regression against independent variables (ingredients, factor scores) to create models has been accepted in sensometrics. Experimental designs are widely used for product optimization, since the developer can systematically vary the independent variables, create the models, optimize, and then identify the speci®c, operational levels to use in order to attain speci®c objectives. When analyses of unrelated stimuli are done, there is no experimental design. Occasionally the researcher might choose a limited set of attributes as independent variables. Often, however, these attributes are intercorrelated. No easily identi®ed subset of attributes can be found that are at once uncorrelated with each other, and `span the space' (viz. account for the di€erent and often large array of attributes]. Hence, factor scores are often used instead of the actual attributes in order to remove problems due to intercorrelations among the sensory attributes. [It makes more sense to use a set of uncorrelated variables (and especially a parsimonious set) than to use a set of correlated variables (often far less parsimonious)]. Furthermore, one can rotate the factors (e.g. by quartimax) in order to create a simple set of orthogonal variables in which the original variables load on relatively few factors, making the interpretation of the factors easy.] What is new about the reverse engineering approach is that it adjusts the levels of attributes in the basis set so that the pro®le predicted by the basis set is as close as possible to a desired or goal pro®le. Thus, regression modeling is used simply as a convenient way to provide an estimation of a set of dependent variables. The algorithm itself can be simply a brute force exploration of many di€erent combinations of factor scores in the

108

H. Moskowitz / Food Quality and Preference 11 (2000) 105±119

Table 1 Algorithm for reverse engineering Step

Activity

1

Array the data in a rectangular array, with rows corresponding to products and columns corresponding to formulations, consumer sensory ratings, consumer liking and image ratings, expert panel ratings, and instrumental ratings. The greater the number of attributes measured (whether sensory, liking, expert, instrument, respectively), the greater the number of columns in the rectangular array.

2

Create a basis set, corresponding to ingredients, or if the products are not connected by design, then corresponding to factor scores obtained by principal components analysis (see Step 3a).

3a

When using sensory attributes, perform a principal components analysis on the basis set of attributes identi®ed in Step 2, save the factor scores (after rotation to a simple solution, e.g. through quartimax rotation) (Systat, 1994). Quartimax is useful because it provides a simple solution, with each variable loading on just one or two factors.

3b

Adjoin the factor scores to the data matrix created in Step 1.

4

Create a quadratic model for each attribute in Step 1, relating that attribute to the linear, square and relevant cross terms of the independent variables (basis set). [Keep in mind that the basis set can either be independently varied ingredients or factors that are independent of each other.]

5

Set a target pro®le for a subset of attributes. This is the `goal' pro®le that is to be matched.

6

Identify the levels of the basis set (combination of ingredients or factor scores) that yield an estimated pro®le close to the `goal', using the equations created in Step 4. Make sure that the sensory ratings lie within the range tested (the convex hull), especially if the independent variables are factor scores.

7

Once the levels of the basis set are identi®ed (Step 6), solve the full set of equations (Step 4) to estimate the full set of attributes that correspond to that basis set, and thus to that goal pro®le.

basis set. The exploration searches for that combination of variables in the basis set that minimizes the absolute di€erence between the goal pro®le chosen to be the target and the expected goal pro®le. [Other loss functions can be used. Furthermore, each attribute in the goal pro®le can be weighted independently, so that the algorithm can minimize the weighted absolute di€erence. In situations where there are di€erent units of measurement, the algorithm can be modi®ed to minimize the absolute percent deviation rather than the absolute deviation itself.] 7. Case history objectives The study reported here presents a case history showing how experimental design tied with di€erent data sources (consumer, expert, instrument) generated a powerful product model for a sauce, and how the data sources inter-relate by a set of equations. The objective is to estimate pro®les in one domain (e.g. consumer sensory ratings) given pro®les in another domain (e.g. expert sensory ratings, or instrumental measures). This paper also shows how reverse engineering approaches may overcome limitations of approaches that build the relation between a limited set of independent variables and a full set of dependent variables. These limitations arise because: (a) Conventional approaches typically go `forward' by estimating the likely response pro®le corresponding to a set of independent variables. (b) When conventional regression goes `backward' (from the dependent variable to the independent

variable), the objective is to optimize one variable at a time, rather than to minimize the di€erence between two pro®les. 8. Stimuli In this study there were many variables that could be changed. In a commercial environment, where time and resources are limited and where actionable answers have to be obtained quickly, it is critical to identify the key variables that are likely to have an impact on the product. It is infeasible to create all of the product prototypes, taste them, and then return to the `drawing board' to re-develop new prototypes. Through tasting selected prototypes, and through evaluating prototypes that vary along one ingredient at a time, the commercial product developer can get a good idea of what ranges look most feasible for the ingredients. Although this method of identifying formulation ranges does not seem to be particularly scienti®c or rigorous, usually the product developer can get a fairly good idea of the levels of ingredients, beyond which the product would be unacceptable. [Of course, what is unknown yet is the e€ect of high levels of all key variables, which high levels were not evaluated together by the developer in the early stage of the study design.] It should be emphasized here that the participants in such a `range setting' typically comprise the product developer, the sensory analyst, the marketing director, and the outside research ®rm (if there is one commissioned to do the research). At this early stage it is rare to bring in consumers. The stimuli comprised pasta sauces, created by experimental variation of six di€erent formula variables.

H. Moskowitz / Food Quality and Preference 11 (2000) 105±119

All variables were deemed to be critical from earlier research projects and focus groups in which many different formula variables were explored in a systematic way via Plackett±Burman screening designs (Plackett & Burman, 1946). For this project the six formula variables were ®rst assayed subjectively through informal tasting to identify the upper and lower limits. Then a 1/2 replicate central composite design was constructed (Box, Hunter & Hunter, 1978). This design required 45 prototypes, rather than 77 (for a full replicate design). [It is worth noting here that the informal tasting was exactly that. The product development group identi®ed the physical variables, and then created a set of products to show the upper and lower levels of each variable, as well as creating some levels in between. The upper and lower levels were determined by R&D, and based upon their experience. The tastings to decide upon the ®nal levels took place during working hours (8±4:30) at the product development laboratory. No ®xed protocol was ®nished. Rather, marketing, marketing research, sensory analysis and the outside supplier participated in a set of these tastings, lasting 3 days. Based upon the verbal comments of the group, the product developer decided upon the ®nal range of each ingredient.] Since the project was originally run for commercial purposes it was important to cover the ingredient space (many prototypes), while at the same time maintaining cost of the project within bounds. Unlike many smaller scale tests, this study was run in multiple markets, with many panelists rating each product. Commercial consumer research is expensive, and so there is always the tension between economy in the study and the desire to design and execute a perfect study (many ingredients, many prototypes, and many panelists). A 1/2 replicate design creates the necessary set of combinations to span the ingredient space while not draining corporate resources. The six variables were acid, sugar, spice, pieces, paste, and process (treatment). The levels of each variable were coded 1±3 for con®dentiality. 9. Panelists Two groups of panelists participated in the study. The ®rst group comprised 200 consumers, in three dispersed US markets (New York, Chicago, Los Angeles). The panelists were to be female, head of household, between the ages of 21 and 55. The screening requirements dictated that the consumer had to purchase and consume red pasta sauce at least three times per month, and had to have purchased a red pasta sauce within the past 2 weeks. These screening requirements generated a panel of 200 `heavy users' of pasta sauce, the target population for the project. The consumers were pre-recruited a week

109

ahead of the study and invited to participate for a 2-day, 3 h/day, test session in which they would evaluate these 18 pasta sauces (nine pasta sauces per day, over 3 h). [Part of the original study was to discover the existence of sensory segments and to see how di€erent these segments were from already de®ned subgroups in the populations. These sensory preference segments were developed for this data, but are not discussed here. The preference patterns of these segments were quite di€erent from each other. The sensory ratings of the segments were similar, albeit not identical.] The expert panel comprised 11 in-house employees who were specially trained in descriptive analysis as part of the corporation's on-going sensory analysis program. These panelists were trained to report the di€erent sensory notes of the product. The attribute list was generated by these panelists prior to the study, and without knowledge that a separate consumer evaluation of the products was to be done. Each panelist had at least 6 months experience in descriptive analysis through participation in other studies. The prototypes were also measured instrumentally by a variety of objective measures commonly used for pasta sauces. These measures assessed the appearance (viz. Hunter colorimetry; Hutchings, 1994), chemical (e.g. proximate physical analysis for sugar level, acid level, etc.) and rheological (e.g. Bostwick viscosity) characteristics of the product. 10. Panelist activities The consumer panel was recruited to participate for a two-day session. During that session the panelists were oriented in evaluation, and then tested 18 of the 45 products, nine on each of 2 days. The products were blocked in di€erent ways because it was impossible to totally randomize the order of products at all times for physical reasons (viz. there cannot be 45 burners going simultaneously). Each day the panelists participated for 3 h, testing three products per hour. All products were served freshly made (no older than 5 min). In these consumer sessions the panelists participated in groups of 20±25 individuals, following the protocol presented by Moskowitz (1985). This test design yielded approximately 80 ratings per product, sucient for analysis of data from the total panel and from key subgroups. [The data from key subgroups, such as sensory segments and users of di€erent in-market products were tabulated, but are not germane to this paper.] This ®eld method has been previously described (Moskowitz, 1985), and has been in use for over 25 years (e.g. Moskowitz, Wolfe & Beck, 1978). The approach represents a modi®cation of earlier ®eld methods used in psychophysics (Moskowitz & Klarman, 1975). The panelists rated both sensory intensity

110

H. Moskowitz / Food Quality and Preference 11 (2000) 105±119

and liking, as well as image. [There is no clear data to suggest that panelists cannot validly rate sensory attributes and liking attributes in the same session, and in an interdigitated order for the same products.] In these studies, whether using model systems or real foods, the panelists are highly motivated because they are paid to participate. The study is run `double blind' so neither the panelists nor the research sta€ know what products correspond to speci®c formulations, yet the panelist's sensory ratings usually `track' the physical stimulus levels when the stimuli are systematically varied by experimental design. [It is worth noting here that the evaluations were conducted at the times when pasta sauce might be consumed Ð viz. afternoon and evening. There is continuing debate among practitioners in the industry as to the best time to evaluate a product. Although there may not be a best time, per se, the prudent researcher should try to schedule consumer product evaluations at appropriate times, not at inappropriate times. Thus, no morning sessions were run. It is also worth noting that these product evaluation sessions are not run in the context of a meal, but rather run as evaluations of the product with the proper carrier. In most commercial product evaluations done today (1999), the study is carried out in a test facility, and the context is one of a rigidly choreographed test rather than in the context of a meal.] The expert panel evaluated the same 45 prototypes over a 2-week period, with each panelist rating every product twice in a blocked order (total of 90 samples). The blocking for the experts was done for the same reason of practicality. The composition of the blocks was changed for the two replicates. 11. Activities during the evaluation The products were tested during the afternoon and evening. The afternoon sessions began at 1:00 pm, and the evening sessions began at 6:00 pm. Although there is often an issue of time of data, body state (hunger versus satiated), and context in product testing, these were not factored into the study. The study was run in the same way as many other studies were run Ð with the product tested at times that were perceived as `not inappropriate'. The actual context of the session was a test session. Panelists were simply told that they would be evaluating a variety of pasta sauces, and that they should rate these sauces in the same way for each sauce. [The panelists were not told to imagine that this was a dinner or a lunch.] Finally, the panelists were told not to eat for 1 hour prior to the test. The questionnaire inter-mixed sensory and liking attributes. The attributes were rated in the order that they appeared, with one exception. Panelists rated overall liking after the ®rst taste of the product. Exit

interviews from this study (and from other studies), conducted on behalf of the project sponsor, revealed that panelists felt that they had no problem rating one product on many attributes, and did not mention that they felt confused between sensory attributes and liking attributes. [All attributes were presented in anchored form, with both ends anchored, which may have reduced ambiguity about the meaning of the attribute.] 12. Results 12.1. Analytic strategy The analytic strategy follows these steps. (a) Experimental design 1: Create models relating formula variables to ratings using regression analysis. The regression analysis uses linear, squares and cross terms. [The consumer liking ratings will be treated on a `whole panel' basis, for the purposes of this paper. It should be noted, however, that the results revealed the existence of three sensory preference segments, with panelists in a particular preference segment showing a distinctive pro®le of attribute levels that they found most acceptable. Since this paper focuses on analytic method, rather than on substantive results for the particular sauce study, that sensory preference segmentation will not be considered.] (b) Experimental design 2: Identify target or goal pro®les for one set of data (e.g. consumers), and estimate the likely pro®le of ratings for the other sets of data (viz. experts and consumers). (c) Factor scores: Follow steps a and b, this time using factor scores as the independent variables. These factor scores can be created from the consumer sensory ratings, expert sensory ratings, or objective measures. The objective of step c is to determine whether the inter-relation of data sets can be accomplished using products without an underlying experimental design. Instead of using the ingredients as the independent variables, the researcher uses factor scores, which are orthogonal to each other, parsimonious, but not arrayed by experimental design. 13. Information content of attributes The ®rst analysis, even before model building, determines the `amount of information' in the attribute. The amount of information measures the degree to which panelists di€erentiate products on a speci®c

H. Moskowitz / Food Quality and Preference 11 (2000) 105±119

attribute. If panelists rate all products similarly on the attribute then the attribute conveys very little information. By itself, the fact that there is little information is unimportant. On the other hand, when the researcher tries to relate two data sets, on an a priori basis there will be less success with attributes conveying limited information than with attributes conveying more information. In the worst situation the attribute may contain no information, meaning that all the products achieve the same mean on the attribute. It will be impossible to inter-relate two data sets if the attributes in one data set convey information but the attributes in another set convey no information. There are at least two measures of information in the means Ð the range and the standard deviation. The greater the range or the standard deviation the more the products di€er from each other. [Other measures could include the F ratio for products, which compares the variance for the product to the error variance. The F ratio may be problematic, however, if the products are rated by di€erent panelists, so that the design is not totally `within subjects'.] Table 2 presents the range (maximum±minimum) and the standard deviations for the various measures (ingredients, ratings, instrumental measures). The statistics for the consumer data are computed from the means for the products, based upon the ratings from the panelists who evaluated the products. No e€ort was made to correct for the fact that each panelist only evaluated 18 of the samples. The attributes are arrayed in di€erent sets (ingredients, liking, consumer sensory, expert sensory, instrumental measure, and consumer image). Table 2 shows that for these prototypes the attributes di€er from each other in their ability to differentiate these particular 45 products. For example, the prototypes di€er most on liking of texture, and least on liking of aroma. Similarly, the prototypes di€er most on the textural attributes, and less on the attributes of taste. Continuing this line of analysis with the expert panelists, Table 2 shows the greatest di€erentiation in texture/appearance (with standard deviations around 12±15) and least in the trigeminal attributes of burning, peppery, and heat. The implication is that when interrelating attributes, the prudent researcher will probably have greater success with the textural attributes, and less success with the trigeminal attributes. [The relation between standard deviation on an attribute, the measure of information here, and success on the task of interrelating data sets will be dealt with below.] 14. Building a product model using systematically varied ingredients as independent variables A more productive approach, generating fewer problems, builds a single set of equations relating one set of

111

Table 2 Range and standard deviation of the attributes and ingredients. The attributes are sorted within class (e.g. expert panel ratings), in descending order according to the standard deviation (SD)

Ingrediant Acid Sugar Pieces Paste Spice Treatment COST Liking Texture Appearance Total Flavor Aroma Consumer (sensory) Thick Chunky Visible spice Pieces Spicy Size Dark Sweet Taste Brown O€ taste Grassy Tart Gritty Aroma Tomato ¯avour After taste Salty Expert (sensory) Spreadable Pieces Residue Thick Visible spice Coarse Green Hay Salt Caramel Bitter Onion Sweet Sour Astringent Burn Pepper Heat Cooked taste Instrument Hunter L Hunter a Hunter b PH Solids Bostwick Sugar Image Home made Fresh Children

Max

Min

Range

SD

4 4 4 4 4 3 189

1 1 1 1 1 1 101

3 3 3 3 3 2 88

0.9 0.9 0.9 0.9 0.9 0.9 17.7

77 76 73 73 69

13 18 27 34 38

65 58 45 38 31

12 10 9 9 7

82 69 82 59 79 50 80 73 81 75 51 44 61 48 62 72 59 50

7 15 36 8 36 6 46 28 43 43 18 10 29 15 37 36 28 23

75 54 46 51 43 44 34 45 38 31 33 34 32 33 25 36 32 26

15 12 12 12 10 9 8 8 8 7 7 7 7 6 6 6 6 5

88 74 86 90 90 89 61 39 68 34 22 42 41 74 40 28 13 26 58

25 0 16 29 30 41 16 1 28 12 1 14 12 48 21 13 2 12 49

63 74 70 61 59 48 45 38 40 22 21 28 29 26 19 15 11 14 9

15 15 14 12 12 11 10 9 6 6 5 5 5 5 4 3 3 2 2

2593 2704 1484 4221 307 120 142

2239 1901 1228 3700 172 34 71

354 803 256 521 135 86 71

79 180 62 109 24 17 16

70 75 69

24 31 32

45 44 38

9 8 7

112

H. Moskowitz / Food Quality and Preference 11 (2000) 105±119

independent variables to the full set of dependent variables. The product model comprises a set of equations, one per dependent variables, which in concert describe the relation between the independent variables (ingredients here) and the dependent variables (viz. attribute ratings by consumers or experts, or instrumental readings, respectively). The equations may be linear, quadratic, or more complicated, depending upon either one's theoretical view of the underlying relation between variables, or one's desire to describe the relation parsimoniously, even if there is no theory to suggest a speci®c equation type. Researchers working in the chemical senses and in food know that as a sensory attribute (or formulation) increases, the liking rating often describes an inverted U shaped curve or at least a non-linear function, with an optimal point, or perhaps with an asymptote (Moskowitz, 1981). This relation calls for at least a quadratic function. Furthermore, there are often interactions among independent variables, so that the quadratic function should allow for, but not necessarily force in, interaction terms. The simplest interaction terms to allow are pair-wise cross terms (e.g. ingredient Aingredient B). For these six variables, the product model was constructed by a simple strategy that the author has used previously in many studies, and that has been previously described (Moskowitz, 1994b). The independent variables comprise the six ingredients, the dependent variables comprise the ratings and instrumental measures, and the equation comprises linear and square terms, along with an additive constant. With 6 linear and 6 quadratic terms and an additive constant the number of predictors forced into the equation totals 13. There are 45 total observations (one per product), allowing 32 degrees of freedom. The predictor equation is not saturated. After the 13 terms are forced into the equation, the modeling strategy tests each additional pair-wise interaction term (15 in total from the six ingredient variable) in order to determine whether the term materially correlates with the residuals. If the term correlates 0.2 or higher with the absolute value of the unexplained residuals then the interaction term enters into the equation. The process repeats, constructing the equation until no further interaction terms can be added. This strategy mixes theory and pragmatic curve ®tting. On the theoretical side the approach creates a quadratic function, allowing any curvi-linearity in the basic ingredient-rating relation to emerge, even if the curvi-linearity is not statistically signi®cant. [In most cases the quadratic term is not signi®cant, and in ordinary curve ®tting the quadratic term would not enter the equation.] As noted above, quite often with food products, as a formula variable or sensory attribute increases, liking ®rst goes up, then peaks at an optimum, and then drops down with further sensory increases. As such the model building allows for a curvilinear relation (viz. quadratic) between the independent and dependent

variables. On the pragmatic side the approach allows new interaction terms to enter the equation if they provide additional predictability beyond the linear and the square terms. That is, the interactions among ingredients or sensory attributes may not be understood from a scienti®c viewpoint, but incorporating them can add more descriptive power to the equation. There is no theoretical requirement that interaction terms enter the model, but pragmatically the interaction terms may increase the proportion of variability accounted for by the equation. Fig. 1 presents the distribution of goodness of ®t statistics for the models. Fig. 1 shows that for the most part the models ®t quite well, with the preponderance of the multiple R2 lying above 0.70. [Note that this is the unadjusted value for R2. The adjusted values would, of course, be lower, but the pattern would be similar. The unadjusted R2 value is shown here to provide an idea of the percent variability accounted for by the equation that describes the relation among the variables]. It is worthwhile noting here that the choice of terms to use in the model is a thorny one, with many, equally appropriate points of view. At one extreme the researcher can use all of the variables at one's disposal, including all terms that increase the predictability (at least signi®cantly). At the other extreme the researcher can use a very limited number of linear factors, along with their squares and cross terms. There are other objectives that one might set for oneself in curve ®tting Ð speci®cally maximizing some goodness of ®t statistic versus maximizing

Fig. 1. Distribution of goodness of ®t (multiple R2, not adjusted for degrees of freedom) for equations relating ingredients to attribute ratings and instrumental readings. The goodness of ®t is calculated for the full set of 45 prototypes.

H. Moskowitz / Food Quality and Preference 11 (2000) 105±119

predictive performance. There is no right choice here. . . just a series of alternative strategies for ®tting equations to data. 15. Validating the reverse engineering models using ingredients as the independent variables One of the key issues in creating a model is to validate the model. When it comes to interrelating di€erent sets of variables, the issue of validation becomes even more important because the model will be used to make decisions with signi®cant economic import. [Some of these decisions include accepting or rejecting a prototype based upon expert panel pro®ling, and the translation of that pro®le to consumer responses; accepting or rejecting a batch based upon instrumental measures and the translation of that instrumental pro®le to consumer acceptance.] This issue has been raised by Dugle (1997) and is addressed here. These data provide an opportunity to further explore the validity of the modeling and reverse engineering approach. The validation comprises these ®ve steps: Step 1 Ð Select seven random products from the set of 45 prototypes, and eliminate them from the data set. Step 2 Ð For the remaining 38 products create the model inter-relating systematically formula variables and attributes (sensory, liking, image, instrumental, expert). Use linear terms and quadratic terms for each variable, and allow cross-terms to enter the equation if the cross terms correlate 0.2 or higher in absolute value with the residual. Keep in mind that the independent variables will no longer be strictly orthogonal to each other, nor neatly laid out in an experimental design since seven of the 45 products have been excluded. Step 3 Ð For each of the seven products set aside in Step 1, set the consumer sensory pro®le as the goal, and estimate the likely ingredient pro®le corresponding to that goal. Step 4 Ð Given the ingredient pro®le, estimate the full attribute pro®le for the product. Step 5 Ð Correlate the actual and the expected values for the attributes, on a product by product basis, and on an attribute category basis.

113

As Table 3 shows, the correlations between the predicted and the actual values are very high for most of the products, and most of the attribute categories. In order to better understand the degree to which the reverse engineering approach holds, one can look in more detail at the relation between the predicted and the obtained ratings for the seven hold-out samples. Merging all of the attributes of a like type together (e.g. all consumer sensory attributes) could distort the degree of validity, since on an attribute by attribute basis the correlation between estimated and actual may be low. Yet, if the di€erent attributes of a like type fall in di€erent regions of the scale, this could lead to a spuriously high correlation. The more rigorous analysis computes the correlation between estimated and actual for each attribute. Fig. 2a shows these correlations. The majority of correlations exceed 0.5. The only correlations that lie below 0 are half of the expert panel ratings. The instrumental measures, the expert ratings, and the consumer image/liking ratings are all well predicted (even though these were not used to create the basis set). Fig. 2a by itself may be insucient because the correlation can be a€ected by one or two outliers. Fig. 2b shows the distribution of leverage values for each attribute, and suggests that there are relatively few outlier points that signi®cantly a€ect the correlation value. It is worth noting that the leverage statistic was computed for each product and each attribute type separately. 16. Building a product model with factor scores as independent variables Quite often the stimuli in studies comprise products that cannot be related to each other through an underlying set of systematically varied ingredients. In such cases there is no experimental design whose independent variables can be used to create the model. One way to circumvent the lack of an experimental design uses principal components analysis on a set of attributes, in order to create the necessary set of orthogonal variables. Once the factors are created, they form the basis set. These factor scores then take the place of the independent variables. Every product in the set generates its own factor score pro®le. To develop equations, one follows the

Table 3 Pearson correlations between actual and estimated ratings, for seven holdout products (P107. . .P131). The consumer sensory ratings were used as the goals in reverse engineering Category

Cases

P107

P111

P120

P123

P112

P127

P131

Ingredient Image Liking Consumer sensory Expert sensory Instrument

6 3 5 19 19 8

0.50 0.45 0.95 0.98 0.98 0.99

0.05 0.96 0.95 0.97 0.90 1.00

0.59 0.99 0.92 0.98 0.96 1.00

0.64 0.96 0.33 0.98 0.98 0.99

0.36 0.98 0.93 0.91 0.97 0.99

0.39 0.74 0.85 0.98 0.99 1.00

0.33 0.54 0.93 0.99 0.99 1.00

114

H. Moskowitz / Food Quality and Preference 11 (2000) 105±119

Fig. 2. (a) Distribution of correlations between estimated and actual values, based upon reverse engineering. Each symbol corresponds to a speci®c attribute. Each correlation was computed from seven holdout products. (b) Leverage of speci®c data points. Each leverage statistic was calculated from a speci®c attribute type and a speci®c product.

same step as before, with the factor scores substituting for ingredients, and with the same conventions about the terms that appear in the equation (viz. force in linear and square terms, and allow additional cross terms to enter). 17. Validating the model created with factor scores as the basis set The reverse engineering approach using factor scores can be validated by the same method used to validate the approach based upon ingredients. The validation follows the same general steps used to validate the reverse engineering model, except that a principal components analysis transforms the consumer sensory attributes into the basis set.

(a) Identify the sensory attributes to be used to create the basis set. (b) Select the seven hold-out samples (same ones selected for validating the model based upon ingredients), and use the remaining 38 products in the factor analysis. (c) Perform a principal components analysis on the consumer sensory attributes to generate the factors (conventions: minimum eigenvalue=1.2, rotation by quartimax). [Other rotations can be used, but the quartimax rotation provides a simple factor structure, easy to understand.] This yields four factors, shown in Table 4. [The principal components factor analysis could have been performed on the expert panel data, on the combination of the expert panel and consumer panel data, and on that combination along with the instrumental measures.] (d) Compute the factor scores corresponding to the 38 products. (e) Create models relating the factor scores, their squares and additional signi®cant cross terms to each attribute. [Again, the terms in the equation are introduced as a convention. They can be changed, if the assumptions are changed regarding what constitutes the appropriate predictor terms in the equation.] Create regression models for all attributes, viz. instrumental readings, and ingredients (which could be modeled as dependent variables). (f) On a product by product basis, use the consumer sensory pro®le as a goal, and estimate the full pro®le of attributes for the particular product. (g) Correlate the expected rating pro®les for the seven holdout samples versus the actual rating pro®les for the same seven products, on an attribute type basis. There are four sets of correlations: consumer sensory, expert sensory, ingredients, and instruments. Table 5 shows that the correlations between goal (actual) and expected pro®les are quite high for the consumer sensory and expert sensory pro®les, lower for liking pro®les, and lowest for ingredient pro®les. Fig. 3a shows that the leverage (or chance of being an outlier) is fairly low for the di€erent points, suggesting that the high correlations between estimated and actual are real, rather than spurious. The basis set for Fig. 3a comprises the factor scores obtained from the consumer sensory ratings (h) Repeat analyses f and g, but change the goal pro®le. Now use the expert sensory pro®le as the goal, for the seven holdout samples. Similar results emerge from this second exercise. Fig. 3b shows the same analysis of the individual points. This time, however, the basis set comprises the factor scores obtained from the expert panel data. Again the leverage of individual points is low, suggesting that the high correlations are not spurious.

H. Moskowitz / Food Quality and Preference 11 (2000) 105±119

18. Illustrative exercise Ð relating data sets to each other One example of the approach appears in Table 6, which shows the results of three exercises in reverse engineering. The goal pro®le to be matched comes from an arti®cially created product, corresponding to the average of the ®rst eight prototypes in the study. Column B shows the results using the consumer sensory pro®le as a goal, Column C shows the results using the expert sensory pro®le as a goal, and Column D shows the results using the instrumental pro®le as a goal. In all three cases the loss function was the percentage absolute deviation between the estimated and the actual values (absolute [(estimated±actual)/(actual)]). Once the goal and the loss function are speci®ed, (here minimizing percentage deviation of predicted versus obtained) the reverse engineering algorithm identi®es the ingredients corresponding to the goal pro®le, and then estimates the full range.

Table 4 Factor structure from consumer sensory data Consumer

Factor 1

Factor 2

Factor 3

Factor 4

Spicy Aroma Brown Dark Visible spice Taste intensity Gritty Grassy Thick Aftertaste Chunky

0.94 0.94 0.94 0.94 0.92 0.90 0.89 0.76 0.68 0.61 0.56

ÿ0.14 0.00 0.19 0.27 0.02 ÿ0.16 ÿ0.15 ÿ0.40 0.16 ÿ0.68 0.09

ÿ0.05 0.12 0.04 0.05 0.21 ÿ0.01 0.17 0.02 0.40 ÿ0.11 0.72

0.16 ÿ0.02 ÿ0.09 ÿ0.01 ÿ0.07 0.30 ÿ0.01 ÿ0.42 0.46 ÿ0.06 0.33

Tart Sweet O€-taste Salty

0.08 0.29 0.08 0.25

ÿ0.94 0.82 ÿ0.68 ÿ0.64

ÿ0.04 ÿ0.05 ÿ0.13 ÿ0.22

0.10 0.15 ÿ0.56 0.44

Size Pieces

0.34 0.41

0.11 0.09

0.88 0.87

0.05 0.08

Tomato ¯avour % Variation

0.28 45%

0.00 18%

0.37 14%

0.72 9%

115

As Table 2 shows, no matter which pro®le is the goal to be matched, the remaining attributes were predicted quite closely. This should come as no surprise, given the strong evidence of the validity of the model from the validation exercise with the seven holdout samples. The use of a single arti®cial product generates a pro®le that is, by necessity, closer to the middle of the sensory space than would be the case for a single product that is created at an extreme. To obtain a good idea of the performance of the reverse engineering approach one might wish to do these exercises for a wide variety of di€erent pro®les located at di€erent parts of the space. Time and energy do not permit this additional exercise. On the other hand, the validation exercise selected di€erent products in di€erent parts of the space. One might surmise that the performance of the reverse engineering approach would be as good as that achieved in the validation phase. 19. `Information' in the attribute versus success in inter-relating databases Is there a relation between the information in an attribute (as represented by the standard deviation across the 45 prototypes) and the degree to which the reverse engineering model successfully estimates the hold-out sample values? One approach computes the correlation between actual and estimated, for each attribute, across the seven holdout samples, and then compare this goodness of prediction to the standard deviation. Fig. 4 shows this plot. The plot is not conclusive, but does suggest that the reverse engineering model fails when there is little information in the rating attribute, but often (but not always) succeeds when there is a lot of information in the rating attribute 20. Discussion 20.1. Models that describe relations versus models that explain relations When inter-relating two sets of variables, the researcher can opt for at least two approaches Ð descriptive models

Table 5 Correlation between the actual and the expected pro®le for seven holdout products, by type of attribute. The models were developed using factor scores as the predictors. The factor scores were developed from the consumer sensory attributes. There were two sets of goals Ð consumer sensory and expert sensory Attribute type

Number of attributes

Total number of observations

(a) Goal Ð consumer sensory pro®le

(b) Goal Ð expert sensory pro®le

Consumer sensory Expert sensory Ingredients Liking

19 19 6 5

126 133 42 35

0.98 0.96 0.41 0.86

0.93 0.99 0.38 0.63

116

H. Moskowitz / Food Quality and Preference 11 (2000) 105±119

Fig. 3. Leverage of (a) consumer and (b) expert speci®c data points. Each leverage statistic was calculated from a speci®c attribute type and a speci®c product. The basis set was created from consumer sensory ratings.

or explanatory models. Descriptive models simply account for the relations. There is no supposition that the models do anything more than describe relations among variables. The models allow one to predict the value of a dependent variable, given a set of values for the independent variables, but do not provide the underlying reason, hypothesis, or theory for this prediction. There is no underlying mechanism that the model attempts to capture. In contrast, explanatory models are assumed to represent underlying processes. The parameters of the explanatory models are assumed to re¯ect actual processes, and can be interpreted as meaningful beyond simply data description. Reverse engineering uses descriptive modeling, so that the equations simply act as a convenient way to capture the relations among variables. The di€erence between descriptive and explanatory is important here. In the food industry (as in many other consumer goods industries) the developer must know how pro®les of attributes in one domain (e.g. instrumental measures or expert panel ratings) co-vary with pro®les of attributes in another domain (e.g. consumer ratings). The developer may or may not understand the fundamentals of the product, but the model can nonetheless guide development, or quality assurance (viz. accept or reject batches on a production line). One further point needs to be made. In sensory science researchers are very far from knowing the true relation between the physico/chemical properties of a stimulus and the likely sensory reaction. In the sense of taste, for example, it is dicult to predict the quality

pro®le of a chemical (e.g. the proportion of sweet, salty, sour, bitter in the taste sensation) from knowing the molecular con®guration. When it comes to texture perceptions we are equally far away from knowing the inter-relations. Thus, any inter-relations between the two are, at today's level of knowledge, at best descriptive with perhaps some hints of ®rst order, underlying principles. It should come as no surprise that for more complex stimuli, such as that represented by foods, our level of basic mechanisms underlying the perception of the food is even more meager. Yet, models can and do help predict likely responses to the food, given formulation levels, sensory pro®le ratings, or instruments. 20.2. Do other currently popular methods yield the same type of results? One of the key issues in understanding and adopting a new procedure is to establish the degree to which the method adds value to the research. For instance, one can ask whether the other methods for relating databases to each other would have yielded comparable information. The published approaches do appear not to provide an estimate of the values for a subset of attributes in a data set given the values provided by other attributes in a data set. [Approaches such as partial least squares may do so, but this capability has not been emphasized.] The current approaches using regression do estimate the likely pro®le of one set of attributes from a set of factor scores. However, they do not look for the set of factor scores that, in concert, and

H. Moskowitz / Food Quality and Preference 11 (2000) 105±119

117

Table 6 Comparison of the actual pro®le (A) to the pro®les estimated from the model when the goal comprises the full consumer pro®le (B), the full expert pro®le (C) or the full instrumental pro®le, respectively. The actual pro®le was synthesized for each attribute by averaging the attribute ratings for the ®rst eight products A

B

C

D

Attribute

Attribute type

Actual pro®le

Consumer attributes as goals

Expert attributes as goals

Instrumental measures as goals

Thick Dark Visible spice Taste intensity Tomato ¯ovour Chunky Brown Spicy Pieces Aroma Sweet Size Salty Tart Aftertaste Gritty O€-taste Grassy Thick Coarse Pieces Residue Visible spice Cooked taste Sour Salty Green Spreadable Onion Sweet Caramel Burn Asringent Heat Hay Pepper Bitter pH Hunter L Hunter a Hunter H Solids Sugar Bostwick

Consumer Consumer Consumer Consumer Consumer Consumer Consumer Consumer Consumer Consumer Consumer Consumer Consumer Consumer Consumer Consumer Consumer Consumer Expert Expert Expert Expert Expert Expert Expert Expert Expert Expert Expert Expert Expert Expert Expert Expert Expert Expert Expert Instrument Instrument Instrument Instrument Instrument Instrument Instrument

79 70 70 69 66 64 64 63 54 54 52 45 39 39 37 31 21 20 79 73 67 58 57 53 53 47 35 33 31 31 29 24 24 21 18 12 9 4164 2450 2450 1405 247 11 48

75 69 71 70 64 61 64 65 51 56 40 42 41 46 45 34 35 29 80 76 65 63 64 53 53 49 39 35 32 25 27 21 24 19 23 11 13 4171 2511 2339 1423 217 96 54

82 71 71 72 67 65 65 67 51 58 40 42 42 47 46 37 35 30 85 79 70 65 64 53 53 49 39 30 33 25 28 21 24 19 23 11 14 4229 2540 2363 1446 213 94 48

84 74 75 78 68 66 67 71 55 57 51 47 41 47 46 36 31 27 85 80 70 66 66 55 56 53 43 29 34 32 32 24 28 21 24 13 12 4042 2445 2332 1399 264 122 45

with interactions and quadratic terms, generate a predesignated pro®le (or come as close as possible to doing so). 20.3. Applications: product development The key current application for reverse engineering lies in product development, and speci®cally in the use of expert panels to replace the more expensive consumer

evaluations. Although nothing can replace consumers in terms of rating the acceptance of a new product, it becomes increasingly expensive to test product after product, to determine whether or not the product is acceptable. A panel of in-house employees can do routine evaluations of product sensory characteristics, submit the pro®le to the reverse engineering algorithm, and then compute the expected liking scores, and the expected sensory pro®le. Reverse engineering becomes an

118

H. Moskowitz / Food Quality and Preference 11 (2000) 105±119

Fig. 4. How information in the attribute (shown by the standard deviation) relates to performance on inter-relating data sets. Results are from the seven holdout samples. When the attribute has no information the correlation is low. When the attribute has a lot of information the correlation is often moderate to high.

analytical adjunct to the screening process that helps identify promising candidates. This process is currently being used by a number of manufacturers in just such a fashion. 20.4. Applications: quality control Quality control attempts to ensure that the products from batch product maintain their sensory limits and consumer acceptance. Currently, expert panels are trained at the plant to taste products pulled from a production batch in order to ensure that the batch is within the sensory speci®cation (see Munoz, Civille & Carr, 1992 for a discussion of sensory analysis and quality control). In many companies the practice is for speci®cations to be set for each of the attributes assigned by the expert panel, values beyond which the production batch is deemed to be out of speci®cation, and is thus rejected. What is not known, however, is the translation of the full expert panel pro®le to the consumer sensory and liking pro®le. [This translation is certainly possible, if the researcher creates the equations relating the expert data to the consumer data.] Furthermore, in many applications the quality control system is often a `legacy' system, so that even the plant personnel responsible for the test do not really know how departures from expert panel speci®cations translate into changes in consumer sensory perceptions, or consumer liking. The `legacy' aspect comes from the fact that these speci®cations were once set, but the rationale was never made clear.

The approach presented in this paper makes the translation between pro®les quite straightforward. Once the basis set is developed and the equations created, the quality control problem becomes one of ®nding the likely consumer pro®le corresponding to a speci®ed expert panel pro®le. Armed with the translation, one may discover that the expert panelists focus on the wrong sensory aspects, reject products that should be accepted (at least according to the consumer), or accept products that should be rejected. That is, even though the product lies within speci®cations, the pro®le of the product on all expert attributes may suggest that the product deviations from speci®cation are being compounded. In the aggregate these small departures from speci®cation, insigni®cant in themselves, may, in fact, generate a product that is out of speci®cation. The reverse engineering approach calibrates the expert panel results, without interfering with the expert panel training. Experts (or instruments) are free to operate within their own language and measurement system. The reverse engineering technology then acts as a translation device. 20.5. Applications: future databasing The utility of the approach comes from its ability to inter-relate data sets using any subset of variables in one set to estimate the likely pro®le of all the variables in the set. The method as presented here assumes that all of the equations used in the modeling must pertain to the same set of products, whether the equations be for consumer sensory, liking or image, expert panel sensory, or instrumental measures. This assumption may not be necessary. Since it is the set of equations that is important, it may well be possible to develop these equations on similar sets of products (but not necessarily on the same set of products). As long as the products in the study have similar attributes and similar levels it may be possible to merge di€erent sets of equations into one database. 20.5.1. A note on terminology Researchers often use terms idiosyncratically. Before launching into a discussion of the approaches presented in this paper, it will be instructive to set forth a number of de®nitions of terms used throughout the paper. 20.5.2. Descriptive attributes Attributes that describe the amount of an attribute (e.g. from none at all to an extreme amount). Descriptive attributes are not judgmental Ð they do not pertain to liking, nor do they assume an optimal level in the middle range of intensity. 20.5.3. Consumer descriptive attributes Descriptive attributes used by a consumer panel. The attributes may be generated by consumers, by experts,

H. Moskowitz / Food Quality and Preference 11 (2000) 105±119

by marketing or by R&D. The word `consumer' refers to the individual who uses the attribute. 20.5.4. Expert descriptive attributes Descriptive attributes used by a so-called expert panel. [Expertise is de®ned by the user group, and generally comprises individuals who have received training.] 20.5.5. Rating These refer to numerical assessments assigned by the panelist. All of the rating scales referred to in this paper are either descriptive scales (amount of. . .) or liking scales (dislike versus like, on a scale).

References Box, G. E. P., Hunter, J., & Hunter, S. (1989). Statistics for experimenters. New York: John Wiley. Burton, J. (1989). Towards the digital cheese grader. Dairy Institute International, 5(4), 7±21. Conner, M. T., & Booth, D. A. (1992). Combined measurement of food taste and consumer reference in the individual: Reliability, precision and stability data. Journal of Food Quality, 15, 1±17. Cunningham, D. G., Acree, T. E., Barnard, J., Butts, R. M., & Barell, P. A. (1986). Charm analysis of apple volatiles. Food Chemistry, 39, 7±148. Dugle, J. (1997). Note on ``Experts versus consumers: A comparison''. Journal of Sensory Studies, 12, 147±154. Hutchings, J. B. (1994). Food colour and appearance. London: Blackie Academic and Professional. Kermadec, F. H. D, Durand, J. F., & Sabatier, R. (1997). Comparison between linear and non-linear PLS methods to explain overall liking from characteristics. Food Quality and Preference, 8, 395±402. Moskowitz, H. R. (1981). Relative importance of perceptual factors to consumer acceptance: Linear versus quadratic analysis. Journal of Food Science, 46, 244±248. Moskowitz, H. R. (1985). New directions in product testing and sensory evaluation of foods. Westport, CT: Food and Nutrition Press. Moskowitz, H. R. (1994). Food concept and products: just in time development. Trumbull, CT: Food and Nutrition Press.

119

Moskowitz, H. R. (1994). Product testing 2: Modeling versus mapping and their integration. Journal of Sensory Studies, 9, 323±336. Moskowitz, H. R., Kapsalis, J. G., Cardello, A. V., Fishken, D., Maller, O., & Segars, R. A. (1979). Determining relationships among objective, expert, and consumer measures of texture. Food Technology, 33, 84±88. Moskowitz, H. R., & Klarman, L. (1975). The tastes of arti®cial sweeteners and their mixtures. Chemical Senses and Flavor, 1, 411±421. Moskowitz, H. R., Wolfe, K., & Beck, C. (1978). Sweetness and acceptance optimization in cola ¯avored beverages using combinations of arti®cial sweeteners Ð a psychophysical approach. Journal of Food Quality, 2, 17±26. Munoz, A. M., Chambers IV, E., & Hummber, S. (1996). A multifaceted category research study: how to understand a product category and its consumer responses. Journal of Sensory Studies, 11, 261±294. Munoz, A. M., Civille, G. V., & Carr, B. T. (1992). Sensory evaluation in quality control, Van Nostrand Reinhold. Naes, T., & Kowalski, B. (1989). Predicting sensory pro®les from external instrumental measures. Food Quality and Preference, 4/5, 135±147. Piggott, J. R. (1990). Relating sensory and chemical data to understand ¯avor. Journal of Sensory Studies, 4, 261±272. Plackett, R. L., & Burman, J. D. (1946). The design of optimum multifactorial experiments. Biometrika, 33, 305±325. Powers, J. J. (1976). Experiences with subjective/objective correlations. In J. J. Powers, & H. R. Moskowitz, Correlating Sensory And Objective Measurements Ð New Methods For Answering Old Problems (pp. 111±129). Philadelphia: American Society For Testing And Materials. Schutz, H. G. (1983). Multiple regression approach to optimization. Food Technology, 37, 47±62. Schutz, H. G. (1987). Predicting preferences from sensory and analytical data. In M. Martens, G. A. Dalen, & H. Russwurn Jr., Flavour science and technology (pp. 399±406). Chichester, UK: John Wiley and Sons. Systat (1994). Systat, the system for statistics. Evanston: Systat Corporation. Szczesniak, A. S., Brandt, J. A., & Friedman, H. H. (1963). Development of standard rating scales for mechanical parameters of texture and correlation between the objective and sensory methods of texture evaluation. Journal of Food Science, 28, 397±403. Taguchi, G., & Wu, Y. (1980). Introduction to o€-line quality control. Central Japan Quality Control Association. Williams, A. A., Rogers, C. A., & Collins, A. J. (1988). Relating chemical/physical and sensory data in food acceptance studies. Food Quality and Preference, 1, 25±31.