- Email: [email protected]
The Sensory Quality System: a global quality control solution
Food Quality and Preference 13 (2002) 385–395 www.elsevier.com/locate/foodqual
The Sensory Quality System: a global quality control solution Silvia Kinga,*, Marianne Gillettea, David Titmanb, Julie Adamsa, Maria Ridgelya a McCormick and Company, 204 Wight Avenue, Hunt Valley, Maryland, USA McCormcik (UK) PLC, Thame Road, Haddenham, Aylesbury, Buckinghamshire HP17 8LB, UK
b
Received 26 January 2001; received in revised form 4 June 2001; accepted 20 November 2001
Abstract Assuring product quality is paramount to the success of any consumer product company that competes in the global marketplace. Quality Control/Quality Assurance programs must be in place to guarantee the consistent delivery of goods to consumers. However, establishing a successful QC/QA program has numerous obstacles within a single country’s borders and crossing international lines exponentially increases the hurdles. The Sensory Quality System described in this paper has proven to be an effective program to maintain product quality in an international environment for a global food company. # 2002 Elsevier Science Ltd. All rights reserved. Keywords: Sensory; Quality control; Quality assurance; Global
1. Introduction The expanding global market has resulted in a myriad of shifts in the way companies operate. However, delivering consistent products to consumers, whether within or across country borders, remains a critical element to a company’s success. An effective Quality Control (QC)/Quality Assurance (QA) program can help assure success to an organization. Considering the resources and efforts invested by an organization to launch a new product, the need to maintain product integrity is paramount. An organization’s reputation for quality is one of its most prized assets. Delivering products that are similar over time is essential to the consumer perception of quality, and key to brand loyalty. Distributing a product that consistently meets consumers expectations can mean the difference between a company thriving in the global market, or not. Methods to initiate and issues with establishing a QC/QA program have been well-documented (ASTM, 1992; Mun˜oz, 1992). In addition to the hurdles described by these authors, the impact of crossing international borders offers a unique set of challenges. Language may be the most difficult issue facing those setting up an
international program (Cardello, 1993). Companies must also be aware of the accepted communication methods, cultural biases, ingredient availability and familiarity, and reporting formats (Carlton, 1985). Another universal challenge in setting up any Sensory QC program has been the concept of the control. Simply identifying and securing a physical ‘‘control’’ is often a major research project in itself. For many food products, a single control sample is misleading and results in the rejection of adequate commercial product. This is particularly true for agricultural products and food service menu items. An effective global QC/QA program must describe an ‘‘acceptable range’’ of deviation, clearly identify ‘‘unacceptable’’ production samples, and be simple enough for all testing locations to take into account resource sophistication, monitoring and auditing. Management must also have a plan in place as to how to respond to sensory results. The program described in this paper has met the requirements for a sound program and has addressed the international challenges. It has been successful for an international food ingredients supplier at multiple global locations.
2. Sensory quality system * Corresponding author. Tel.: +1-401-771-7390; fax: +1- 410-5276527. E-mail address: [email protected] (S. King).
The Sensory Quality System (SQS) was developed by Gillette and Beckley (Beckley & Kroll, 1996) initially to
0950-3293/02/$ - see front matter # 2002 Elsevier Science Ltd. All rights reserved. PII: S0950-3293(01)00074-X
386
S. King et al. / Food Quality and Preference 13 (2002) 385–395
satisfy the demands of industrial customers requiring quality control on their products and was later optimized to meet other business demands. The long-term intent was to provide assurance of product quality across multiple plants and multiple companies. Critical elements of the program’s design were the prevention of flavor drift over time as well as providing an inter-plant communication tool to match target product. As the markets expanded, so did the SQS system, proving to be a powerful tool at all levels of the process, from ingredient monitoring to finished product evaluation, as well as facilitating communication among production and Quality Assurance labs throughout the world.
3. Method 3.1. Panelist selection
A score of 9–10 is equal to a match. The sample has virtually identical sensory characteristics to the control by appearance, aroma, flavor and texture. Any differences that may be noted are insignificant, and would not be detected without careful side by side comparison to the control. A score of 6–7–8 is equal to acceptable. The sample meets the definition for the product, but has differences that are easily spotted when comparing side by side with the control. These differences might not be discerned it the sample were not directly compared with the control. The sample is acceptable for shipping. A score of 3–4–5 is equal to unacceptable. The sample does not meet the definition of the product, but possibly can be reworked. Differences from the control are easily noted, but no foreign flavor has contaminated the sample. The sample is not acceptable for shipping.
Initially, 10–20 potential panelists are selected based upon their availability, motivation and ability to participate in regular product evaluations. Standard screening procedures can be used to assure panelists have a normal sense of smell and taste acuity. Potential panelists are typically selected from a pool of office, laboratory or plant personnel, depending on the site. Alternately, non-employee panelists have been utilized because of their consistent availability; these panelists can be easily trained on the ballot and references. 3.2. Panelist training The training objectives are to familiarize the panel with the QC program, the product being evaluated, the product’s ingredients, the product’s key characteristics and the ballot. The training process is also used to evaluate critical ingredients and physical references to familiarize the panel with the key ingredient characteristics. The initial training session takes about six 1/2-h sessions. Introduction of subsequent products takes 2–3 sessions per product, given that the products fall under the same product category. The panelists are familiarized with a 10-point quality scale, which measures quality based on degree of difference from the control. A control should be available at each panel session. The scale is divided into four categories. Panelists are calibrated on the scale using standard references, representing the range within each category. The ranges on the scale are defined where (Fig. 1):
Fig. 1. Example of a 10-point rating scale used to measure degree of difference from the control.
Fig. 2. Graphical representation of the typical variation found in a product, where the control is not a single point, and acceptable, unacceptable and reject regions vary in shape and distance from the control.
Fig. 3. Graphical representation of the variation found in a product, including reference standards used to represent those variations.
S. King et al. / Food Quality and Preference 13 (2002) 385–395
A score of 1–2 is equal to a reject. The sample has obvious defects rendering it non-shippable as product. The product could not be considered for rework. It may contain foreign flavors/ingredients. If the panelist rates the sample a 9 or a 10, they must consider it a ‘‘match’’ to the control, and the analysis stops here. If a panelist marks the sample 8 or below, the panelist completes the next section of the ballot. This section provides diagnostic information on how the production sample differs from the control. This section contains a list of product attributes, with boxes indicating strength levels of the attributes. Panelists examine this list and determine whether any of the words given are the reason for the difference from the control. If the panelist fails to find the cause for the difference on this list, they use an area marked ‘‘other’’ to describe the perceived difference. The cornerstone of this procedure is the calibration of the panel. This includes teaching the panel to appreciate the range of acceptable, unacceptable and reject pro-
387
duct. A perfect sensory match is typically not feasible in a production environment; therefore, a definition of what is an acceptable variation is required. It is important to start by acknowledging that the target is a range, and not a point. Targets drift. Additionally, minor deviations from the target are acceptable. At some point differences from the target become borderline, then unacceptable (Fig. 2). These ranges are defined by formulating a set of laboratory samples for calibration purposes. The set of calibration samples is initially graded by the customer, using the 10-point scale for ‘‘acceptable’’ to ‘‘unacceptable.’’ Usually, 15–20 calibration samples are prepared by Product Development. They are designed to demonstrate in minor, moderate, and major variations from the target. These calibration standards represent the expected range of deviations from the control. They illustrate the effects of likely production errors, and most importantly, the acceptable range of deviation (Fig. 3). These calibration standards must be created through formula modification, as well as aging or abusing the product. All steps used to create these stan-
Fig. 4. Example of a ballot used by QA and QC to evaluate vanilla extract.
388
S. King et al. / Food Quality and Preference 13 (2002) 385–395
dards must be carefully documented to insure that these samples can be recreated at any test location to recalibrate panels or train new panels. When feasible, the calibration set is scored by the customer, Product Development, discrimination panels and/or consumer. It is ideal to have the customer do the grading so the panel can be calibrated based upon the customer’s expectations. Once the calibration set is generated and scored, 8–10 of the samples are used to calibrate subsequent QC panels. On-going recalibration also occurs when the customer or Corporate QA forwards examples of ‘‘unacceptable’’, ‘‘acceptable’’ and ‘‘match’’ products from their cuttings back to the plants.
fectly replicated. The customer and supplier can work closely in the selection of the gold standard(s) using a range of sensory approaches, from descriptive analysis to consumer testing. In some cases, the customer identifies the control or gold standard from a series of samples. It is then the responsibility of the supplier to consistently produce this product concept and maintain a constant supply of new control to replace old/aged control. When updating the control, timing is critical so new controls can be selected before the shelf life of the existing control expires. Panelists evaluate potential controls and provide feedback regarding their similarity to the current gold standard.
3.3. Selection of the control
3.4. Ballot development
As previously suggested, the selection and maintenance of the control can be a major project in itself. For many product classes a range of ‘‘controls’’ is necessary because no single control sample can be per-
The selection of ballot attributes used to evaluate the samples is handled in a joint effort between Product Development and the Sensory Group. The ballot incorporates only those critical attributes that directly
Fig. 5. Example of manual data collection including quality scores and frequencies.
S. King et al. / Food Quality and Preference 13 (2002) 385–395
influence the quality score of the product as well as attributes associated with critical ingredients and or processing steps during production. The following is an example of a ballot used for QC and QA evaluations of vanilla (Fig. 4).
389
4. Data handling Data handling is kept simple and incorporates a concensus approach. This minimal analysis aspect of the method is a key to the successful implementation of the
Fig. 6. Bar chart representing panel scores for different lots of the same product. These data can be utilized to select potential gold standards as well as monitoring specific attributes that could cause flavor drift.
Fig. 7. Bar chart representing attribute scores for a single sample. These data are can be collected over time to monitor specific attributes and potential drift.
390
S. King et al. / Food Quality and Preference 13 (2002) 385–395
SQS process. Frequencies and mean scores for each sample are collected at the end of each evaluation to determine the status of the sample (Fig. 5). Panel means can be charted over time to track product performance. The SQS program can be used to select potential new controls as part of the routine sample evaluations. A sample rating a quality score of 8.5 or higher has the potential of becoming a new control sample. Panelists’ comments and score deviation are all key components in determining whether the sample should be considered as a future gold standard (Fig. 6). Diagnostic attributes can also be tracked and charted over time to measure product drift due to an attribute, ingredient change or issues during processing (Fig. 7).
Flavor changes and possible product drift can be documented through panelists’ comments. Panelists’ comments may suggest changes in the raw materials or process, which require investigation and or re-calibration. One or more panelists’ rejection scores will flag a questionable sample, setting a verification process in motion including additional testing, if needed, to identify potential problems. This verification takes place even when the average score for the sample may suggest an acceptable product (mean 5 6.0; Table 1). When different plant panels are calibrated using the same system, inter-plant lab audits are possible, and the results are much more easily communicated.
Fig. 8. Example of an SQS ballot translated into Spanish for Latin America.
391
S. King et al. / Food Quality and Preference 13 (2002) 385–395
5. Applications This method and several adaptations are used in production facilities around the world. The SQS method has been applied to the evaluation of key ingredients as well as production of seasonings, wet sauces and flavors. This system has also been adapted for shelf life studies, food service product cuttings and commercialization/ scale-up of new technologies. 5.1.1. Europe SQS has had a considerable effect in assisting in the globalization and control of products that were originally developed for the American market and have since found success in Europe. For the globalization of snack seasonings, SQS proved particularly valuable. Differences between US and EC legislation governing maximum levels and usage of permitted colors in foods meant that it was not possible for the formulation of a blend to be transferred without some modification of a key ingredient. Import duties also meant that commercial benefits would arise from sourcing the ingredient locally. However, any alterations made to the product due to formulation change or supplier would only be permitted if the flavor and color of the finished blend remained unchanged.
The local supplier was provided with training in SQS along with a suitable quantity of the original US ingredient to be used as a target for development. The SQS method provided a quick method for calibrating the local supplier to effectively evaluate and screen any pilot samples of the modified ingredient against the standard. As a result, there was increased confidence in any samples submitted for approval and considerably reduced time spent on validation through more time-intensive discrimination testing. By using standardized ballots with a common list of attributes, communication was more effective when discussing variation from the gold standard and pathways for development. Interestingly, the attributes used in the common ballot were only fully understood after training. Some of the flavor attributes, essential to the overall profile, were defined by grocery items that were household names in the USA, but unheard of in Europe. Universal understanding of these attributes was only achieved after the dispatch of several grocery parcels from the USA. The same challenge has been encountered when defining sensory attributes among panels from other European countries. Standard physical references for ballot attributes are as essential as providing the gold standard and calibration set. Another example of how SQS has benefited the supply of raw material occurred when a major global food
Table 1 SQS results for several lots of the same product, including average scores, number of panelists rating each sample a pass (quality of 6 and higher) or fail (quality of 2 or less)
Table 3 Key extract attributes for SQS panel training Attribute
Reference
Date
Average score
No. pass
5/4/00 5/8/00 5/8/00 5/8/00 5/8/00 5/9/00 5/9/00 5/9/00 5/9/00 5/9/00
7.0 7.0 6.3 7.0 7.2 6.2 7.2 6.2 6.2 7.0
5 6 4 6 6 5 5 5 5 7
Vanillin Fruity Floral Musty
Ethyl vanillin Ethyl butyrate Linalool Alpha-fenchol
No. fail
2
1 1 1 1
Comments Too much acid Too much dairy Too much dairy Too much dairy Too much dairy Too much dairy Too much vanilla Too much acid Too much acid Too much dairy
Table 4 Data from the evaluation of three production runs of extract using the SQS method Test No. 1
Table 2 SQS calibration set for extract Range
Sample
Match Acceptable Unacceptable
Control Two previously ‘‘accepted’’ lots Two previously ‘‘unacceptable’’ lots Control diluted 75% Extract spiked with maple flavor
Reject
Judge 1 Judge 2 Judge 3 Judge 4 Judge 5 Judge 6 Judge 7 Judge 8 Mean S.D.
Test No. 2
Test No. 3
Blind control
Lot No. 6
Blind control
Lot No. 5
Blind control
Lot No. 3
10 9 8 10 9 9 7 – 8.9 1.1
5 6 6 8 4 5 5 – 5.6 1.3
9 8 9 7 10 8 8 8 8.4 0.9
6 5 7 6 7 4 9 6 6.3 1.5
9 10 10 6 8 10 9 – 8.9 1.4
10 8 7 5 8 7 7 – 7.4 1.5
392
S. King et al. / Food Quality and Preference 13 (2002) 385–395
company was buying an ingredient from a supplier in Europe. Routine SQS results for the incoming raw material demonstrated a declining quality trend and comments from the ballots indicated the presence of Marmite-type off-flavors (Marmite# is a well-known consumer brand of a yeast extract). This information was shared with the supplier but had no effect on the quality of the incoming raw material. Action was taken before the raw material quality became unacceptable and a meeting was arranged with the supplier. Armed with historical SQS data and a jar of Marmite#, a Quality Assurance representative set off to talk to the supplier. SQS provided clear evidence that there was an issue with the declining flavor quality of the ingredient. The Marmite# reference standard clearly illustrated the nature of the flavor defect and it was quickly established that the problem was caused by over-heating of the ingredient during drying. The quality of the raw material improved immediately after the SQS discussion. 5.1.2. Latin America The SQS method was used to transfer quality information for a wet sauce from the USA to Mexico for a global customer. The customer’s expectation dictated similar flavor profiles within and between countries.
This expectation, although it presented some challenges for the product development team in Mexico, helped facilitate the transfer of the QC process from the USA. Once again, ingredient sourcing varied between countries, requiring that the ingredient references be re-evaluated. For example, when the reference library was sent to Mexico, it was found that some of the ingredients used as references were different from the local ingredients known by the same name. Differences were associated with sourcing of the spices. Key ingredients, causing flavor differences in the sauce, were identified and modified or changed, to insure that the Mexican product matched the gold standard (sourced from the USA). Implementation of this process facilitated discussion between developers and QA from the two countries, providing a forum for on-going communication regarding other issues related to production that affected the sensory characteristics of the product. A common language, selected from the SQS ballot, was the common ground for communication between the two producing countries. In addition, the ballot required translation into Spanish (Fig. 8). Twenty employees were trained in Mexico, including manufacturing personnel. During training, information was gathered regarding production issues, which were not evident in the US product and incorporated into the Mexican SQS ballot.
Fig. 9. Example of an SQS ballot used to evaluate shelf life of an orange flavor.
S. King et al. / Food Quality and Preference 13 (2002) 385–395
5.1.3. Australia A new seasoning, unique to the Australian market, was used to introduce the SQS method into the QC program for a company. Development and selection of ingredient references and calibration standards were conducted in the US using Australian ingredients. Product Development and QA Sensory participated in the selection of standards. Triangle tests were conducted to select acceptable from unacceptable samples and create a training kit to be transferred to the production site. This step eliminated some of the issues
393
encountered sourcing ingredients in the UK or Mexico. The supplier of one of the fat-based components was supplying below standard ingredient, which was quickly identified by the SQS panel. Diagnostic information was provided to the supplier to modify the ingredient and improve the quality. The supplier of another key ingredient was unable to provide a consistent ingredient, making the SQS process even more important. It was also recognized that the gold standard had a 6-month shelf life requiring that it be refreshed on a biannual basis.
Fig. 10. Example on an SQS ballot used to monitor commercialization of a new herb process. Flavor and appearance quality scores were evaluated
394
S. King et al. / Food Quality and Preference 13 (2002) 385–395
5.1.4. United States Products that are shipped within the borders of a county require careful monitoring to prevent customer rejection and possible production shut down resulting from receiving an unacceptable raw material. The SQS method has been utilized for a key raw material, an extract, shipped across the country to various customers. The control extract was first identified by the ingredient supplier as a ‘‘gold standard’’ and accepted by the set of customers. A group of panelists at the supplier’s R&D site were trained and calibrated using a range of samples (Table 2). Panelists were also shown reference standards for key attributes (Table 3). Incoming extracts from the plant were submitted to QA for evaluation. For each SQS session, a sub-set of the panelist pool was drawn. At each session, panelists tasted a labeled control and proceeded to rate a blind control and production sample. Table 4 shows data from a series of tests conducted over several weeks. Lot No. 3 was deemed an acceptable substitute for the control. Extract from Lot No. 6 was found to be an unacceptable substitute and comments suggested that this sample was strong in critical off-notes and weaker in positive characteristics. In this session, Judge 4 rated this production lot unacceptable (5) and the blind
control acceptable (6). It was concluded that calibration standards should be reviewed with this individual and his/her performance monitored. The mean scores (mean=6.3) for Lot No. 5 suggest that is was an acceptable substitute for this control. However, because two panelists rated Lot No. 5 unacceptable, the panel’s comments should be noted and this sample re-evaluated. 5.1.5. Other applications 5.1.5.1. Shelf life testing. The SQS method is used to evaluate new and reformulated products during their shelf life cycle. The method is particularly useful for this application since a control product is available throughout the study. SQS measures the degree of difference between the control and the stored samples, as well as identifying the differences. One of the challenges for this application is the generation of calibration standards; a concerted effort must be made to develop practical standards that will help identify acceptable and unacceptable ranges for the product. Information collected from these panels can be correlated to chemical and physical measurements to help select other methods that complement the SQS method (Fig. 9). In the following example, the stability of a new flavor carrier was investigated using SQS. The current flavor
Fig. 11. Example of an SQS ballot modified for Food Service applications during product cuttings.
S. King et al. / Food Quality and Preference 13 (2002) 385–395 Table 5 Degree of difference from the control for flavor carrier shelf life study after 2 months using SQS method 23 C 23 C 32 C 32 C Temperature 4 C Sample Control Control New carrier Control New carrier Judge 1 Judge 2 Judge 3 Judge 4 Judge 5 Mean S.D.
10 10 10 10 10 10 –
10 8 8 7 7 8.0 1.2
9 9 8 6 5 7.4 1.8
9 9 8 8 5 7.8 1.6
2 2 3 2 2 2.2 0.4
carrier was used as the control. Initially, descriptive analysis tests were conducted to assure that the zero time control and new carrier samples were similar in all sensory characteristics. Key attributes and references were identified and applied to the SQS ballot. The control sample was held at 4 C. The control and new carrier were also stored at 23 C and 32 C and tested at predetermined time intervals. Failure criteria were defined as the presence of off-notes and/or decrease in flavor impact. Results for the 2-month evaluation suggest that the new carrier was performing satisfactorily compared to the control at 23 C (Table 5). However, at elevated temperatures, the new carrier was rejected due the presence of off-notes described as rancid, sour and bitter. 5.1.5.2. Commercialization. The SQS method was successfully used during the scale-up and commercialization step of a new product. A control sample was created during the initial stages of development at the bench. Once the control sample was approved by the developers, scale-up samples were compared to the control to determine the variability in the process and help guide commercialization. For example, when a new herb process was commercialized, a control sample was created and properly stored. The commercialization took place at a distant location, requiring the training of a local panel to monitor scale-up. Calibration samples
395
were created in the development laboratory based on known potential defects in this process. References and calibration standards were transferred to the new location and the panel was trained (Fig. 10). The method was quickly implemented at this location. Monitoring of the panel took place at the headquarter facilities. Once the commercialization step was completed, the SQS method was assimilated into the quality control function. 5.1.5.3. Product cuttings. Another benefit of the SQS method is found in the food service industry. The method is used by food service operations to facilitate selection of alternate suppliers for their current products. Product cuttings are typically conducted with chefs and quality control personnel. The SQS method was successfully adapted to meet the challenges of this industry: definitions, rather than physical standards, are used to describe product attributes, since physical references are very difficult to obtain and retain in this environment. However, the discussions surrounding the attributes and quality definitions have proven valuable in achieving group consensus on the definition of product quality (Fig. 11). The SQS process continues to be well received by production facilities worldwide. It minimizes the potential loss of product due to variations in key ingredients and that the SQS method is easily understood and applied by our professional counterparts throughout the world, making it a truly global quality control method.
References ASTM. (1992). In J. Yantis, The role of sensory analysis in quality control MNL 14, 1–15. West Constohocken, PA, USA: American Society for Testing and Material. Beckley, J. P., & Kroll, D. (1996). R, Searching for sensory research excellence. Food Technology, 50(2), 61–63. Cardello, A. V. (1993). Cross-cultural sensory testing: a changing tide?. Cereal Foods World, 38(9), 699–701. Carlton, D. K. (1985). Plant sensory evaluation within a multiplant international organization. Food Technology, 39(11), 130–132. Mun˜oz, A. M., Civille, G. V., & Carr, B. T. (1992). Sensory evaluation in quality control. New York: Van Nostrand Reinhold.
The Sensory Quality System: a global quality control solution Silvia Kinga,*, Marianne Gillettea, David Titmanb, Julie Adamsa, Maria Ridgelya a McCormick and Company, 204 Wight Avenue, Hunt Valley, Maryland, USA McCormcik (UK) PLC, Thame Road, Haddenham, Aylesbury, Buckinghamshire HP17 8LB, UK
b
Received 26 January 2001; received in revised form 4 June 2001; accepted 20 November 2001
Abstract Assuring product quality is paramount to the success of any consumer product company that competes in the global marketplace. Quality Control/Quality Assurance programs must be in place to guarantee the consistent delivery of goods to consumers. However, establishing a successful QC/QA program has numerous obstacles within a single country’s borders and crossing international lines exponentially increases the hurdles. The Sensory Quality System described in this paper has proven to be an effective program to maintain product quality in an international environment for a global food company. # 2002 Elsevier Science Ltd. All rights reserved. Keywords: Sensory; Quality control; Quality assurance; Global
1. Introduction The expanding global market has resulted in a myriad of shifts in the way companies operate. However, delivering consistent products to consumers, whether within or across country borders, remains a critical element to a company’s success. An effective Quality Control (QC)/Quality Assurance (QA) program can help assure success to an organization. Considering the resources and efforts invested by an organization to launch a new product, the need to maintain product integrity is paramount. An organization’s reputation for quality is one of its most prized assets. Delivering products that are similar over time is essential to the consumer perception of quality, and key to brand loyalty. Distributing a product that consistently meets consumers expectations can mean the difference between a company thriving in the global market, or not. Methods to initiate and issues with establishing a QC/QA program have been well-documented (ASTM, 1992; Mun˜oz, 1992). In addition to the hurdles described by these authors, the impact of crossing international borders offers a unique set of challenges. Language may be the most difficult issue facing those setting up an
international program (Cardello, 1993). Companies must also be aware of the accepted communication methods, cultural biases, ingredient availability and familiarity, and reporting formats (Carlton, 1985). Another universal challenge in setting up any Sensory QC program has been the concept of the control. Simply identifying and securing a physical ‘‘control’’ is often a major research project in itself. For many food products, a single control sample is misleading and results in the rejection of adequate commercial product. This is particularly true for agricultural products and food service menu items. An effective global QC/QA program must describe an ‘‘acceptable range’’ of deviation, clearly identify ‘‘unacceptable’’ production samples, and be simple enough for all testing locations to take into account resource sophistication, monitoring and auditing. Management must also have a plan in place as to how to respond to sensory results. The program described in this paper has met the requirements for a sound program and has addressed the international challenges. It has been successful for an international food ingredients supplier at multiple global locations.
2. Sensory quality system * Corresponding author. Tel.: +1-401-771-7390; fax: +1- 410-5276527. E-mail address: [email protected] (S. King).
The Sensory Quality System (SQS) was developed by Gillette and Beckley (Beckley & Kroll, 1996) initially to
0950-3293/02/$ - see front matter # 2002 Elsevier Science Ltd. All rights reserved. PII: S0950-3293(01)00074-X
386
S. King et al. / Food Quality and Preference 13 (2002) 385–395
satisfy the demands of industrial customers requiring quality control on their products and was later optimized to meet other business demands. The long-term intent was to provide assurance of product quality across multiple plants and multiple companies. Critical elements of the program’s design were the prevention of flavor drift over time as well as providing an inter-plant communication tool to match target product. As the markets expanded, so did the SQS system, proving to be a powerful tool at all levels of the process, from ingredient monitoring to finished product evaluation, as well as facilitating communication among production and Quality Assurance labs throughout the world.
3. Method 3.1. Panelist selection
A score of 9–10 is equal to a match. The sample has virtually identical sensory characteristics to the control by appearance, aroma, flavor and texture. Any differences that may be noted are insignificant, and would not be detected without careful side by side comparison to the control. A score of 6–7–8 is equal to acceptable. The sample meets the definition for the product, but has differences that are easily spotted when comparing side by side with the control. These differences might not be discerned it the sample were not directly compared with the control. The sample is acceptable for shipping. A score of 3–4–5 is equal to unacceptable. The sample does not meet the definition of the product, but possibly can be reworked. Differences from the control are easily noted, but no foreign flavor has contaminated the sample. The sample is not acceptable for shipping.
Initially, 10–20 potential panelists are selected based upon their availability, motivation and ability to participate in regular product evaluations. Standard screening procedures can be used to assure panelists have a normal sense of smell and taste acuity. Potential panelists are typically selected from a pool of office, laboratory or plant personnel, depending on the site. Alternately, non-employee panelists have been utilized because of their consistent availability; these panelists can be easily trained on the ballot and references. 3.2. Panelist training The training objectives are to familiarize the panel with the QC program, the product being evaluated, the product’s ingredients, the product’s key characteristics and the ballot. The training process is also used to evaluate critical ingredients and physical references to familiarize the panel with the key ingredient characteristics. The initial training session takes about six 1/2-h sessions. Introduction of subsequent products takes 2–3 sessions per product, given that the products fall under the same product category. The panelists are familiarized with a 10-point quality scale, which measures quality based on degree of difference from the control. A control should be available at each panel session. The scale is divided into four categories. Panelists are calibrated on the scale using standard references, representing the range within each category. The ranges on the scale are defined where (Fig. 1):
Fig. 1. Example of a 10-point rating scale used to measure degree of difference from the control.
Fig. 2. Graphical representation of the typical variation found in a product, where the control is not a single point, and acceptable, unacceptable and reject regions vary in shape and distance from the control.
Fig. 3. Graphical representation of the variation found in a product, including reference standards used to represent those variations.
S. King et al. / Food Quality and Preference 13 (2002) 385–395
A score of 1–2 is equal to a reject. The sample has obvious defects rendering it non-shippable as product. The product could not be considered for rework. It may contain foreign flavors/ingredients. If the panelist rates the sample a 9 or a 10, they must consider it a ‘‘match’’ to the control, and the analysis stops here. If a panelist marks the sample 8 or below, the panelist completes the next section of the ballot. This section provides diagnostic information on how the production sample differs from the control. This section contains a list of product attributes, with boxes indicating strength levels of the attributes. Panelists examine this list and determine whether any of the words given are the reason for the difference from the control. If the panelist fails to find the cause for the difference on this list, they use an area marked ‘‘other’’ to describe the perceived difference. The cornerstone of this procedure is the calibration of the panel. This includes teaching the panel to appreciate the range of acceptable, unacceptable and reject pro-
387
duct. A perfect sensory match is typically not feasible in a production environment; therefore, a definition of what is an acceptable variation is required. It is important to start by acknowledging that the target is a range, and not a point. Targets drift. Additionally, minor deviations from the target are acceptable. At some point differences from the target become borderline, then unacceptable (Fig. 2). These ranges are defined by formulating a set of laboratory samples for calibration purposes. The set of calibration samples is initially graded by the customer, using the 10-point scale for ‘‘acceptable’’ to ‘‘unacceptable.’’ Usually, 15–20 calibration samples are prepared by Product Development. They are designed to demonstrate in minor, moderate, and major variations from the target. These calibration standards represent the expected range of deviations from the control. They illustrate the effects of likely production errors, and most importantly, the acceptable range of deviation (Fig. 3). These calibration standards must be created through formula modification, as well as aging or abusing the product. All steps used to create these stan-
Fig. 4. Example of a ballot used by QA and QC to evaluate vanilla extract.
388
S. King et al. / Food Quality and Preference 13 (2002) 385–395
dards must be carefully documented to insure that these samples can be recreated at any test location to recalibrate panels or train new panels. When feasible, the calibration set is scored by the customer, Product Development, discrimination panels and/or consumer. It is ideal to have the customer do the grading so the panel can be calibrated based upon the customer’s expectations. Once the calibration set is generated and scored, 8–10 of the samples are used to calibrate subsequent QC panels. On-going recalibration also occurs when the customer or Corporate QA forwards examples of ‘‘unacceptable’’, ‘‘acceptable’’ and ‘‘match’’ products from their cuttings back to the plants.
fectly replicated. The customer and supplier can work closely in the selection of the gold standard(s) using a range of sensory approaches, from descriptive analysis to consumer testing. In some cases, the customer identifies the control or gold standard from a series of samples. It is then the responsibility of the supplier to consistently produce this product concept and maintain a constant supply of new control to replace old/aged control. When updating the control, timing is critical so new controls can be selected before the shelf life of the existing control expires. Panelists evaluate potential controls and provide feedback regarding their similarity to the current gold standard.
3.3. Selection of the control
3.4. Ballot development
As previously suggested, the selection and maintenance of the control can be a major project in itself. For many product classes a range of ‘‘controls’’ is necessary because no single control sample can be per-
The selection of ballot attributes used to evaluate the samples is handled in a joint effort between Product Development and the Sensory Group. The ballot incorporates only those critical attributes that directly
Fig. 5. Example of manual data collection including quality scores and frequencies.
S. King et al. / Food Quality and Preference 13 (2002) 385–395
influence the quality score of the product as well as attributes associated with critical ingredients and or processing steps during production. The following is an example of a ballot used for QC and QA evaluations of vanilla (Fig. 4).
389
4. Data handling Data handling is kept simple and incorporates a concensus approach. This minimal analysis aspect of the method is a key to the successful implementation of the
Fig. 6. Bar chart representing panel scores for different lots of the same product. These data can be utilized to select potential gold standards as well as monitoring specific attributes that could cause flavor drift.
Fig. 7. Bar chart representing attribute scores for a single sample. These data are can be collected over time to monitor specific attributes and potential drift.
390
S. King et al. / Food Quality and Preference 13 (2002) 385–395
SQS process. Frequencies and mean scores for each sample are collected at the end of each evaluation to determine the status of the sample (Fig. 5). Panel means can be charted over time to track product performance. The SQS program can be used to select potential new controls as part of the routine sample evaluations. A sample rating a quality score of 8.5 or higher has the potential of becoming a new control sample. Panelists’ comments and score deviation are all key components in determining whether the sample should be considered as a future gold standard (Fig. 6). Diagnostic attributes can also be tracked and charted over time to measure product drift due to an attribute, ingredient change or issues during processing (Fig. 7).
Flavor changes and possible product drift can be documented through panelists’ comments. Panelists’ comments may suggest changes in the raw materials or process, which require investigation and or re-calibration. One or more panelists’ rejection scores will flag a questionable sample, setting a verification process in motion including additional testing, if needed, to identify potential problems. This verification takes place even when the average score for the sample may suggest an acceptable product (mean 5 6.0; Table 1). When different plant panels are calibrated using the same system, inter-plant lab audits are possible, and the results are much more easily communicated.
Fig. 8. Example of an SQS ballot translated into Spanish for Latin America.
391
S. King et al. / Food Quality and Preference 13 (2002) 385–395
5. Applications This method and several adaptations are used in production facilities around the world. The SQS method has been applied to the evaluation of key ingredients as well as production of seasonings, wet sauces and flavors. This system has also been adapted for shelf life studies, food service product cuttings and commercialization/ scale-up of new technologies. 5.1.1. Europe SQS has had a considerable effect in assisting in the globalization and control of products that were originally developed for the American market and have since found success in Europe. For the globalization of snack seasonings, SQS proved particularly valuable. Differences between US and EC legislation governing maximum levels and usage of permitted colors in foods meant that it was not possible for the formulation of a blend to be transferred without some modification of a key ingredient. Import duties also meant that commercial benefits would arise from sourcing the ingredient locally. However, any alterations made to the product due to formulation change or supplier would only be permitted if the flavor and color of the finished blend remained unchanged.
The local supplier was provided with training in SQS along with a suitable quantity of the original US ingredient to be used as a target for development. The SQS method provided a quick method for calibrating the local supplier to effectively evaluate and screen any pilot samples of the modified ingredient against the standard. As a result, there was increased confidence in any samples submitted for approval and considerably reduced time spent on validation through more time-intensive discrimination testing. By using standardized ballots with a common list of attributes, communication was more effective when discussing variation from the gold standard and pathways for development. Interestingly, the attributes used in the common ballot were only fully understood after training. Some of the flavor attributes, essential to the overall profile, were defined by grocery items that were household names in the USA, but unheard of in Europe. Universal understanding of these attributes was only achieved after the dispatch of several grocery parcels from the USA. The same challenge has been encountered when defining sensory attributes among panels from other European countries. Standard physical references for ballot attributes are as essential as providing the gold standard and calibration set. Another example of how SQS has benefited the supply of raw material occurred when a major global food
Table 1 SQS results for several lots of the same product, including average scores, number of panelists rating each sample a pass (quality of 6 and higher) or fail (quality of 2 or less)
Table 3 Key extract attributes for SQS panel training Attribute
Reference
Date
Average score
No. pass
5/4/00 5/8/00 5/8/00 5/8/00 5/8/00 5/9/00 5/9/00 5/9/00 5/9/00 5/9/00
7.0 7.0 6.3 7.0 7.2 6.2 7.2 6.2 6.2 7.0
5 6 4 6 6 5 5 5 5 7
Vanillin Fruity Floral Musty
Ethyl vanillin Ethyl butyrate Linalool Alpha-fenchol
No. fail
2
1 1 1 1
Comments Too much acid Too much dairy Too much dairy Too much dairy Too much dairy Too much dairy Too much vanilla Too much acid Too much acid Too much dairy
Table 4 Data from the evaluation of three production runs of extract using the SQS method Test No. 1
Table 2 SQS calibration set for extract Range
Sample
Match Acceptable Unacceptable
Control Two previously ‘‘accepted’’ lots Two previously ‘‘unacceptable’’ lots Control diluted 75% Extract spiked with maple flavor
Reject
Judge 1 Judge 2 Judge 3 Judge 4 Judge 5 Judge 6 Judge 7 Judge 8 Mean S.D.
Test No. 2
Test No. 3
Blind control
Lot No. 6
Blind control
Lot No. 5
Blind control
Lot No. 3
10 9 8 10 9 9 7 – 8.9 1.1
5 6 6 8 4 5 5 – 5.6 1.3
9 8 9 7 10 8 8 8 8.4 0.9
6 5 7 6 7 4 9 6 6.3 1.5
9 10 10 6 8 10 9 – 8.9 1.4
10 8 7 5 8 7 7 – 7.4 1.5
392
S. King et al. / Food Quality and Preference 13 (2002) 385–395
company was buying an ingredient from a supplier in Europe. Routine SQS results for the incoming raw material demonstrated a declining quality trend and comments from the ballots indicated the presence of Marmite-type off-flavors (Marmite# is a well-known consumer brand of a yeast extract). This information was shared with the supplier but had no effect on the quality of the incoming raw material. Action was taken before the raw material quality became unacceptable and a meeting was arranged with the supplier. Armed with historical SQS data and a jar of Marmite#, a Quality Assurance representative set off to talk to the supplier. SQS provided clear evidence that there was an issue with the declining flavor quality of the ingredient. The Marmite# reference standard clearly illustrated the nature of the flavor defect and it was quickly established that the problem was caused by over-heating of the ingredient during drying. The quality of the raw material improved immediately after the SQS discussion. 5.1.2. Latin America The SQS method was used to transfer quality information for a wet sauce from the USA to Mexico for a global customer. The customer’s expectation dictated similar flavor profiles within and between countries.
This expectation, although it presented some challenges for the product development team in Mexico, helped facilitate the transfer of the QC process from the USA. Once again, ingredient sourcing varied between countries, requiring that the ingredient references be re-evaluated. For example, when the reference library was sent to Mexico, it was found that some of the ingredients used as references were different from the local ingredients known by the same name. Differences were associated with sourcing of the spices. Key ingredients, causing flavor differences in the sauce, were identified and modified or changed, to insure that the Mexican product matched the gold standard (sourced from the USA). Implementation of this process facilitated discussion between developers and QA from the two countries, providing a forum for on-going communication regarding other issues related to production that affected the sensory characteristics of the product. A common language, selected from the SQS ballot, was the common ground for communication between the two producing countries. In addition, the ballot required translation into Spanish (Fig. 8). Twenty employees were trained in Mexico, including manufacturing personnel. During training, information was gathered regarding production issues, which were not evident in the US product and incorporated into the Mexican SQS ballot.
Fig. 9. Example of an SQS ballot used to evaluate shelf life of an orange flavor.
S. King et al. / Food Quality and Preference 13 (2002) 385–395
5.1.3. Australia A new seasoning, unique to the Australian market, was used to introduce the SQS method into the QC program for a company. Development and selection of ingredient references and calibration standards were conducted in the US using Australian ingredients. Product Development and QA Sensory participated in the selection of standards. Triangle tests were conducted to select acceptable from unacceptable samples and create a training kit to be transferred to the production site. This step eliminated some of the issues
393
encountered sourcing ingredients in the UK or Mexico. The supplier of one of the fat-based components was supplying below standard ingredient, which was quickly identified by the SQS panel. Diagnostic information was provided to the supplier to modify the ingredient and improve the quality. The supplier of another key ingredient was unable to provide a consistent ingredient, making the SQS process even more important. It was also recognized that the gold standard had a 6-month shelf life requiring that it be refreshed on a biannual basis.
Fig. 10. Example on an SQS ballot used to monitor commercialization of a new herb process. Flavor and appearance quality scores were evaluated
394
S. King et al. / Food Quality and Preference 13 (2002) 385–395
5.1.4. United States Products that are shipped within the borders of a county require careful monitoring to prevent customer rejection and possible production shut down resulting from receiving an unacceptable raw material. The SQS method has been utilized for a key raw material, an extract, shipped across the country to various customers. The control extract was first identified by the ingredient supplier as a ‘‘gold standard’’ and accepted by the set of customers. A group of panelists at the supplier’s R&D site were trained and calibrated using a range of samples (Table 2). Panelists were also shown reference standards for key attributes (Table 3). Incoming extracts from the plant were submitted to QA for evaluation. For each SQS session, a sub-set of the panelist pool was drawn. At each session, panelists tasted a labeled control and proceeded to rate a blind control and production sample. Table 4 shows data from a series of tests conducted over several weeks. Lot No. 3 was deemed an acceptable substitute for the control. Extract from Lot No. 6 was found to be an unacceptable substitute and comments suggested that this sample was strong in critical off-notes and weaker in positive characteristics. In this session, Judge 4 rated this production lot unacceptable (5) and the blind
control acceptable (6). It was concluded that calibration standards should be reviewed with this individual and his/her performance monitored. The mean scores (mean=6.3) for Lot No. 5 suggest that is was an acceptable substitute for this control. However, because two panelists rated Lot No. 5 unacceptable, the panel’s comments should be noted and this sample re-evaluated. 5.1.5. Other applications 5.1.5.1. Shelf life testing. The SQS method is used to evaluate new and reformulated products during their shelf life cycle. The method is particularly useful for this application since a control product is available throughout the study. SQS measures the degree of difference between the control and the stored samples, as well as identifying the differences. One of the challenges for this application is the generation of calibration standards; a concerted effort must be made to develop practical standards that will help identify acceptable and unacceptable ranges for the product. Information collected from these panels can be correlated to chemical and physical measurements to help select other methods that complement the SQS method (Fig. 9). In the following example, the stability of a new flavor carrier was investigated using SQS. The current flavor
Fig. 11. Example of an SQS ballot modified for Food Service applications during product cuttings.
S. King et al. / Food Quality and Preference 13 (2002) 385–395 Table 5 Degree of difference from the control for flavor carrier shelf life study after 2 months using SQS method 23 C 23 C 32 C 32 C Temperature 4 C Sample Control Control New carrier Control New carrier Judge 1 Judge 2 Judge 3 Judge 4 Judge 5 Mean S.D.
10 10 10 10 10 10 –
10 8 8 7 7 8.0 1.2
9 9 8 6 5 7.4 1.8
9 9 8 8 5 7.8 1.6
2 2 3 2 2 2.2 0.4
carrier was used as the control. Initially, descriptive analysis tests were conducted to assure that the zero time control and new carrier samples were similar in all sensory characteristics. Key attributes and references were identified and applied to the SQS ballot. The control sample was held at 4 C. The control and new carrier were also stored at 23 C and 32 C and tested at predetermined time intervals. Failure criteria were defined as the presence of off-notes and/or decrease in flavor impact. Results for the 2-month evaluation suggest that the new carrier was performing satisfactorily compared to the control at 23 C (Table 5). However, at elevated temperatures, the new carrier was rejected due the presence of off-notes described as rancid, sour and bitter. 5.1.5.2. Commercialization. The SQS method was successfully used during the scale-up and commercialization step of a new product. A control sample was created during the initial stages of development at the bench. Once the control sample was approved by the developers, scale-up samples were compared to the control to determine the variability in the process and help guide commercialization. For example, when a new herb process was commercialized, a control sample was created and properly stored. The commercialization took place at a distant location, requiring the training of a local panel to monitor scale-up. Calibration samples
395
were created in the development laboratory based on known potential defects in this process. References and calibration standards were transferred to the new location and the panel was trained (Fig. 10). The method was quickly implemented at this location. Monitoring of the panel took place at the headquarter facilities. Once the commercialization step was completed, the SQS method was assimilated into the quality control function. 5.1.5.3. Product cuttings. Another benefit of the SQS method is found in the food service industry. The method is used by food service operations to facilitate selection of alternate suppliers for their current products. Product cuttings are typically conducted with chefs and quality control personnel. The SQS method was successfully adapted to meet the challenges of this industry: definitions, rather than physical standards, are used to describe product attributes, since physical references are very difficult to obtain and retain in this environment. However, the discussions surrounding the attributes and quality definitions have proven valuable in achieving group consensus on the definition of product quality (Fig. 11). The SQS process continues to be well received by production facilities worldwide. It minimizes the potential loss of product due to variations in key ingredients and that the SQS method is easily understood and applied by our professional counterparts throughout the world, making it a truly global quality control method.
References ASTM. (1992). In J. Yantis, The role of sensory analysis in quality control MNL 14, 1–15. West Constohocken, PA, USA: American Society for Testing and Material. Beckley, J. P., & Kroll, D. (1996). R, Searching for sensory research excellence. Food Technology, 50(2), 61–63. Cardello, A. V. (1993). Cross-cultural sensory testing: a changing tide?. Cereal Foods World, 38(9), 699–701. Carlton, D. K. (1985). Plant sensory evaluation within a multiplant international organization. Food Technology, 39(11), 130–132. Mun˜oz, A. M., Civille, G. V., & Carr, B. T. (1992). Sensory evaluation in quality control. New York: Van Nostrand Reinhold.