- Email: [email protected]
Ignorant experts and erudite novices: Exploring the Dunning-Kruger effect in wine consumers
Accepted Manuscript Ignorant Experts and Erudite Novices: Exploring the Dunning-Kruger Effect in Wine Consumers Claudio Aqueveque PII: DOI: Reference:
S0950-3293(17)30299-9 https://doi.org/10.1016/j.foodqual.2017.12.007 FQAP 3434
To appear in:
Food Quality and Preference
Received Date: Revised Date: Accepted Date:
24 January 2017 5 December 2017 16 December 2017
Please cite this article as: Aqueveque, C., Ignorant Experts and Erudite Novices: Exploring the Dunning-Kruger Effect in Wine Consumers, Food Quality and Preference (2017), doi: https://doi.org/10.1016/j.foodqual. 2017.12.007
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Ignorant Experts and Erudite Novices: Exploring the Dunning-Kruger Effect in Wine Consumers.
Claudio Aqueveque Associate Professor Business School, Universidad Adolfo Ibáñez Av. Padre Hurtado 750, Viña del Mar Región de Valparaíso, Chile Phone: +56-32-2503818 [email protected]
Abstract Research devoted to identify differences between expert and nonexpert consumers in terms of wine quality perceptions, preferences, and information use and processing, have been prolific during the last decade. Many of these studies have used subjective or selfreported measures of knowledge to distinguish between expert and non-expert consumers. However, this approach can be problematic due to the existence of the Dunning-Kruger effect, a cognitive bias in which incompetent or unaware subjects tend to overestimate their knowledge or expertise, whereas high-ability individuals tend to underestimate it. The objective of this study was to explore the presence of this cognitive bias within the wine-knowledge domain. Using a sample of wine consumers (n = 193) and through different statistical analyses, the presence of the Dunning-Kruger effect was confirmed, raising important concerns regarding the use of subjective or self-reported measures of knowledge to classify consumers as experts or non-experts.
1
Keywords: Wine; Dunning-Kruger; cognitive bias; objective knowledge; subjective knowledge.
1. Introduction During the last decades, the emergence of new world producers and the growing consumption of wine (Barber, Ismail, & Todd, 2008) have created a new scenario with many more product options in terms of brands, types, and varieties. These circumstances, combined with the unique and highly complex nature of wine as a product category (Hollebeek, Jaeger, Brodie, & Balemi, 2007), and the fact that it is regarded as a complicated product from the view-point of consumers (Johnson & Bruwer, 2007), make the purchase of wine a process that, in order to be successful, requires a certain level of knowledge from the consumer. Consequently, and as a way to understand how consumer knowledge influences the purchase process of wine, researchers and practitioners have investigated how different levels of subjective or self-reported knowledge determine differences in terms of consumers’ information use and processing to infer quality (Cox, 2009; Perrouty, d’Hauteville, & Lockshin, 2006; Sáenz-Navajas, Ballester, Peyron, & Valentin, 2014). For example, it has been shown that expert and novice wine consumers differ in terms of the use of extrinsic cues to infer quality (Aqueveque, 2015; Perrouty, d’Hauteville, & Lockshin, 2006), and in terms of the number and type of attributes considered important, as well as the size of their consideration sets (Viot, 2012). Also, is has been shown that connoisseurs used price as a signal of quality less intensively than nonconnoisseurs (Gergaud and Livat, 2007), and that the beliefs of naïve consumers regarding the relationship between the weight of the bottle and the price (and quality) of the wine are
2
significantly different than the beliefs of amateur and expert consumers (Piqueras-Fiszman and Spence, 2012). All of the examples mentioned above relied on subjective or self-reported measures of consumer knowledge to classify consumers according to their level of knowledge (i.e. novices or experts; connoisseurs or non-connoisseurs; naïve, amateurs, or expert consumers). Moreover, according to Perrouty et al. (2006), subjective knowledge is a key determinant of wine consumers’ behaviour and an appropriate basis for determining whether consumers are wine experts or novices. However, and notwithstanding the importance of these studies and their findings, it is important to note that the use of subjective or self-reported knowledge as an indicator of consumer knowledge might be problematic due to a cognitive bias known as the Dunning– Kruger effect (Kruger & Dunning, 1999). The Dunning–Kruger effect has been defined as a cognitive bias in which incompetent or unaware subjects overestimate their knowledge or expertise, considering themselves as more adept than they really are, whereas high-ability individuals underestimate their relative competence and may erroneously assume that tasks which are easy for them are also easy for others (Kruger & Dunning, 1999). According to these authors “[...] unskilled individuals suffer a dual burden: not only do they perform poorly, but they fail to realize it. It thus appears that extremely competent individuals suffer a burden as well. Although they perform competently, they fail to realize that their proficiency is not necessarily shared by their peers” (Kruger & Dunning, 1999, p. 1131). This effect has been observed in varied domains, including academic knowledge, grammar skills, emotional intelligence, driving tests, professional examinations, among others (for a review, see Dunning, 2011). 3
Therefore, and considering the characteristic complexity of wine as a category and the different levels of wine-related knowledge that consumers can have, there is a strong possibility that the previously mentioned cognitive bias might be present within this domain, affecting consumers’ subjective or self-reported measures of category knowledge. Consequently, the main objective of the present study is to examine how present the Dunning–Kruger effect is within the wine knowledge realm, and its potential practical and theoretical implications.
2. Methods 2.1.
Participants Two hundred and four students, all enrolled at a large Chilean university in different
academic programs for executives, participated in the study. The study employed a survey design. A research assistant administrated the questionnaire during students’ normal class time. Participants volunteered to participate in the research, and were provided with a paper-and-pencil questionnaire. As a way to avoid potential Middle Response Style effects, eleven questionnaires that showed this precise response pattern on the Subjective Knowledge Scale were removed. Therefore, the final sample was composed of 126 men and 67 women, aged 26 to 55 (men: M = 37.85, SD = 6.23; women: M = 34.73, SD = 6.17).
2.2.
Materials and procedure Participants were asked to participate in a study oriented to investigate wine
consumption habits. Subjects were randomly assigned to one of the two conditions that differed in terms of the measurement order of objective and subjective knowledge. These
4
two conditions were developed with the goal of controlling for possible priming effects due to the nature of the objective and subjective knowledge measures. Participants were provided with a paper-and-pencil questionnaire that included an introduction with the instructions to complete it. Within this part, the importance of doing it in the established order was emphasized. After this introduction, the questionnaire included five short sections. The first section was a “warm up” task in which they were asked to rate the quality of nine wines based on basic information (brand, valley, price, and bottle). In the second section, they were asked to answer a set of questions oriented to measure either their subjective or objective knowledge about wines, depending on the condition they were assigned. Third, and in order to minimize carry over effects from one measure of knowledge to the other, a set of questions related to measure their wine consumption habits was included. Fourth, a set of questions oriented to measure the type of knowledge not measured in the second section was included. Finally, demographic queries and a question oriented to identify if participants were able to infer the objective of the study were included. It is relevant to note that a pilot study and pre-test were conducted with subjects from the population under study in order to test the material. The pilot study (n = 30) was conducted to test if questions were well-written, if objective knowledge questions were adequate, and if the questionnaire procedure was clear. Based on pre-test results, questionnaire was considered appropriate, and therefore was applied to the final sample.
2.3.
Knowledge measures Respondents answered a series of scale items intended to measure their objective and
subjective (i.e. self-reported) knowledge of wine. Objective knowledge was measured using 5
a set of seven multiple-choice questions. These questions were developed with the assistance of three wine experts (two sommeliers and one oenologist), whom were asked to provide questions with different degrees of difficulty. Participants were assigned one point for each correct answer and cero points for incorrect or no answer. Scores from participants ranged from 0 to 7 (Mean = 3.678; S.D. = 1.768). Subjective knowledge was measured with a four-item summated scale adapted from Mitchell and Dacin (1996) using a 5-point Likert scale ranging from 1 to 5, and was found to be highly reliable (α = 0.904). Scores from participants ranged from 4 to 20 (Mean = 11.907; S.D. = 3.676). Then, and as a way to facilitate statistical comparisons, this scale was linearly transformed to a 0 to 7 scale (Mean = 3.459; S.D. = 1.608). Both objective and subjective knowledge scales are presented in Table 1. [Insert Table 1 Here]
3. Results In order to compare the differences between objective and subjective knowledge, respondents were divided into four groups based on their scores of objective knowledge, with subjects performing from zero to one point (correct answers) as “Lowest Objective Knowledge” (n = 28), subjects performing from two or three points as “Low Objective Knowledge” (n = 59), subjects performing from four or five points as “High Objective Knowledge” (n = 70) and subjects performing from six or seven points as “Highest Objective Knowledge” (n = 36). Table 2 shows the mean and standard deviations of objective and subjective knowledge scores for the four groups, whereas Figure 1 examines the relationship between subjective knowledge versus objective knowledge for the four groups. 6
[Insert Table 2 Here] [Insert Figure 1 Here] A paired-samples t-test was conducted within each group to compare the scores of objective and subjective knowledge. Results of these tests showed significant differences in the scores of objective and subjective knowledge for all groups. Our main focus, however, are the extreme groups, defined as those composed by respondents with the lowest and highest levels of Objective Knowledge. For those two groups, results of the paired-samples t-test provide evidence of very significant overestimation of wine knowledge within the “Lowest Objective Knowledge” group (MO-S = -1.069, t = -4.642, p < 0.001), and very significant underestimation of wine knowledge within the “Highest Objective Knowledge” (MO-S = 1.767, t = 8.490, p < 0.001) group. These results provide evidence of the presence of the Dunning–Kruger effect within the wine knowledge domain. Specifically, the results confirm the fact that subjects with low levels of wine knowledge overestimate their knowledge of the category, while individuals with high levels of wine knowledge underestimate it. Additionally, a Pearson product-moment correlation coefficient was computed to assess the relationship between objective and subjective knowledge measures. This analysis shows a significant positive correlation between the two variables (r = 0.507, p < 0.001). This correlation is in line with the results of a meta-analysis of 51 empirical findings associated with the relationship between objective and subjective knowledge performed by Carlson et al. (2009). Results of this meta-analysis show that objective and subjective knowledge are significantly – although moderately – positively related (r=0.37) and that this correlation is stronger for products (r=0.57) versus non‐products (r=0.32).
7
However, it is interesting to note that the shared variance between the two variables is less than 25% (R2 = 0.257). This means that it is possible to assert that merely 25% of the variance in subjective knowledge measures can be attributable to variance in objective knowledge.
4. Discussion and conclusion During the last years, wine-related knowledge differences and their impact in the buying process of wine have been an important area of research, with many studies devoted to identify differences between expert and non-expert consumers in terms of quality perceptions, preferences, information use and processing, among other aspects. Notwithstanding the importance of these studies, a problematic issue related with them is the wide use of subjective or self-reported knowledge measures as a way to distinguish between experts and non-experts consumers. This issue is problematic due to the existence of cognitive biases affecting consumer self-assessment processes, such as the DunningKruger effect. Considering this relevant matter, this study provides strong evidence of the existence of this cognitive bias among wine consumers. The results of the paired-samples t-tests conducted provide evidence of significant differences between subjective knowledge and objective knowledge measures for consumers with the lowest and highest levels of wine knowledge. In particular, and confirming the existence of the Dunning-Kruger effect, consumers with extremely low levels of wine knowledge overestimate their knowledge of the category, whereas consumers with extremely high levels of wine knowledge underestimate their knowledge of the category.
8
Finally, results of the correlation analysis between the two knowledge measures show a low correlation and shared variance between subjective and objective measures of knowledge, increasing the evidence of the problematic use of subjective measures. This study has some limitations. In particular, and although the development process and scores demonstrated that the scale was adequate, the objective knowledge scale utilized is one of many objective knowledge measurement procedures that could be used to assess the amount of product-related knowledge of consumers. While our scale measured knowledge related to wine types, wine consumption, and wine production processes, other scales could focus only in product and consumption related knowledge, excluding wine production knowledge. Additionally, our scale has questions that are country specific, and therefore could be useless in other countries. Therefore, future research could use alternative measurement procedures, such as open-ended questions, or use differentiated measures for objective product class information and objective expertise (Philippe & Ngobo, 1999). This study’s results have important implications for researchers and practitioners. First, and considering the findings related to the poor ability of subjective measures of knowledge to reflect the real level of knowledge of consumers, researchers studying behavioural and attitudinal differences between expert and non-expert consumers should try to favour the use of objective knowledge measures. Second, the results of previous studies that used subjective measures of knowledge to distinguish between expert and non-expert consumers should be interpreted cautiously, considering the fact that the dissimilarity between these groups might be spurious. Finally, practitioners in the wine industry, and especially those involved in market research activities, should start moving from subjective measures of
9
consumer knowledge to objective measurement scales in order to obtain non-biased responses and therefore useful data and insightful results.
References Aqueveque, C. (2015). The influence of experts' positive word-of-mouth on a wine's perceived quality and value: the moderator role of consumers' expertise. Journal of Wine Research, 26, 181-191. Barber, N., Ismail, J., & Dodd, T. (2008). Purchase attributes of wine consumers with low involvement. Journal of Food Products Marketing, 14, 69-86. Carlson, J. P., Vincent, L. H., Hardesty, D. M., & Bearden, W. O. (2008). Objective and subjective knowledge relationships: A quantitative analysis of consumer research findings. Journal of Consumer Research, 35, 864-876. Cox, D. (2009). Predicting consumption, wine involvement and perceived quality of Australian red wine, Journal of Wine Research, 20, 209-229. Dunning, D. (2011). The Dunning-Kruger Effect: On Being Ignorant of One's Own Ignorance. In J. Olson & M. P. Zanna (Eds.), Advances in experimental social psychology Volume 4 (pp. 247-296). New York: Elsevier Gergaud, O., & Livat, F. (2007). How do consumers use signals to assess quality? American Association of Wine Economists Working Paper, 3, 1-22. Hollebeek, L. D., Jaeger, S. R., Brodie, R. J., & Balemi, A. (2007). The influence of involvement on purchase intention for new world wine. Food Quality and Preference, 18, 1033-1049.
10
Johnson, R., & Bruwer, J. (2007). Regional brand image and perceived wine quality: the consumer perspective. International Journal of Wine Business Research, 19, 276297. Kruger, J. M., & Dunning, D. (1999). Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77, 1121-1134. Mitchell, A., & Dacin, P. (1996). The assessment of alternative measures of consumer expertise. Journal of Consumer Research, 23, 219-239. Perrouty, J. P., d’Hauteville, F., & Lockshin, L. (2006). The influence of wine attributes on region of origin equity: an analysis of the moderating effect of consumer’s perceived expertise. Agribusiness, 22, 323-341. Philippe, A., & Ngobo, P. V. (1999). Assessment of consumer knowledge and its consequences: A multi-component approach. In E.J. Arnould & L.M. Scott (Eds.), NA - Advances in Consumer Research Volume 26 (pp. 569-575). Provo, UT: Association for Consumer Research. Piqueras-Fiszman, B., & Spence, C. (2012). The weight of the bottle as a possible extrinsic cue with which to estimate the price (and quality) of the wine? Observed correlations. Food Quality and Preference, 25(1), 41-45. Sáenz-Navajas, M. P., Ballester, J., Peyron, D., & Valentin, D. (2014). Extrinsic attributes responsible for red wine quality perception: a cross-cultural study between France and Spain. Food Quality and Preference, 35, 70-85.
11
Viot, C. (2012). Subjective knowledge, product attributes and consideration set: a wine application. International Journal of Wine Business Research, 24, 219-248.
12
Figure 1. Participants’ Objective and Subjective Knowledge
Scores 6.17 4.44 3.91
3.100 1.89
4.40
2.61
.82 Lowest Objective Knowledge
Low Objective Knowledge
High Objective Knowledge
Objective Knowledge
Highest Objective Knowledge
Subjective Knowledge
13
Table 1. Objective and Subjective Knowledge Scales Objective Knowledge Scalea Q1: Which of the following wine varieties is a red wine? Chardonnay Riesling Merlot Don't Know / Don't Answer Q2: Which of the following wine varieties is produced only in Chile? Cabernet-Sauvignon Carmenere Merlot Don't Know / Don't Answer Q3: Which of the following wine varieties it is usually considered a "full-bodied" wine? Cabernet-Sauvignon Carmenere Merlot Don't Know / Don't Answer Q4: "Letting wine breathe" is: Uncorking a bottle and letting it sit undisturbed for period of time before drink Letting the wine improve its flavour Aerating the wine so it can react with oxygen Don't Know / Don't Answer Q5: Tannins are responsible of wine's: Sweetness Astringency Acidity Don't Know / Don't Answer Q6: Which of the following groups of characteristics is commonly associated to Chardonnay wine? Apple/Pineapple/Melon/Peach/Lemon Tomato/Peppers/Apple/Cherry Strawberry/Vanilla/Lime/Cinnamon Don't Know / Don't Answer Q7: In the production process of wine, racking means: Moving wine from one container to another, to leave any sediment behind Extracting the colour of wine The grape has reached its optimal maturation point Don't Know / Don't Answer Subjective Knowledge Scaleb (α = 0.904) How would you rate your knowledge about wines compared to the rest of the population? (One of the least knowledgeable people – One of the most knowledgeable people) How interested are you in wines? (Not very interested – Very interested) How familiar are you with wines? (Not familiar at all – Extremely familiar) Your knowledge about wines is (Very low – Very high) a. The correct answers are written in bold and italic. b. All items measured using five-point scales ranging from 1 to 5. Anchors are between parentheses.
14
Table 2. Means (standard deviations) of participants’ Objective and Subjective Knowledge
Objective Knowledge Subjective Knowledge
Lowest Objective Knowledge (n = 28) 0.821 (0.390)
Low Objective Knowledge (n = 59) 2.610 (0.492)
High Objective Knowledge (n = 70) 4.443 (0.501)
Highest Objective Knowledge (n = 36) 6.167 (0.378)
1.891 (1.283)
3.100 (1.494)
3.906 (1.431)
4.399 (1.298)
15
Highlights The Dunning-Kruger effect exists among wine consumers. Consumers with low levels of wine knowledge overestimate their knowledge Consumers with high levels of wine knowledge underestimate their knowledge. There is low shared variance between subjective and objective measures of knowledge
16
S0950-3293(17)30299-9 https://doi.org/10.1016/j.foodqual.2017.12.007 FQAP 3434
To appear in:
Food Quality and Preference
Received Date: Revised Date: Accepted Date:
24 January 2017 5 December 2017 16 December 2017
Please cite this article as: Aqueveque, C., Ignorant Experts and Erudite Novices: Exploring the Dunning-Kruger Effect in Wine Consumers, Food Quality and Preference (2017), doi: https://doi.org/10.1016/j.foodqual. 2017.12.007
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Ignorant Experts and Erudite Novices: Exploring the Dunning-Kruger Effect in Wine Consumers.
Claudio Aqueveque Associate Professor Business School, Universidad Adolfo Ibáñez Av. Padre Hurtado 750, Viña del Mar Región de Valparaíso, Chile Phone: +56-32-2503818 [email protected]
Abstract Research devoted to identify differences between expert and nonexpert consumers in terms of wine quality perceptions, preferences, and information use and processing, have been prolific during the last decade. Many of these studies have used subjective or selfreported measures of knowledge to distinguish between expert and non-expert consumers. However, this approach can be problematic due to the existence of the Dunning-Kruger effect, a cognitive bias in which incompetent or unaware subjects tend to overestimate their knowledge or expertise, whereas high-ability individuals tend to underestimate it. The objective of this study was to explore the presence of this cognitive bias within the wine-knowledge domain. Using a sample of wine consumers (n = 193) and through different statistical analyses, the presence of the Dunning-Kruger effect was confirmed, raising important concerns regarding the use of subjective or self-reported measures of knowledge to classify consumers as experts or non-experts.
1
Keywords: Wine; Dunning-Kruger; cognitive bias; objective knowledge; subjective knowledge.
1. Introduction During the last decades, the emergence of new world producers and the growing consumption of wine (Barber, Ismail, & Todd, 2008) have created a new scenario with many more product options in terms of brands, types, and varieties. These circumstances, combined with the unique and highly complex nature of wine as a product category (Hollebeek, Jaeger, Brodie, & Balemi, 2007), and the fact that it is regarded as a complicated product from the view-point of consumers (Johnson & Bruwer, 2007), make the purchase of wine a process that, in order to be successful, requires a certain level of knowledge from the consumer. Consequently, and as a way to understand how consumer knowledge influences the purchase process of wine, researchers and practitioners have investigated how different levels of subjective or self-reported knowledge determine differences in terms of consumers’ information use and processing to infer quality (Cox, 2009; Perrouty, d’Hauteville, & Lockshin, 2006; Sáenz-Navajas, Ballester, Peyron, & Valentin, 2014). For example, it has been shown that expert and novice wine consumers differ in terms of the use of extrinsic cues to infer quality (Aqueveque, 2015; Perrouty, d’Hauteville, & Lockshin, 2006), and in terms of the number and type of attributes considered important, as well as the size of their consideration sets (Viot, 2012). Also, is has been shown that connoisseurs used price as a signal of quality less intensively than nonconnoisseurs (Gergaud and Livat, 2007), and that the beliefs of naïve consumers regarding the relationship between the weight of the bottle and the price (and quality) of the wine are
2
significantly different than the beliefs of amateur and expert consumers (Piqueras-Fiszman and Spence, 2012). All of the examples mentioned above relied on subjective or self-reported measures of consumer knowledge to classify consumers according to their level of knowledge (i.e. novices or experts; connoisseurs or non-connoisseurs; naïve, amateurs, or expert consumers). Moreover, according to Perrouty et al. (2006), subjective knowledge is a key determinant of wine consumers’ behaviour and an appropriate basis for determining whether consumers are wine experts or novices. However, and notwithstanding the importance of these studies and their findings, it is important to note that the use of subjective or self-reported knowledge as an indicator of consumer knowledge might be problematic due to a cognitive bias known as the Dunning– Kruger effect (Kruger & Dunning, 1999). The Dunning–Kruger effect has been defined as a cognitive bias in which incompetent or unaware subjects overestimate their knowledge or expertise, considering themselves as more adept than they really are, whereas high-ability individuals underestimate their relative competence and may erroneously assume that tasks which are easy for them are also easy for others (Kruger & Dunning, 1999). According to these authors “[...] unskilled individuals suffer a dual burden: not only do they perform poorly, but they fail to realize it. It thus appears that extremely competent individuals suffer a burden as well. Although they perform competently, they fail to realize that their proficiency is not necessarily shared by their peers” (Kruger & Dunning, 1999, p. 1131). This effect has been observed in varied domains, including academic knowledge, grammar skills, emotional intelligence, driving tests, professional examinations, among others (for a review, see Dunning, 2011). 3
Therefore, and considering the characteristic complexity of wine as a category and the different levels of wine-related knowledge that consumers can have, there is a strong possibility that the previously mentioned cognitive bias might be present within this domain, affecting consumers’ subjective or self-reported measures of category knowledge. Consequently, the main objective of the present study is to examine how present the Dunning–Kruger effect is within the wine knowledge realm, and its potential practical and theoretical implications.
2. Methods 2.1.
Participants Two hundred and four students, all enrolled at a large Chilean university in different
academic programs for executives, participated in the study. The study employed a survey design. A research assistant administrated the questionnaire during students’ normal class time. Participants volunteered to participate in the research, and were provided with a paper-and-pencil questionnaire. As a way to avoid potential Middle Response Style effects, eleven questionnaires that showed this precise response pattern on the Subjective Knowledge Scale were removed. Therefore, the final sample was composed of 126 men and 67 women, aged 26 to 55 (men: M = 37.85, SD = 6.23; women: M = 34.73, SD = 6.17).
2.2.
Materials and procedure Participants were asked to participate in a study oriented to investigate wine
consumption habits. Subjects were randomly assigned to one of the two conditions that differed in terms of the measurement order of objective and subjective knowledge. These
4
two conditions were developed with the goal of controlling for possible priming effects due to the nature of the objective and subjective knowledge measures. Participants were provided with a paper-and-pencil questionnaire that included an introduction with the instructions to complete it. Within this part, the importance of doing it in the established order was emphasized. After this introduction, the questionnaire included five short sections. The first section was a “warm up” task in which they were asked to rate the quality of nine wines based on basic information (brand, valley, price, and bottle). In the second section, they were asked to answer a set of questions oriented to measure either their subjective or objective knowledge about wines, depending on the condition they were assigned. Third, and in order to minimize carry over effects from one measure of knowledge to the other, a set of questions related to measure their wine consumption habits was included. Fourth, a set of questions oriented to measure the type of knowledge not measured in the second section was included. Finally, demographic queries and a question oriented to identify if participants were able to infer the objective of the study were included. It is relevant to note that a pilot study and pre-test were conducted with subjects from the population under study in order to test the material. The pilot study (n = 30) was conducted to test if questions were well-written, if objective knowledge questions were adequate, and if the questionnaire procedure was clear. Based on pre-test results, questionnaire was considered appropriate, and therefore was applied to the final sample.
2.3.
Knowledge measures Respondents answered a series of scale items intended to measure their objective and
subjective (i.e. self-reported) knowledge of wine. Objective knowledge was measured using 5
a set of seven multiple-choice questions. These questions were developed with the assistance of three wine experts (two sommeliers and one oenologist), whom were asked to provide questions with different degrees of difficulty. Participants were assigned one point for each correct answer and cero points for incorrect or no answer. Scores from participants ranged from 0 to 7 (Mean = 3.678; S.D. = 1.768). Subjective knowledge was measured with a four-item summated scale adapted from Mitchell and Dacin (1996) using a 5-point Likert scale ranging from 1 to 5, and was found to be highly reliable (α = 0.904). Scores from participants ranged from 4 to 20 (Mean = 11.907; S.D. = 3.676). Then, and as a way to facilitate statistical comparisons, this scale was linearly transformed to a 0 to 7 scale (Mean = 3.459; S.D. = 1.608). Both objective and subjective knowledge scales are presented in Table 1. [Insert Table 1 Here]
3. Results In order to compare the differences between objective and subjective knowledge, respondents were divided into four groups based on their scores of objective knowledge, with subjects performing from zero to one point (correct answers) as “Lowest Objective Knowledge” (n = 28), subjects performing from two or three points as “Low Objective Knowledge” (n = 59), subjects performing from four or five points as “High Objective Knowledge” (n = 70) and subjects performing from six or seven points as “Highest Objective Knowledge” (n = 36). Table 2 shows the mean and standard deviations of objective and subjective knowledge scores for the four groups, whereas Figure 1 examines the relationship between subjective knowledge versus objective knowledge for the four groups. 6
[Insert Table 2 Here] [Insert Figure 1 Here] A paired-samples t-test was conducted within each group to compare the scores of objective and subjective knowledge. Results of these tests showed significant differences in the scores of objective and subjective knowledge for all groups. Our main focus, however, are the extreme groups, defined as those composed by respondents with the lowest and highest levels of Objective Knowledge. For those two groups, results of the paired-samples t-test provide evidence of very significant overestimation of wine knowledge within the “Lowest Objective Knowledge” group (MO-S = -1.069, t = -4.642, p < 0.001), and very significant underestimation of wine knowledge within the “Highest Objective Knowledge” (MO-S = 1.767, t = 8.490, p < 0.001) group. These results provide evidence of the presence of the Dunning–Kruger effect within the wine knowledge domain. Specifically, the results confirm the fact that subjects with low levels of wine knowledge overestimate their knowledge of the category, while individuals with high levels of wine knowledge underestimate it. Additionally, a Pearson product-moment correlation coefficient was computed to assess the relationship between objective and subjective knowledge measures. This analysis shows a significant positive correlation between the two variables (r = 0.507, p < 0.001). This correlation is in line with the results of a meta-analysis of 51 empirical findings associated with the relationship between objective and subjective knowledge performed by Carlson et al. (2009). Results of this meta-analysis show that objective and subjective knowledge are significantly – although moderately – positively related (r=0.37) and that this correlation is stronger for products (r=0.57) versus non‐products (r=0.32).
7
However, it is interesting to note that the shared variance between the two variables is less than 25% (R2 = 0.257). This means that it is possible to assert that merely 25% of the variance in subjective knowledge measures can be attributable to variance in objective knowledge.
4. Discussion and conclusion During the last years, wine-related knowledge differences and their impact in the buying process of wine have been an important area of research, with many studies devoted to identify differences between expert and non-expert consumers in terms of quality perceptions, preferences, information use and processing, among other aspects. Notwithstanding the importance of these studies, a problematic issue related with them is the wide use of subjective or self-reported knowledge measures as a way to distinguish between experts and non-experts consumers. This issue is problematic due to the existence of cognitive biases affecting consumer self-assessment processes, such as the DunningKruger effect. Considering this relevant matter, this study provides strong evidence of the existence of this cognitive bias among wine consumers. The results of the paired-samples t-tests conducted provide evidence of significant differences between subjective knowledge and objective knowledge measures for consumers with the lowest and highest levels of wine knowledge. In particular, and confirming the existence of the Dunning-Kruger effect, consumers with extremely low levels of wine knowledge overestimate their knowledge of the category, whereas consumers with extremely high levels of wine knowledge underestimate their knowledge of the category.
8
Finally, results of the correlation analysis between the two knowledge measures show a low correlation and shared variance between subjective and objective measures of knowledge, increasing the evidence of the problematic use of subjective measures. This study has some limitations. In particular, and although the development process and scores demonstrated that the scale was adequate, the objective knowledge scale utilized is one of many objective knowledge measurement procedures that could be used to assess the amount of product-related knowledge of consumers. While our scale measured knowledge related to wine types, wine consumption, and wine production processes, other scales could focus only in product and consumption related knowledge, excluding wine production knowledge. Additionally, our scale has questions that are country specific, and therefore could be useless in other countries. Therefore, future research could use alternative measurement procedures, such as open-ended questions, or use differentiated measures for objective product class information and objective expertise (Philippe & Ngobo, 1999). This study’s results have important implications for researchers and practitioners. First, and considering the findings related to the poor ability of subjective measures of knowledge to reflect the real level of knowledge of consumers, researchers studying behavioural and attitudinal differences between expert and non-expert consumers should try to favour the use of objective knowledge measures. Second, the results of previous studies that used subjective measures of knowledge to distinguish between expert and non-expert consumers should be interpreted cautiously, considering the fact that the dissimilarity between these groups might be spurious. Finally, practitioners in the wine industry, and especially those involved in market research activities, should start moving from subjective measures of
9
consumer knowledge to objective measurement scales in order to obtain non-biased responses and therefore useful data and insightful results.
References Aqueveque, C. (2015). The influence of experts' positive word-of-mouth on a wine's perceived quality and value: the moderator role of consumers' expertise. Journal of Wine Research, 26, 181-191. Barber, N., Ismail, J., & Dodd, T. (2008). Purchase attributes of wine consumers with low involvement. Journal of Food Products Marketing, 14, 69-86. Carlson, J. P., Vincent, L. H., Hardesty, D. M., & Bearden, W. O. (2008). Objective and subjective knowledge relationships: A quantitative analysis of consumer research findings. Journal of Consumer Research, 35, 864-876. Cox, D. (2009). Predicting consumption, wine involvement and perceived quality of Australian red wine, Journal of Wine Research, 20, 209-229. Dunning, D. (2011). The Dunning-Kruger Effect: On Being Ignorant of One's Own Ignorance. In J. Olson & M. P. Zanna (Eds.), Advances in experimental social psychology Volume 4 (pp. 247-296). New York: Elsevier Gergaud, O., & Livat, F. (2007). How do consumers use signals to assess quality? American Association of Wine Economists Working Paper, 3, 1-22. Hollebeek, L. D., Jaeger, S. R., Brodie, R. J., & Balemi, A. (2007). The influence of involvement on purchase intention for new world wine. Food Quality and Preference, 18, 1033-1049.
10
Johnson, R., & Bruwer, J. (2007). Regional brand image and perceived wine quality: the consumer perspective. International Journal of Wine Business Research, 19, 276297. Kruger, J. M., & Dunning, D. (1999). Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77, 1121-1134. Mitchell, A., & Dacin, P. (1996). The assessment of alternative measures of consumer expertise. Journal of Consumer Research, 23, 219-239. Perrouty, J. P., d’Hauteville, F., & Lockshin, L. (2006). The influence of wine attributes on region of origin equity: an analysis of the moderating effect of consumer’s perceived expertise. Agribusiness, 22, 323-341. Philippe, A., & Ngobo, P. V. (1999). Assessment of consumer knowledge and its consequences: A multi-component approach. In E.J. Arnould & L.M. Scott (Eds.), NA - Advances in Consumer Research Volume 26 (pp. 569-575). Provo, UT: Association for Consumer Research. Piqueras-Fiszman, B., & Spence, C. (2012). The weight of the bottle as a possible extrinsic cue with which to estimate the price (and quality) of the wine? Observed correlations. Food Quality and Preference, 25(1), 41-45. Sáenz-Navajas, M. P., Ballester, J., Peyron, D., & Valentin, D. (2014). Extrinsic attributes responsible for red wine quality perception: a cross-cultural study between France and Spain. Food Quality and Preference, 35, 70-85.
11
Viot, C. (2012). Subjective knowledge, product attributes and consideration set: a wine application. International Journal of Wine Business Research, 24, 219-248.
12
Figure 1. Participants’ Objective and Subjective Knowledge
Scores 6.17 4.44 3.91
3.100 1.89
4.40
2.61
.82 Lowest Objective Knowledge
Low Objective Knowledge
High Objective Knowledge
Objective Knowledge
Highest Objective Knowledge
Subjective Knowledge
13
Table 1. Objective and Subjective Knowledge Scales Objective Knowledge Scalea Q1: Which of the following wine varieties is a red wine? Chardonnay Riesling Merlot Don't Know / Don't Answer Q2: Which of the following wine varieties is produced only in Chile? Cabernet-Sauvignon Carmenere Merlot Don't Know / Don't Answer Q3: Which of the following wine varieties it is usually considered a "full-bodied" wine? Cabernet-Sauvignon Carmenere Merlot Don't Know / Don't Answer Q4: "Letting wine breathe" is: Uncorking a bottle and letting it sit undisturbed for period of time before drink Letting the wine improve its flavour Aerating the wine so it can react with oxygen Don't Know / Don't Answer Q5: Tannins are responsible of wine's: Sweetness Astringency Acidity Don't Know / Don't Answer Q6: Which of the following groups of characteristics is commonly associated to Chardonnay wine? Apple/Pineapple/Melon/Peach/Lemon Tomato/Peppers/Apple/Cherry Strawberry/Vanilla/Lime/Cinnamon Don't Know / Don't Answer Q7: In the production process of wine, racking means: Moving wine from one container to another, to leave any sediment behind Extracting the colour of wine The grape has reached its optimal maturation point Don't Know / Don't Answer Subjective Knowledge Scaleb (α = 0.904) How would you rate your knowledge about wines compared to the rest of the population? (One of the least knowledgeable people – One of the most knowledgeable people) How interested are you in wines? (Not very interested – Very interested) How familiar are you with wines? (Not familiar at all – Extremely familiar) Your knowledge about wines is (Very low – Very high) a. The correct answers are written in bold and italic. b. All items measured using five-point scales ranging from 1 to 5. Anchors are between parentheses.
14
Table 2. Means (standard deviations) of participants’ Objective and Subjective Knowledge
Objective Knowledge Subjective Knowledge
Lowest Objective Knowledge (n = 28) 0.821 (0.390)
Low Objective Knowledge (n = 59) 2.610 (0.492)
High Objective Knowledge (n = 70) 4.443 (0.501)
Highest Objective Knowledge (n = 36) 6.167 (0.378)
1.891 (1.283)
3.100 (1.494)
3.906 (1.431)
4.399 (1.298)
15
Highlights The Dunning-Kruger effect exists among wine consumers. Consumers with low levels of wine knowledge overestimate their knowledge Consumers with high levels of wine knowledge underestimate their knowledge. There is low shared variance between subjective and objective measures of knowledge
16