- Email: [email protected]
Arbitrary Coherence in Theoretical Decision Making About Surgical Training: The Effect of Irrelevant Subliminal Anchoring
ORIGINAL REPORTS
Arbitrary Coherence in Theoretical Decision Making About Surgical Training: The Effect of Irrelevant Subliminal Anchoring David D. Pothier, MBChB, MSc, FRCS (ORL-HNS), DOHNS, Tanya Mangal, MBCHB, BSc (hons), Peter Loizou, BM, BSc (Hons), and Katrina A. Mason, BSc (hons) OBJECTIVE: The extent of surgical training given to under-
COMPETENCY: Professionalism, Interpersonal and Commu-
graduates is variable and the decisions taken on the adequacy of the amount to be delivered are often made using little evidence. Therefore, these decisions are vulnerable to bias. Arbitrary coherence is a recently recognized behavior that can be induced by influencing subjects to anchor their answers to a series of questions to an unrelated anchor. We set out to assess the impact of arbitrary numerical anchors on answers given to questions concerning surgical training.
nication Skills, Medical Knowledge
DESIGN: Prospective Anchoring Study. SETTING: Nonclinical environment. PARTICIPANTS: A group of 41 fourth year medical students. MAIN OUTCOME MEASURES: Each participant was asked to write the last 2 digits of their mobile telephone numbers on the proforma. This was used as an arbitrary anchor to their decisions. They were then asked a series of questions, the answers to which required an estimate to be made. RESULTS: There was a statistically significant correlation (r ⫽ 0.36, p ⫽ 0.04) between the mean of the answers given by each student and the value of the last 2 digits of their mobile telephone number. Despite the fact that the students knew that these numbers could not be related to the questions that were asked, they were influenced by them in their responses. CONCLUSIONS: The introduction of a subliminal anchor affects decision making in participants who are given questions where estimation is required. Simple anchors will substantially sway responses in a subliminal fashion. This pilot study shows how powerful this effect can be. This effect is under-recognized and may be the source of considerable bias in research into training needs. Steps should be taken to guard against this effect in similar surveys. (J Surg 66:129-131. © 2009 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.)
Correspondence: David D. Pothier, MD, Department of Otolaryngology, Southmead Hospital, Westbury-on-Trym BS10 5NB, United Kingdom; e-mail: [email protected]
INTRODUCTION The opportunities for medical undergraduates to obtain a broad-based training in the surgical disciplines is diminishing. There is little evidence concerning how to apportion undergraduate training to the various disciplines that need to be taught as part of the process of becoming a well-rounded doctor, ready to be a generalist as well as ready to enter specialty training. Because of this, decisions need to be made that required judgments to be made. When asking students what they felt they need from an undergraduate curriculum, it is accepted that the responses obtained will be subject to biases inherent in such a system of data collection. These biases, such as career preference, are well known but can be controlled for to a large extent by careful sampling and thereby keep decision-making well informed. It seems, however, from recent research that decisions are easily and substantially affected by arbitrary anchors which are only now being fully explored. Decisions where an estimate must be made of proportions are most likely to be affected by these influences. This makes decision-making concerning the provision of teaching particularly vulnerable. Apart from known sources of bias, recent work has shown that entirely arbitrary forces may influence decision-making where judgment is required. This research has been undertaken largely in the field of behavioral economics, but the implications for medicine are undeniable. In a dramatic example of the phenomenon, Ariely1 undertook research where he showed a significant correlation between the amount students in his economics class would pay for a number of small items and the last 2 digits of their US social security numbers, when these numbers were used as arbitrary anchors. This effect is of particular importance when decision-making is undertaken in the context of education as the estimation of the importance and relative prominence of surgical education is likely to be vulnerable to arbitrary effects. A rational approach is required when undertaking an analysis of the role of surgical education within general medical training and any untoward influence on decision making surrounding this topic should be
Journal of Surgical Education • © 2009 Association of Program Directors in Surgery Published by Elsevier Inc. All rights reserved.
1931-7204/09/$30.00 doi:10.1016/j.jsurg.2009.03.004
129
avoided. Given the strong effect that arbitrary factors have been shown to exert on similar decision making, we undertook a small study to examine any influence that this effect may have on responses to questions on views on surgical training where judgment and estimation was required. Aim To determine the coherence to an arbitrary anchor when medical students estimate answers to questions about the provision of surgical training.
RESULTS Forty-five proformas were collected, 4 participants did not complete their proformas; these results were discarded. Of the remaining 41 participants, all successfully completed all 10 questions. The last 2 digits of the mobile numbers of the participants were distributed in a fairly uniform fashion across the possible range (see Figure 1). Spearman’s rho for the correlation between the last 2 digits of the mobile numbers of the participants and their mean response to the questions was calculated as r ⫽ 0.36 (p ⫽ 0.04). Figure 2 shows the relationship between these 2 variables.
METHODS A group of fourth year medical students at the University of Bristol was invited to take part in a questionnaire-based study into decision-making and estimation. Participants were given a proforma to complete with the space for 10 answers. A series of 10 questions relating to medical students’ views on the provision of surgical training was produced that called for participants to estimate an answer to each question (see Appendix 1). Each question called for a proportion or percentage and had no correct answer as such and the answers represented a range of opinion. These questions formed the basis of the study to determine if responses could be affected by an arbitrary number. Each student was asked to write the last 2 digits of their mobile telephone numbers in the top right hand corner of their proforma. These numbers served as an arbitrary anchor. For each question participants were asked the following questions: 1. Were the last 2 digits of their mobile phone number, taken in the form of a percentage, the answer to the question? 2. If the last 2 digits of their mobile phone number, taken in the form of a percentage, were not the answer to the question:
DISCUSSION This is the first research to investigate the effect of arbitrary anchors on decision-making in surgical education. The results from this preliminary research suggest that the phenomenon of arbitrary coherence need to be investigated as a potential source of bias in the area of educational research. The effect of entirely arbitrary numbers on the answers given by participants to the questions was strong. Although the questions were not taken from validated questionnaires on surgical training, they serve to illustrate how estimates can be swayed when an anchor is provided, no matter how arbitrary the anchoring variable is. Arbitrary coherence has been described in the field of behavioral economics, but has not yet been investigated in the context of healthcare to any great extent. One of the few studies that has used arbitrary numerical anchors in the medical context was undertaken by Brewer,2 where it was demonstrated that both patients and doctors, when given arbitrary anchors, estimated very different levels of risk in a health scenario. Chapman and Johnson3 have postulated that anchors are a psychological reference point that allow our subconscious a focus of comparison that we can use to justify a point of view that we may already hold in our minds. This is supported by work undertaken by
a. Was the answer larger or smaller than this number? b. What was the actual answer to the question? Analysis Data were collected and entered into a custom Access 2003 database (Microsoft, Redmond, US) and exported into SPSS v11.1 (SPSS, Chicago, IL, US) and Stata V10 (Stata, College Station, TX, US). Answers were pooled for each participant and no subgroup or question-specific analysis was undertaken owing to the limited sample size of this pilot group. The mean response for each participant was used to allow the maximum effect from the subliminal anchor to be shown for each participant in a limited sample. Data were tested for normality using the Kolmogorov–Smirnov test and were found to be nonparametrically distributed. Spearman’s rho was used to test for correlation. 130
FIGURE 1. The last 2 digits of the participant’s mobile phone numbers were distributed in a fairly uniform fashion across the possible range.
Journal of Surgical Education • Volume 66/Number 3 • May/June 2009
ACKNOWLEDGMENTS The authors thank Mr. David Grant, MBChB, FRCS (ORLHNS), Dr. Ed Toll, MBChB and Dr. Paula Bradley, MBBS, BA (hons) for their assistance in the production of this report.
REFERENCES 1. Ariely D. Predictably Irrational: The Hidden Forces That
Shape Our Decisions. London: HarperCollins; 2008. 2. Brewer NT, Chapman GB, Schwartz JA, et al. The influ-
ence of irrelevant anchors on the judgments and choices of doctors and patients. Med Decis Mak. 2007;27:203-211. 3. Chapman GB, Johnson EJ. Anchoring, activation, and the FIGURE 2. The relationship between the last 2 digits of the mobile numbers of the participants and their mean response to the questions.
construction of values. Organ Behav Hum Decis Process. 1999;79:115-153. 4. Bodenhausen GV, Gabriel S, Lineberger M. Sadness and
susceptibility to judgmental bias: The case of anchoring. Psychol Sci. 2000;11:320-323.
4
Bodenhausen et al that shows that if an arbitrary anchor is actively considered and made a point of conscious reference it gains in its power to distort decision making, rather than eliminating its effect, as would seem logical. Galdi et al5 analyzed decision making at a conscious level and determined that undecided participants could be assessed subconsciously to show that decisions could be predicted reliably from subconscious responses and that therefore many “undecided” participants had actually made a decision in their subconscious and were merely looking for justification for their choice. This was further supported by Dijksterhuis et al6 who analyzed complex decision making that required a large number of interacting variables; they found that conscious decision making was more likely to result in a regrettable choice than a subconscious one. Given that a number received at random will influence specialist surgeons where no evidence can be relied upon, it is even more likely that recognized anchors will have a more powerful effect. Even where specific anchors are recognized, the effect of anchoring can be so nonspecific that it can influence decision making across modalities.7 Even when an arbitrary factor is recognized as being entirely arbitrary, it is not easy to ignore its subliminal effects.
CONCLUSIONS The risks of arbitrary coherence are new to health science research and pose a very real risk to reliable and rational decisionmaking. These effects on our decision-making and that of our patients are considerable; more research needs to be undertaken to determine the full extent of the influence of this effect on the day-to-day decisions made by doctors and their patients. Strategies to reduce the impact that the effect of arbitrary coherence may have on our choice should be sought.
5. Galdi S, Arcuri L, Gawronski B. Automatic mental associ-
ations predict future choices of undecided decision-makers. Science 2008;321:1100-2. 6. Dijksterhuis A, Bos MW, Nordgren LF, et al. On making
the right choice: The deliberation-without-attention effect. Science. 2006;311:1005-1007. 7. Oppenheimer DM, LeBoeuf RA, Brewer NT. Anchors
aweigh: A demonstration of cross-modality anchoring and magnitude priming. Cognition. 2008;106:13-26.
APPENDIX 1 Questions Asked of Participants 1. What proportion of surgeons decide on surgery as a career while in medical school? 2. What proportion of clinical time should be spent in a surgical discipline? 3. What proportion of time spent by students in a surgical discipline should be spent in the operating theater? 4. What proportion of time spent by students in the operating theater should be spent scrubbed into the surgery? 5. What proportion of medical student surgical training should be spent obtaining basic surgical skills? 6. What proportion of consultant surgeons are studentfriendly? 7. What proportion of surgical lists should be dedicated to training 8. What proportion of surgeons should be allowed to have part-time training? 9. What proportion of surgical placements were encouraging about medical students becoming surgeons? 10. What proportion of surgeons do you think are happy in their jobs?
Journal of Surgical Education • Volume 66/Number 3 • May/June 2009
131
Arbitrary Coherence in Theoretical Decision Making About Surgical Training: The Effect of Irrelevant Subliminal Anchoring David D. Pothier, MBChB, MSc, FRCS (ORL-HNS), DOHNS, Tanya Mangal, MBCHB, BSc (hons), Peter Loizou, BM, BSc (Hons), and Katrina A. Mason, BSc (hons) OBJECTIVE: The extent of surgical training given to under-
COMPETENCY: Professionalism, Interpersonal and Commu-
graduates is variable and the decisions taken on the adequacy of the amount to be delivered are often made using little evidence. Therefore, these decisions are vulnerable to bias. Arbitrary coherence is a recently recognized behavior that can be induced by influencing subjects to anchor their answers to a series of questions to an unrelated anchor. We set out to assess the impact of arbitrary numerical anchors on answers given to questions concerning surgical training.
nication Skills, Medical Knowledge
DESIGN: Prospective Anchoring Study. SETTING: Nonclinical environment. PARTICIPANTS: A group of 41 fourth year medical students. MAIN OUTCOME MEASURES: Each participant was asked to write the last 2 digits of their mobile telephone numbers on the proforma. This was used as an arbitrary anchor to their decisions. They were then asked a series of questions, the answers to which required an estimate to be made. RESULTS: There was a statistically significant correlation (r ⫽ 0.36, p ⫽ 0.04) between the mean of the answers given by each student and the value of the last 2 digits of their mobile telephone number. Despite the fact that the students knew that these numbers could not be related to the questions that were asked, they were influenced by them in their responses. CONCLUSIONS: The introduction of a subliminal anchor affects decision making in participants who are given questions where estimation is required. Simple anchors will substantially sway responses in a subliminal fashion. This pilot study shows how powerful this effect can be. This effect is under-recognized and may be the source of considerable bias in research into training needs. Steps should be taken to guard against this effect in similar surveys. (J Surg 66:129-131. © 2009 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.)
Correspondence: David D. Pothier, MD, Department of Otolaryngology, Southmead Hospital, Westbury-on-Trym BS10 5NB, United Kingdom; e-mail: [email protected]
INTRODUCTION The opportunities for medical undergraduates to obtain a broad-based training in the surgical disciplines is diminishing. There is little evidence concerning how to apportion undergraduate training to the various disciplines that need to be taught as part of the process of becoming a well-rounded doctor, ready to be a generalist as well as ready to enter specialty training. Because of this, decisions need to be made that required judgments to be made. When asking students what they felt they need from an undergraduate curriculum, it is accepted that the responses obtained will be subject to biases inherent in such a system of data collection. These biases, such as career preference, are well known but can be controlled for to a large extent by careful sampling and thereby keep decision-making well informed. It seems, however, from recent research that decisions are easily and substantially affected by arbitrary anchors which are only now being fully explored. Decisions where an estimate must be made of proportions are most likely to be affected by these influences. This makes decision-making concerning the provision of teaching particularly vulnerable. Apart from known sources of bias, recent work has shown that entirely arbitrary forces may influence decision-making where judgment is required. This research has been undertaken largely in the field of behavioral economics, but the implications for medicine are undeniable. In a dramatic example of the phenomenon, Ariely1 undertook research where he showed a significant correlation between the amount students in his economics class would pay for a number of small items and the last 2 digits of their US social security numbers, when these numbers were used as arbitrary anchors. This effect is of particular importance when decision-making is undertaken in the context of education as the estimation of the importance and relative prominence of surgical education is likely to be vulnerable to arbitrary effects. A rational approach is required when undertaking an analysis of the role of surgical education within general medical training and any untoward influence on decision making surrounding this topic should be
Journal of Surgical Education • © 2009 Association of Program Directors in Surgery Published by Elsevier Inc. All rights reserved.
1931-7204/09/$30.00 doi:10.1016/j.jsurg.2009.03.004
129
avoided. Given the strong effect that arbitrary factors have been shown to exert on similar decision making, we undertook a small study to examine any influence that this effect may have on responses to questions on views on surgical training where judgment and estimation was required. Aim To determine the coherence to an arbitrary anchor when medical students estimate answers to questions about the provision of surgical training.
RESULTS Forty-five proformas were collected, 4 participants did not complete their proformas; these results were discarded. Of the remaining 41 participants, all successfully completed all 10 questions. The last 2 digits of the mobile numbers of the participants were distributed in a fairly uniform fashion across the possible range (see Figure 1). Spearman’s rho for the correlation between the last 2 digits of the mobile numbers of the participants and their mean response to the questions was calculated as r ⫽ 0.36 (p ⫽ 0.04). Figure 2 shows the relationship between these 2 variables.
METHODS A group of fourth year medical students at the University of Bristol was invited to take part in a questionnaire-based study into decision-making and estimation. Participants were given a proforma to complete with the space for 10 answers. A series of 10 questions relating to medical students’ views on the provision of surgical training was produced that called for participants to estimate an answer to each question (see Appendix 1). Each question called for a proportion or percentage and had no correct answer as such and the answers represented a range of opinion. These questions formed the basis of the study to determine if responses could be affected by an arbitrary number. Each student was asked to write the last 2 digits of their mobile telephone numbers in the top right hand corner of their proforma. These numbers served as an arbitrary anchor. For each question participants were asked the following questions: 1. Were the last 2 digits of their mobile phone number, taken in the form of a percentage, the answer to the question? 2. If the last 2 digits of their mobile phone number, taken in the form of a percentage, were not the answer to the question:
DISCUSSION This is the first research to investigate the effect of arbitrary anchors on decision-making in surgical education. The results from this preliminary research suggest that the phenomenon of arbitrary coherence need to be investigated as a potential source of bias in the area of educational research. The effect of entirely arbitrary numbers on the answers given by participants to the questions was strong. Although the questions were not taken from validated questionnaires on surgical training, they serve to illustrate how estimates can be swayed when an anchor is provided, no matter how arbitrary the anchoring variable is. Arbitrary coherence has been described in the field of behavioral economics, but has not yet been investigated in the context of healthcare to any great extent. One of the few studies that has used arbitrary numerical anchors in the medical context was undertaken by Brewer,2 where it was demonstrated that both patients and doctors, when given arbitrary anchors, estimated very different levels of risk in a health scenario. Chapman and Johnson3 have postulated that anchors are a psychological reference point that allow our subconscious a focus of comparison that we can use to justify a point of view that we may already hold in our minds. This is supported by work undertaken by
a. Was the answer larger or smaller than this number? b. What was the actual answer to the question? Analysis Data were collected and entered into a custom Access 2003 database (Microsoft, Redmond, US) and exported into SPSS v11.1 (SPSS, Chicago, IL, US) and Stata V10 (Stata, College Station, TX, US). Answers were pooled for each participant and no subgroup or question-specific analysis was undertaken owing to the limited sample size of this pilot group. The mean response for each participant was used to allow the maximum effect from the subliminal anchor to be shown for each participant in a limited sample. Data were tested for normality using the Kolmogorov–Smirnov test and were found to be nonparametrically distributed. Spearman’s rho was used to test for correlation. 130
FIGURE 1. The last 2 digits of the participant’s mobile phone numbers were distributed in a fairly uniform fashion across the possible range.
Journal of Surgical Education • Volume 66/Number 3 • May/June 2009
ACKNOWLEDGMENTS The authors thank Mr. David Grant, MBChB, FRCS (ORLHNS), Dr. Ed Toll, MBChB and Dr. Paula Bradley, MBBS, BA (hons) for their assistance in the production of this report.
REFERENCES 1. Ariely D. Predictably Irrational: The Hidden Forces That
Shape Our Decisions. London: HarperCollins; 2008. 2. Brewer NT, Chapman GB, Schwartz JA, et al. The influ-
ence of irrelevant anchors on the judgments and choices of doctors and patients. Med Decis Mak. 2007;27:203-211. 3. Chapman GB, Johnson EJ. Anchoring, activation, and the FIGURE 2. The relationship between the last 2 digits of the mobile numbers of the participants and their mean response to the questions.
construction of values. Organ Behav Hum Decis Process. 1999;79:115-153. 4. Bodenhausen GV, Gabriel S, Lineberger M. Sadness and
susceptibility to judgmental bias: The case of anchoring. Psychol Sci. 2000;11:320-323.
4
Bodenhausen et al that shows that if an arbitrary anchor is actively considered and made a point of conscious reference it gains in its power to distort decision making, rather than eliminating its effect, as would seem logical. Galdi et al5 analyzed decision making at a conscious level and determined that undecided participants could be assessed subconsciously to show that decisions could be predicted reliably from subconscious responses and that therefore many “undecided” participants had actually made a decision in their subconscious and were merely looking for justification for their choice. This was further supported by Dijksterhuis et al6 who analyzed complex decision making that required a large number of interacting variables; they found that conscious decision making was more likely to result in a regrettable choice than a subconscious one. Given that a number received at random will influence specialist surgeons where no evidence can be relied upon, it is even more likely that recognized anchors will have a more powerful effect. Even where specific anchors are recognized, the effect of anchoring can be so nonspecific that it can influence decision making across modalities.7 Even when an arbitrary factor is recognized as being entirely arbitrary, it is not easy to ignore its subliminal effects.
CONCLUSIONS The risks of arbitrary coherence are new to health science research and pose a very real risk to reliable and rational decisionmaking. These effects on our decision-making and that of our patients are considerable; more research needs to be undertaken to determine the full extent of the influence of this effect on the day-to-day decisions made by doctors and their patients. Strategies to reduce the impact that the effect of arbitrary coherence may have on our choice should be sought.
5. Galdi S, Arcuri L, Gawronski B. Automatic mental associ-
ations predict future choices of undecided decision-makers. Science 2008;321:1100-2. 6. Dijksterhuis A, Bos MW, Nordgren LF, et al. On making
the right choice: The deliberation-without-attention effect. Science. 2006;311:1005-1007. 7. Oppenheimer DM, LeBoeuf RA, Brewer NT. Anchors
aweigh: A demonstration of cross-modality anchoring and magnitude priming. Cognition. 2008;106:13-26.
APPENDIX 1 Questions Asked of Participants 1. What proportion of surgeons decide on surgery as a career while in medical school? 2. What proportion of clinical time should be spent in a surgical discipline? 3. What proportion of time spent by students in a surgical discipline should be spent in the operating theater? 4. What proportion of time spent by students in the operating theater should be spent scrubbed into the surgery? 5. What proportion of medical student surgical training should be spent obtaining basic surgical skills? 6. What proportion of consultant surgeons are studentfriendly? 7. What proportion of surgical lists should be dedicated to training 8. What proportion of surgeons should be allowed to have part-time training? 9. What proportion of surgical placements were encouraging about medical students becoming surgeons? 10. What proportion of surgeons do you think are happy in their jobs?
Journal of Surgical Education • Volume 66/Number 3 • May/June 2009
131