- Email: [email protected]
To operate or not to operate? A multi-method analysis of decision-making in emergency surgery
The American Journal of Surgery (2010) 200, 298 –304
Surgical Education
To operate or not to operate? A multi-method analysis of decision-making in emergency surgery Peter Szatmary, M.B.B.Chir., M.Ed., M.A.,a,* Sonal Arora, B.Sc., M.B.B.S., M.R.C.S.,a Nick Sevdalis, B.Sc., M.Sc., Ph.D.a,b a
Department of Biosurgery and Surgical Technology and bImperial Centre for Patient Safety and Service Quality, Imperial College of Medicine, London, UK KEYWORDS: Decision making; Emergency surgery; Qualitative outcome assessment; Surgical education
Abstract BACKGROUND: The ability to decide when to operate and when not to operate is a key surgical skill. The aim of this study was to investigate factors affecting that decision. METHODS: In phase 1, semistructured interviews were used to investigate how expert surgeons decide when to operate. In phase 2, clinical case vignettes were constructed, and 22 general surgeons at various stages of their training indicated whether they would operate and their confidence in patient outcomes. RESULTS: Interviews answers centered on the theme of “patient outcome,” which was defined similarly by all surgeons. In phase 2, surgeons chose to operate when they perceived the outcome with an operation to be better than the outcome without. Surgeons with ⬍5 years of experience were less certain about what outcomes might be. These surgeons opted to perform significantly more operations (40 ⫾ 4%) than surgeons with ⱖ5 years of experience (18 ⫾ 2%). CONCLUSIONS: A subjective, balanced assessment of the likelihood of patient outcome is crucial in deciding whether to operate. Novices face higher degrees of uncertainty, explaining differences in decisions taken. © 2010 Elsevier Inc. All rights reserved.
Research into surgical decision making is often carried out with the aim of producing consensus or guidelines for the management of common surgical conditions such as appendicitis1 or cholecystitis.2,3 More recently, qualitative “decision mapping” has been used to investigate and identify the decisions involved in caring for a surgical patient from point of admission to point of discharge,4 and quanDr Sevdalis is a member of the Imperial Centre for Patient Safety and Service Quality at Imperial College Healthcare NHS Trust, which is funded by the National Institute of Health Research. Dr Sevdalis is also funded by the Economic and Social Research Council Centre for Economic Learning and Social Evolution. * Corresponding author: Tel.: 44-789-6223424; fax: 44-151-7065828. E-mail address: [email protected] Manuscript received June 1, 2009; revised manuscript October 20, 2009
0002-9610/$ - see front matter © 2010 Elsevier Inc. All rights reserved. doi:10.1016/j.amjsurg.2009.10.020
titative approaches from behavioral science have been successfully applied to modeling surgical judgment.5,6 To date, little research has been carried out into the thought processes of surgeons’ decisions of whether or not to operate in given situations. Despite this, the decision of whether to operate is seen by many as fundamental to the practice of surgery and the ability to make that decision appropriately as a key tenet of surgical skill.7 Indeed, in the well-renowned phrase from Kirk’s textbook of general surgical operations,8 the best surgeon is one who “knows when not to operate.” Understanding the factors that influence this decision-making process is critical if this hallmark of expertise is to be distilled into a teachable format. The aim of this study was to investigate the process involved in deciding whether a patient should be operated on or not in an emergency setting. Specifically, the first aim
P. Szatmary et al. Table 1
Making the decision to operate
299
Decision to operate by surgical outcome matrix Poor outcome with operation
Poor Outcome Without Operation Good Outcome Without Operation
Good outcome with operation
Scenario 1: elderly woman, multiple comorbidities, Scenario 2: small bowel obstruction in fit young adult, perforated ulcer disease, septic shock 14-h history, early signs of shock Scenario 3: elderly patient, fresh rectal bleed from Scenario 4: middle-aged adult, arteriopathy, severe likely diverticulosis, hemodynamically stable, no abdominal pain of unclear origin, no signs of signs of peritonism hemodynamic instability
was to elucidate factors considered important to the decision to operate (phase 1). The second aim was to investigate if and how these factors applied in clinical practice using surgical scenarios (phase 2).
Methods Design This was a prospective, descriptive, multimethod study. In phase 1 of the study, semistructured interviews with expert surgeons were carried out to delineate the key factors influencing the decision to operate. In the light of the findings of phase 1, in phase 2 of the study, 5 clinical scenarios (vignettes) were constructed following an approach taken by other researchers,9 and participating surgeons were asked to indicate their decisions to operate or not in each of them.
ate. The complete list of questions can be found in Appendix A. Interviews lasted between 40 and 50 minutes and were audio-recorded. They were then transcribed verbatim and checked for accuracy against original recordings. In phase 2, 5 clinical scenarios were constructed (Appendix B). The scenarios were designed to reflect 4 common general surgical problems according to a 2 ⫻ 2 “decision to operate”– by–“surgical outcome” matrix (Table 1). A fifth scenario based on an orthopedic emergency (septic arthritis) was used as a control scenario. Participating surgeons were instructed to read the scenarios carefully and indicate whether they would operate (yes or no), what they felt the expected outcome would be with and without an operation (on a visual analogue scale [VAS] ranging from 0 to 100 with 5% intervals anchored at “poor outcome” and “good outcome”), and their confidence in the outcome (on a VAS ranging from 0 to 100 with 5% intervals anchored at “not at all confident” and “very confident”).
Data analysis Participants Interviews (phase 1) were conducted with 3 attending surgeons (2 general/colorectal, 1 orthopedic; all surgeons had ⱖ15 years of surgical experience) across 2 large UK district general hospitals. Twenty-two additional surgeons at various stages of their general surgical training were presented with the clinical scenarios (phase 2). Participants were grouped into those with ⬎5 years of experience, not counting their year of internship (the expert group, n ⫽ 12; range, 6 –34 years) and those with ⱕ5 years of experience (the novice group, n ⫽ 10; range, 0 –5 years). Five years was the average length of specialty training for general surgery in the United Kingdom and was therefore chosen as transition point between novice and expert surgeons.
Interview transcripts were content analyzed and emerging themes tabulated. The themes that emerged from the interviews were fed back to participants in the form of a presentation to validate the researchers’ findings (memberchecking validation technique for qualitative studies). Participants’ responses to the scenarios were measured on the VAS. Answers to every clinical scenario were analyzed as independent events, therefore leading to analysis of 110 data sets (22 participants, 5 scenarios). All data were analyzed using Microsoft Excel spreadsheets (Microsoft Corporation, Redmond, WA). Analysis of variance was used to assess differences between expert and novice participants. Obtained findings were deemed significant at P ⬍ .05.
Results Research materials and procedure In phase 1, the interview protocol consisted of a set of open-ended questions relating to extreme surgical situations (eg, “Describe a situation where an operation is undoubtedly necessary and one where it is likely to be harmful”). Reasons for not offering surgery immediately were explored (eg, “waiting for imaging results”), and attending surgeons were asked to describe their thought processes in making a decision to oper-
We first describe the findings from the interviews, followed by the findings from the clinical scenarios.
Factors affecting the decision to operate Table 2 summarizes the common themes influencing the decision to operate and gives example quotations from the interviews. The following themes emerged.
300 Table 2
The American Journal of Surgery, Vol 200, No 2, August 2010 Common factors influencing the decision to operate
Theme
Comments
Patient outcome
“The first consideration is the clinical condition of the patient. What the patient requires in order to make them well.” “(You have to ensure) you’re doing no further harm to the patient (when deciding to operate) and are trying to minimise the trauma that they experience.” “. . . those with large bowel obstruction (may require urgent operation). You have a patient who has a hugely distended caecum and the outcome would go from very good if you operate within a few hours . . . to one where they’ll almost certainly die if the caecum ruptures and they have generalised faecel peritonitis.” “. . . in the olden days . . . you open them up, ’cause they have generalised peritonitis and you don’t know what it is . . . nowadays we’d do a CT and get the diagnosis.” “Some fracture-neck-of-femur patients who need medically optimising (may benefit from delaying surgery). Sometimes it is better to delay (and) treat a chest infection.” “You want to optimise a patient’s cardio-vascular status . . . where you have a patient who could develop renal failure requiring renal support even . . . because you’ve piled in to them too soon before catching up with the fluid that they’ve lost inside the abdomen . . . you don’t do them any good.”
Disease progression Diagnostic certainty Resuscitation
Patient outcome. Surgery was thought to be positive if the condition of the patient improves because of it. An improvement in the condition of the patient was described as depending on a complex set of factors, including reduced pain, reduced chance of death, and improved function of a limb. Disease progression. Surgery was similarly described as positive if it slows or halts natural disease progression, thereby stabilizing “patient outcome” at the status quo. Diagnostic certainty. If there is serious doubt in the diagnosis and a potential for a disorder that may lead to a significant deterioration in patient outcome, an intervention was seen as indicated despite it being of potentially no benefit. Similarly, a usually indicated intervention might be considered detrimental if the diagnosis is not as expected, in which case delaying the procedure for further investigation was seen as the best choice.
Resuscitation. If current physiologic parameters (eg, circulatory volume, cardiac output, renal perfusion, coagulopathies) are seen to have a negative impact on operative outcome but can be optimized, a period of resuscitation was seen as positive before operating, and in this circumstance, the optimal decision was to delay the operation.
Decision to operate in the clinical scenarios Figure 1 shows how participants’ confidence in their judgment varied with experience. In this figure, “confidence” is the level of certainty indicated by participants about a given outcome on the VAS of the questionnaire. There is a rapid, significant increase in perceived confidence about outcome with or without operation in the first 3 to 5 years of surgical training (P ⫽ .03). This plateaus at a level of about 70% to 80% certainty, even with ⬎30 years of experience. However, there appears to be a cohort whose
Figure 1 Plot of percentage confidence versus surgeon experience in years. Confidence values are taken from VAS scores from questionnaires, on which surgeons were specifically asked to state the probability of patient outcome in a given scenario and then to self-assess how certain they were about patient outcome with (triangles) or without (crosses) an operation.
P. Szatmary et al.
Making the decision to operate
confidence peaks at about 10 years of training, before slowly declining. This may represent the point at which trainees become fully unsupervised, independent practitioners without the constant support of a senior colleague. To capture simultaneously the impact of surgeons’ judgment of outcome quality with or without surgery and their subjective confidence in that outcome, we computed the arithmetic product of their outcome perception (poor vs good) times surgeons’ self-reported confidence (low vs high). This simple algebraic manipulation reveals how surgeons’ views of potential outcome and their certainty in these views determine their decision to operate or not. Figure 2 summarizes our findings. All values above the diagonal line indicate that the outcome with an operation is perceived as better than the outcome without an operation and vice versa. In most cases in which surgeons stated they would operate, their perception markers fall above this line. Looking specifically at cases in which surgeons stated they would choose to operate, “likelihood of positive outcome with operation” values are significantly higher than those of cases where surgeons chose not to operate (51 ⫾ 5% vs 31 ⫾ 3%, P ⫽ .002). Similarly, “likelihood of positive outcome without operation” values were significantly lower (21 ⫾ 3% vs 48 ⫾ 3%, P ⬍ .001). Novice surgeons judged the likelihood of a positive outcome with operation similarly to expert surgeons (56% ⫾ 5% vs 55 ⫾ 8%, P ⫽ .46) but tended to judge the likelihood of a positive outcome without operation worse than experts, although this finding just failed to reach significance (49 ⫾ 8% vs 66 ⫾ 6%, P ⫽ .06). Novice surgeons also significantly favored operative approaches compared with experts: given the same scenarios, novice surgeons opted to operate
301 40 ⫾ 4% of the time, whereas experts would operate only 18 ⫾ 2% of the time (P ⫽ .001).
Comments The findings of both studies suggest that the perceived likelihood of a positive outcome given a certain intervention is fundamental in making the decision whether to operate or not. This is seen when attending surgeons state this directly with little or no prompting (phase 1 interviews) and when surgeons are seen to choose to operate when they perceive an outcome without an operation to be worse than one with an operation (phase 2 vignettes). It is important to note that a surgeon’s confidence in outcome with or without an operation increases rapidly in the first 3 to 5 years of training and also that certainty appears to plateau at 70% to 80%. It is not surprising that with increasing experience and increasing diagnostic skill such confidence increases. It is also not surprising that there is a plateau in confidence, as surgeons would rarely claim to be 100% correct in any patient. One should note, however, that the higher level of confidence may represent falsely elevated self-confidence in expert surgeons. There is also a cohort of surgeons with about 10 years of experience who appear to show declining confidence. This may reflect the knowledge that they are no longer supervised and therefore entirely responsible for their actions. Alternatively, it may represent a shift in educational style, as younger surgeons undergo modern training that focuses on self-evaluation and reflective practice. These questions might be answered by repeating the exercise with real cases, where
Figure 2 Plot of patient outcome with operation ⫻ certainty of that outcome versus patient outcome without operation ⫻ certainty of that outcome as stated by surgeons on a questionnaire relating to a specific clinical scenario. Surgeons were also asked to decide whether they would perform urgent operations on the patients in the scenarios: square data points represent decisions to operate and diamond data points decisions not to operate. The hashed line is the line of unity, where the perceived outcome with an operation equals the perceived outcome without an operation.
302 actual eventual outcomes could be measured and compared with outcomes predicted by surgeons. This finding does, however, have other important implications for clinical practice. It may suggest that surgeons treating uncommon conditions outside specialist centers may offer operations more readily than their subspecialist colleagues. This prediction could also be tested in future work. It appears that the product of predicted outcome and confidence in that outcome correlates very well with the decision to operate or not. This means that the more certain a surgeon is that an outcome with and operation will be better than one without, the more likely that surgeon is to propose an operation. This might mean that surgeons could refine their proposed treatments by using their own collected outcome data to guide their decision making, thus increasing their own confidence in any given outcome. Interestingly, novices tended to perceive outcomes without operation as worse than experts in our scenarios. This may lead to novice surgeons’ opting to perform operations in a far greater number of cases. In addition, novice surgeons were less sure about what an outcome might be with or without an operation than their expert peers. This might in part be explained by the desire to do “something” in view of an uncertain future. Although this study is not designed to address this particular point, these findings may indicate that novice surgeons may offer operations when experts deem them unnecessary, even though they think they are acting in patients’ best interest. If and how this finding translates to actual practice remains to be demonstrated in larger scale studies, but one can see how patient safety concerns may arise in this setting. An additional limitation to the generalizability of this finding is that in this study novice and expert surgeons responded independently, whereas in practice they would work as part of a team. Further limitations of this study include the relatively small sample size. Nevertheless, it should be noted that given the small sample, there appear to be significant differences in decision making between novice and expert surgeons. Also, the qualitative design of the study may be a limitation in that it attempts to quantify data by asking individuals to self-assess. This may be misleading if self-assessment skills vary between novices and experts for some reason (eg, differences in medical school curricula) and may also represent idealized actions instead of those taken in the real world. Quantitative studies with larger sample sizes are needed to further assess the validity of the findings, but the qualitative data presented here provides some insight into the thought processes involved. Further research should examine whether these findings translate into clinical practice and if confidence in outcome can be specifically developed in ways other than by passive gaining of experience. An important next step would be to investigate the clinical transferability of these findings by using real patient encounters as opposed to clinical scenarios.
The American Journal of Surgery, Vol 200, No 2, August 2010 In conclusion, appropriate decision making in determining whether or not operate is crucial to high-quality care. Surgeons evidently decide to operate when they perceive patient outcomes with an operation to be better than patient outcomes without an operation. The main difference between experts and novices is that experts are more certain about their assessments, probably a feature of their greater experience. This could explain the increased readiness to operate in novice surgeons, who might feel the need to “do something” in the absence of certainty of disease progression. Understanding the factors that influence this critical decision and the uncertainties faced by novices is an important first step toward better training, improved decision making, and ultimately safer patient care.
References 1. Kharbanda AB, Taylor GA, Fishman SJ, et al. A clinical decision rule to identify children at low risk for appendicitis. Paediatrics 2005;116:709 –16. 2. Miura F, Takada T, Kawarada Y, et al. Flowcharts for the diagnosis and treatment of acute cholangitis and cholecystitis. J Hepatobiliary Pancreat Surg 2007;14:27–34. 3. Weiland DE, Caruso DM, Kassir A, et al. Using delta/DRG diagrams and decision tree analysis to select a cost-effective surgery for cholecystitis. JSLS 1997;1:175– 80. 4. Jacklin R, Sevdalis N, Harries C, et al. Judgment analysis: a method for quantitative evaluation of trainee surgeons’ judgments of surgical risk. Am J Surg 2008;195:183– 8. 5. Jacklin R, Sevdalis N, Darzi A, et al. Efficacy of cognitive feedback in improving operative risk estimation. Am J Surg 2009;197:76 – 81. 6. Jacklin R, Sevdalis N, Darzi A, et al. Mapping surgical practice decision making: an interview study to evaluate decisions in surgical care. Am J Surg 2008;195:689 –96. 7. Spencer FC, The Gibbon lecture— competence and compassion: two qualities of surgical excellence. Bull Am Coll Surg 1979;64:15–22. 8. Kirk RM, editor General surgical operations. 5th ed. New York: Elsevier; 2006. 9. MacCormick AD, Parry BR, Judgement analysis of surgeons’ prioritization of patients for elective general surgery. Med Decis Making 2006;26:255– 64.
Appendix A: Interview Protocol 1. Of concern is the decision to operate or not to operate in emergency surgery of your specialty field 2. Please describe a patient presenting with an acute problem who will need urgent surgery (same day) 3. Please describe a patient presenting with an acute problem who would come to harm if operated on urgently (same day) 4. What would have to change or become apparent for urgent surgery to be postponed/cancelled in the case of the first patient you described 5. What would have to change or become apparent for urgent surgery to be immediately organised for the second patient you described 6. Describe the decision/decisions that you make when reaching the conclusion that urgent surgery is required
P. Szatmary et al.
Making the decision to operate
7. Are there any factors that you have known to alter the above decision making process in any way? 8. Describe the factors that you take into consideration when reaching the conclusion that urgent surgery is required. How would you categorise/group these factors? 9. Do your colleagues or junior staff ever change your decision about the necessity of urgent surgery? 10. Do biochemical markers and/or imaging techniques ever change your decision about the necessity of urgent surgery - if yes, in what type of scenario do you think they are particularly effective? 11. Are you guided mainly by your own experience, or mainly by clinical guidelines and published evidence? Can you think of a situation where that balance is shifted in any way? 12. How do external pressures (financial, staff-related, timerelated) affect your decision to perform an urgent operation? 13. Are there any other points relating to the way you reach a decision about whether to operate in an acute emergency that you would like to make?
Appendix B: Scenarios Background You are a surgeon at your current level of training at a large district general hospital. It is late afternoon and routine services (ie clinics, routine lists, routine imaging services) have ceased, however out of hours imaging and theatre staff are available. It would take between 30 and 120 minutes to perform a CT scan and between 15 and 30 minutes to prepare theatres if no concurrent operation is in progress. There is a Consultant/Attending led general anaesthetic service and a 6 bed ITU/HDU. Specialist intensive care (neurosurgical, paediatric, burns) is available at the tertiary referral centre 80 minutes away.
303 Blood tests reveal a mild renal impairment, but are otherwise normal. An erect chest XR reveals overdistended lung fields as well as a small amount of free intra-peritoneal air under the right hemi-diaphragm.
Scenario 2 A previously fit and well 32 year old man attends A&E with a 14 hour history of worsening abdominal pain and severe, unrelenting bilious vomiting. He has last opened his bowels this morning. He has been unable to eat or drink anything all day due to his severe nausea and recurrent vomiting. He is a keen sportsman; currently training for national swimming championships. He has only been in hospital once before - when he had an operation for abdominal sepsis secondary to a perforated appendix 10 years ago. He is on no regular medication and has no allergies. Observations are as follows: Pulse rate 102bpm regular, BP 116/83, respiratory rate 24 (saturation 100% on air). Abdominal examination reveals a slightly distended and tympanic abdomen. Abdominal X-ray shows several distended loops of small bowel with no gas in the rectum. You are called to see the patient for a surgical opinion.
Scenario 3 A GP refers a 63 year old lady with a 2 day history of per-rectum bleeding associated with lower abdominal pain. She is not complaining of nausea and has had her bowels open twice today - stools are loose in consistency and mixed with both dark and bright red blood. Her past medical history includes angina, hypertension and a myocardial infarction 4 years ago, from which she has recovered well. She is independent and is the main carer for her elderly father. Observations are: pulse rate 88 regular, BP 89/36, respiratory rate 26 (saturation 100% on air). The patient appears pale and slightly anxious. Abdominal examination reveals a soft abdomen with tenderness in the left lower quadrant and suprapubic area. Plain chest and abdominal radiographs are unremarkable.
Scenario 1 Scenario 4 You are asked to see an 86 year old lady who presents with an 18 hour history of severe epigastric pains and vomiting. She has been transferred from a residential home where she has spent the last 6 years due to mild memory problems. She has no living relatives. She has a past medical history of COPD requiring home nebulisers, but not home oxygen. She is also on long term oral steroid therapy due to giant cell arteritis. Nursing observations are as follows: a pulse rate of 92 bpm, irregularly irregular, a blood pressure of 147/72, a respiratory rate of 14 (saturation 93% on air) and a temperature of 36.7°C. The patient appears comfortable after 5 mg of iv morphine. Abdominal examination reveals localised epigastric peritonism.
You are asked to see a 52 year old man who presents to the emergency services with a 3 hours history of sudden onset severe central and upper abdominal pain associated with repeated vomiting and relentless diarrhoea. He is an investment banker and smoker of 40 cigarettes per day and drinks between 15 and 25 units of alcohol per week. He is clinically obese. He has recently been diagnosed as suffering from angina and an angiogram from 8 months ago confirms 2 vessel disease. Observations are: pulse rate 112 regular, BP 98/60, respiratory rate 28 (saturation 100% on air). Abdominal examination reveals mild peri-umbilical and epigastric tenderness with no signs of peritonism. Plain chest and abdominal
304 radiography are unremarkable except for a suspicious absence of bowel gas.
Scenario 5 A 5 year old girl presents to the emergency department having fallen off a trampoline, complaining of severe pain in her right elbow and supporting her right arm with the left hand. She has no known medical problems. She is right hand dominant. Unfortunately the emergency department has
The American Journal of Surgery, Vol 200, No 2, August 2010 been very busy and by the time you get the referral it is 02:00 in the morning. The girls elbow is now supported in a plaster slab. On examination you find her hand cool to touch, but pulses are present and sensation and motor function is present, but limited by pain. She is also complaining of some altered sensation over the dorsum of the first web space. X ray of the elbow confirms your suspicion of a supracondylar fracture of her right humerus. It is a closed injury with minimal displacement on plain radiographs.
Surgical Education
To operate or not to operate? A multi-method analysis of decision-making in emergency surgery Peter Szatmary, M.B.B.Chir., M.Ed., M.A.,a,* Sonal Arora, B.Sc., M.B.B.S., M.R.C.S.,a Nick Sevdalis, B.Sc., M.Sc., Ph.D.a,b a
Department of Biosurgery and Surgical Technology and bImperial Centre for Patient Safety and Service Quality, Imperial College of Medicine, London, UK KEYWORDS: Decision making; Emergency surgery; Qualitative outcome assessment; Surgical education
Abstract BACKGROUND: The ability to decide when to operate and when not to operate is a key surgical skill. The aim of this study was to investigate factors affecting that decision. METHODS: In phase 1, semistructured interviews were used to investigate how expert surgeons decide when to operate. In phase 2, clinical case vignettes were constructed, and 22 general surgeons at various stages of their training indicated whether they would operate and their confidence in patient outcomes. RESULTS: Interviews answers centered on the theme of “patient outcome,” which was defined similarly by all surgeons. In phase 2, surgeons chose to operate when they perceived the outcome with an operation to be better than the outcome without. Surgeons with ⬍5 years of experience were less certain about what outcomes might be. These surgeons opted to perform significantly more operations (40 ⫾ 4%) than surgeons with ⱖ5 years of experience (18 ⫾ 2%). CONCLUSIONS: A subjective, balanced assessment of the likelihood of patient outcome is crucial in deciding whether to operate. Novices face higher degrees of uncertainty, explaining differences in decisions taken. © 2010 Elsevier Inc. All rights reserved.
Research into surgical decision making is often carried out with the aim of producing consensus or guidelines for the management of common surgical conditions such as appendicitis1 or cholecystitis.2,3 More recently, qualitative “decision mapping” has been used to investigate and identify the decisions involved in caring for a surgical patient from point of admission to point of discharge,4 and quanDr Sevdalis is a member of the Imperial Centre for Patient Safety and Service Quality at Imperial College Healthcare NHS Trust, which is funded by the National Institute of Health Research. Dr Sevdalis is also funded by the Economic and Social Research Council Centre for Economic Learning and Social Evolution. * Corresponding author: Tel.: 44-789-6223424; fax: 44-151-7065828. E-mail address: [email protected] Manuscript received June 1, 2009; revised manuscript October 20, 2009
0002-9610/$ - see front matter © 2010 Elsevier Inc. All rights reserved. doi:10.1016/j.amjsurg.2009.10.020
titative approaches from behavioral science have been successfully applied to modeling surgical judgment.5,6 To date, little research has been carried out into the thought processes of surgeons’ decisions of whether or not to operate in given situations. Despite this, the decision of whether to operate is seen by many as fundamental to the practice of surgery and the ability to make that decision appropriately as a key tenet of surgical skill.7 Indeed, in the well-renowned phrase from Kirk’s textbook of general surgical operations,8 the best surgeon is one who “knows when not to operate.” Understanding the factors that influence this decision-making process is critical if this hallmark of expertise is to be distilled into a teachable format. The aim of this study was to investigate the process involved in deciding whether a patient should be operated on or not in an emergency setting. Specifically, the first aim
P. Szatmary et al. Table 1
Making the decision to operate
299
Decision to operate by surgical outcome matrix Poor outcome with operation
Poor Outcome Without Operation Good Outcome Without Operation
Good outcome with operation
Scenario 1: elderly woman, multiple comorbidities, Scenario 2: small bowel obstruction in fit young adult, perforated ulcer disease, septic shock 14-h history, early signs of shock Scenario 3: elderly patient, fresh rectal bleed from Scenario 4: middle-aged adult, arteriopathy, severe likely diverticulosis, hemodynamically stable, no abdominal pain of unclear origin, no signs of signs of peritonism hemodynamic instability
was to elucidate factors considered important to the decision to operate (phase 1). The second aim was to investigate if and how these factors applied in clinical practice using surgical scenarios (phase 2).
Methods Design This was a prospective, descriptive, multimethod study. In phase 1 of the study, semistructured interviews with expert surgeons were carried out to delineate the key factors influencing the decision to operate. In the light of the findings of phase 1, in phase 2 of the study, 5 clinical scenarios (vignettes) were constructed following an approach taken by other researchers,9 and participating surgeons were asked to indicate their decisions to operate or not in each of them.
ate. The complete list of questions can be found in Appendix A. Interviews lasted between 40 and 50 minutes and were audio-recorded. They were then transcribed verbatim and checked for accuracy against original recordings. In phase 2, 5 clinical scenarios were constructed (Appendix B). The scenarios were designed to reflect 4 common general surgical problems according to a 2 ⫻ 2 “decision to operate”– by–“surgical outcome” matrix (Table 1). A fifth scenario based on an orthopedic emergency (septic arthritis) was used as a control scenario. Participating surgeons were instructed to read the scenarios carefully and indicate whether they would operate (yes or no), what they felt the expected outcome would be with and without an operation (on a visual analogue scale [VAS] ranging from 0 to 100 with 5% intervals anchored at “poor outcome” and “good outcome”), and their confidence in the outcome (on a VAS ranging from 0 to 100 with 5% intervals anchored at “not at all confident” and “very confident”).
Data analysis Participants Interviews (phase 1) were conducted with 3 attending surgeons (2 general/colorectal, 1 orthopedic; all surgeons had ⱖ15 years of surgical experience) across 2 large UK district general hospitals. Twenty-two additional surgeons at various stages of their general surgical training were presented with the clinical scenarios (phase 2). Participants were grouped into those with ⬎5 years of experience, not counting their year of internship (the expert group, n ⫽ 12; range, 6 –34 years) and those with ⱕ5 years of experience (the novice group, n ⫽ 10; range, 0 –5 years). Five years was the average length of specialty training for general surgery in the United Kingdom and was therefore chosen as transition point between novice and expert surgeons.
Interview transcripts were content analyzed and emerging themes tabulated. The themes that emerged from the interviews were fed back to participants in the form of a presentation to validate the researchers’ findings (memberchecking validation technique for qualitative studies). Participants’ responses to the scenarios were measured on the VAS. Answers to every clinical scenario were analyzed as independent events, therefore leading to analysis of 110 data sets (22 participants, 5 scenarios). All data were analyzed using Microsoft Excel spreadsheets (Microsoft Corporation, Redmond, WA). Analysis of variance was used to assess differences between expert and novice participants. Obtained findings were deemed significant at P ⬍ .05.
Results Research materials and procedure In phase 1, the interview protocol consisted of a set of open-ended questions relating to extreme surgical situations (eg, “Describe a situation where an operation is undoubtedly necessary and one where it is likely to be harmful”). Reasons for not offering surgery immediately were explored (eg, “waiting for imaging results”), and attending surgeons were asked to describe their thought processes in making a decision to oper-
We first describe the findings from the interviews, followed by the findings from the clinical scenarios.
Factors affecting the decision to operate Table 2 summarizes the common themes influencing the decision to operate and gives example quotations from the interviews. The following themes emerged.
300 Table 2
The American Journal of Surgery, Vol 200, No 2, August 2010 Common factors influencing the decision to operate
Theme
Comments
Patient outcome
“The first consideration is the clinical condition of the patient. What the patient requires in order to make them well.” “(You have to ensure) you’re doing no further harm to the patient (when deciding to operate) and are trying to minimise the trauma that they experience.” “. . . those with large bowel obstruction (may require urgent operation). You have a patient who has a hugely distended caecum and the outcome would go from very good if you operate within a few hours . . . to one where they’ll almost certainly die if the caecum ruptures and they have generalised faecel peritonitis.” “. . . in the olden days . . . you open them up, ’cause they have generalised peritonitis and you don’t know what it is . . . nowadays we’d do a CT and get the diagnosis.” “Some fracture-neck-of-femur patients who need medically optimising (may benefit from delaying surgery). Sometimes it is better to delay (and) treat a chest infection.” “You want to optimise a patient’s cardio-vascular status . . . where you have a patient who could develop renal failure requiring renal support even . . . because you’ve piled in to them too soon before catching up with the fluid that they’ve lost inside the abdomen . . . you don’t do them any good.”
Disease progression Diagnostic certainty Resuscitation
Patient outcome. Surgery was thought to be positive if the condition of the patient improves because of it. An improvement in the condition of the patient was described as depending on a complex set of factors, including reduced pain, reduced chance of death, and improved function of a limb. Disease progression. Surgery was similarly described as positive if it slows or halts natural disease progression, thereby stabilizing “patient outcome” at the status quo. Diagnostic certainty. If there is serious doubt in the diagnosis and a potential for a disorder that may lead to a significant deterioration in patient outcome, an intervention was seen as indicated despite it being of potentially no benefit. Similarly, a usually indicated intervention might be considered detrimental if the diagnosis is not as expected, in which case delaying the procedure for further investigation was seen as the best choice.
Resuscitation. If current physiologic parameters (eg, circulatory volume, cardiac output, renal perfusion, coagulopathies) are seen to have a negative impact on operative outcome but can be optimized, a period of resuscitation was seen as positive before operating, and in this circumstance, the optimal decision was to delay the operation.
Decision to operate in the clinical scenarios Figure 1 shows how participants’ confidence in their judgment varied with experience. In this figure, “confidence” is the level of certainty indicated by participants about a given outcome on the VAS of the questionnaire. There is a rapid, significant increase in perceived confidence about outcome with or without operation in the first 3 to 5 years of surgical training (P ⫽ .03). This plateaus at a level of about 70% to 80% certainty, even with ⬎30 years of experience. However, there appears to be a cohort whose
Figure 1 Plot of percentage confidence versus surgeon experience in years. Confidence values are taken from VAS scores from questionnaires, on which surgeons were specifically asked to state the probability of patient outcome in a given scenario and then to self-assess how certain they were about patient outcome with (triangles) or without (crosses) an operation.
P. Szatmary et al.
Making the decision to operate
confidence peaks at about 10 years of training, before slowly declining. This may represent the point at which trainees become fully unsupervised, independent practitioners without the constant support of a senior colleague. To capture simultaneously the impact of surgeons’ judgment of outcome quality with or without surgery and their subjective confidence in that outcome, we computed the arithmetic product of their outcome perception (poor vs good) times surgeons’ self-reported confidence (low vs high). This simple algebraic manipulation reveals how surgeons’ views of potential outcome and their certainty in these views determine their decision to operate or not. Figure 2 summarizes our findings. All values above the diagonal line indicate that the outcome with an operation is perceived as better than the outcome without an operation and vice versa. In most cases in which surgeons stated they would operate, their perception markers fall above this line. Looking specifically at cases in which surgeons stated they would choose to operate, “likelihood of positive outcome with operation” values are significantly higher than those of cases where surgeons chose not to operate (51 ⫾ 5% vs 31 ⫾ 3%, P ⫽ .002). Similarly, “likelihood of positive outcome without operation” values were significantly lower (21 ⫾ 3% vs 48 ⫾ 3%, P ⬍ .001). Novice surgeons judged the likelihood of a positive outcome with operation similarly to expert surgeons (56% ⫾ 5% vs 55 ⫾ 8%, P ⫽ .46) but tended to judge the likelihood of a positive outcome without operation worse than experts, although this finding just failed to reach significance (49 ⫾ 8% vs 66 ⫾ 6%, P ⫽ .06). Novice surgeons also significantly favored operative approaches compared with experts: given the same scenarios, novice surgeons opted to operate
301 40 ⫾ 4% of the time, whereas experts would operate only 18 ⫾ 2% of the time (P ⫽ .001).
Comments The findings of both studies suggest that the perceived likelihood of a positive outcome given a certain intervention is fundamental in making the decision whether to operate or not. This is seen when attending surgeons state this directly with little or no prompting (phase 1 interviews) and when surgeons are seen to choose to operate when they perceive an outcome without an operation to be worse than one with an operation (phase 2 vignettes). It is important to note that a surgeon’s confidence in outcome with or without an operation increases rapidly in the first 3 to 5 years of training and also that certainty appears to plateau at 70% to 80%. It is not surprising that with increasing experience and increasing diagnostic skill such confidence increases. It is also not surprising that there is a plateau in confidence, as surgeons would rarely claim to be 100% correct in any patient. One should note, however, that the higher level of confidence may represent falsely elevated self-confidence in expert surgeons. There is also a cohort of surgeons with about 10 years of experience who appear to show declining confidence. This may reflect the knowledge that they are no longer supervised and therefore entirely responsible for their actions. Alternatively, it may represent a shift in educational style, as younger surgeons undergo modern training that focuses on self-evaluation and reflective practice. These questions might be answered by repeating the exercise with real cases, where
Figure 2 Plot of patient outcome with operation ⫻ certainty of that outcome versus patient outcome without operation ⫻ certainty of that outcome as stated by surgeons on a questionnaire relating to a specific clinical scenario. Surgeons were also asked to decide whether they would perform urgent operations on the patients in the scenarios: square data points represent decisions to operate and diamond data points decisions not to operate. The hashed line is the line of unity, where the perceived outcome with an operation equals the perceived outcome without an operation.
302 actual eventual outcomes could be measured and compared with outcomes predicted by surgeons. This finding does, however, have other important implications for clinical practice. It may suggest that surgeons treating uncommon conditions outside specialist centers may offer operations more readily than their subspecialist colleagues. This prediction could also be tested in future work. It appears that the product of predicted outcome and confidence in that outcome correlates very well with the decision to operate or not. This means that the more certain a surgeon is that an outcome with and operation will be better than one without, the more likely that surgeon is to propose an operation. This might mean that surgeons could refine their proposed treatments by using their own collected outcome data to guide their decision making, thus increasing their own confidence in any given outcome. Interestingly, novices tended to perceive outcomes without operation as worse than experts in our scenarios. This may lead to novice surgeons’ opting to perform operations in a far greater number of cases. In addition, novice surgeons were less sure about what an outcome might be with or without an operation than their expert peers. This might in part be explained by the desire to do “something” in view of an uncertain future. Although this study is not designed to address this particular point, these findings may indicate that novice surgeons may offer operations when experts deem them unnecessary, even though they think they are acting in patients’ best interest. If and how this finding translates to actual practice remains to be demonstrated in larger scale studies, but one can see how patient safety concerns may arise in this setting. An additional limitation to the generalizability of this finding is that in this study novice and expert surgeons responded independently, whereas in practice they would work as part of a team. Further limitations of this study include the relatively small sample size. Nevertheless, it should be noted that given the small sample, there appear to be significant differences in decision making between novice and expert surgeons. Also, the qualitative design of the study may be a limitation in that it attempts to quantify data by asking individuals to self-assess. This may be misleading if self-assessment skills vary between novices and experts for some reason (eg, differences in medical school curricula) and may also represent idealized actions instead of those taken in the real world. Quantitative studies with larger sample sizes are needed to further assess the validity of the findings, but the qualitative data presented here provides some insight into the thought processes involved. Further research should examine whether these findings translate into clinical practice and if confidence in outcome can be specifically developed in ways other than by passive gaining of experience. An important next step would be to investigate the clinical transferability of these findings by using real patient encounters as opposed to clinical scenarios.
The American Journal of Surgery, Vol 200, No 2, August 2010 In conclusion, appropriate decision making in determining whether or not operate is crucial to high-quality care. Surgeons evidently decide to operate when they perceive patient outcomes with an operation to be better than patient outcomes without an operation. The main difference between experts and novices is that experts are more certain about their assessments, probably a feature of their greater experience. This could explain the increased readiness to operate in novice surgeons, who might feel the need to “do something” in the absence of certainty of disease progression. Understanding the factors that influence this critical decision and the uncertainties faced by novices is an important first step toward better training, improved decision making, and ultimately safer patient care.
References 1. Kharbanda AB, Taylor GA, Fishman SJ, et al. A clinical decision rule to identify children at low risk for appendicitis. Paediatrics 2005;116:709 –16. 2. Miura F, Takada T, Kawarada Y, et al. Flowcharts for the diagnosis and treatment of acute cholangitis and cholecystitis. J Hepatobiliary Pancreat Surg 2007;14:27–34. 3. Weiland DE, Caruso DM, Kassir A, et al. Using delta/DRG diagrams and decision tree analysis to select a cost-effective surgery for cholecystitis. JSLS 1997;1:175– 80. 4. Jacklin R, Sevdalis N, Harries C, et al. Judgment analysis: a method for quantitative evaluation of trainee surgeons’ judgments of surgical risk. Am J Surg 2008;195:183– 8. 5. Jacklin R, Sevdalis N, Darzi A, et al. Efficacy of cognitive feedback in improving operative risk estimation. Am J Surg 2009;197:76 – 81. 6. Jacklin R, Sevdalis N, Darzi A, et al. Mapping surgical practice decision making: an interview study to evaluate decisions in surgical care. Am J Surg 2008;195:689 –96. 7. Spencer FC, The Gibbon lecture— competence and compassion: two qualities of surgical excellence. Bull Am Coll Surg 1979;64:15–22. 8. Kirk RM, editor General surgical operations. 5th ed. New York: Elsevier; 2006. 9. MacCormick AD, Parry BR, Judgement analysis of surgeons’ prioritization of patients for elective general surgery. Med Decis Making 2006;26:255– 64.
Appendix A: Interview Protocol 1. Of concern is the decision to operate or not to operate in emergency surgery of your specialty field 2. Please describe a patient presenting with an acute problem who will need urgent surgery (same day) 3. Please describe a patient presenting with an acute problem who would come to harm if operated on urgently (same day) 4. What would have to change or become apparent for urgent surgery to be postponed/cancelled in the case of the first patient you described 5. What would have to change or become apparent for urgent surgery to be immediately organised for the second patient you described 6. Describe the decision/decisions that you make when reaching the conclusion that urgent surgery is required
P. Szatmary et al.
Making the decision to operate
7. Are there any factors that you have known to alter the above decision making process in any way? 8. Describe the factors that you take into consideration when reaching the conclusion that urgent surgery is required. How would you categorise/group these factors? 9. Do your colleagues or junior staff ever change your decision about the necessity of urgent surgery? 10. Do biochemical markers and/or imaging techniques ever change your decision about the necessity of urgent surgery - if yes, in what type of scenario do you think they are particularly effective? 11. Are you guided mainly by your own experience, or mainly by clinical guidelines and published evidence? Can you think of a situation where that balance is shifted in any way? 12. How do external pressures (financial, staff-related, timerelated) affect your decision to perform an urgent operation? 13. Are there any other points relating to the way you reach a decision about whether to operate in an acute emergency that you would like to make?
Appendix B: Scenarios Background You are a surgeon at your current level of training at a large district general hospital. It is late afternoon and routine services (ie clinics, routine lists, routine imaging services) have ceased, however out of hours imaging and theatre staff are available. It would take between 30 and 120 minutes to perform a CT scan and between 15 and 30 minutes to prepare theatres if no concurrent operation is in progress. There is a Consultant/Attending led general anaesthetic service and a 6 bed ITU/HDU. Specialist intensive care (neurosurgical, paediatric, burns) is available at the tertiary referral centre 80 minutes away.
303 Blood tests reveal a mild renal impairment, but are otherwise normal. An erect chest XR reveals overdistended lung fields as well as a small amount of free intra-peritoneal air under the right hemi-diaphragm.
Scenario 2 A previously fit and well 32 year old man attends A&E with a 14 hour history of worsening abdominal pain and severe, unrelenting bilious vomiting. He has last opened his bowels this morning. He has been unable to eat or drink anything all day due to his severe nausea and recurrent vomiting. He is a keen sportsman; currently training for national swimming championships. He has only been in hospital once before - when he had an operation for abdominal sepsis secondary to a perforated appendix 10 years ago. He is on no regular medication and has no allergies. Observations are as follows: Pulse rate 102bpm regular, BP 116/83, respiratory rate 24 (saturation 100% on air). Abdominal examination reveals a slightly distended and tympanic abdomen. Abdominal X-ray shows several distended loops of small bowel with no gas in the rectum. You are called to see the patient for a surgical opinion.
Scenario 3 A GP refers a 63 year old lady with a 2 day history of per-rectum bleeding associated with lower abdominal pain. She is not complaining of nausea and has had her bowels open twice today - stools are loose in consistency and mixed with both dark and bright red blood. Her past medical history includes angina, hypertension and a myocardial infarction 4 years ago, from which she has recovered well. She is independent and is the main carer for her elderly father. Observations are: pulse rate 88 regular, BP 89/36, respiratory rate 26 (saturation 100% on air). The patient appears pale and slightly anxious. Abdominal examination reveals a soft abdomen with tenderness in the left lower quadrant and suprapubic area. Plain chest and abdominal radiographs are unremarkable.
Scenario 1 Scenario 4 You are asked to see an 86 year old lady who presents with an 18 hour history of severe epigastric pains and vomiting. She has been transferred from a residential home where she has spent the last 6 years due to mild memory problems. She has no living relatives. She has a past medical history of COPD requiring home nebulisers, but not home oxygen. She is also on long term oral steroid therapy due to giant cell arteritis. Nursing observations are as follows: a pulse rate of 92 bpm, irregularly irregular, a blood pressure of 147/72, a respiratory rate of 14 (saturation 93% on air) and a temperature of 36.7°C. The patient appears comfortable after 5 mg of iv morphine. Abdominal examination reveals localised epigastric peritonism.
You are asked to see a 52 year old man who presents to the emergency services with a 3 hours history of sudden onset severe central and upper abdominal pain associated with repeated vomiting and relentless diarrhoea. He is an investment banker and smoker of 40 cigarettes per day and drinks between 15 and 25 units of alcohol per week. He is clinically obese. He has recently been diagnosed as suffering from angina and an angiogram from 8 months ago confirms 2 vessel disease. Observations are: pulse rate 112 regular, BP 98/60, respiratory rate 28 (saturation 100% on air). Abdominal examination reveals mild peri-umbilical and epigastric tenderness with no signs of peritonism. Plain chest and abdominal
304 radiography are unremarkable except for a suspicious absence of bowel gas.
Scenario 5 A 5 year old girl presents to the emergency department having fallen off a trampoline, complaining of severe pain in her right elbow and supporting her right arm with the left hand. She has no known medical problems. She is right hand dominant. Unfortunately the emergency department has
The American Journal of Surgery, Vol 200, No 2, August 2010 been very busy and by the time you get the referral it is 02:00 in the morning. The girls elbow is now supported in a plaster slab. On examination you find her hand cool to touch, but pulses are present and sensation and motor function is present, but limited by pain. She is also complaining of some altered sensation over the dorsum of the first web space. X ray of the elbow confirms your suspicion of a supracondylar fracture of her right humerus. It is a closed injury with minimal displacement on plain radiographs.