- Email: [email protected]
Frequent Fliers, Internal and External Validity, and Problems With Making Continuous Variables Binary
ANNALS OF EMERGENCY MEDICINE JOURNAL CLUB
Frequent Fliers, Internal and External Validity, and Problems With Making Continuous Variables Binary Answers to the May 2009 Journal Club Questions Sanjay Arora, MD David L. Schriger, MD, MPH
From the Department of Emergency Medicine, Keck School of Medicine/University of Southern California, Los Angeles, CA (Arora); and the University of California, Los Angeles, CA (Schriger).
0196-0644/$-see front matter Copyright © 2009 by the American College of Emergency Physicians. doi:10.1016/j.annemergmed.2009.04.013
Editor’s Note: You are reading answers to the ninth installment of Annals of Emergency Medicine Journal Club. The questions and the article they are about (Aisiku et al. Ann Emerg Med. 2009;53:587-593.) were published in the May 2009 issue. Information about journal club can be found at http://www.annemergmed.com/content/journalclub. Readers should recognize that these are suggested answers. We hope they are accurate; we know that they are not comprehensive. There are many other points that could be made about these questions or about the article in general. Questions are rated “novice,” ( ) “intermediate,” ( ) and “advanced” ( ) so that individuals planning a journal club can assign the right question to the right student. The “novice” rating does not imply that a novice should be able to spontaneously answer the question. “Novice” means we expect that someone with little background should be able to do a bit of reading, formulate an answer, and teach the material to others. Intermediate and advanced questions also will likely require some reading and research, and that reading will be sufficiently difficult that some background in clinical epidemiology will be helpful in understanding the reading and concepts. We are interested in receiving feedback about this feature. Please e-mail [email protected] with your comments.
DISCUSSION POINTS 1. As stated in its title, this study1 seeks to compare patients with sickle cell disease who are frequent and infrequent users of the emergency department (ED). The authors recruited patients from “established [sickle cell disease] clinics, health fairs, referrals, and targeted mailings.” A. Contrast this study design to one in which all patients were recruited from ED logs. Draw a Venn diagram comparing the patients likely to be included in these 2 designs. How do the populations differ? B. Describe the pros and cons of each approach focusing on internal and external validity. C. What techniques might be used to overcome the limitations of each approach? D. What ethical issues arise in each study design? 628 Annals of Emergency Medicine
E. Read the abstract’s conclusions. How might you modify them? How might you modify the limitations section? 2. Patients were asked to keep a daily diary for 6 months, though investigators included patients who completed as few as 30 of the 188 study days. Each subject’s data were extrapolated to create 1-year values. Examine the paper’s figure. A. What pattern do you see on the far left of the figure about the number of ED visits by infrequent users? How do you explain this pattern? Does this figure represent the true frequencies or an artifact of the study methods? How could a patient have 1 ED visit per year, given the way data were handled in this study? B. These authors defined frequent users as patients who had at least 3 ED visits per year. According to your clinical experience, are there alternate definitions that might be used? Considering Figure 1, do you think that other definitions might be warranted? 3. The authors write, “[Al]though the data were not statistically significant, high ED utilizers trended toward a higher incidence of anxiety, higher incidence of avascular necrosis and higher WBC count.” Write a sentence that starts, “Although data were not statistically significant, low ED utilizers trended toward . . .” Discuss the meaning of their and your sentence. What are some problems with this approach? How else might the authors have conveyed this information? 4. The authors suggest that the “frequent flier” designation that some emergency physicians give to patients who visit often for sickle cell pain is unjustified because, on average, these patients were sicker than infrequent users. A. Honestly examine your own feelings about the “frequent flier” designation. Do you use the term? How do you feel when others use it? Do you view patients who come in often differently? Do you manage their treatment differently? Do you worry that you may miss a change in their chronic condition? How do you balance pragmatic and moral concerns related to this issue? Volume , . : October
Journal Club
Figure 1. Distribution of frequent ED users among hypothetical populations of study subjects.
B. Similarly, what is your approach to pain management in patients who frequently come to the ED complaining of chronic or acute-on-chronic pain? How does your behavior compare with that of your peers? What arguments do you invoke to justify your approach?
ANSWER 1 Q1. As stated in its title, this study seeks to compare patients with sickle cell disease who are frequent and infrequent users of the emergency department (ED). The authors recruited patients from “established [sickle cell disease] clinics, health fairs, referrals, and targeted mailings.” Q1.a Contrast this study design to one in which all patients were recruited from ED logs. Draw a Venn diagram comparing the patients likely to be included in these 2 designs. How do the populations differ? Figure 1, the Venn diagram we had in mind, illustrates 2 seemingly contradictory conclusions that could both be true: (a) the study is valid and, in the patients recruited for this study, frequent users are a minority and the frequency of visits is associated with disease severity and little else; (b) there exists a large group of patients who use the ED frequently and whose visit frequency is unrelated to disease severity. This paradox highlights the importance of being very clear about the relationship of the study design to the research question. The Aisuku et al1 study examines the ED use behaviors of a group of patients who attend sickle cell clinics, health fairs, etc, and were willing to participate in the study. It does not study the behavior of patients with sickle cell disease who do not participate in the activities targeted for recruitment. It also does not capture the behavior of patients who were identified for recruitment but refused to participate in the study. The sample in the Aisuku et al1 study and the sample that would result from combing ED logs likely differ in many of the characteristics that determine the utilization of health care.2 We are not surprised that patients who are engaged with the health care system—as many of the patients in the Aisuku et al1 sample Volume , . : October
were— used the ED infrequently. It is also wholly conceivable that the kinds of patients who were not recruited for this study may use the ED frequently. An alternative study design that recruited patients from the ED would be a better proxy for patients with sickle cell disease who use the ED for some part of their care, assuming that that study was successful in recruiting and retaining subjects. The important point is that the 2 approaches are complementary and answer different questions. A study of ED users might document an abundance of frequent users whose frequency of use had little correlation with other proxies for severity of illness but would fail to capture the patients who manage their disease with few or no trips to the ED. The current study may have missed the patients who use the ED as their primary source of care regardless of their frequency of use and may underestimate the frequent user problem. Q1.b Describe the pros and cons of each approach focusing on internal and external validity. There are many textbooks that nicely define external and internal validity, and readers are directed to these for detailed study3,4 or, for those who want to study the opposite of validity, to an article that describes 53 distinct forms of bias, each of which can compromise the validity of a study.5 In brief, by “internal validity” we mean the extent to which a study provides an unbiased estimate of the true value, with bias defined as “any process at any stage of inference which tends to produce results or conclusions that differ systematically from the truth.”5 An internally valid study, if repeatedly executed, will produce values that approach, on average, the truth for that population. If one measured the weight of a random sample of female US newborns with a scale that was out of calibration, one would have a study that lacked internal validity because the average weight of the children reported in the study would not approach the average weight of the children in the population. Similarly, because the study by Aisuku et al1 is based on patient diaries, if the subjects systematically over- or underreported their ED visits, then the study would lack internal validity. For randomized trials, the expectation is that if all study limbs were given the same treatment (exposure), then, apart from random error, all would have the same outcome. If this is not the case, then the study lacks internal validity. A study has external validity if the results apply to the population identified in the study question. In the example above, if the baby scale were recalibrated to perfection, the study would have excellent internal validity and external validity as long as the question were, what is the weight of newborn females? However, if the study question were, “what is the weight of newborn infants?”, then the study might have poor external validity if female and male infants differ in weight. External validity issues typically predominate in large, wellconducted, randomized trials because these trials may have great internal validity but may have patient selection issues that render the study population a poor homologue for the population defined in the study question. Annals of Emergency Medicine 629
Journal Club The internal validity of the study by Aisuku et al1 could be increased by actually measuring events (ie, ED visits), rather than relying on self-report. This is true regardless of how the population was recruited. Alternatively, the magnitude of measurement bias could be estimated by reviewing the medical records of a sample of enrolled patients to compare actual ED visits with patient diaries. Our assessment of the external validity of the study depends on our understanding of the study question. The authors write “. . . we sought to better characterize sickle cell disease patients who frequently utilize the ED.” If we assume this is the study question, the study’s sample may be a poor proxy for the target population and the study may lack external validity. The “pro” of the authors’ approach is that they recruited patients who were generally willing to complete the diaries (see below). This made the study feasible. We suspect that the patients missed by the study’s recruitment procedures would be unlikely to participate in the study or complete the diary should they consent to participate. The “con” is that the study may have missed the patients who were most likely to be frequent users of ED services. The considerable pro of the alternate approach is that it would identify the patients most likely to be frequent users. The cons are that it is wholly unclear whether these patients would be willing to consent and participate and that there may be ethical issues about how these patients are identified and approached (see Q1.e below). Q1.c What techniques might be used to overcome the limitations of each approach? As discussed above, the major threat to the external validity of the study by Aisuku et al1 is that it missed the patients most likely to be frequent users The authors could have examined this possibility by stratifying their data on the recruitment method. One might hypothesize that the patients most connected to the health care system (ie, those recruited from dedicated sickle cell clinics) would be least likely to be frequent users, whereas those with weaker ties (eg, those identified at a health fair who had no regular physician) would be more likely. If heterogeneity were found among strata, one would have a better sense of how different the ED-use behavior of those not captured in the study might be. The absence of heterogeneity would suggest that the missed patients may not be that different from those captured. The major limitations of recruitment through the ED are logistics and ethics (see Q1.d), selection bias, and compliance. If patients most likely to use the ED are unwilling to participate or fail to complete the diaries, then the study would be infeasible. Alternate means of outcome measurement (eg, getting permission from patients to use insurance billing data or ED logs to directly measure their utilization) might prove a better strategy for measuring utilization in this group. Q1.d What ethical issues arise in each study design? We begin by reviewing some facts about HIPAA, the Health Insurance Portability and Accountability Act of 1996. HIPAA was created in anticipation of the proliferation of electronic 630 Annals of Emergency Medicine
medical record systems and the concern that patient records produced by such systems could be accessed by unauthorized users. Congress recognized that additional safeguards were needed to keep personal health information private, so they added provisions for further layers of protection. The HIPAA Privacy Rule (Standards for Privacy of Individually Identifiable Health Information) had a mandatory compliance date of April 14, 2003. HIPAA specifies what information is “protected health information”—information of any form (oral, written, electronic) that identifies an individual—and when and with whom such information may be shared. In the design by Aisuku et al,1 there are no obvious HIPAA or ethical issues because the patients keep their own diaries, do not have to tell their treating physicians that they are part of a research study, and are deidentified when the data are presented. There are, however, aspects of the recruitment phase of the study that, if improperly conducted, could be in violation of HIPAA. First, the authors indicate that some patients were recruited by “referral.” It is not explicitly stated where these referrals came from, but if they came from the patient’s physician without the patient’s knowledge, that would be a HIPAA violation. Similarly, it is unclear how recipient lists for the targeted mailings were created. In the alternative strategy, there are several ways in which patients could be identified from the ED. Research assistants could be summoned to the ED to approach patients with sickle cell disease in real time. However, unless an ED has full-time research assistant coverage, this strategy could be costly and infeasible. An alternate approach would be to review billing records for recent visits with International Classification of Diseases, Ninth Revision code 282.6 (sickle cell anemia) and then contact these patients after the fact. Such contact would violate HIPAA (45 CFR 164.514) unless investigators first obtain “a partial waiver of consent/HIPAA” for screening and recruitment purposes from their institutional review board. Several requirements must be met to obtain this waiver: (1) there must be no more than minimal risk to the subjects; (2) the enrollment could not be effectively and practically carried out without obtaining a waiver; (3) waiving consent will not adversely affect the subject’s rights and welfare (ie, they have the right to refuse to participate, they will be provided the opportunity to fully consent to participation, and those who say no will immediately have their recorded identifying information destroyed). Federal regulations further require that if a waiver is granted that researchers obtain only the minimal personal health information that is necessary for recruitment. In this case, that would be name, telephone number, and possibly address. After patients are identified, they should be approached with a standardized letter or verbal script that is institutional review board approved and designed to eliminate intrusive questions and coercion and allows the subject to easily say no. Although somewhat cumbersome, this is a commonly used recruitment Volume , . : October
Journal Club strategy in prospective trials and could have been used to identify ED users in this trial. Q1.e Read the abstract’s conclusions. How might you modify them? The abstract’s conclusion reads “[a] substantial minority of sickle cell disease patients are high ED utilizers. However, high ED utilizers with sickle cell disease are more severely ill as measured by laboratory variables, have more pain, more distress, and have a lower quality of life.” Given the discussion in Q1.a to Q1.d, the truth would be better served if the words “included in this study” were inserted after “patients.” Abstracts are short (Annals has a 250-word limit), and many details conveyed in an article are necessarily omitted from the abstract. Unfortunately, important caveats and qualifiers are often among the omitted details. As a result, those who read only the abstract may get a false sense of the meaning or applicability of an article’s conclusions. How might you modify the limitations section? The article’s limitations section considers how self-reported ED use may differ from actual use but fails to consider the most important limitation of the study, the recruitment strategy. As discussed above, the patients in the study may be poorly representative of the patients who actually visit the ED. This threat to external validity should have been considered either in the methods and results, where the analysis suggested in Q1.c would be reported, or in the limitations section. The limitations section also failed to discuss how the treatment of a continuous outcome variable (number of visits) as a binary one (high versus low use) and the choice of cut point (greater than 2 visits) might affect study results. Authors also failed to address the myriad problems that can arise when extrapolating as little as 1 month’s worth of data to 12 months. Finally, the authors argue that if there is bias in the selfreports of ED use, the bias is toward underreporting. We don’t see why this is necessarily so because those patients who missed many of the diary days (they could miss up to 157 of the 188 days and still be in the study) may be more likely to complete the diary on days when they were in pain or days when they consulted a provider. This could lead to an overestimation of the number of ED visits and to a spuriously enhanced correlation between severity of illness and ED visit frequency. If there is a body of literature that supports their claim, the authors should have referenced it to strengthen their argument.
ANSWER 2 Q2. Patients were asked to keep a daily diary for 6 months, though investigators included patients who completed as few as 30 of the 188 study days. Each subject’s data were extrapolated to create 1-year values. Examine the article’s figure. Q2.a What pattern do you see on the far left of the figure about the number of ED visits by infrequent users? A magnified version of the far left of the article’s figure is shown in Figure 2. We note that most patients had an even number of annual visits (0, 2, 4), whereas few had an odd Volume , . : October
Figure 2. Highly magnified view of left side of the Figure in Aisiku et al.1
number (1, 3, 5). We also note that more than 50% of subjects in this study of ED users actually had no ED visits. How do you explain this pattern? This pattern can be explained by looking closely at the method by which annual visit rates were calculated. The authors collected at most 6 months and as little as 1 month of data on each patient. Annual rates were calculated by multiplying the number of reported visits by the fraction: 365/number of daily diaries completed. For example, if a patient reported 2 ED visits in 3 months of completed diaries, his or her annual total would be 8 visits (2⫻365/91⬇2⫻4⫽8). The Table shows the possible annual rates achieved by this method and reveals the cause of the undulating pattern. Is it representative of the true frequencies or an artifact of the study methods? As explained above, this is an artifact of the multiplier effect. For patients who have no events, we have little idea whether they kept 30, 60, or 188 daily diaries. Similarly, a patient with 1 visit in 1 month of recorded diary would be listed as having 12 visits, but this is a very imprecise estimate of the actual visit rate. How could a patient have 1 ED visit per year, given the way data were handled in this study? According to the multiplier method described and the 6month maximum for data collection, it is not possible for a Annals of Emergency Medicine 631
Journal Club Table. Method for calculating visits/year. Months Completed
No. of Visits
Multiplier
Annual Total
1, 2, 3, 4, 5, or 6 1 2 3 4 5 6 1 2 3 4 5 6
0 1 1 1 1 1 1 2 2 2 2 2 2
12, 6, 4, 3, 2.4, 2 12 6 4 3 2.4 2 12 6 4 3 2.4 2
0 12 6 4 3 ⬇2 2 24 12 8 6 ⬇5 4
Most multipliers are even, and consequently, most rates (annual totals) are even.
patient to have just 1 ED visit per year. As shown in the Table above, the minimum multiplier value is 2; hence, the minimum number of annual visits for a patient with at least 1 visit is 2. We do not know how a patient was given a value of 1, but the existence of this data point reminds us that the scientific literature is imperfect and both authors and peer reviewers can miss what should be easily detectable errors. (Editor’s note: We queried the authors and they explained that rather than rounding, they simply took the integer part of any decimal numbers. Hence 1.96 [a patient with 1 visit in a full 188 days of diaries) was rounded to 1, not 2]. They did not provide a justification for this practice.) Q2.b These authors defined frequent users as patients who had at least 3 ED visits per year. According to your clinical experience, are there alternate definitions that might be used? Considering Figure 1, do you think that other definitions might be warranted? The definition of “frequent user” is certainly in the eye of the beholder. An emergency physician who had a medical condition that required 3 ED visits each year might consider himself a frequent user, but that same physician, told by the nurse that this was a patient’s third visit in 12 months, would not make much of it. Here lies a paradox of this article. The authors based their cut point on the Multicenter Study of Hydroxyurea, which considered patients with 3 or more visits per year sick enough to warrant this treatment. This choice of cut point is somewhat tautologic. Their goal is to show that frequent ED users are sicker than infrequent users, and they use a cut point that has already been shown to define a sicker group of patients by virtue of their frequency of ED use. We can think of many other ways to define frequent users: patients who are treated at least once each month, patients treated several times each week, patients who are known to the ED staff on a first-name basis, patients who have a protocol in place for treatment to be administered when they arrive, patients who self-report that they come to the ED “all the time,” or patients who visit multiple EDs in a given area in a short period. 632 Annals of Emergency Medicine
The authors’ results might be very different had they selected a cut point of 12 or 24. A way out of this problem is to resist the temptation to take a continuous variable, the annual visit rate, and make it binary (high versus low).6 An analysis conducted using 4 groups (no use, 1 to 6 visits per year, 7 to 19 visits per year, and ⱖ20 visits per year) would provide richer information that might illuminate important patterns that cannot be seen with this binary treatment. An alternative method would be to use graphics to show each patient’s disease severity versus ED use so that readers could see patterns in the data. Another strategy for making a continuous variable categorical is to look at the data before making the grouping. Although it is generally advisable to let theory guide the creation of categories, pragmatic considerations related to sample size are sometimes important. Consider a study of children. According to theory, one might argue that meaningful groupings would be age younger than 6 months, 6 months to 2 years, 3 to 12 years, and 13 to 17 years. Age quartiles (created by making 4 equally sized groups) might be 0 to 4 months, 4 months to 3 years, 3 to 15 years, and 16 to 17 years. The quartiles create equal-sized groups but groups that may make little sense (eg, 3 to 15 years), depending on the clinical question. Investigators need to balance sample size considerations and theoretic considerations, with the latter given emphasis in questionable situations. In this case (see the figure in Aisuku et al1), the first 2 quartiles would include 53% of the patients (because of ties), all of whom had no ED visits. The third quartile (22%) would include patients with 1 to 5 visits, and the fourth quartile (25%) would be greater than or equal to 6 visits. One would have to wrestle with the cut points suggested by theory (such as those mentioned 2 paragraphs above) and the need to have an adequate N in each group. In this case, we would let theory prevail and accept that the high-use groups will have substantially less precision (because of smaller Ns) than the lowuse groups.
ANSWER 3 Q3. The authors write “[Al]though the data were not statistically significant, high ED utilizers trended toward a higher incidence of anxiety, higher incidence of avascular necrosis and higher WBC count.” Write a sentence that starts, “Although data were not statistically significant, low ED utilizers trended toward.. . .” Although data were not statistically significant, infrequent ED utilizers trended toward being wealthier (33% versus 49% earned ⬍$10,000), older (53% ⱖ35 years of age versus 39%), and receiving care at a specialty center (53% versus 43%). (Editor’s note: There are other possible sentences but this one serves to make our general point.) Discuss the meaning of their and your sentence. If this were a randomized trial with no bias and no measurement error, then, in a classic statistics framework, the absence of statistical significance would be taken to mean that we have insufficient evidence to reject the null hypotheses about the factors mentioned in these sentences. This failure could Volume , . : October
Journal Club represent a good decision (a true negative) because there exists no true difference in these factors in low and high ED users, and the observed differences are solely due to random error. It could also represent a type II (false negative), or , error. The likelihood of this being a type II  error depends on the power of the study (the sample size relative to the variance of the measure in question). For example, if the investigators of this hypothetical randomized trial had powered their study to have a  of .1 (power of .9), then there is only a 10% chance that the failure to find statistical significance would be a type II error, and the more reasonable interpretation is that the observed differences represent random variation. This is not, however, a large randomized trial. This is an observational study of self-reported ED use, subject to any number of systematic biases and systematic and random forms of measurement error. In this case, the interpretation of observed differences is far more complicated than in a randomized controlled trial because differences could be real (approximating the truth except for random error), due to bias, due to measurement error, due to random error, or due to any combination of the above. Furthermore, because the authors provide no description of why their study is this particular size (no sample size or power calculation), the failure to achieve statistical significance could be due to the absence of an important effect (true negative) or to a lack of power (type II error). We are therefore left in a bit of a quandary about how to interpret these nonsignificant results and, for that matter, the significant ones. We lack access to highly stratified data that might help us decide whether results represent truths, random error, or bias. (Editor’s note: We suspect that some of our readers had questions about the use of the Bonferroni method to reduce ␣ to account for multiple testing. We promise to cover this in detail in a future journal club article but here acknowledge that this may not be the best way to handle this problem.7) What are some problems with this approach? The main problem with the authors’ sentence is that it tells only half the story. If authors highlight the “nonsignificant” results that support their argument, should they not, in the name of balance, also mention results that are contrary? What scientific basis is there for including their sentence but not ours? How else might the authors have conveyed this information? Our preferred alternative is to say nothing. The data tables (particularly Tables 2 and 3 that provide between-group differences with their confidence intervals) give readers a sense for the magnitude of each variable in each group and how they differ. Readers can interpret the meaning of these differences as they wish. Readers might be aided by more detailed tables and figures that show further stratification of the results but are unlikely to be helped by sentences that cherry pick selected results.
ANSWER 4 Q4. The authors suggest that the “frequent flier” designation that some emergency physicians give to patients who visit often for Volume , . : October
sickle cell pain is unjustified because, on average, these patients were sicker than infrequent users. Q4.a Honestly examine your own feelings about the “frequent flier” designation. Meditation cushions are available for this purpose at http:// annemergmed.com. Just kidding. Do you use the term? We will not attempt to answer these questions in any comprehensive way or attempt to say which opinions are “correct.” We posed these questions to stimulate journal club participants to have a candid discussion about these difficult issues. In the name of complete disclosure, we do our best to answer the questions honestly. We make no claim that our answers are more important than anyone else’s answers. The following is simply a summary of our opinions, which can be used as a conversation starter. We both admit to using the term at times and that it was difficult to write these answers without resorting to it. We chose not to because “frequent flier” means different things to different people. The literal slang meaning is that someone uses a service often, be it popping into Starbucks daily at 7:05 AM for a java fix or appearing frequently at a neighborhood ED. The words have no inherent judgmental quality. The negative connotation is instilled by context— how it is used and who is using it. In our experience, the term is occasionally used nonjudgmentally, (for certain nonsmelly, undemanding, non– drug-seeking, anxious [but not too anxious] little old ladies who like to check in periodically) but more frequently is loaded with judgments about the appropriateness and motivations of the patient’s visit. How do you feel when others use it? Although we both use the term on occasion, it worries us when undergraduate research assistants, medical students, and junior residents use it. Neither of us has a degree in semiotics, but we believe that changes in language often precede cultural change and worry that the use of this language by those whose attitudes are still being shaped may foster the development of a more callous attitude toward all kinds of needy patients. As much as we hate to admit it, we think we can all agree that most of the time when we call a patient a “frequent flier,” we do not mean it as a compliment. This logic leads us to question our own use of the term and whether, as teachers, we should be avoiding language that could foster undesirable attitudes in our students. Do you view patients who come in often differently? Do you treat their disease differently? Do you worry that you may miss a change in their chronic condition? We strive to avoid prejudging patients, but for select individuals, we have strong opinions before entering the room. We hope that our self-awareness of our tendency to do this helps us avoid errors that can occur when one assumes that the patient is here “for the same old thing” or when one fails to hear the patient saying that “something is different.” Neither of us feels that we systematically manage frequent users differently Annals of Emergency Medicine 633
Journal Club from infrequent users and believe that our knowledge of a patient’s frequent-user status affects our treatment more than our assessment. How do you balance pragmatic and moral issues related to this issue? We struggle to define what constitutes an appropriate balance of “pragmatic and moral issues” and leave you to work out your own comfort zone with respect to these issues. Q4.b Similarly, what is your approach to pain management in patients who frequently come to the ED complaining of chronic or acute-on-chronic pain? Patients are generally assumed to have legitimate pain concerns that warrant treatment unless there is a specific history to suggest otherwise. How does your behavior compare with that of your peers? The behaviors exhibited by emergency physicians are as wide as the day is long. Some of our peers regularly give patients whatever they want (within a “3 shots and be admitted or discharged” policy). These faculty members believe that even if they question the patient’s authenticity, it is better to treat the patient than risk failing to treat a patient in need or create a confrontation. The other group believes that there are a small number of patients who resist all attempts to be shepherded into appropriate continuing care. These attending physicians, after giving patients ample warning that they will not be given narcotics unless they establish a demonstrable relationship with a continuing care provider, will not give narcotics to patients who fail to do so. We fall in the latter group. What arguments do you invoke to justify your approach? Being an emergency physician for a frequent user who is seeking narcotics but refuses to establish a relationship with a primary care provider is a bit like being the parent of a toddler or teenager. In the short term, saying yes is the easiest path; it makes all parties happy and eliminates all of the conflict and drama. But one has cause to worry about the children of parents who consistently use this strategy. Saying no is harder in the moment but may lead to better long-term outcomes. Many emergency physicians for whom we have great respect argue that
634 Annals of Emergency Medicine
they would rather have many false-positive results (giving narcotics to someone who is merely drug seeking) than 1 falsenegative result (denying someone in pain adequate analgesia).8 They also argue that there is little harm in giving a narcotics addict narcotics. However, we find these arguments a bit shortsighted because they fail to consider the negative long-term consequences of these strategies. We believe that many experienced emergency physicians will learn to trust their gut, rather than always erring on the side of giving narcotics. Each of us knows patients whom we refuse to treat with narcotics. In our experience, when these patients arrive in the ED and learn that one of us is the attending physician on duty, they leave. We are each satisfied with this arrangement. Section editors: Tyler W. Barrett, MD; David L. Schriger, MD, MPH
REFERENCES 1. Aisiku IP, Smith WR, McClish DK, et al. Comparisons of high versus low emergency department utilizers in sickle cell disease. Ann Emerg Med. 2009;53:587-593. 2. Aday LA, Andersen R. A framework for the study of access to medical care. Health Serv Res. 1974;9:208-220. 3. Rothman KJ, Greenland S, Lash TL. Validity in epidemiologic studies. In: Rothman KJ, Greenland S, Lash TL, eds. Modern Epidemiology. 3rd ed. Philadelphia, PA: Lippincott; 2008:128-129. 4. Altman D. Clinical trials. In: Practical Statistics for Medical Research. London: Chapman & Hall; 1991. 5. Sackett DL. Bias in analytic research. J Chronic Dis. 1979;32:5163. 6. Altman DG, Royston P. The cost of dichotomizing continuous variables. BMJ. 2006;332:1080. 7. Rothman KJ, Greenland S. Fundamentals of epidemiologic data analysis. In: Rothman KJ, Greenland S, Lash TL, eds. Modern Epidemiology. 3rd ed. Philadelphia, PA: Lippincott; 2008:236-237. 8. Henry G. What’s your pain care philosophy? Emergency Physicians Monthly Web site. Available at: http://www.epmonthly.com/index. php?option⫽com_content&task⫽view&id⫽453&Itemid⫽91. Accessed April 13, 2009. (Archived by WebCite at http://www. webcitation.org/5g0KuiMcP).
Volume , . : October
Frequent Fliers, Internal and External Validity, and Problems With Making Continuous Variables Binary Answers to the May 2009 Journal Club Questions Sanjay Arora, MD David L. Schriger, MD, MPH
From the Department of Emergency Medicine, Keck School of Medicine/University of Southern California, Los Angeles, CA (Arora); and the University of California, Los Angeles, CA (Schriger).
0196-0644/$-see front matter Copyright © 2009 by the American College of Emergency Physicians. doi:10.1016/j.annemergmed.2009.04.013
Editor’s Note: You are reading answers to the ninth installment of Annals of Emergency Medicine Journal Club. The questions and the article they are about (Aisiku et al. Ann Emerg Med. 2009;53:587-593.) were published in the May 2009 issue. Information about journal club can be found at http://www.annemergmed.com/content/journalclub. Readers should recognize that these are suggested answers. We hope they are accurate; we know that they are not comprehensive. There are many other points that could be made about these questions or about the article in general. Questions are rated “novice,” ( ) “intermediate,” ( ) and “advanced” ( ) so that individuals planning a journal club can assign the right question to the right student. The “novice” rating does not imply that a novice should be able to spontaneously answer the question. “Novice” means we expect that someone with little background should be able to do a bit of reading, formulate an answer, and teach the material to others. Intermediate and advanced questions also will likely require some reading and research, and that reading will be sufficiently difficult that some background in clinical epidemiology will be helpful in understanding the reading and concepts. We are interested in receiving feedback about this feature. Please e-mail [email protected] with your comments.
DISCUSSION POINTS 1. As stated in its title, this study1 seeks to compare patients with sickle cell disease who are frequent and infrequent users of the emergency department (ED). The authors recruited patients from “established [sickle cell disease] clinics, health fairs, referrals, and targeted mailings.” A. Contrast this study design to one in which all patients were recruited from ED logs. Draw a Venn diagram comparing the patients likely to be included in these 2 designs. How do the populations differ? B. Describe the pros and cons of each approach focusing on internal and external validity. C. What techniques might be used to overcome the limitations of each approach? D. What ethical issues arise in each study design? 628 Annals of Emergency Medicine
E. Read the abstract’s conclusions. How might you modify them? How might you modify the limitations section? 2. Patients were asked to keep a daily diary for 6 months, though investigators included patients who completed as few as 30 of the 188 study days. Each subject’s data were extrapolated to create 1-year values. Examine the paper’s figure. A. What pattern do you see on the far left of the figure about the number of ED visits by infrequent users? How do you explain this pattern? Does this figure represent the true frequencies or an artifact of the study methods? How could a patient have 1 ED visit per year, given the way data were handled in this study? B. These authors defined frequent users as patients who had at least 3 ED visits per year. According to your clinical experience, are there alternate definitions that might be used? Considering Figure 1, do you think that other definitions might be warranted? 3. The authors write, “[Al]though the data were not statistically significant, high ED utilizers trended toward a higher incidence of anxiety, higher incidence of avascular necrosis and higher WBC count.” Write a sentence that starts, “Although data were not statistically significant, low ED utilizers trended toward . . .” Discuss the meaning of their and your sentence. What are some problems with this approach? How else might the authors have conveyed this information? 4. The authors suggest that the “frequent flier” designation that some emergency physicians give to patients who visit often for sickle cell pain is unjustified because, on average, these patients were sicker than infrequent users. A. Honestly examine your own feelings about the “frequent flier” designation. Do you use the term? How do you feel when others use it? Do you view patients who come in often differently? Do you manage their treatment differently? Do you worry that you may miss a change in their chronic condition? How do you balance pragmatic and moral concerns related to this issue? Volume , . : October
Journal Club
Figure 1. Distribution of frequent ED users among hypothetical populations of study subjects.
B. Similarly, what is your approach to pain management in patients who frequently come to the ED complaining of chronic or acute-on-chronic pain? How does your behavior compare with that of your peers? What arguments do you invoke to justify your approach?
ANSWER 1 Q1. As stated in its title, this study seeks to compare patients with sickle cell disease who are frequent and infrequent users of the emergency department (ED). The authors recruited patients from “established [sickle cell disease] clinics, health fairs, referrals, and targeted mailings.” Q1.a Contrast this study design to one in which all patients were recruited from ED logs. Draw a Venn diagram comparing the patients likely to be included in these 2 designs. How do the populations differ? Figure 1, the Venn diagram we had in mind, illustrates 2 seemingly contradictory conclusions that could both be true: (a) the study is valid and, in the patients recruited for this study, frequent users are a minority and the frequency of visits is associated with disease severity and little else; (b) there exists a large group of patients who use the ED frequently and whose visit frequency is unrelated to disease severity. This paradox highlights the importance of being very clear about the relationship of the study design to the research question. The Aisuku et al1 study examines the ED use behaviors of a group of patients who attend sickle cell clinics, health fairs, etc, and were willing to participate in the study. It does not study the behavior of patients with sickle cell disease who do not participate in the activities targeted for recruitment. It also does not capture the behavior of patients who were identified for recruitment but refused to participate in the study. The sample in the Aisuku et al1 study and the sample that would result from combing ED logs likely differ in many of the characteristics that determine the utilization of health care.2 We are not surprised that patients who are engaged with the health care system—as many of the patients in the Aisuku et al1 sample Volume , . : October
were— used the ED infrequently. It is also wholly conceivable that the kinds of patients who were not recruited for this study may use the ED frequently. An alternative study design that recruited patients from the ED would be a better proxy for patients with sickle cell disease who use the ED for some part of their care, assuming that that study was successful in recruiting and retaining subjects. The important point is that the 2 approaches are complementary and answer different questions. A study of ED users might document an abundance of frequent users whose frequency of use had little correlation with other proxies for severity of illness but would fail to capture the patients who manage their disease with few or no trips to the ED. The current study may have missed the patients who use the ED as their primary source of care regardless of their frequency of use and may underestimate the frequent user problem. Q1.b Describe the pros and cons of each approach focusing on internal and external validity. There are many textbooks that nicely define external and internal validity, and readers are directed to these for detailed study3,4 or, for those who want to study the opposite of validity, to an article that describes 53 distinct forms of bias, each of which can compromise the validity of a study.5 In brief, by “internal validity” we mean the extent to which a study provides an unbiased estimate of the true value, with bias defined as “any process at any stage of inference which tends to produce results or conclusions that differ systematically from the truth.”5 An internally valid study, if repeatedly executed, will produce values that approach, on average, the truth for that population. If one measured the weight of a random sample of female US newborns with a scale that was out of calibration, one would have a study that lacked internal validity because the average weight of the children reported in the study would not approach the average weight of the children in the population. Similarly, because the study by Aisuku et al1 is based on patient diaries, if the subjects systematically over- or underreported their ED visits, then the study would lack internal validity. For randomized trials, the expectation is that if all study limbs were given the same treatment (exposure), then, apart from random error, all would have the same outcome. If this is not the case, then the study lacks internal validity. A study has external validity if the results apply to the population identified in the study question. In the example above, if the baby scale were recalibrated to perfection, the study would have excellent internal validity and external validity as long as the question were, what is the weight of newborn females? However, if the study question were, “what is the weight of newborn infants?”, then the study might have poor external validity if female and male infants differ in weight. External validity issues typically predominate in large, wellconducted, randomized trials because these trials may have great internal validity but may have patient selection issues that render the study population a poor homologue for the population defined in the study question. Annals of Emergency Medicine 629
Journal Club The internal validity of the study by Aisuku et al1 could be increased by actually measuring events (ie, ED visits), rather than relying on self-report. This is true regardless of how the population was recruited. Alternatively, the magnitude of measurement bias could be estimated by reviewing the medical records of a sample of enrolled patients to compare actual ED visits with patient diaries. Our assessment of the external validity of the study depends on our understanding of the study question. The authors write “. . . we sought to better characterize sickle cell disease patients who frequently utilize the ED.” If we assume this is the study question, the study’s sample may be a poor proxy for the target population and the study may lack external validity. The “pro” of the authors’ approach is that they recruited patients who were generally willing to complete the diaries (see below). This made the study feasible. We suspect that the patients missed by the study’s recruitment procedures would be unlikely to participate in the study or complete the diary should they consent to participate. The “con” is that the study may have missed the patients who were most likely to be frequent users of ED services. The considerable pro of the alternate approach is that it would identify the patients most likely to be frequent users. The cons are that it is wholly unclear whether these patients would be willing to consent and participate and that there may be ethical issues about how these patients are identified and approached (see Q1.e below). Q1.c What techniques might be used to overcome the limitations of each approach? As discussed above, the major threat to the external validity of the study by Aisuku et al1 is that it missed the patients most likely to be frequent users The authors could have examined this possibility by stratifying their data on the recruitment method. One might hypothesize that the patients most connected to the health care system (ie, those recruited from dedicated sickle cell clinics) would be least likely to be frequent users, whereas those with weaker ties (eg, those identified at a health fair who had no regular physician) would be more likely. If heterogeneity were found among strata, one would have a better sense of how different the ED-use behavior of those not captured in the study might be. The absence of heterogeneity would suggest that the missed patients may not be that different from those captured. The major limitations of recruitment through the ED are logistics and ethics (see Q1.d), selection bias, and compliance. If patients most likely to use the ED are unwilling to participate or fail to complete the diaries, then the study would be infeasible. Alternate means of outcome measurement (eg, getting permission from patients to use insurance billing data or ED logs to directly measure their utilization) might prove a better strategy for measuring utilization in this group. Q1.d What ethical issues arise in each study design? We begin by reviewing some facts about HIPAA, the Health Insurance Portability and Accountability Act of 1996. HIPAA was created in anticipation of the proliferation of electronic 630 Annals of Emergency Medicine
medical record systems and the concern that patient records produced by such systems could be accessed by unauthorized users. Congress recognized that additional safeguards were needed to keep personal health information private, so they added provisions for further layers of protection. The HIPAA Privacy Rule (Standards for Privacy of Individually Identifiable Health Information) had a mandatory compliance date of April 14, 2003. HIPAA specifies what information is “protected health information”—information of any form (oral, written, electronic) that identifies an individual—and when and with whom such information may be shared. In the design by Aisuku et al,1 there are no obvious HIPAA or ethical issues because the patients keep their own diaries, do not have to tell their treating physicians that they are part of a research study, and are deidentified when the data are presented. There are, however, aspects of the recruitment phase of the study that, if improperly conducted, could be in violation of HIPAA. First, the authors indicate that some patients were recruited by “referral.” It is not explicitly stated where these referrals came from, but if they came from the patient’s physician without the patient’s knowledge, that would be a HIPAA violation. Similarly, it is unclear how recipient lists for the targeted mailings were created. In the alternative strategy, there are several ways in which patients could be identified from the ED. Research assistants could be summoned to the ED to approach patients with sickle cell disease in real time. However, unless an ED has full-time research assistant coverage, this strategy could be costly and infeasible. An alternate approach would be to review billing records for recent visits with International Classification of Diseases, Ninth Revision code 282.6 (sickle cell anemia) and then contact these patients after the fact. Such contact would violate HIPAA (45 CFR 164.514) unless investigators first obtain “a partial waiver of consent/HIPAA” for screening and recruitment purposes from their institutional review board. Several requirements must be met to obtain this waiver: (1) there must be no more than minimal risk to the subjects; (2) the enrollment could not be effectively and practically carried out without obtaining a waiver; (3) waiving consent will not adversely affect the subject’s rights and welfare (ie, they have the right to refuse to participate, they will be provided the opportunity to fully consent to participation, and those who say no will immediately have their recorded identifying information destroyed). Federal regulations further require that if a waiver is granted that researchers obtain only the minimal personal health information that is necessary for recruitment. In this case, that would be name, telephone number, and possibly address. After patients are identified, they should be approached with a standardized letter or verbal script that is institutional review board approved and designed to eliminate intrusive questions and coercion and allows the subject to easily say no. Although somewhat cumbersome, this is a commonly used recruitment Volume , . : October
Journal Club strategy in prospective trials and could have been used to identify ED users in this trial. Q1.e Read the abstract’s conclusions. How might you modify them? The abstract’s conclusion reads “[a] substantial minority of sickle cell disease patients are high ED utilizers. However, high ED utilizers with sickle cell disease are more severely ill as measured by laboratory variables, have more pain, more distress, and have a lower quality of life.” Given the discussion in Q1.a to Q1.d, the truth would be better served if the words “included in this study” were inserted after “patients.” Abstracts are short (Annals has a 250-word limit), and many details conveyed in an article are necessarily omitted from the abstract. Unfortunately, important caveats and qualifiers are often among the omitted details. As a result, those who read only the abstract may get a false sense of the meaning or applicability of an article’s conclusions. How might you modify the limitations section? The article’s limitations section considers how self-reported ED use may differ from actual use but fails to consider the most important limitation of the study, the recruitment strategy. As discussed above, the patients in the study may be poorly representative of the patients who actually visit the ED. This threat to external validity should have been considered either in the methods and results, where the analysis suggested in Q1.c would be reported, or in the limitations section. The limitations section also failed to discuss how the treatment of a continuous outcome variable (number of visits) as a binary one (high versus low use) and the choice of cut point (greater than 2 visits) might affect study results. Authors also failed to address the myriad problems that can arise when extrapolating as little as 1 month’s worth of data to 12 months. Finally, the authors argue that if there is bias in the selfreports of ED use, the bias is toward underreporting. We don’t see why this is necessarily so because those patients who missed many of the diary days (they could miss up to 157 of the 188 days and still be in the study) may be more likely to complete the diary on days when they were in pain or days when they consulted a provider. This could lead to an overestimation of the number of ED visits and to a spuriously enhanced correlation between severity of illness and ED visit frequency. If there is a body of literature that supports their claim, the authors should have referenced it to strengthen their argument.
ANSWER 2 Q2. Patients were asked to keep a daily diary for 6 months, though investigators included patients who completed as few as 30 of the 188 study days. Each subject’s data were extrapolated to create 1-year values. Examine the article’s figure. Q2.a What pattern do you see on the far left of the figure about the number of ED visits by infrequent users? A magnified version of the far left of the article’s figure is shown in Figure 2. We note that most patients had an even number of annual visits (0, 2, 4), whereas few had an odd Volume , . : October
Figure 2. Highly magnified view of left side of the Figure in Aisiku et al.1
number (1, 3, 5). We also note that more than 50% of subjects in this study of ED users actually had no ED visits. How do you explain this pattern? This pattern can be explained by looking closely at the method by which annual visit rates were calculated. The authors collected at most 6 months and as little as 1 month of data on each patient. Annual rates were calculated by multiplying the number of reported visits by the fraction: 365/number of daily diaries completed. For example, if a patient reported 2 ED visits in 3 months of completed diaries, his or her annual total would be 8 visits (2⫻365/91⬇2⫻4⫽8). The Table shows the possible annual rates achieved by this method and reveals the cause of the undulating pattern. Is it representative of the true frequencies or an artifact of the study methods? As explained above, this is an artifact of the multiplier effect. For patients who have no events, we have little idea whether they kept 30, 60, or 188 daily diaries. Similarly, a patient with 1 visit in 1 month of recorded diary would be listed as having 12 visits, but this is a very imprecise estimate of the actual visit rate. How could a patient have 1 ED visit per year, given the way data were handled in this study? According to the multiplier method described and the 6month maximum for data collection, it is not possible for a Annals of Emergency Medicine 631
Journal Club Table. Method for calculating visits/year. Months Completed
No. of Visits
Multiplier
Annual Total
1, 2, 3, 4, 5, or 6 1 2 3 4 5 6 1 2 3 4 5 6
0 1 1 1 1 1 1 2 2 2 2 2 2
12, 6, 4, 3, 2.4, 2 12 6 4 3 2.4 2 12 6 4 3 2.4 2
0 12 6 4 3 ⬇2 2 24 12 8 6 ⬇5 4
Most multipliers are even, and consequently, most rates (annual totals) are even.
patient to have just 1 ED visit per year. As shown in the Table above, the minimum multiplier value is 2; hence, the minimum number of annual visits for a patient with at least 1 visit is 2. We do not know how a patient was given a value of 1, but the existence of this data point reminds us that the scientific literature is imperfect and both authors and peer reviewers can miss what should be easily detectable errors. (Editor’s note: We queried the authors and they explained that rather than rounding, they simply took the integer part of any decimal numbers. Hence 1.96 [a patient with 1 visit in a full 188 days of diaries) was rounded to 1, not 2]. They did not provide a justification for this practice.) Q2.b These authors defined frequent users as patients who had at least 3 ED visits per year. According to your clinical experience, are there alternate definitions that might be used? Considering Figure 1, do you think that other definitions might be warranted? The definition of “frequent user” is certainly in the eye of the beholder. An emergency physician who had a medical condition that required 3 ED visits each year might consider himself a frequent user, but that same physician, told by the nurse that this was a patient’s third visit in 12 months, would not make much of it. Here lies a paradox of this article. The authors based their cut point on the Multicenter Study of Hydroxyurea, which considered patients with 3 or more visits per year sick enough to warrant this treatment. This choice of cut point is somewhat tautologic. Their goal is to show that frequent ED users are sicker than infrequent users, and they use a cut point that has already been shown to define a sicker group of patients by virtue of their frequency of ED use. We can think of many other ways to define frequent users: patients who are treated at least once each month, patients treated several times each week, patients who are known to the ED staff on a first-name basis, patients who have a protocol in place for treatment to be administered when they arrive, patients who self-report that they come to the ED “all the time,” or patients who visit multiple EDs in a given area in a short period. 632 Annals of Emergency Medicine
The authors’ results might be very different had they selected a cut point of 12 or 24. A way out of this problem is to resist the temptation to take a continuous variable, the annual visit rate, and make it binary (high versus low).6 An analysis conducted using 4 groups (no use, 1 to 6 visits per year, 7 to 19 visits per year, and ⱖ20 visits per year) would provide richer information that might illuminate important patterns that cannot be seen with this binary treatment. An alternative method would be to use graphics to show each patient’s disease severity versus ED use so that readers could see patterns in the data. Another strategy for making a continuous variable categorical is to look at the data before making the grouping. Although it is generally advisable to let theory guide the creation of categories, pragmatic considerations related to sample size are sometimes important. Consider a study of children. According to theory, one might argue that meaningful groupings would be age younger than 6 months, 6 months to 2 years, 3 to 12 years, and 13 to 17 years. Age quartiles (created by making 4 equally sized groups) might be 0 to 4 months, 4 months to 3 years, 3 to 15 years, and 16 to 17 years. The quartiles create equal-sized groups but groups that may make little sense (eg, 3 to 15 years), depending on the clinical question. Investigators need to balance sample size considerations and theoretic considerations, with the latter given emphasis in questionable situations. In this case (see the figure in Aisuku et al1), the first 2 quartiles would include 53% of the patients (because of ties), all of whom had no ED visits. The third quartile (22%) would include patients with 1 to 5 visits, and the fourth quartile (25%) would be greater than or equal to 6 visits. One would have to wrestle with the cut points suggested by theory (such as those mentioned 2 paragraphs above) and the need to have an adequate N in each group. In this case, we would let theory prevail and accept that the high-use groups will have substantially less precision (because of smaller Ns) than the lowuse groups.
ANSWER 3 Q3. The authors write “[Al]though the data were not statistically significant, high ED utilizers trended toward a higher incidence of anxiety, higher incidence of avascular necrosis and higher WBC count.” Write a sentence that starts, “Although data were not statistically significant, low ED utilizers trended toward.. . .” Although data were not statistically significant, infrequent ED utilizers trended toward being wealthier (33% versus 49% earned ⬍$10,000), older (53% ⱖ35 years of age versus 39%), and receiving care at a specialty center (53% versus 43%). (Editor’s note: There are other possible sentences but this one serves to make our general point.) Discuss the meaning of their and your sentence. If this were a randomized trial with no bias and no measurement error, then, in a classic statistics framework, the absence of statistical significance would be taken to mean that we have insufficient evidence to reject the null hypotheses about the factors mentioned in these sentences. This failure could Volume , . : October
Journal Club represent a good decision (a true negative) because there exists no true difference in these factors in low and high ED users, and the observed differences are solely due to random error. It could also represent a type II (false negative), or , error. The likelihood of this being a type II  error depends on the power of the study (the sample size relative to the variance of the measure in question). For example, if the investigators of this hypothetical randomized trial had powered their study to have a  of .1 (power of .9), then there is only a 10% chance that the failure to find statistical significance would be a type II error, and the more reasonable interpretation is that the observed differences represent random variation. This is not, however, a large randomized trial. This is an observational study of self-reported ED use, subject to any number of systematic biases and systematic and random forms of measurement error. In this case, the interpretation of observed differences is far more complicated than in a randomized controlled trial because differences could be real (approximating the truth except for random error), due to bias, due to measurement error, due to random error, or due to any combination of the above. Furthermore, because the authors provide no description of why their study is this particular size (no sample size or power calculation), the failure to achieve statistical significance could be due to the absence of an important effect (true negative) or to a lack of power (type II error). We are therefore left in a bit of a quandary about how to interpret these nonsignificant results and, for that matter, the significant ones. We lack access to highly stratified data that might help us decide whether results represent truths, random error, or bias. (Editor’s note: We suspect that some of our readers had questions about the use of the Bonferroni method to reduce ␣ to account for multiple testing. We promise to cover this in detail in a future journal club article but here acknowledge that this may not be the best way to handle this problem.7) What are some problems with this approach? The main problem with the authors’ sentence is that it tells only half the story. If authors highlight the “nonsignificant” results that support their argument, should they not, in the name of balance, also mention results that are contrary? What scientific basis is there for including their sentence but not ours? How else might the authors have conveyed this information? Our preferred alternative is to say nothing. The data tables (particularly Tables 2 and 3 that provide between-group differences with their confidence intervals) give readers a sense for the magnitude of each variable in each group and how they differ. Readers can interpret the meaning of these differences as they wish. Readers might be aided by more detailed tables and figures that show further stratification of the results but are unlikely to be helped by sentences that cherry pick selected results.
ANSWER 4 Q4. The authors suggest that the “frequent flier” designation that some emergency physicians give to patients who visit often for Volume , . : October
sickle cell pain is unjustified because, on average, these patients were sicker than infrequent users. Q4.a Honestly examine your own feelings about the “frequent flier” designation. Meditation cushions are available for this purpose at http:// annemergmed.com. Just kidding. Do you use the term? We will not attempt to answer these questions in any comprehensive way or attempt to say which opinions are “correct.” We posed these questions to stimulate journal club participants to have a candid discussion about these difficult issues. In the name of complete disclosure, we do our best to answer the questions honestly. We make no claim that our answers are more important than anyone else’s answers. The following is simply a summary of our opinions, which can be used as a conversation starter. We both admit to using the term at times and that it was difficult to write these answers without resorting to it. We chose not to because “frequent flier” means different things to different people. The literal slang meaning is that someone uses a service often, be it popping into Starbucks daily at 7:05 AM for a java fix or appearing frequently at a neighborhood ED. The words have no inherent judgmental quality. The negative connotation is instilled by context— how it is used and who is using it. In our experience, the term is occasionally used nonjudgmentally, (for certain nonsmelly, undemanding, non– drug-seeking, anxious [but not too anxious] little old ladies who like to check in periodically) but more frequently is loaded with judgments about the appropriateness and motivations of the patient’s visit. How do you feel when others use it? Although we both use the term on occasion, it worries us when undergraduate research assistants, medical students, and junior residents use it. Neither of us has a degree in semiotics, but we believe that changes in language often precede cultural change and worry that the use of this language by those whose attitudes are still being shaped may foster the development of a more callous attitude toward all kinds of needy patients. As much as we hate to admit it, we think we can all agree that most of the time when we call a patient a “frequent flier,” we do not mean it as a compliment. This logic leads us to question our own use of the term and whether, as teachers, we should be avoiding language that could foster undesirable attitudes in our students. Do you view patients who come in often differently? Do you treat their disease differently? Do you worry that you may miss a change in their chronic condition? We strive to avoid prejudging patients, but for select individuals, we have strong opinions before entering the room. We hope that our self-awareness of our tendency to do this helps us avoid errors that can occur when one assumes that the patient is here “for the same old thing” or when one fails to hear the patient saying that “something is different.” Neither of us feels that we systematically manage frequent users differently Annals of Emergency Medicine 633
Journal Club from infrequent users and believe that our knowledge of a patient’s frequent-user status affects our treatment more than our assessment. How do you balance pragmatic and moral issues related to this issue? We struggle to define what constitutes an appropriate balance of “pragmatic and moral issues” and leave you to work out your own comfort zone with respect to these issues. Q4.b Similarly, what is your approach to pain management in patients who frequently come to the ED complaining of chronic or acute-on-chronic pain? Patients are generally assumed to have legitimate pain concerns that warrant treatment unless there is a specific history to suggest otherwise. How does your behavior compare with that of your peers? The behaviors exhibited by emergency physicians are as wide as the day is long. Some of our peers regularly give patients whatever they want (within a “3 shots and be admitted or discharged” policy). These faculty members believe that even if they question the patient’s authenticity, it is better to treat the patient than risk failing to treat a patient in need or create a confrontation. The other group believes that there are a small number of patients who resist all attempts to be shepherded into appropriate continuing care. These attending physicians, after giving patients ample warning that they will not be given narcotics unless they establish a demonstrable relationship with a continuing care provider, will not give narcotics to patients who fail to do so. We fall in the latter group. What arguments do you invoke to justify your approach? Being an emergency physician for a frequent user who is seeking narcotics but refuses to establish a relationship with a primary care provider is a bit like being the parent of a toddler or teenager. In the short term, saying yes is the easiest path; it makes all parties happy and eliminates all of the conflict and drama. But one has cause to worry about the children of parents who consistently use this strategy. Saying no is harder in the moment but may lead to better long-term outcomes. Many emergency physicians for whom we have great respect argue that
634 Annals of Emergency Medicine
they would rather have many false-positive results (giving narcotics to someone who is merely drug seeking) than 1 falsenegative result (denying someone in pain adequate analgesia).8 They also argue that there is little harm in giving a narcotics addict narcotics. However, we find these arguments a bit shortsighted because they fail to consider the negative long-term consequences of these strategies. We believe that many experienced emergency physicians will learn to trust their gut, rather than always erring on the side of giving narcotics. Each of us knows patients whom we refuse to treat with narcotics. In our experience, when these patients arrive in the ED and learn that one of us is the attending physician on duty, they leave. We are each satisfied with this arrangement. Section editors: Tyler W. Barrett, MD; David L. Schriger, MD, MPH
REFERENCES 1. Aisiku IP, Smith WR, McClish DK, et al. Comparisons of high versus low emergency department utilizers in sickle cell disease. Ann Emerg Med. 2009;53:587-593. 2. Aday LA, Andersen R. A framework for the study of access to medical care. Health Serv Res. 1974;9:208-220. 3. Rothman KJ, Greenland S, Lash TL. Validity in epidemiologic studies. In: Rothman KJ, Greenland S, Lash TL, eds. Modern Epidemiology. 3rd ed. Philadelphia, PA: Lippincott; 2008:128-129. 4. Altman D. Clinical trials. In: Practical Statistics for Medical Research. London: Chapman & Hall; 1991. 5. Sackett DL. Bias in analytic research. J Chronic Dis. 1979;32:5163. 6. Altman DG, Royston P. The cost of dichotomizing continuous variables. BMJ. 2006;332:1080. 7. Rothman KJ, Greenland S. Fundamentals of epidemiologic data analysis. In: Rothman KJ, Greenland S, Lash TL, eds. Modern Epidemiology. 3rd ed. Philadelphia, PA: Lippincott; 2008:236-237. 8. Henry G. What’s your pain care philosophy? Emergency Physicians Monthly Web site. Available at: http://www.epmonthly.com/index. php?option⫽com_content&task⫽view&id⫽453&Itemid⫽91. Accessed April 13, 2009. (Archived by WebCite at http://www. webcitation.org/5g0KuiMcP).
Volume , . : October