Publication Bias in Surgery: Implications for Informed Consent

Journal of Surgical Research 143, 88 –93 (2007) doi:10.1016/j.jss.2007.03.035

Publication Bias in Surgery: Implications for Informed Consent Dora Syin, B.S.,*,§ Tinsay Woreta, B.S.,* David C. Chang, Ph.D., M.P.H., M.B.A.,*,‡ John L. Cameron, M.D.,* Peter J. Pronovost, M.D., Ph.D.,†,‡,§ and Martin A. Makary, M.D., M.P.H.*,‡,§,1 *Center for Outcomes Research, Department of Surgery, and †Department of Anesthesiology and Critical Care Medicine, John Hopkins University School of Medicine; ‡Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health; and §Johns Hopkins Quality and Safety Research Group, Johns Hopkins Medical Institutions, Baltimore, Maryland Submitted for publication January 8, 2007

5601), P < 0.0001), but still higher than the literaturebased rate of 3.2% (P < 0.0001). Conclusions. Mortality rates for pancreatic resections in actual practice are 2.4-fold higher than those reported in the literature. Proper informed consent for surgical procedures should include an accurate description of the risks, using actual local and national mortality rates. © 2007 Elsevier Inc. All rights reserved. Key Words: publication bias; surgery; pancreatic cancer; pancreaticoduodenectomy; informed consent.

Background. Patients consenting for pancreas surgery are often quoted an operative risk of 1% to 3% based on the literature. However, these results are often from centers of excellence, and as a result the literature mortality rates may not be representative or generalizable. Methods. A MEDLINE search was performed to identify the major studies of pancreaticoduodenectomy (PD) and total pancreatectomy (TP) over a 6-y period (January 1998 –December 2003). To obtain a literaturebased mortality rate, we performed a meta-analysis of these published series and compared them with actual in-hospital mortality rates based on a representative 20% sample of hospital data in 37 states (the Nationwide Inpatient Sample). The sample included approximately 8 million patient records per year. Literature versus actual mortality rates were compared for the same 6-y period and stratified by academic versus nonacademic medical centers. Results. We identified 16 major studies on PD and TP totaling 3641 patients with an overall mortality rate of 3.2% (range 0%–9.3%). The actual mortality rate based on the Nationwide Inpatient Sample (n ⴝ 7604) was 2.4-fold higher than the literature rate (adjusted rate of 7.6% versus 3.2%, P < 0.0001). All literature-based series were published from academic medical centers. By contrast, in the national database, 26.3% of PDs (2003/7604) were performed at nonacademic medical centers with a mortality rate of 11.4% (229/2003). The actual mortality rate at academic medical centers was lower than nonacademic medical centers (6.4% (360/

INTRODUCTION

Mortality rates quoted to patients prior to surgery are often based on outcomes reported in the surgical literature. However, these complication rates may be subject to publication bias since series demonstrating good patient outcomes are more likely to be submitted than research producing poor or negative findings [1– 8]. In addition, journal reviewers and editors are more likely to accept studies with positive results for publication, and publishers have a financial incentive to print articles with the largest appeal. The cumulative result is that negative trials and series with poor outcomes are under-represented on Internet medical search engines. Because understating the risk of a procedure can alter surgical decision-making for providers as well as patients, measuring the degree of publication bias in surgery is an important issue. Significant advances have been made in the safety of pancreatic resection, but the overall prognosis remains poor and the decision to operate can be difficult for patients with advanced age or multiple comorbidities. Patients consenting for surgery are often quoted an operative risk of 1% to 3% based on the wellestablished and commonly cited series in the literature [9 –14]. To better understand the extent to which liter-

1 To whom correspondence and reprint requests should be addressed at Surgery and Health Policy and Management, Johns Hopkins University School of Medicine, Johns Hopkins Medical Institutions, 600 N. Wolfe Street, Carnegie 683, Baltimore, MD 21287. E-mail: [email protected].

0022-4804/07 $32.00 © 2007 Elsevier Inc. All rights reserved.

88

89

SYIN ET AL.: PUBLICATION BIAS IN SURGERY

ature mortality rates may not be generalizable, we designed a study to measure the degree of publication bias in surgery, by comparing mortality rates following pancreatic resections. METHODS A MEDLINE search was performed to identify the major studies of pancreaticoduodenectomy (PD) over a 6-y period (January 1998 – December 2003) using the search terms “operative mortality of pancreaticoduodenectomy”. To be included in this study, an article had to provide the results for the postoperative mortality of a group of patients undergoing PD. Postoperative mortality was defined as in-hospital deaths or deaths within 30 d of surgery. We then crossreviewed the citation lists of all publications found on MEDLINE to identify additional large series. Analyses were limited to Englishlanguage publications. To obtain a literature-based mortality rate, a meta-analysis of these published series was performed and a pooled estimate for mortality rate was calculated. Actual mortality rates were calculated using six years of the Nationwide Inpatient Sample database (1998 –2003), a 20% representative sample of hospital admissions in 37 states. The sample included approximately 8 million patient records per year. Inclusion criteria for the analysis were patients who underwent pancreatic resections with radical PD (International Classification of Diseases, Ninth Revision [ICD-9] procedure code 52.7) or total pancreatectomy (TP) (procedure code 52.6). The actual mortality rate was adjusted for age, gender, and percentage with malignancy to match the literature series. Criteria for malignancy included malignant neoplasms of the duodenum (ICD diagnosis code: 152.0), extrahepatic bile ducts (ICD diagnosis code: 156.1), Ampulla of Vater (ICD diagnosis code: 156.2), and pancreas (ICD diagnosis: 157.x). Literature versus actual mortality rates were compared for the same 6-y period and stratified by academic versus nonacademic medical center. Academic medical centers were identified by possession of an American Medical Association (AMA) approved residency program, membership in the Council of Teaching Hospitals (COTH), or an interns and residents to beds (IRB) ratio of 0.25 or higher. ␹ 2 and Fisher’s exact tests were used to perform statistical analysis in Stata ver. 9.2 (Stata Corp., College Station, TX).

RESULTS

We identified 16 major studies of PD and TP published within the 6-y time period (January 1998 – December 2003), comprising 3641 patients [15–30] (Table 1). The mean age of literature patients was 63.5 y, and 40% were female; 84.1% of patients underwent pancreatic resection for malignancy. Most series were retrospective (56.3%, 9/16) and observational (75.0%, 12/16). All series were performed at single institutions (100.0%, 16/16) and academic medical centers (100.0%, 16/16). Since several studies did not differentiate between in-hospital and/or 30-d mortality, both definitions were included in determining postoperative mortality. From the 16 series published within the designated time period, we found an overall postoperative mortality rate of 3.2% (116/3641), with a range 0 to 9.3%. Using the Nationwide Inpatient Sample (NIS), we identified 7604 patients who underwent PD or TP during the same 6-y time period (Table 2). The mean age of

NIS patients was 62.1 y, and 47.1% were female; 71.5% of patients underwent surgery for malignancy. Outcomes in the NIS were limited to in-hospital mortality. The unadjusted in-hospital mortality rate was 7.7% (589/7604), which was significantly higher than the corresponding literature mortality rate (3.2%) during the same time period (P ⬍ 0.0001). The adjusted inhospital mortality rate was 7.6%. Most cases identified in the NIS were performed at academic medical centers (73.7%) with a mortality rate of 6.4% (360/5601); the mortality rate at nonacademic medical centers was 11.4% (229/2003). While the actual mortality rate at academic medical centers was lower than at nonacademic medical centers (6.4% versus 11.4%, P ⬍ 0.0001), it was still significantly higher than the literature-derived mortality rate of academic medical centers (6.4% versus 3.2%, P ⬍ 0.0001). Thus, the actual mortality rate was 2.4-fold higher than the literature rate (2-fold higher for academic centers and 3.6-fold higher for nonacademic centers). CONCLUSIONS

Studies with positive or favorable results are more likely to be submitted, expedited through review, and published than negative or less favorable studies [31]. In fact, negative or null studies have been shown to be 2.3 times less likely to reach publication [2]. Observational studies such as case series, which are highly prevalent among top surgery journals [32], are particularly at risk—their exposure to publication bias is 3.8 times greater than randomized clinical trials. In the present study, we confirm these findings and report that actual mortality for PD and TP is over 2-fold higher than published studies suggest. These findings have implications for the way we report research findings as a profession and, moreover, for the way we communicate operative risk to our patients. The Problem

Complication rates obtained from the published literature are often quoted as standard complication rates for a procedure. These best practice results are commonly cited by other authors and adopted by the surgical community. For example, pancreatic surgery is frequently described in textbooks with mortality rates based on series from the top specialty centers in the world. Because information in the surgical literature disseminates quickly, publication bias must be recognized and addressed. Sources of Bias

Biased selection can occur at many different stages in the publication process, as summarized in Table 3. One source of bias that is gaining attention is the

90

JOURNAL OF SURGICAL RESEARCH: VOL. 143, NO. 1, NOVEMBER 2007

TABLE 1 Literature Series, Patient Characteristics, and Mortality Rates for PD and TP, 1998 –2003 Study Sohn et al. [15]

Povoski et al. [16] Bottger et al. [17] Schwarz et al. [18] Millikan et al. [19]

Yeo et al. [20]

Study design

1998

Retrospective Age ⱖ 80 years Age ⬍ 80 years Retrospective Prospective, case-control Retrospective Retrospective In-hospital 30-day Prospective, randomized Standard PD Radical PD Prospective, randomized Classic Whipple Pylorus-preserving PD Prospective, randomized Classic Whipple Pylorus-preserving PD Retrospective Preoperative biliary stenting No preoperative biliary stenting Retrospective Age ⱖ 70 years Age ⬍ 70 years Prospective Retrospective Prospective, randomized Standard PD Radical PD Retrospective Retrospective Prospective

1998 1999 1999 1999

1999

Lin et al. [21]

1999

Seiler et al. [22]

2000

Sohn et al. [23]

Hodul et al. [24]

Bassi et al. [25] Pisters et al. [26] Yeo et al. [27]

Duffy et al. [28] Aranha et al. [29] Billingsley et al. [30] Totals

Patients N (%)

Year

2000

2001

2001 2001 2002

2003 2003 2003

1998–2003 Retrospective 9 (56.3) Observational 12 (75.0) Single institution 16 (100.0) Academic center 16 (100.0)

46 (1.3) 681 (18.7) 240 (6.6) 221 (6.1) 54 (1.5)

Age (Mean) Gender female, Malignancy Mortality (SD) N (%) N (%) N (%)

82.9 (2.7) 60.9 (12.4) 66 Median 61 63 Median 61.9

27 (59) 304 (45) 106 (46) 93 (42) 28 (52) 41 (55)

75 (2.1)

43 (93.5) 544 (79.9) 190 (79.2) 193 (87.3) 50 (92.6)

2 (4.3) 11 (1.6) 12 (5.0) 7 (3.1) 0 (0.0)

75 (100.0)

3 (4.0) 1 (1.3)

56 (1.5) 58 (1.6)

64.6 (1.4) 65.4 (1.2)

21 (37) 32 (55)

56 (100.0) 58 (100.0)

3 (5.4) 2 (3.4)

15 (0.4) 16 (0.4)

59 60 66

4 (27) 9 (56) 41 (53)

15 (100.0) 16 (100.0) 61 (79.2)

0 (0.0) 1 (6.3)

40 (1.1) 37 (1.0)

2 (5.0) 1 (2.7)

408 (11.2) 159 (4.4)

63.8 (0.6) 61.4 (1.2)

188 (46) 81 (51)

301 (73.8) 90 (56.6)

7 (1.7) 4 (2.5)

48 (1.3) 74 (2.0) 150 (4.1) 300 (8.2)

74.7 (3.5) 57.7 (10.0) 59 Median 62 Median

19 (40) 37 (50) 64 (43) 136 (45)

44 (92.0) 67 (91.0) 90 (60.0) 244 (81.3)

0 (0.0) 1 (1.4) 3 (2.0) 4 (1.3)

146 (4.0) 148 (4.1) 55 (1.5) 152 (4.2) 462 (12.7)

66.2 (0.9) 65.2 (0.9) 67.3 65.7 (11.7) 65.3 (9.6)

146 (100.0) 148 (100.0) 55 (100.0) 114 (75.0) 462 (100.0)

6 (4.1) 3 (2.0) 0 (0.0) 0 (0.0) 43 (9.3)

3641 (100.0)

63.5

61 (42) 73 (49) 21 (38.2) 67 (44) 7 (1.5) 1460 (40)

3062 (84.1)

116 (3.2)

Note. Some studies stratified their patient populations and reported separate mortality rates for each group (e.g., 1.6% for patients ⬍80 years old, 4.3% for patients ⱖ80 years old). For this table, all patients and mortality rates were reported as in the literature, to illustrate the raw numbers used in meta-analysis.

incomplete reporting of outcomes within published series [33–35]. Chan et al. identified 102 trials that ultimately led to 122 published journal articles and 3736 outcome reports [33]. They found that 50% of efficacy outcomes and 65% of harm outcomes per trial were

incompletely reported. Moreover, statistically significant outcomes were more likely to be fully reported than nonsignificant outcomes. When final publications were compared with study protocols, 62% of trials were found to have changed, introduced, or omitted at least

TABLE 2 Nationwide Inpatient Sample, Patient Characteristics, and Mortality Rates for PD and TP, January 1998 –December 2003 Institution

Patients N (%)

Academic medical centers Non-academic medical centers Totals

5601 (73.7) 2003 (26.3) 7604 (100.0)

Age (mean)

62.1

Gender female, N (%)

Malignancy N (%)

3581 (47.1) 5437 (71.5) Adjusted for age, gender, and malignancy (7.6)

Mortality N (%) 360 (6.4) 229 (11.4) 589 (7.7)

91

SYIN ET AL.: PUBLICATION BIAS IN SURGERY

TABLE 3 Potential Sources of Bias, Literature versus Administrative Databases Stage of publication

Literature

Study design Study recruitment Study implementation Data reporting Data collection Data analysis Authors’ decision to publish Journal submission Journal peer review Editors’ decision to publish

X X X X X X X X X X

Administrative databases X

X X X X X X X

one primary outcome. The tendency for investigators to internally censor unfavorable results has been cited as a major root cause in the problem of publication bias [7, 36]. Limitations

There are some important limitations to this study. First, mortality data in some literature series were reported only as 30-d mortality and were included in the analysis; however mortality data in the NIS was limited to in-hospital deaths. Consequently, our results do not equitably capture out-of-hospital 30-d mortality in the NIS and, thus, understate the difference between published and actual mortality. The real-world difference between literature and actual mortality rates is likely even greater than our conservative estimate. Second, we chose to focus on only pancreatic resection. This was based on the understanding that pancreatic surgery is a surrogate for other major procedures, which have large variations in outcomes. Academic versus Nonacademic Medical Centers

Academic medical centers tend to be high-volume referral institutions with greater subspecialty experience and refined protocols for procedures or conditions seen with high frequency. These infrastructure components have all been associated with improved outcomes [37– 43], and academic medical centers frequently outperform nonacademic centers using standardized metrics of quality. However, academic status is not a requisite for high quality care [44 – 46]; additionally, many hospitals lie within a continuum of academic status and do not fit well into a dichotomous distinction. Alternately, the disparity between academic and nonacademic hospitals may not reflect true differences in outcomes, but differences between patient populations. This is reflected in currently unpublished results of a recent multivariate analysis, which indicates that there is no difference in outcomes between academic and nonacademic medical centers, after adjusting for

Type of bias Sampling bias Selection bias Institutional (referral) bias, Surgeon (experience) bias Selective reporting within studies (outcomes) Sampling bias Selective reporting within studies (outcomes) Publication bias Publication bias Publication bias Publication bias

patient age, gender, race, Charlson Comorbidity Index, and annual hospital volume (Chang DC, et al. Personal communication. January 2, 2007. Unpublished data). Furthermore, smaller hospitals are often nonacademic yet critical to our healthcare system. Many such hospitals deliver surgical care to rural communities, which lack access, resources, and insurance coverage to be treated far away. Thus, the difference in outcomes by academic status should be regarded with caution and should not be used for policy-making. Implications for Patients and Providers

Publication bias has important implications for patient care. Quoting the results of selected best series in the surgical literature or textbooks can be misleading. To address this issue, some surgeons maintain personal databases to track their patient outcomes. However, this is not always feasible or informative. An institutional, community, or regional database may be more practical, particularly for lower-volume procedures. Yoshimoto proposed a prospective, communitybased registry of all patients undergoing a high-risk neurosurgical procedure to provide an objective sampling frame for surgical decision-making [5]. The American College of Surgeons’ National Safety and Quality Improvement Program was established to capture complication rates specific to each institution, to tailor improvements in the delivery of care [47]. These reporting systems aim to provide complete follow-up of the tracked patients so that accurate outcomes can be viewed by providers. It is important for surgeons to recognize that national mortality rates may be higher than the subset represented in the literature, so that they can give a more global perspective to their patients during preoperative discussion. Proper informed consent for surgical procedures should include an accurate description of the risks, using actual local and national mortality rates. For example, a surgeon might suggest that “Nationally, operative

92

JOURNAL OF SURGICAL RESEARCH: VOL. 143, NO. 1, NOVEMBER 2007

mortality can be as high as 7% to 9%, but at this institution our rates are about 1%.” Implications for Researchers

Although it takes time to change an academic culture, journal editors and the NIH have made much progress. It may be difficult to completely eliminate the occurrence of publication bias, but we can minimize its influence. A variety of practical approaches have been suggested. For trials funded by the NIH, failure to report all research findings is now considered scientific misconduct, since patient volunteers and funding agencies participate in these studies for their contribution to medical knowledge [48, 49]. Incomplete reporting also complicates practice recommendations by skewing systematic reviews, meta-analyses, and evidence-based practice guidelines based on the published literature. In response to increasing concern about this practice, the Journal of the American Medical Informatics Association issued a special call for papers reporting “null, negative, or disappointing results” in February 2000 [50]. This resulted in publication of two such articles, with encouragement of the medical community to recognize all potentially informative work. Chalmers et al. had several recommendations for decreasing publication bias: (1) insisting on highquality research and thorough literature reviews, (2) eliminating the double standard applied to clinical research but not to clinical practice, (3) publishing legitimately conducted trials regardless of their results, (4) increased oversight of authors and peer reviewers with conflicts of interest, and (5) shifting away from review articles toward meta-analyses [51]. Chalmers and Dickersin recommended prospective registration of all clinical trials [1, 51, 52]. A comprehensive registry would benefit not only clinicians and patients seeking therapeutic guidance, but patients in search of appropriate trials for enrollment, reviewers establishing practice guidelines, and researchers seeking study questions. Widespread support exists for expansion to an international level, for the most complete data and analytical pool [3]. Significant progress has been made toward these ends on multiple levels, but efforts are still fractured and many challenges remain [52]. The Open Source Model

“Open source” refers to information that is publicly accessible. The model was pioneered in the physics community and has recently been popular in the information technology/software industry. In medicine, the Journal of Burns and Wound Care created a web-based repository for all burn and wound research, allowing physician researchers to instantly report their experiences with colleagues [53]. The portal is free and ac-

cessible worldwide. Such an information revolution has many barriers, however; namely the publishing industry and professional societies, which gain revenue from selling research findings. Additionally, the removal of peer review may allow incorrect inferences to be propagated, as well as the reporting of studies with potential conflicts of interest. CONCLUDING REMARKS

Providers should be aware of the disparity between real and reported outcomes of major index surgical procedures, and factor this into their surgical decisionmaking. Furthermore, patients and their families must be properly informed when obtaining their consent. Researchers and journals should recognize that negative results may have equal clinical utility to positive results when considering papers for publication, and that negative trials and less favorable outcome studies must be available to clinicians. Appropriately considering the degree of bias in the literature, minimizing conflicts of interest in research, and creating an open source model for medical information are potential solutions aimed at addressing publication bias. REFERENCES 1. 2. 3. 4. 5.

6. 7. 8.

9.

10.

11.

12.

13.

14.

Dickersin K. The existence of publication bias and risk factors for its occurrence. JAMA 1990;263:1385. Easterbrook PJ, Berlin JA, Gopalan R, et al. Publication bias in clinical research. Lancet 1991;337:867. Simes RJ. Publication bias: The case for an international registry of clinical trials. J Clin Oncol 1986;4:1529. Kiroff GK. Publication bias in presentations to the Annual Scientific Congress. ANZ J Surg 2001;71:167. Yoshimoto Y. Publication bias in neurosurgery: Lessons from series of unruptured aneurysms. Acta Neurochir 2003;145: 45– 48. Dickersin K, Chan S, Chalmers TC, et al. Publication bias and clinical trials. Control Clin Trials 1987;8:343. Dickersin K, Min YI. Publication bias: The problem that won’t go away. Ann NY Acad Sci 1993;703:135. Dickersin K, Min YI, Meinert CL. Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. JAMA 1992;267:374. Yeo CJ, Cameron JL, Sohn TA, et al. Six hundred fifty consecutive pancreaticoduodenectomies in the 1990s: Pathology, complications, and outcomes. Ann Surg 1997;226:248. Fernandez-del-Castillo C, Rattner DW, Warshaw AL. Standards for pancreatic resection in the 1990s. Arch Surg 1995; 130:295. Trede M, Schwall G, Saeger HD. Survival after pancreatoduodenectomy. One hundred eighteen consecutive resections without an operative mortality. Ann Surg 1990;211:447. Geer RJ, Brennan MF. Prognostic indicators for survival after resection of pancreatic adenocarcinoma. Am J Surg 1993; 165:68. Cameron JL, Pitt HA, Yeo CJ, et al. One hundred forty-five consecutive pancreaticoduodenectomies without mortality. Ann Surg 1993;217:430. Yeo CJ, Cameron JL, Lillemoe KD, et al. Pancreaticoduodenec-

SYIN ET AL.: PUBLICATION BIAS IN SURGERY

15.

16.

17.

18.

19.

20.

21.

22.

23.

24. 25.

26.

27.

28.

29.

30.

31. 32.

tomy for cancer of the head of the pancreas. Two hundred one patients. Ann Surg 1995;221:721. Sohn TA, Yeo CJ, Cameron JL, et al. Should pancreaticoduodenectomy be performed in octogenarians? J Gastrointest Surg 1998;2:207. Povoski SP, Karpeh MS, Jr., Conlon KC, et al. Association of preoperative biliary drainage with postoperative outcome following pancreaticoduodenectomy. Ann Surg 1999;230:131. Bottger TC, Junginger T. Factors influencing morbidity and mortality after pancreaticoduodenectomy: Critical analysis of 221 resections. World J Surg 1999;23:164. Schwarz RE, Hemant K, Ellenhorn JDI. A mortality-free decade of pancreatoduodenectomy: Is quality independent of quantity? Am Surg 1999;65:949. Millikan KW, Deziel DJ, Silverstein JC, et al. Prognostic factors associated with resectable adenocarcinoma of the head of the pancreas. Am Surg 1999;65:618. Yeo CJ, Cameron JL, Sohn TA, et al. Pancreaticoduodenectomy with or without extended retroperitoneal lymphadenectomy for periampullary adenocarcinoma: Comparison of morbidity and mortality and short-term outcome. Ann Surg 1999;229:613. Lin PW, Lin YJ. Prospective randomized comparison between pylorus-preserving and standard pancreaticoduodenectomy. Br J Surg 1999;86:603. Seiler CA, Wagner M, Sadowski C, et al. Randomized prospective trial of pylorus-preserving versus classic duodenopancreatectomy (Whipple procedure): Initial clinical results. J Gastrointest Surg 2000;4:443. Sohn TA, Yeo CJ, Cameron JL, et al. Do preoperative biliary stents increase post-pancreaticoduodenectomy complications? J Gastrointest Surg 2000;4:258. Hodul P, Tansey J, Golts E, et al. Age is not a contraindication to pancreaticoduodenectomy. Am Surg 2001;67:270. Bassi C, Falconi M, Salvia R, et al. Management of complications after pancreaticoduodenectomy in a high volume center: Results on 150 consecutive patients. Dig Surg 2001;18:453. Pisters PWT, Hudec WA, Hess KR, et al. Effect of preoperative biliary decompression on pancreaticoduodenectomy-associated morbidity in 300 consecutive patients. Ann Surg 234:47. Yeo CJ, Cameron JL, Lillemoe KD, et al. Pancreaticoduodenectomy with or without distal gastrectomy and extended retroperitoneal lymphadenectomy for periampullary adenocarcinoma, Part 2: Randomized controlled trial evaluating survival, morbidity, and mortality. Ann Surg 2002;236:355. Duffy JP, Hines OJ, Liu JH, et al. Improved survival for adenocarcinoma of the ampulla of Vater: Fifty-five consecutive resections. Arch Surg 2003;138:941. Aranha GV, Hodul PJ, Creech S, et al. Zero mortality after 152 consecutive pancreaticoduodenectomies with pancreaticogastrostomy. J Am Coll Surg 2003;197:223. Billingsley KG, Hur K, Henderson WG, et al. Outcome after pancreaticoduodenectomy for periampullary cancer: An analysis from the Veterans Affairs National Surgical Quality Improvement Program. J Gastrointest Surg 2003;7:484. Egger M, Smith GD. Misleading meta-analysis. BMJ 1995;310: 752. Chang DC, Matsen SL, Simpkins CE. Why should surgeons care about clinical research methodology? J Am Coll Surg 2006; 203:827. Epub 2006 Oct 25.

33.

34.

35.

36. 37.

38.

39.

40.

41.

42.

43.

44. 45.

46. 47.

48. 49.

50. 51. 52. 53.

93

Chan A, Hrobjartsson A, Haahr MT, et al. Empirical evidence for selective reporting of outcomes in randomized trials: Comparison of protocols to published articles. JAMA 2004; 291:2457. Chan A, Krleza-Jeric K, Schmid I, et al. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ 2004;171:735. Chan A, Altman DG. Identifying outcome reporting bias in randomized trials on PubMed: Review of publications and survey of authors. BMJ 2005;330:753. Olson CM, Rennie D, Cook D, et al. Publication bias in editorial decision making. JAMA 2002;287:2825. Birkmeyer JD, Finlayson SR, Tosteson AN, et al. Effect of hospital volume on in-hospital mortality with pancreaticoduodenectomy. Surgery 1999;125:250. Gordon TA, Bowman HM, Bass EB, et al. Complex gastrointestinal surgery: Impact of provider experience on clinical and economic outcomes. J Am Coll Surg 1999;189:46. Kotwall CA, Maxwell JG, Brinker CC, et al. National estimates of mortality rates for radical pancreaticoduodenectomy in 25,000 patients. Ann Surg Oncol 2002;9:847. Ho V, Heslin MJ. Effect of hospital volume and experience on in-hospital mortality for pancreaticoduodenectomy. Ann Surg 2003;237:509. Pearce WH, Parker MA, Feinglass J, et al. The importance of surgeon volume and training in outcomes for vascular surgical procedures. J Vasc Surg 1999;29:768. Cowan JA, Jr., Dimick JB, Thompson BG, et al. Surgeon volume as an indicator of outcomes after carotid endarterectomy: An effect independent of specialty practice and hospital volume. J Am Coll Surg 2002;195:814. Cowan JA, Jr., Dimick JB, Leveque JC, et al. The impact of provider volume on mortality after intracranial tumor resection. Neurosurgery 2003;52:48. Vij A. The Malcolm Baldrige National Quality Award for healthcare organizations. Hosp Top 1995;73:4. Goldstein SM, Schweikhart SB. Empirical support for the Baldrige Award framework in U.S. hospitals. Health Care Manage Rev 2002;27:62. Griffith JR, White KR. The revolution in hospital management. J Health Care Manag 2005;50:170; Discussion 189. Hutter MM, Rowell KS, Devaney LA, et al. Identification of surgical complications and deaths: An assessment of the traditional surgical morbidity and mortality conference compared with the American College of Surgeons—National Surgical Quality Improvement Program. J Am Coll Surg 2006;203:618. Chalmers I. Underreporting research is scientific misconduct. JAMA 1990;263:1405. Policy on enhancing public access to archived publications for NIH funded research. Available at: http://grants.nih.gov/ grants/guide/notice-files/NOT-OD-05-022.html. Accessed January 1, 2007. Friedman CP, Wyatt JC. Editorial comments: Publication bias in medical informatics. J Am Med Inform Assoc 2001;8:189. Chalmers TC, Frank CS, Reitman D. Minimizing the three stages of publication bias. JAMA 1990;263:1392. Dickersin K, Rennie D. Registering clinical trials. JAMA 2003; 290:516. Milner SM. Editorial: Setting science free. J Burns Wounds 2006;5:39.