Unrecognized necrosis at same admission cholecystectomy for pancreatitis increases organ failure and infected necrosis

Unrecognized necrosis at same admission cholecystectomy for pancreatitis increases organ failure and infected necrosis

Pancreatology xxx (2016) 1e4 Contents lists available at ScienceDirect Pancreatology journal homepage: www.elsevier.com/locate/pan Original Article...

204KB Sizes 0 Downloads 15 Views

Pancreatology xxx (2016) 1e4

Contents lists available at ScienceDirect

Pancreatology journal homepage: www.elsevier.com/locate/pan

Original Article

Unrecognized necrosis at same admission cholecystectomy for pancreatitis increases organ failure and infected necrosis Wilson Tak-Yu Kwong a, Santhi Swaroop Vege b, * a b

Division of Gastroenterology, University of California San Diego Health Sciences, 9500 Gilman Drive (MC 0956), La Jolla, CA 92093, United States Division of Gastroenterology, Mayo Clinic Rochester, 2001st St SW, Rochester, MN 55902, United States

a r t i c l e i n f o

a b s t r a c t

Article history: Received 25 July 2016 Received in revised form 1 October 2016 Accepted 21 October 2016 Available online xxx

Background and aims: Guidelines recommend same admission cholecystectomy (SAC) in the management of mild acute gallstone pancreatitis (AGP) with a recent randomized trial supporting this recommendation. However, the push for early cholecystectomy will lead a subset of patients with evolving, unrecognized necrotizing pancreatitis (NP) to undergo laparoscopic cholecystectomy (LC) with unknown consequences. With concerns about potentially serious outcomes, we studied the outcomes in patients with unrecognized NP who underwent SAC and identified predictors of unrecognized NP at the time of SAC. Methods: Retrospective study of patients who appeared to have mild AGP but subsequently discovered to have unrecognized NP after SAC (study group). Outcomes were compared to a similar cohort with necrotizing AGP who did not undergo SAC (control group 1). Predictors for unrecognized NP at the time of SAC were identified through logistic regression using a second control group with truly mild AGP undergoing SAC. Results: Patients in the study group (N ¼ 46) undergoing SAC demonstrated higher rates of persistent organ failure (p ¼ 0.0003), infected necrosis (p ¼ 0.02), and length of hospital stay (p ¼ 0.049) compared to a similar group (N ¼ 48) with necrotizing AGP who did not undergo SAC. Persistent SIRS (p < 0.0001) and WBC >12  109/L (p < 0.0001) on the day of cholecystectomy were associated with evolving/unrecognized NP. Conclusions: Unrecognized NP at the time of SAC is associated with increased rates of subsequent persistent organ failure, infected necrosis, and length of hospital stay. Persistent leukocytosis and SIRS at the time of proposed cholecystectomy are predictive of unrecognized NP and should prompt contrast enhanced CT prior to proceeding with LC. © 2016 Published by Elsevier B.V. on behalf of IAP and EPC.

Keywords: Gallstone pancreatitis Laparoscopic cholecystectomy Timing SIRS Infected necrosis Pancreas

1. Introduction The optimal timing of cholecystectomy for acute gallstone pancreatitis (AGP) has been debated for several decades [1,2,3,4]. Recurrences of gallstone related adverse events occur in 30% of patients not undergoing same admission cholecystectomy (SAC)

Abbreviations: AGP, acute gallstone pancreatitis; BMI, body mass index; BUN, blood urea nitrogen; CT, computed tomography; IQR, interquartile range; LC, laparoscopic cholecystectomy; MRI, magnetic resonance imaging; NP, necrotizing pancreatitis; ROC, receiver operator characteristic; SAC, same admission cholecystectomy; SIRS, systemic inflammatory response syndrome; SD, standard deviation; WBC, white blood cell count. * Corresponding author. E-mail address: [email protected] (S.S. Vege).

and the minimally invasive nature of the operation are reasons for increasing rates of SAC in mild AGP [5]. Recent from the American College of Gastroenterology, International Association of Pancreatology, and American Pancreatic Association all recommend laparoscopic cholecystectomy (LC) during the initial hospitalization for mild AGP and recommend delayed LC in the setting of necrotizing pancreatitis (NP) [6,7]. Two randomized trials including the recent PONCHO trial have reported that SAC can be safely performed with regards to surgical complications [3,4]. However, these trials were not powered to detect uncommon complications related to pancreatitis which can be clinically significant especially if SAC approach becomes widely adopted. A potential danger of SAC is subjecting patients with evolving, unrecognized NP to an additional inflammatory insult of surgery which could increase the risk of subsequent organ failure, infection,

http://dx.doi.org/10.1016/j.pan.2016.10.009 1424-3903/© 2016 Published by Elsevier B.V. on behalf of IAP and EPC.

Please cite this article in press as: Kwong WT-Y, Vege SS, Unrecognized necrosis at same admission cholecystectomy for pancreatitis increases organ failure and infected necrosis, Pancreatology (2016), http://dx.doi.org/10.1016/j.pan.2016.10.009

2

W.T.-Y. Kwong, S.S. Vege / Pancreatology xxx (2016) 1e4

or death. The outcomes of LC in the setting of NP has not been well studied. One study reported two deaths out of 12 patients with NP who underwent LC during their initial hospitalization [8]. Unlike organ failure which is easily recognized, NP can be clinically silent and imaging may take up to 5 days to reveal necrosis [7]. There are no reliable predictors of necrosis in the first few days if imaging fails to demonstrate necrosis [7]. With SAC becoming increasingly common in accordance with guidelines, it seems inevitable that a subset of patients who appear to have mild pancreatitis, yet harbor unrecognized NP, will undergo LC. The evolving NP then can manifest with more serious outcomes in subsequent days, due to the added insult of surgery. There is no data in the literature regarding this scenario which is likely to become increasingly recognized. A clinical trial powered to detect this complication would require several hundred patients in the SAC arm and will be hard to accomplish. The aim of this study is to determine the clinical outcomes of patients with unrecognized NP who underwent SAC. Additional aims are to estimate the magnitude of this problem among patients who undergo SAC for AGP and also identify predictors for undiagnosed NP prior to SAC. 2. Methods 2.1. Study group This is a retrospective study of patients over 18 years of age treated at Mayo Clinic Rochester from January 2000 to August 2015 for AGP. Patients were retrospectively identified through a search of clinical databases including clinical notes, diagnosis codes, and procedure codes. The study group included consecutively admitted patients who had a presentation consistent with mild AGP and underwent SAC but were subsequently discovered to have (peri) pancreatic necrosis within 3 months of the diagnosis of AGP. Diagnosis of acute pancreatitis required 2 of the following 3 criteria: acute onset of upper abdominal pain consistent with pancreatic pain, amylase and/or lipase levels at least 3 times the upper limit of normal, or imaging evidence of pancreatic inflammation. A gallstone etiology was based on acute elevations of liver function tests and the presence of gallstones on ultrasound imaging. The assessment of mild pancreatitis was consistent with the revised Atlanta classification and included patients without evidence of organ failure, acute necrotic collections or acute peripancreatic fluid collections [9]. The definition of organ failure included acutely elevated creatinine greater than 2 mg/dL after fluid resuscitation or renal replacement therapy, a systolic blood pressure less than 90 mmHg after fluid resuscitation or pressor support, or requirement of high flow oxygen by facemask, positive airway pressure ventilation, or intubation. Severe pancreatitis was defined as persistent organ failure beyond 48 h. The diagnosis of NP was based on either contrast enhanced computed tomography or magnetic resonance imaging (MRI) performed with gadolinium. The timing of LC was at the discretion of the consulting surgeon but only patients undergoing SAC were included in the study group. 2.2. Control groups Two control groups were created through a retrospective clinical database search of admitted patients to Mayo Clinic Rochester over the same time period. Control group 1 consisted of consecutively admitted patients with AGP who appeared to have mild pancreatitis within the first 48 h of presentation but were later discovered to have NP on imaging. These patients with NP did not demonstrate organ failure within the first 48 h and did not undergo LC during the same admission. Comparisons between the study

group and control group 1 were performed to assess the effect of SAC on outcomes including length of hospital stay, development of persistent organ failure, infected necrosis, conversion to open cholecystectomy, need for intervention on (peri)pancreatic necrosis, and death during hospitalization. Control group 2 consisted of consecutively admitted patients with mild AGP who underwent SAC and were not diagnosed with NP at any point. Logistic regression analysis between the study group and control group 2 were performed to identify predictors of unrecognized necrosis in patients who present with what appears to be mild AGP. 2.3. Data collection Medical records of patients included in this study were retrospectively reviewed and data regarding age, sex, BMI, vital signs, laboratory values, length of hospital stay, hospital readmission, persistent organ failure, infected necrosis, cholecystectomy technique, need for intervention for NP, and death were extracted. 2.4. Statistical analysis Descriptive data is presented as median with interquartile ranges (IQR) or mean with standard deviations (SD) for continuous variables. Statistical analyses were performed using JMP and SAS statistical software (Cary, NC, USA). Wilcoxon rank-sum test was used for comparisons of continuous variables. c2 and Fisher's exact tests were used for comparisons of categorical variables. Univariate logistic regression analysis was performed to identify potential predictors of necrosis on the day of cholecystectomy. All reported P values are 2 sided with a P < 0.05 level of significance. Assessment of the predictive accuracy of potential predictors including SIRS and WBC were assessed by calculation of serial area under receiver operating characteristic (ROC) curves on day 1, day 2, and on day of cholecystectomy. 3. Results 3.1. Baseline patient characteristics The study group included 46 patients with mild appearing AGP who underwent SAC and subsequently discovered to harbor unrecognized necrosis. Control group 1 included 48 similar patients with mild appearing AGP in the first 48 h and were subsequently discovered to have NP but did not undergo SAC. Control group 2 included 48 patients with mild AGP who underwent SAC and were not diagnosed with NP at any point. Clinical characteristics of patients are presented in Table 1. 3.2. Outcome comparisons Outcome comparisons between the study group and control group 1 are provided in Table 2. The study group (unrecognized NP who underwent SAC) had a significantly longer length of hospital stay (median: 26 days, IQR 19e70) compared to control group 1 (unrecognized NP without SAC) (median: 24.5 days, IQR 16e33.5; p ¼ 0.049). The study group had a higher rate of subsequent persistent organ failure (24%) compared to control group 1 (0%, p ¼ 0.0003). Subsequent persistent organ failure in the study group was defined as organ failure with onset after cholecystectomy and persisting at least 48 h. Subsequent persistent organ failure in the control group 1 was organ failure with onset after the first 48 h of admission and persisting for at least 48 h. Patients in the study group demonstrated a higher rate of culture confirmed infected necrosis (52%) compared to control group 1 (27%, p ¼ 0.02). There were no differences in the rates of intervention on necrosis,

Please cite this article in press as: Kwong WT-Y, Vege SS, Unrecognized necrosis at same admission cholecystectomy for pancreatitis increases organ failure and infected necrosis, Pancreatology (2016), http://dx.doi.org/10.1016/j.pan.2016.10.009

W.T.-Y. Kwong, S.S. Vege / Pancreatology xxx (2016) 1e4 Table 1 Baseline patient characteristics.

N Age e years (IQR) Sex - Male BMI CCY timing (days) Laparoscopic technique

Study group

Control group 1

Control group 2

46 58.5 (53e66) 33 (72%) 28.9 (25e34) 3 (2e5) 42 (91%)

48 59 (52e71) 30 (63%) 29 (27-e35) 152 (97e269) 30 (86%)

48 56.5 (48e73) 19 (40%) 28.7 (26e36) 2 (1e3) 46 (96%)

Table 2 Outcome comparisons of patients with initially unrecognized necrotizing pancreatitis who did (study group) and did not (control group 1) undergo same admission cholecystectomy.

Persistent organ failure Length of stay e median (IQR) Infected necrosis Need for intervention Laparoscopic technique Death

Study group

Control group 1

P value

11 (24%) 26 (19e70) 24 (52%) 37 (80%) 42 (91%) 2 (4%)

0 (0%) 24.5 (16e34) 13 (27%) 33 (69%) 30 (86%) 1 (2%)

0.0003 0.049 0.02 0.46 0.48 0.53

Table 3 Laboratory and SIRS parameters during the first 2 days of admission and on day of cholecystectomy. Day 1 Study group SIRS (þ) 6/10 (60%) WBC 21.2 ± 5.6 BUN 16.7 ± 7.8 Cr 0.9 ± 0.3 HCT 43.9 ± 4.0 Control group 2 SIRS (þ) 12/48 (26%) WBC 12.4 ± 5.9 BUN 17.7 ± 7.4 Cr 0.9 ± 0.3 HCT 40.3 ± 3.9

Day 2

Cholecystectomy

7/11 (64%) 19.8 ± 6.1 16.1 ± 5.5 0.9 ± 0.2 42.4 ± 6.1

10/11 (91%) 18.2 ± 5.3 12 ± 3.1 0.8 ± 0.3 42.5 ± 5.9

7/48 (15%) 10.2 ± 5.4 14.8 ± 6.4 0.9 ± 0.3 36.3 ± 3.8

0/48 (0%) 8.3 ± 5.5 12.0 ± 5.8 0.8 ± 0.3 35.9 ± 3.9

conversion to open cholecystectomy, and death between the study group and control group 1. Among patients in control group 2 with true mild pancreatitis, the median length of stay was 3 days and no patients developed necrosis. 3.3. Calculating the rate of unrecognized necrosis at same admission cholecystectomy From January 2014 to August 2015, 102 patients with AGP were directly admitted with AGP and underwent SAC. After SAC, 7 of these patients were subsequently discovered to have previously unrecognized necrosis. The rate of patients with unrecognized necrosis from AGP who underwent SAC is therefore 7% (7/102) during this recent period. 3.4. Predictors of evolving/unrecognized NP at time of cholecystectomy Laboratory and SIRS parameters were available for 11 study group patients (remainder were transferred from other hospitals). Laboratory and SIRS parameters from the study group and control group 2 (true mild AGP undergoing SAC) are provided in Table 3. 10 of 11 (91%) study group patients demonstrated SIRS on the day of cholecystectomy while none of the 48 patients in control group 2 with true mild AGP demonstrated SIRS on the day of cholecystectomy (p < 0.0001). 9 or 11 (82%) of patients in the study group demonstrated the persistent SIRS (SIRS score 2 starting from the

3

second day, or earlier, which persisted until the day of cholecystectomy). All 11 (100%) patients in the study group for whom data was available had a WBC count greater than 12  109/L on the day of cholecystectomy while 7 of 48 (21%) in the second control group had a WBC count greater than 12  109/L on the day of surgery (p < 0.0001). Univariate logistic regression identified WBC count greater than 12  109/L and SIRS positivity (SIRS score 2) as statistically significant predictors of unrecognized necrosis after AGP. Odds ratio with confidence intervals are provided in Table 4. Multivariate logistic regression was not feasible due to the limited number of patients in the study group for whom data was available in the first 48 h. Serial area under receiver operating characteristic curves (AUC) were calculated to assess the predictive accuracy of SIRS and WBC counts for unrecognized NP on day 1, day 2, and the day of cholecystectomy (Table 4). The AUC values improved as time went on (AUC was higher on day of cholecystectomy compared to day 2 which was higher than day 1). AUC for WBC >12  109/L and SIRS positivity on the day of cholecystectomy were 0.92 and 0.96, respectively. 4. Discussion Management guidelines from several societies recommend SAC for AGP [6,7]. However, only 2 randomized clinical trials have studied the safety of such an approach and neither was powered to detect differences in adverse outcomes related to evolving/unrecognized NP [3,4]. Our data indicates that 7% of patients undergoing SAC for gallstone pancreatitis harbor evolving, unrecognized NP which becomes symptomatic after surgery. The current study examines 46 such patients and compared their outcomes with a group of control patients with a similar initial presentation, but did not undergo SAC. Patients with unrecognized NP undergoing SAC had significantly worse outcomes with increased rates of new organ failure, increased length of hospital stay, and increased rates of infected NP compared to those with NP but did not undergo SAC. The present study provides the first evidence that not all patients who appear to have mild AGP are safe to undergo SAC. The NP that evolves after SAC is more serious than the NP when LC is not done, indicating an adverse effect of LC in patients with unrecognized NP. This also had not been previously demonstrated in the laparoscopic era of cholecystectomy. SAC is likely appropriate and safe for the majority of patients with mild AGP based on available studies. However, it is crucial to identify the subset of patients with unrecognized NP so that SAC can be avoided. A traditional approach has been to proceed with LC when all lab parameters and pain have resolved, although the data to support this practice is weak. Based on our results, persistent leukocytosis is an inexpensive and readily available parameter able to identify patients at risk for undiagnosed necrosis before and at the time of cholecystectomy. All patients who underwent SAC with unrecognized pancreatic necrosis and had WBC count greater than Table 4 Predictors of Evolving/Unrecognized Necrotizing Pancreatitise Univariate Logistic Regression and Area under ROC curve comparison.

WBC day 1 > 12 WBC day 2 > 12 WBC CCY > 12 SIRS (þ) day 1 SIRS (þ) day 2 SIRS (þ) CCY

Odds ratio (CI)

P value

7.0  109 (1.7-∞) 28.6 (4.65e559) 6.0  108 (22.9e∞) 3.6 (0.83e16.98) 8.4 (1.9e40) 1.4  109, (132e∞)

p p p p p p

¼ 0.02 < 0.001 < 0.0001 ¼ 0.08 ¼ 0.005 < 0.0001

Area under ROC 0.81 0.83 0.92 0.65 0.72 0.96

CCY e Day of cholecystectomy.

Please cite this article in press as: Kwong WT-Y, Vege SS, Unrecognized necrosis at same admission cholecystectomy for pancreatitis increases organ failure and infected necrosis, Pancreatology (2016), http://dx.doi.org/10.1016/j.pan.2016.10.009

4

W.T.-Y. Kwong, S.S. Vege / Pancreatology xxx (2016) 1e4

12  109/L on the morning of surgery. SIRS is also correlated with unrecognized NP in AGP as 91% demonstrated SIRS on the day of surgery and 82% demonstrated persistent SIRS in the days leading up to LC. ROC analysis shows that accuracy of WBC count and SIRS to detect unrecognized necrosis improved as time went suggesting it is the persistence of leukocytosis and SIRS that is most indicative of evolving pancreatic necrosis. For patients with leukocytosis or persistent SIRS who are otherwise candidates for SAC, contrast imaging prior to LC should be considered. Recent studies have demonstrated both perfusion CT scan and subtraction CT scan techniques to be superior to conventional CT for early detection of NP and may allow earlier and more reliable diagnosis of NP [10,11]. Those patients found to have NP and remain symptomatic after several weeks can then undergo laparoscopic debridement and cholecystectomy in the same operation, which can reduce the number of interventions. For those patients found to have NP but resolution of necrotic collections after several weeks, LC alone can be safely performed without debridement. Despite a 7% rate of apparently mild AGP with evolving, unrecognized NP undergoing SAC in our series, no such cases were reported in the two prospective trials examining early LC, nor has this complication been described in the literature thus far. In Aboulian's study, there were 25 patients in the early cholecystectomy arm. The small number of patient's in the study arm would explain why an adverse event present in 7% of patients was not detected. In the recent PONCHO trial, 129 patients with mild pancreatitis underwent cholecystectomy after a median of 7 days. Hospitalization for 7 days prior to cholecystectomy for mild AGP is longer than in our study and prior studies where the median length of hospitalization including cholecystectomy for mild AGP was 3e4 days [4,12]. The 7 day period before LC in the PONCHO trial would allow detection of most cases of NP including those that were missed during the first 48e72 h and explain the absence of unrecognized NP in any of their patients. While waiting 7 inpatient days prior to proceeding with LC may be more feasible in the Netherlands where the PONCHO trial was conducted for various reasons, it appears less practical in the United States to wait 7 days in presumed mild AGP at least in part due to issues of cost and reimbursement. Future studies should focus on the optimal timing of SAC. The current study has several limitations. Only patients with symptomatic NP after SAC would have sought medical attention and received a diagnosis of NP. It is possible that there were more patients with silent NP who underwent SAC without complications and were therefore never diagnosed with NP. If true, then the rate the 7% rate reported in this study only reflects those who have symptomatic necrosis after SAC and the actual fraction of those with unrecognized NP undergoing SAC is even higher. The magnitude of this problem may also be larger than calculated in this study as Mayo Clinic Rochester is a referral center with many patients traveling from other states. Patients who subsequently developed symptomatic NP after SAC would not be captured if they received medical care locally. Our study is limited in its ability to identify predictors of necrosis given the small number of patients for whom initial laboratory data was available from the first 48 h. Due to a limited number of patients for whom initial WBC and SIRS data was available, our statistical analysis were limited to univariate as opposed to multivariate analyses and are also vulnerable to a type I error. Therefore, while a WBC count greater than 12  109/L and SIRS on the day of proposed LC are significantly associated with subsequent NP, these predictors require validation from larger, prospective cohorts. In conclusion, the current study defines, for the first time, a subset of patients with unrecognized NP who would be harmed by

SAC for presumed mild AGP. Patients who have unrecognized NP at the time of LC demonstrated significantly higher rates of subsequent persistent organ failure, a longer hospital stay, and higher rates of infected necrosis. With most major gastroenterology societies recommending SAC, the practice of SAC for mild AGP is likely to increase worldwide. However, accurate identification of those with evolving, unrecognized NP is paramount to prevent them from developing severe complications as a result of SAC. Based on our results, we suggest a persistent WBC count greater than 12  109/L or the presence of SIRS on the morning of proposed LC or the preceding day as clues that evaluation for NP should be undertaken prior to surgery. Disclosures The authors have no conflicts of interest to disclose. Writing assistance None. Author contributions Wilson Kwong e study concept and design, acquisition of data, analysis and interpretation. Santhi Swaroop Vege e study concept and design, analysis, interpretation and revision of manuscript. Grant support University of California San Diego Health System. References €m J. The timing of cholecystectomy in patients with gallstone pancre[1] Elfstro atitis. A retrospective analysis of 89 patients. Acta Chir Scand 1978;144(7e8): 487e90. [2] Kelly TR, Wagner DS. Gallstone pancreatitis: a prospective randomized trial of the timing of surgery. Surgery 1988 Oct;104(4):600e5. [3] da Costa DW, Dijksman LM, Bouwense SA, Schepers NJ, Besselink MG, van Santvoort HC, et al. Same-admission versus interval cholecystectomy for mild gallstone pancreatitis (PONCHO): a multicentre randomised controlled trial. Lancet 2015 Sep 26;386(10000):1261e8. [4] Aboulian A, Chan T, Yaghoubian A, Kaji AH, Putnam B, Neville A, et al. Early cholecystectomy safely decreases hospital stay in patients with mild gallstone pancreatitis: a randomized prospective study. Ann Surg 2010 Apr;251(4): 615e9. [5] Ito K, Ito H, Whang EE. Timing of cholecystectomy for biliary pancreatitis: do the data support current guidelines? J Gastrointest Surg 2008 Dec;12(12): 2164e70. [6] Working Group IAP/APA Acute Pancreatitis Guidelines. IAP/APA evidencebased guidelines for the management of acute pancreatitis. Pancreatology 2013 Jul-Aug;13(4 Suppl 2):e1e15. [7] Tenner S, Baillie J, DeWitt J, Vege SS. American College of Gastroenterology guideline: management of acute pancreatitis. Am J Gastroenterol 2013 Sep;108(9). 1400e15; 1416.  ski A, Zasada J. Early endoscopic [8] Panek J, Karcz D, Rembiasz K, Budzyn sphincterotomy and early laparoscopic cholecystectomy in the treatment of severe acute biliary pancreatitisea preliminary report. Adv Med Sci 2006;51: 103e4. [9] Acute Pancreatitis Classification Working Group. Classification of acute pancreatitisd2012: revision of the Atlanta classification and definitions by international consensus. Gut 2013;62:102e11. [10] Tsuji Y, Takahashi N, Fletcher JG, Hough DM, McMenomy BP, Lewis DM, et al. Subtraction color map of contrast-enhanced and unenhanced CT for the prediction of pancreatic necrosis in early stage of acute pancreatitis. AJR Am J Roentgenol 2014 Apr;202(4):W349e56. [11] Tsuji Y, Yamamoto H, Yazumi S, Watanabe Y, Matsueda K, Yamamoto H, et al. Perfusion computerized tomography can predict pancreatic necrosis in early stages of severe acute pancreatitis. Clin Gastroenterol Hepatol 2007 Dec;5(12):1484e92. [12] Taylor E, Wong C. The optimal timing of laparoscopic cholecystectomy in mild gallstone pancreatitis. Am Surg 2004 Nov;70(11):971e5.

Please cite this article in press as: Kwong WT-Y, Vege SS, Unrecognized necrosis at same admission cholecystectomy for pancreatitis increases organ failure and infected necrosis, Pancreatology (2016), http://dx.doi.org/10.1016/j.pan.2016.10.009