A Novel Screening Method to Identify Late-Stage Dementia Patients for Palliative Care Research and Practice

Accepted Manuscript A novel screening method to identify late-stage dementia patients for palliative care research and practice Natalie C. Ernecoff, MPH, Kathryn L. Wessell, Stacey Gabriel, MPH, Timothy S. Carey, MD, MPH, Laura C. Hanson, MD, MPH PII:

S0885-3924(17)31243-5

DOI:

10.1016/j.jpainsymman.2017.12.480

Reference:

JPS 9681

To appear in:

Journal of Pain and Symptom Management

Received Date: 6 November 2017 Revised Date:

14 December 2017

Accepted Date: 18 December 2017

Please cite this article as: Ernecoff NC, Wessell KL, Gabriel S, Carey TS, Hanson LC, A novel screening method to identify late-stage dementia patients for palliative care research and practice, Journal of Pain and Symptom Management (2018), doi: 10.1016/j.jpainsymman.2017.12.480. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

ACCEPTED MANUSCRIPT Brief Methodological Report

12/13/17

A novel screening method to identify late-stage dementia patients for palliative care research and practice

RI PT

Natalie C. Ernecoff, MPH1,2 Kathryn L. Wessell1 Stacey Gabriel, MPH1 Timothy S. Carey, MD, MPH1,3 Laura C. Hanson, MD, MPH1,4

G. Sheps Center for Health Services Research, University of North Carolina, Chapel Hill, North Carolina 2Department of Health Policy and Management, Gillings School of Global Public Health, University of North Carolina, Chapel Hill, North Carolina 3Departments of Medicine and Social Medicine, School of Medicine, University of North Carolina, Chapel Hill, North Carolina 4Division of Geriatric Medicine & Palliative Care Progam, University of North Carolina, Chapel Hill, North Carolina

M AN U

SC

1Cecil

AC C

EP

Word Count: 2411/2500 Figures: 1 Tables: 2 Appendix: 1 References: 33

TE D

Acknowledgements: The authors would like to thank Jasper Benoit Becker, Joseph Dobgima Njinimbam, and Loretta Fearrington for their support in building, refining, and implementing the EHR phenotype. The authors would like to thank Emily Pfaff for providing early conceptual guidance for this manuscript.

Page 1 of 16

ACCEPTED MANUSCRIPT Brief Methodological Report

12/13/17

Abstract (250/250 words)

AC C

EP

TE D

M AN U

SC

RI PT

Context. Investigators need novel methods for timely identification of patients with serious illness to test or implement new palliative care models. Objective. The study aim was to develop an electronic health record (EHR) phenotype to identify patients with late-stage dementia for a clinical trial of palliative care consultation. Methods. We developed a computerized method to identify patients with dementia on hospital admission. Within a data warehouse derived from the hospital’s EHR, we used search terms of age, admission date, and ICD-9 and ICD-10 diagnosis codes to create a EHR dementia phenotype, followed by brief medical record review to confirm late-stage dementia. We calculated positive predictive value, false discovery rate, and false negative rate of this novel screening method. Results. The EHR phenotype screening method had a positive predictive value of 76.3% for dementia patients and 24.5% for late-stage dementia patients; a false discovery rate of 23.7% for dementia patients and 75.5% for late-stage dementia patients compared to physician assesement. The sensitivity of this screening method was 59.7% to identify hospitalized patients with dementia. Daily screening—including confirmatory chart reviews—averaged 20 minutes and was more feasible, efficient and more complete than manual screening. Conclusion. A novel method using an EHR phenotype plus brief medical record review is effective to identify hospitalized patients with late-stage dementia. In healthcare systems with similar clinical data warehouses, this method may be applied to serious illness populations to improve enrollment in clinical trials of palliative care or to facilitate access to palliative care services.

Page 2 of 16

ACCEPTED MANUSCRIPT Brief Methodological Report

12/13/17

M AN U

SC

RI PT

Introduction Specialty palliative care improves quality of life, reduces hospitalizations and invasive treatments, yet results in similar or better survival for patients with serious illnesses.1,2 Palliative care may benefit patients and families facing advanced stage Alzheimer’s disease and other neurodegenerative dementias, most of whom prefer comfort-focused care.3 These incurable and progressive diseases affect more than 5 million Americans and have extraordinary societal and family costs.4,5 Dementia is associated with high rates of hospitalization, care transitions, and 30day readmissions.6 Dementia is also the only leading cause of death in the U.S. with no meaningful preventive or curative intervention; as a result, prevalence is expected to double by 2030. 7,8 From a population health perspective, specialty palliative care is most likely to benefit the subset of dementia patients who have symptomatic late-stage disease. Certain groups of seriously ill patients are clustered within a specific nursing unit, clinic or physician practice, but many serious illness populations are diffused across a hospital or health system. For example, while patients with advanced stage cancer are found in oncology clinics or specialty hospital services, patients with congestive heart failure, chronic lung disease, or dementia are found in primary care and specialty care clinics and on various hospital services. Novel methods are needed to identify these patients for palliative care clinical services or research. 9,10

EP

TE D

EHR computable phenotypes for clinical research Electronic health records (EHRs) offer new potential to identify high-risk patient populations. Since 2000, EHR-based research has increased, particularly for chronic illnesses or for rare health conditions. This methodology requires systematic terminology and data presentation in electronic data.11,12 The EHR has been used to facilitate clinical trial enrollment for patients with COPD and chronic pain, and to streamline multisite pragmatic trials. 13,14,15 EHR computable phenotypes are customizable, structured queries of electronic healthcare data used by investigators to identify patients who meet criteria based on standardized terminology and Boolean logic.16,17,18 Existing EHR computable phenotypes largely capitalize on clinical lab values and other structured quantitative data. Building a clinically meaningful EHR phenotype requires extensive refinement and does not yet offer a reliable alternative to manual screening. For dementia, one existing, published algorithm is targeted to all patients with dementia at any stage, including using dementia-related medications as part of their phenotypes, which are more common in earlier stages.19 However, combining a sensitive and efficient EHR phenotype with more specific manual chart review may be an optimal way to find seriously ill patients from within a much larger pool of patients served by a hospital or health system. This combined approach is known as serial testing.

AC C

Computable EHR phenotype to identify patients with late-stage dementia To facilitate a clinical trial of palliative care consultation, we sought to identify late-stage dementia patients at the time of hospitalization. Our study aim was to develop an electronic health record (EHR) computable phenotype to identify serious illness with late-stage dementia, promoting enrollment for this clinical trial.

Page 3 of 16

ACCEPTED MANUSCRIPT Brief Methodological Report

12/13/17

Methods

SC

RI PT

The Triggered Palliative Care for Dementia Study The screening method was developed in the context of a pilot randomized clinical trial of triggered palliative care consultation for patients with dementia. The study site was UNC Hospitals, a 929-bed acute care tertiary care teaching hospital with 25,995 adult admissions in FY17 (excluding psychiatry, obstetrics, and rehabilitation). Patients were eligible if they had late-stage dementia, an acute illness hospitalization, and a family decision-maker; patients with existing hospice or palliative care consultation were excluded. Patients and their family decision makers were randomized to the intervention (triggered specialty palliative care and post-discharge collaborative care) or control (educational materials) arms. Participants were followed for 60 days after enrollment to measure outcomes of hospitalization and emergency room transfers, goals of care communication, completion of advance care planning documentation, symptom distress, and referral to community-based hospice or palliative care services. All study procedures were approved by the University of North Carolina Institutional Review Board and reviewed by a Data Safety Monitor.

EP

TE D

M AN U

Developing a method to screen for patients with late-stage dementia in hospital The novel screening method consisted of an EHR computable phenotype of dementia followed by verification by research staff and palliative care physicians. To construct a computable phenotype, we used EHR data organized by Informatics for Integrating Biology and Bedside (i2b2; Boston, MA) software, designed initially to support integration of genomic research into clinical settings.20 The software searches the EHR based on pre-specified parameters to select patients who meet criteria. We used i2b2 to build an initial EHR phenotype based on diagnoses at admission consistent with Alzheimer’s disease and other neurodegenerative dementias, variables indicating acute care admission, and age. We then linked this phenotype to patient records via the Carolina Data Warehouse (CDW), where information from the EHR is stored using IBM Business Objects (Armonk, NY). Each day, an automated report in Microsoft Excel was exported to a secure research server with medical record numbers and names of patients who met the algorithm’s parameters. Each day, a trained and well-practiced research staff member received the automated report of hospitalized dementia patients, and screened their medical records for evidence of latestage dementia. Patients with potential late-stage dementia as determined by research staff were then sent to a palliative care physician who verified eligibility based on review of the EHR and conversation with the patient’s attending physician to confirm dementia diagnosis and stage 5-7 using the Global Deterioration Scale (GDS) (Figure 1).21

AC C

Feedback to refine the EHR phenotype To refine the algorithm, our EHR phenotype was iteratively evaluated for its accuracy in detecting the target population, and revised twice. We applied serial testing in which research staff reviewed EHR output and compared to the gold standard of physician-verified diagnosis of latestage dementia. Starting with all 158 potential ICD-9 and ICD-10 diagnosis codes, we tested how much utility each had in identifying the first 82 patients with late-stage dementia who met eligibility criteria. The first iteration of the EHR phenotype retrieved 604 patients (3/29/167/13/16), 124 of whom were sent to a palliative care physician to determine that 82 had confirmed late-stage dementia. Screening all patient charts required an average of ~50 minutes/day from research team personnel using this mechanism. (Appendix) For the second iteration we removed 92 codes that were not applied in the first iteration and did not represent a rare disease with progressive dementia, retaining 66 codes that identified at least one eligible patient. Via this process, we sought to improve the specificity of the algorithm without losing sensitivity. Page 4 of 16

ACCEPTED MANUSCRIPT Brief Methodological Report

12/13/17

M AN U

SC

RI PT

The second iteration of the EHR phenotype identified 390 patients (7/14/16-10/31/16), 95 of whom had serious-enough illness to be reviewed by palliative care physician. This yielded 53 patients, requiring 35 minutes of staff time per day. In the final iteration we eliminated ICD-9 and ICD-10 codes for traumatic brain injury (TBI) and stroke when presenting without co-occurrence of a neurodegenerative dementia code. The final algorithm had thirty-four ICD codes (see Appendix) associated with dementia or cognitive impairment, hospital admission in last three days at a hospital participating in the study, and patient age 65 or older. This final EHR phenotype yielded 371 patients (11/1/16-3/31/17; out of 1615 adult patients admitted as inpatients during the timeframe), 174 of whom were referred by research staff for final review by a palliative care physician, who verified late-stage dementia for ninety. The final EHR phenotype required 20 minutes of staff time daily to complete all confirmatory chart reviews. Over the iterations, staff time decreased due to decreases in the average number of patients derived from the EHR phenotype each month; the first iteration of the EHR phenotype yielded an average 173 patients/month, the second iteration yielded an average 111 patients/month, and the final EHR phenotype yielded an average 74 patients/month. The EHR phenotype’s results over each refinement are presented in Table 1. The remainder of the results use data only from this final refinement period. Measuring feasibility and efficiency To assess feasibility and efficiency, research staff tracked the amount of time required to identify potential late-stage dementia patients in the entire hospital using the novel method compared to reviewing census lists for two inpatient services with the largest dementia populations.

TE D

Positive predictive value and false discovery rate Positive predictive value is presented as the percent of cases on the automated daily report that did have dementia of any stage. The false discovery rate (the proportion of cases that were not true dementia) is presented as the percent of cases on the automated daily report without dementia. Because of the specific focus of this study on late-stage disease, we also calculated the positive predictive value and false discovery rate for late-stage dementia.

AC C

EP

Measuring sensitivity During a five-month period, we measured the sensitivity of the EHR computable phenotype method (true positive / (true positive + false negative)) as a ratio of true cases of dementia found by the automated daily report compared to true cases of dementia who were and were not found by this method. False negative cases were identified through daily review of all patients aged 65 and older on three daily census lists: palliative care consults, and two inpatient services with the highest proportion of dementia admissions (Geriatrics and Family Medicine). Because of the study’s specific focus on late-stage dementia, we also calculated sensitivity for the late-stage population separately by examining clinician-confirmed cases of late-stage dementia.

Page 5 of 16

ACCEPTED MANUSCRIPT Brief Methodological Report

12/13/17

M AN U

SC

RI PT

Results Feasibility and efficiency of the screening method We compared manual screening of two high-yield services to hospital-wide screening using the EHR phenotype during a 5-month period (11/1/16-3/31/17). Manual screening of the daily census for two services with higher numbers of dementia cases (Geriatrics and Family Medicine) took an average of 25 minutes/day. From a total of 541 admitted patients, this method yielded 236 patients with dementia and 70 patients with late-stage dementia, yielding positive predictive values of 44% and 13% respectively. In contrast, using the EHR phenotype followed by more selective manual screening across all hospital services averaged 20 minutes per day. This method screened 3174 admitted patients to yield 283 patients with dementia and 91 patients with late-stage dementia. The EHR phenotype presented patients within a median 3 days after admission; it led to research enrollment within a median of 4 days. The average length of stay for these patients was 9 days (median 5 days). To apply the final phenotype in research screening, we then built it into a prompt automatically generated if the criteria are met in the EHR. Instead of receiving a daily output, this mechanism quickly messages the research staff (within 1 day) within the EHR when a patient matching the phenotype enters the hospital. These prompts, often called ‘best practice advisories,’ can be customized so that they alert only the research coordinator, not the care team. The EHR alert was 100% reliable in delivering the list of potential dementia patients 1 day after admission, adding to time efficiency. That is, the list successfully fired to the research staff every day.

AC C

EP

TE D

Postive predictive value, false discovery rate, and sensitivity of the screening method Calculations and final rates appear in Table 2. The EHR phenotype screening method had a positive predictive value (PPV) for any stage dementia of 76.3%. The total number of physicianverified late-stage dementia patients was 91 out of 371 patients pulled by the final EHR phenotype, giving a PPV for late-stage disease of 24.5%. The false discovery rate (FDR) of the EHR phenotype screening method was 23.7%; the FDR for late-stage dementia was 75.5%. Simply stated, three out of four patients identified by this method had dementia, and one of four had late-stage dementia. The sensitivity of the final EHR screening method was 59.7% for any dementia; these were cases for which the EHR phenotype method endorsed real cases of dementia. For late-stage dementia, sensitivity was derived by dividing the number of physician-verified late-stage dementia from the EHR phenotype by the total number of physician-verified late-stage dementia from both the EHR phenotype and Geriatric and Family Medicine census lists, yielding a sensitivity of 64.1 percent.

Page 6 of 16

ACCEPTED MANUSCRIPT Brief Methodological Report

12/13/17

TE D

M AN U

SC

RI PT

Discussion In this study we found that a novel method using an EHR phenotype plus medical record review is effective to identify hospitalized patients with late-stage dementia. Using expert clinician case review as the gold standard, three out of four patients identified by this method had dementia, and one of four had late-stage dementia. Daily screening—including confirmatory chart reviews— averaged 20 minutes and was more feasible, efficient and more complete than manual screening. Our EHR phenotype screening combined with selective manual review is also more complete than reliance on use of triggers by referring clinicians, as it permits rapid review of an entire hospitalized population on a daily basis. We did trade off sensitivity for specificity as we refined the EHR phenotype to shorten time spent manually screening. A recent systematic review of palliative care implementation strategies described extensive use of outreach education, clinical triggers and feedback, and other interpersonal methods to improve access to specialty palliative care, but no prior studies of technology improvements or data mining in the electronic medical record. Thus our approach is a novel method for identifying a seriously ill population in order to offer palliative care consultation, or to research new clinical palliative care interventions.22 Across health care systems with similar clinical data warehouses, this method has the potential to screen for dementia and other serious illness, to facilitate access to palliative care services for those who can meaningfully benefit. Although a majority of patients who are hospitalized do not need palliative care consultations, palliative care remains under-utilized for patients with serious and life-limiting illness.23 Earlier palliative care consultations are more effective to reduce overall cost of hospital care.24,25 We use the example of dementia patients, a population with limited research evidence for palliative intervention.26,27 This approach may be useful for other serious illness populations, including patients with congestive heart failure or cancer.28 The rapidity with which the final method identifies patients—within one day of admission—can also get palliative consultations to patients earlier, which may be the difference in getting a consult, given the relative short median lengths of stay. This study also has implications for the pragmatic application of healthcare databases to improve palliative care research. Since its development, i2b2 software has been applied in diverse medical and biologic databases, including computable phenotypes for COPD and tracking of adverse lab values in chronic kidney disease (CKD).29,30 On a large scale, PCORnet, a multicenter network of EHR data, uses similar mechanisms to identify patients with heart failure and multiple other conditions.31,32

AC C

EP

Although our method worked quite well, there remain limitations to this approach that merit consideration. First, the EHR computable phenotype screening method is dependent on institutional investment in a searchable EHR clinical data warehouse, and requires software and personnel to facilitate algorithmic and manual searches. Second, while this method proved more efficient and complete than manual screening of census lists, it could be even more efficient. Natural language processing is an emerging text search technology that could add efficiency and accuracy to the method we report.33,34 In particular, since variables distinguishing serious illness from chronic illness—disease stage and functional status—are often in text, natural language processing could optimize our methodology. Third, the gold standard for defining late-stage dementia remains imperfect due to persistent clinical failure to record this diagnosis. Fourth, trends in dementia admissions and diagnoses might have changed over time, as the EHR computable phenotype was refined in a stage-wise fashion. The iterative refinements occurred in quick succession, potentially minimizing this risk, though the absence of a counterfactual remains a limitation. Despite these limitations, our novel EHR phenotype screening method is a more accurate and efficient method to find late-stage dementia patients for a palliative care clinical trial. While

Page 7 of 16

ACCEPTED MANUSCRIPT Brief Methodological Report

12/13/17

AC C

EP

TE D

M AN U

SC

RI PT

modest investments in technology and personnel are necessary to implement this approach in the clinical setting, it is feasible and enhances overall efficiency for palliative care research.

Page 8 of 16

ACCEPTED MANUSCRIPT Brief Methodological Report

12/13/17

Disclosure/Conflict of Interest Ms. Ernecoff has nothing to disclose. Ms. Wessell has nothing to disclose.

RI PT

Ms. Gabriel has nothing to disclose. Dr. Carey reports grants from National Palliative Care Research Center, the National Institute on Aging, the Patient-Centered Outcomes Research Institute (PCORI), and the Duke Endowment during the conduct of the study.

SC

Dr. Hanson reports grants from National Palliative Care Research Center, grants from National Institute on Aging, during the conduct of the study. Supported in part by a Pilot & Exploratory Project Support Grant from the National Palliative Care Research Center (NPCRC) and NIH-NIA: R21AG052140.

AC C

EP

TE D

M AN U

The project described was supported by the National Center for Advancing Translational Sciences (NCATS), National Institutes of Health, through Grant Award Number UL1TR001111. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

Page 9 of 16

ACCEPTED MANUSCRIPT Brief Methodological Report

12/13/17

Reference Gade, G., Venohr, I., Conner, D., McGrady, K., Beane, J., Richardson, R. H., ... & Penna, R. D. (2008). Impact of an inpatient palliative care team: a randomized controlled trial. Journal of palliative medicine, 11(2), 180-190. 2 Temel JS, Greer JA, Muzikansky A, Gallagher ER, Admane S, Jackson VA, Dahlin CM, Blinderman CD, Jacobsen J, Pirl WF, Billings JA. Early palliative care for patients with metastatic non–small-cell lung cancer. New England Journal of Medicine. 2010 Aug 19;363(8):733-42. 3 Mitchell SL, Palmer JA, Volandes AE, Hanson LC, Habtemariam D, Shaffer ML. (2017). Level of care preferences among nursing home residents with advanced dementia. J Pain Symptom Manage 54:340-345. 4 Hebert LE, Weuve J, Scherr PA, Evans DA. Alzheimer disease in the United States (2010-2050) estimated using the 2010 census. Neurol 2013; 80:1778-1783. 5 Hurd MD, Martorell P, Delavande A, Mullen KJ, Langa KM. Monetary costs of dementia in the United States. N Engl J Med 2013; 368:1326-1334. 6 Callahan CM, Arling G, Tu W, Rosenman, Counsell SR, Stump TE, Hendrie HC. Transitions in care for older adults with and without dementia. J Am Geriatr Soc 2012; 60:813-820. 7 Tinetti ME, McAvay GJ, Murphy TE, Gross CP, Lin H, Allore HG. Contribution of individual diseases to death in older adults with multiple diseases. J Am Geriatr Soc 2012;60(8):1448–56. 8 Prince M, Bryce R, Albanese E, Wimo A, Ribiero W, Ferri CP. The global prevalence of dementia: a systematic review and meta-analysis. Alzheimers Dement 2013; 9(1):63-75. 9 Fadul, N., Elsayem, A., Palmer, J. L., Zhang, T., Braiteh, F., & Bruera, E. (2007). Predictors of access to palliative care services among patients who died at a Comprehensive Cancer Center. Journal of palliative medicine,10(5), 1146-1152. 10 Nelson JE, Curtis JR, Mulkerin C, Campbell M, Lustbader DR, Mosenthal AC, Puntillo K, Ray DE, Bassett R, Boss RD, Brasel KJ. Choosing and using screening criteria for palliative care consultation in the ICU: a report from the Improving Palliative Care in the ICU (IPAL-ICU) Advisory Board. Critical care medicine. 2013 Oct 1;41(10):2318-27. 11 Dean, B. B., Lam, J., Natoli, J. L., Butler, Q., Aguilar, D., & Nordyke, R. J. (2009). Review: Use of Electronic Medical Records for Health Outcomes Research A Literature Review. Medical Care Research and Review, 66(6), 611-638. 12 Aickin, M. (2011). Patient-centered research from electronic medical records.The Permanente Journal, 15(4), 89. 13 Elkhenini, H. F., Davis, K. J., Stein, N. D., New, J. P., Delderfield, M. R., Gibson, M., ... & Bakerly, N. D. (2015). Using an electronic medical record (EMR) to conduct clinical trials: Salford Lung Study feasibility. BMC medical informatics and decision making, 15(1), 1. 14 Tian, T. Y., Zlateva, I., & Anderson, D. R. (2013). Using electronic health records data to identify patients with chronic pain in a primary care setting.Journal of the American Medical Informatics Association, 20(e2), e275-e280. 15 Joseph, C. L., Ownby, D. R., Zoratti, E., Johnson, D., Considine, S., Bourgeois, R., ... & Lu, M. (2016). Recruitment experience for a pragmatic randomized controlled trial: Using EMR initiatives and minimizing research infrastructure. Clinical Research and Regulatory Affairs, 1-8. 16 Mo, H., Thompson, W. K., Rasmussen, L. V., Pacheco, J. A., Jiang, G., Kiefer, R., ... & Lingren, T. (2015). Desiderata for computable representations of electronic health records-driven phenotype algorithms. Journal of the American Medical Informatics Association, 22(6), 1220-1230. 17 Xu, J., Rasmussen, L. V., Shaw, P. L., Jiang, G., Kiefer, R. C., Mo, H., ... & Pathak, J. (2015). Review and evaluation of electronic health records-driven phenotype algorithm authoring tools for clinical and translational research.Journal of the American Medical Informatics Association, ocv070.

AC C

EP

TE D

M AN U

SC

RI PT

1

Page 10 of 16

ACCEPTED MANUSCRIPT Brief Methodological Report

12/13/17

Berg RL, Jennifer A, Pacheco PP, Rasmussen L, Weston N, Chute CG. Analyzing the heterogeneity and complexity of Electronic Health Record oriented phenotyping algorithms. 19 Carlson C. Group Health Cooperative. Dementia. PheKB; 2012 Available from: https://phekb.org/phenotype/10 20 https://www.i2b2.org/about 21 Reisberg B, Ferris SH, de Leon MJ, Crook T. The Global Deterioration Scale for assessment of primary degenerative dementia. The American journal of psychiatry. 1982 Sep. 22 van Riet Paap, J., Vernooij-Dassen, M., Sommerbakk, R., Moyle, W., Hjermstad, M. J., Leppert, W., ... & Engels, Y. (2015). Implementation of improvement strategies in palliative care: an integrative review.Implementation Science, 10(1), 1. 23 O'Connor, N. R., Moyer, M. E., Behta, M., & Casarett, D. J. (2015). The impact of inpatient palliative care consultations on 30-day hospital readmissions. Journal of Palliative Medicine, 18(11), 956-961. 24 May, P., Garrido, M. M., Cassel, J. B., Kelley, A. S., Meier, D. E., Normand, C., ... & Morrison, R. S. (2015). Prospective cohort study of hospital palliative care teams for inpatients with advanced cancer: earlier consultation is associated with larger cost-saving effect. Journal of Clinical Oncology,33(25), 2745-2752. 25 Hanson LC, Usher B, Spragens L, Bernard S. Clinical and economic impact of palliative care consultation. J Pain Symtom Manage 2008; 35:340-346. 26 Ahronheim JC, Morrison RS, Morris J, Baskin S, Meier DE. Palliative care in advanced dementia: a randomized controlled trial and descriptive analysis. Journal of palliative medicine. 2000 Sep 1;3(3):265-73. 27 Volicer L, Collard A, Hurley A, Bishop C, Kern D, Karon S. Impact of special care unit for patients with advanced Alzheimer's disease on patients' discomfort and costs. Journal of the American Geriatrics Society. 1994 Jun 1;42(6):597-603. 28 Greer, J. A., Jackson, V. A., Meier, D. E., & Temel, J. S. (2013). Early integration of palliative care services with standard oncology care for patients with advanced cancer. CA: a cancer journal for clinicians, 63(5), 349-363. 29 Fu, X., Batista-Navarro, R., Rak, R., & Ananiadou, S. (2015). Supporting the annotation of chronic obstructive pulmonary disease (COPD) phenotypes with text mining workflows. Journal of biomedical semantics, 6(1), 1. 30 Drawz, P. E., Archdeacon, P., McDonald, C. J., Powe, N. R., Smith, K. A., Norton, J., ... & Narva, A. (2015). CKD as a model for improving chronic disease care through electronic health records. Clinical Journal of the American Society of Nephrology, 10(8), 1488-1499. 31 http://www.pcornet.org/about-pcornet/ 32 http://www.clinicalleader.com/doc/duke-pcori-use-ehrs-to-transform-clinical-trials-0001 33 Hirschberg J, Manning CD. Advanced in natural language processing. Science 2015; 349:261-266. 34 Barrett N, Weber-Jahnke JH, Thai V. Engineering natural language processing solutions for structured information from clinical text: extracting sentinel events from palliative care consult letters. Stud Health Techol Inform 2013; 192:594-598.

AC C

EP

TE D

M AN U

SC

RI PT

18

Page 11 of 16

ACCEPTED MANUSCRIPT Methods Brief Report

Screening Mechanism

06.21.17

AC C

EP

TE D

M AN U

SC

RI PT

Figure 1. Data flow for the EHR phenotype. novel screening method. [see Figure document.]

Page 12 of 16

ACCEPTED MANUSCRIPT Methods Brief Report

Screening Mechanism

06.21.17

Table 1. Number of patients identified by the EHR phenotype.

124 (20.5%) 82 (13.6%)

First Refinement 7/14/1610/31/16 390

Second Refinement 11/1/16 3/31/17 371

95 (24.4%)

174 (46.9%)

53 (13.6%)

91 (24.5%)

AC C

EP

TE D

M AN U

SC

Total reviewed CDW list Sent to palliative care clinician for review (dementia) (% of total reviewed) Palliative care clinician confirm late-stage dementia (% of total reviewed)

Initial Method 3/29/167/13/16 604

RI PT

Eligible patients

Page 13 of 16

ACCEPTED MANUSCRIPT Methods Brief Report

Screening Mechanism

06.21.17

Table 2. Calculations for sensitivity and specificity of the final EHR phenotype. Formula

TP= EHR phenotype mild dementia and latestage dementia FP = Patients from EHR phenotype with no dementia FP = Patients from EHR phenotype with no dementia TP = EHR phenotype mild dementia and latestage dementia FN = Mild and late-stage dementia palliative care consults, dementia on geriatrics or family medicine services AND not from EHR phenotype TP = EHR phenotype mild dementia and latestage dementia TP = EHR phenotype mild dementia and latestage dementia FN = Mild and late-stage dementia palliative care consults, dementia on geriatrics or family medicine services AND not from EHR phenotype

283

False Negative Late-stage Rate = False Negative (FNLS)/(False Negative(FN) + True Positive (TP))

Rate

76.3%

88

283

23.7%

191 40.3%

283 283 191

TPLS = Clinician-confirmed late-stage dementia from EHR phenotype TP= EHR phenotype mild dementia and latestage dementia

91

FP = Patients from EHR phenotype with no dementia FPLS = All EHR phenotype patients – clinician-confirmed late-stage dementia TP = EHR phenotype mild dementia and latestage dementia FP = Patients from EHR phenotype with no dementia FNLS = Palliative care consults with moderate/severe dementia or on geriatrics or family medicine services AND not from EHR phenotype FN = Mild and late-stage dementia palliative care consults, dementia on geriatrics or family medicine services AND not from EHR phenotype TP = EHR phenotype mild dementia and late-

88

EP

AC C

FDR for Late-stage Dementia = False Positive (FPLS)/ (True Positive (TP) + False Positive (FP))

88

SC

M AN U

TE D

Sensitivity = True Positive (TP) / (True Positive (TP) + False Negative (FN))

Late-stage Dementia PPV for Late-stage Dementia = True Positive Late-stage (TPLS)/ (True Positive (TP) + False Positive (FP))

ProjectSpecific Result

RI PT

Any Dementia Positive Predictive Value (PPV) = True Positive (TP) / (True Positive (TP) + False Positive (FP)) False Discovery Rate (FDR) = False Positive (FP)/ (True Positive (TP) + False Positive (FP)) False Negative Rate = False Negative (FN)/(False Negative(FN) + True Positive (TP))

Calculation of Formula Components with Project-Specific Data

59.7%

283 24.5%

280 283

75.5%

88 51

191

10.8%

283 Page 14 of 16

ACCEPTED MANUSCRIPT Methods Brief Report

Screening Mechanism

06.21.17

stage dementia TPLS = Clinician-confirmed late-stage dementia from EHR phenotype FNLS = Palliative care consults with moderate/severe dementia or on geriatrics or family medicine services AND not from EHR phenotype

91 51

64.1%

AC C

EP

TE D

M AN U

SC

RI PT

Sensitivity = True Positive Late-Stage (TPLS) / (True Positive LateStage (TPLS) + False Negative Late-Stage (FNLS))

Page 15 of 16

ACCEPTED MANUSCRIPT Methods Brief Report

Screening Mechanism

06.21.17

AC C

EP

TE D

M AN U

SC

RI PT

Appendix. ICD codes before and after refinement. Shading indicates that the code was included. [See Appendix document.]

Page 16 of 16

AC C

EP

TE D

M AN U

SC

RI PT

ACCEPTED MANUSCRIPT

ACCEPTED MANUSCRIPT

Appendix.

Dementias F01.50 F01.51

290.41 291.2

Vascular dementia Dementia, vascular with delirium

F10.27 F10.97

Alcohol-induced persisting dementia Dementia in conditions classified elsewhere with behavioral disturbance

294.11

Dementia, unspecified, with behavioral disturbance

294.8

F99

Mental Disorder, other

331

G30.0 G30.1 G30.8 G30.9

Alzheimer’s disease

331.1

G31.09

Frontotemporal disease

F02.80 F02.81

Dementia in Conditions classified elsewhere

F03.90

Dementia, unspecified, without behavioral disturbance

G31.83

Dementia w/Lewy Bodies

I67.89

Cerebrovascular disease

I69.81

Cerebrovascular disease

TE D

Dementia in Conditions classified elsewhere without behavioral disturbance Dementia, unspecified without behavioral disturbance

EP

294.1 294.2

Lewy body dementia

AC C

331.82

M AN U

F03.91

294.21

A81.00

Creutzfeldt-Jakob disease, unspecified

G10

Huntington's disease

G31.01

Pick's disease

F03.90

Senile dementia, uncomplicated

331.2

G31.1

Senile degeneration of brain

331.83

G31.84

Mild Cognitive Impairment

436

I63.9

Cerebrovascular Accident

438.89

I67.9

Cerebrovascular disease

290

780.93

Memory Loss

Second Round (7/14/1610/31/16)

Final Round (11/1/16 3/31/17)

RI PT

290 290.4

First Round (3/29/167/13/16)

ICD-10 Codes

SC

ICD-9 Codes

ACCEPTED MANUSCRIPT

Cognition sign/sympt NEC Cerebrovascular disease

290.1

Presenile dementia

290.2

Senile dementia with delusional or depressive features

290.3

Senile dementia with delirium

290.42

Dementia, vascular with delusions

290.8

Other specified senile psychotic conditions

290.9

Unspecified senile psychotic condition

310

R41.844

Frontal lobe syndrome

I69.80

Cerebrovascular disease

G46.80

Cerebrovascular disease

ZI68.8

Cerebrovascular disease Creutzfeldt-Jakob disease NEC/NOS

291.1

Alcohol amnestic disorder

300.9

Nonpsychotic disorder, not otherwise specified

333.4

Huntington's chorea

434.91

Cerebral artery occlusion unspecified with cerebral infarction

437.1

AC cerebrovascular insufficiency, NOS

437.8

Cerebrovascular disease NEC

437.9

Cerebrovascular disease NOS

438

Late effects of cerebrovascular disease: cognitive deficits

S06.9X0A

TE D

Unspecified intracranial injury without loss of consciousness, initial encounter

Unspecified intracranial injury without loss of consciousness, subsequent encounter

AC C

S06.9X0D

Anoxic brain damage, not elsewhere classified

EP

G93.1

M AN U

46.19

RI PT

I69.91

SC

799.59

S06.9X0S

Unspecified intracranial injury without loss of consciousness, sequela

S09.90XA

Unspecified injury of head, initial encounter

S09.90XD

Unspecified injury of head, subsequent encounter

S09.90XS

Unspecified injury of head, sequela

I61.9

Nontraumatic intracerebral hemorrhage, unspecified

I63.511

Cerebral infarction due to unspecified occlusion or stenosis of right middle cerebral artery

ACCEPTED MANUSCRIPT

Cerebral infarction due to unspecified occlusion or stenosis of right anterior cerebral artery

I63.9

Cerebral infarction, unspecified

I65.29

Occlusion and stenosis of unspecified carotid artery

AC C

EP

TE D

M AN U

SC

RI PT

I63.521