- Email: [email protected]
A trial of indication based prescribing of antihypertensive medications during computerized order entry to improve problem list documentation
IJB-3019; No. of Pages 8
ARTICLE IN PRESS i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 1 3 ) xxx–xxx
journal homepage: www.ijmijournal.com
A trial of indication based prescribing of antihypertensive medications during computerized order entry to improve problem list documentation Suzanne Falck a,∗ , Sruthi Adimadhyam b , David O. Meltzer c , Surrey M. Walton b , William L. Galanter a,d a
Department of Medicine, Section of General Internal Medicine, University of Illinois Hospital and Health Sciences System (UIHHSS), United States b Department of Pharmacy Administration, College of Pharmacy, UIHHSS, United States c Section of Hospital Medicine, University of Chicago, United States d Department of Pharmacy Practice, College of Pharmacy, UIHHSS, United States
a r t i c l e
i n f o
a b s t r a c t
Article history:
Background: Maintenance of problem lists in electronic medical records is required for the
Received 5 September 2012
meaningful use incentive and by the Joint Commission. Linking indication with prescribed
Received in revised form
medications using computerized physician order entry (CPOE) can improve problem list
20 May 2013
documentation. Prescribing of antihypertensive medications is an excellent target for inter-
Accepted 1 July 2013
ventions to improve indication-based prescribing because antihypertensive medications often have multiple indications and are frequently prescribed.
Keywords:
Objective: To measure the accuracy and completeness of electronic problem list additions
Clinical decision support
using indication-based prescribing of antihypertensives.
CDS
Design: Clinical decision support (CDS) was implemented so that orders of antihypertensives
EMR
prompted ordering physicians to select from problem list additions indicated by that med-
Problem list
ication. An observational analysis of 1000 alerts was performed to determine the accuracy
CPOE
of physicians’ selections.
Indication based
Results: At least one accurate problem was placed 57.5% of the time. Inaccurate problems
Hypertension
were placed 4.8% of the time. Accuracy was lower in medications with multiple indications and the likelihood of omitted problems was higher compared to medications whose only indication was hypertension. Attending physicians outperformed other clinicians. There was somewhat lower accuracy for inpatients compared to outpatients. Conclusion: CDS using indication-based prescribing of antihypertensives produced accurate problem placement roughly two-thirds of time with fewer than 5% inaccurate problems placed. Performance of alerts was sensitive to the number of potential indications of the medication and attendings vs. other clinicians prescribing. Indication-based prescribing during CPOE can be used for problem list maintenance, but requires optimization. © 2013 Elsevier Ireland Ltd. All rights reserved.
∗ Corresponding author at: University of Illinois Hospital and Health Sciences System, Department of Medicine, Section of General Internal Medicine (M/C 718), 840 S. Wood Street, Chicago, IL 60612, United States. Tel.: +1 312 413 5576; fax: +1 312 413 1343. E-mail address: [email protected] (S. Falck). 1386-5056/$ – see front matter © 2013 Elsevier Ireland Ltd. All rights reserved. http://dx.doi.org/10.1016/j.ijmedinf.2013.07.003
Please cite this article in press as: S. Falck, et al., A trial of indication based prescribing of antihypertensive medications during computerized order entry to improve problem list documentation, Int. J. Med. Inform. (2013), http://dx.doi.org/10.1016/j.ijmedinf.2013.07.003
IJB-3019; No. of Pages 8
2
ARTICLE IN PRESS i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 1 3 ) xxx–xxx
1.
Introduction
The problem list is an important tool for patient care, helping physicians organize notes and coordinate patient rounds [1]. Problem lists add value to several clinically relevant activities including patient registries, billing support, data mining research and identifying subjects for clinical trials [2–7]. As originally conceived by Weed [1,8], the problem list, as part of the electronic medical record (EMR), has great potential as a linkable index to facilitate better clinical decisions. The Joint Commission requirements for problem lists [9] currently do not specify if problem lists must be in paper or electronic format, but electronic problem lists of “structured data” are required by Medicare and Medicaid to qualify for incentives for the meaningful use of electronic health records [10]. In either form, all of these uses depend on problem list completeness and accuracy. In addition, for the meaningful use incentive the problem list must be “structured data”, so a system that automatically allows the clinician to add problems in an encoded manner makes meeting this criteria easier. Presently, however, problem lists often are incomplete or inaccurate. For example, a study of the Veteran’s Administrations EMR [11] found that only 49% of patients with hypertension had the diagnosis on the problem list. Further, a study from Intermountain Healthcare reported that their problem lists are “usually incomplete and inaccurate, and are often totally unused” [12]. The completeness and accuracy of electronic problem lists can be improved using clinical decision support (CDS) to semiautomate the creation of problem list entries via indication based prescribing [13,14], or, as recently reported by Wright, using pop-up screens independent from orders [15]. The idea for problem list CDS based on indication based prescribing is that every medication would be linked to a specific indication and supported by evidence to ensure optimal prescribing. However, many medications are used for multiple indications, and not all prescribing is evidence-based. Despite these complications, problem list completeness and accuracy may be improved if medications prescribed through a computerized physician order entry system (CPOE) are used to alert physicians to add potential entries to the patient’s electronic problem list, where alerts prompt the common indications of the prescribed medication. We have previously published results using CPOE and CDS to generate indication-based alerts for diabetes, HIV, hypothyroidism, hyperlipidemia, asthma, COPD, TIA’s and off-label use of medications such as IVIG. The accuracy and yield of problem list additions in our previous research depended on the number of indications for a medication [13,14]. For medications with very few indications, such as metformin, the yield of problem list entries and the accuracy of selected alert generated entries were high [13]. For medications with multiple indications, both labeled and not, such as intravenous immune globulin, both the yield and accuracy were lower [14]. This study analyses antihypertensive medications, which also have a wide range of indications. Over 50 million Americans are affected by hypertension, making it the most frequent primary problem for which patients are seen in clinics in the United States [15]. This
disease commonly is treated with pharmacotherapy, typically with more than one medication per patient [15]. However, antihypertensive medications are diverse, with multiple mechanisms of action that also can be used for congestive heart failure, nephropathy, benign prostatic hypertrophy, and disease processes such as migraine prevention and cirrhosis with esophageal varices. In this study we examined the yield and accuracy of problem list entries selected by physicians using CPOE generated indication alerts for antihypertensive medications. We examined variations in alert accuracy depending on the number of indications for the medication, physician characteristics, and whether the clinical setting was inpatient or outpatient in order to better understand the appropriate uses of this new type of CDS.
2.
Methods
2.1.
EMR, CPOE, CDS environment
The University of Illinois Hospital and Health Sciences System (UIHHSS) has a 450-bed teaching hospital and a large multi-specialty ambulatory clinic utilizing a commercial EMR (Millennium® , Cerner Corporation, Kansas City, MO) as the primary repository for problem lists, clinical notes, test results, medication lists and orders. The EMR is used by all specialties, allowing any clinician to update patient records and problem lists either as free text or using the common discrete coded nomenclatures (ICD-9 CM [16] or SNOMED® [17]). All medication orders are placed by CPOE using a commercially available CDS (Discern Expert® , Cerner Corporation) [13,14,18].
2.2.
Indication based alerts
Similar indication alerts have been described previously [13,14]. For this implementation, antihypertensive medications were categorized by potential indications: HTN (hypertension), CHF (congestive heart failure), BPH (benign prostatic hypertrophy), and NEPH (nephropathy) (see example in Fig. 1 below). The choices presented to the ordering physician varied by medication category, the association linking the medication to indications was done in-house by physicians and pharmacists. Minoxidil and methyldopa, for example, each offered HTN as the only pre-identified indication. ACE inhibitors prompted choices of HTN, CHF or NEPH, any combination of which could be selected. If the clinician chose to enter free text, the free text was not automatically added to the problem list. The CDS was triggered when a clinician initiated a new order, a refill, or modified an order for any designated antihypertensive medication. If an indication supporting the use of the medication was already active on the patient’s problem list, then no indication alert was shown to the ordering clinician. However, if there was no supporting indication on the problem list, then an alert was presented to the ordering clinician. Depending on the medication, the alert displayed one or more problems as outlined in Table 1. The clinician could select one or more of the offered indications, make a free text
Please cite this article in press as: S. Falck, et al., A trial of indication based prescribing of antihypertensive medications during computerized order entry to improve problem list documentation, Int. J. Med. Inform. (2013), http://dx.doi.org/10.1016/j.ijmedinf.2013.07.003
IJB-3019; No. of Pages 8
ARTICLE IN PRESS i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 1 3 ) xxx–xxx
3
Fig. 1 – Example alert.
Table 1 – Medication classes and offered indications. Medication classes
Offered indications
Beta blockers (except carvedilol and metoprolol succinate), calcium channel blockers, thiazide diuretics, methyldopa, nitroprusside, combination drugs, minoxidil, anti-adrenergic central acting.
• HTN • Other (free text)
Carvedilol, hydralazine, metoprolol succinate, spironolactone, epleronone
• HTN • CHF • Other (free text)
Peripheral alpha blockers
• HTN • BPH • Other (free text)
ACE (angiotensinconvertingenzyme inhibitors), ARB (angiotensin II receptor blockers)
• HTN • CHF • NEPH • Other (free text)
entry, or choose not to enter a problem. When selected, offered indications were automatically added to the patient’s problem list. Similar alerts previously had been used at UIHHSS, so no additional physician training was performed.
2.3.
Observational trial and evaluation
The alerts were implemented in March 2007 and were studied for three years. To evaluate the accuracy of the ordering physicians’ selections, two experienced internists reviewed charts in a sample of 1000 alerts selected randomly to create approximately equal sized samples of all four alert categories. The reviewers independently determined the problems the patient had unaware of the choices made by the ordering clinicians or the other reviewer. If the first two reviewers did not agree a third reviewer evaluated the case. The problem agreed upon by two of the reviewers was considered the ‘gold standard’, and used to evaluate ordering physician alert choices for accuracy. In no instance did the third reviewer select problems that did not match one of the initial reviewers. The following rates were calculated for each alert type; Alert error rate: The percentage of alerts where prompts did not include at least one correct problem as determined by the reviewers. This is a measure of design errors in not offering all indications that a medication may be used for and is not a function of clinician response to the alerts. Any correct problem placed: The percentage of alerts where clinicians selected one or more correct problems from the prompts. All Correct problems placed: The percentage of alerts where clinicians selected all the correct problems. Fully accurate: The percentage of alerts where clinicians produced fully accurate problem list additions. This included instances when no problem list addition was warranted and none was placed. Inaccurate problems placed: The percentage of alerts where at least one inaccurate problem was added to the problem list.
Please cite this article in press as: S. Falck, et al., A trial of indication based prescribing of antihypertensive medications during computerized order entry to improve problem list documentation, Int. J. Med. Inform. (2013), http://dx.doi.org/10.1016/j.ijmedinf.2013.07.003
ARTICLE IN PRESS
IJB-3019; No. of Pages 8
4
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 1 3 ) xxx–xxx
Table 2 – Alert distribution by problem category, location & clinician type. Percent of alerts
Type of alert
Number of alerts (N)
HTN HTN/CHF HTN/CHF/Nephropathy HTN/BPH
16,202 1288 11,212 7264
45.0% 3.6% 31.2% 20.2%
Location Inpatient Outpatient ER Unknown
17,444 11,617 5107 1798
48.5% 32.3% 14.2% 5.0%
Receiving clinician Attendings Residents Other
7769 25,248 2949
21.6% 70.2% 8.2%
Total alerts
35,966
These performance rates were examined across the medication groups, clinician types (residents, attending & others) and clinical venues (inpatient versus outpatient) using standard multivariate logistic regression. A P-value of 0.05 was used to determine statistical significance.
3.
Results
3.1.
Distribution of alerts
The total number of alerts during the 3 year study period was 35,966 and these fired on 22,653 patients. The distribution of alerts by type, location and receiving clinician is shown in Table 2. The most common alert type was HTN (45%), and the least frequent alert was HTN/CHF (3.6%). Roughly half the alerts fired in the inpatient setting, with outpatient as the next most frequent, and then ED. Residents received 70% of the alerts and attendings 22%.
3.2.
Validity of alerts
We noted many cases in which two reviewers disagreed on problems from chart review, and relied on a third reviewer 59–88% of the time depending on the type of alert. For example, in some charts it was not clear if well controlled blood pressure was due to an antihypertensive medication or if the medication may have been chosen for a different indication, such as an ACE inhibitor for proteinuria or CHF, a beta-blocker for headache prophylaxis, or an alpha blocker for benign prostatic hypertrophy. Often these medications were started prior to initiation of the alert system, or prior to beginning care at our health system, thus limiting clinical data availability, particularly clinicians’ notes. These issues limited the reviewers’ ability to determine the actual indications.
3.3.
Alert performance
The various performance measures are described in Table 3. The alert error rate was higher than expected with an average of 17.3%, ranging from 12.0% to 36.3% depending on the
alert’s diagnostic group (Table 3). This rate reflects the inability to offer all possible problems in the prompted alerts, as well as omission of some common problems. Two problems which occurred more frequently than anticipated were the use of tamsulosin in patients with lower urinary tract symptoms (LUTS), with or without urolithiasis, but without HTN or BPH, and the use of spironolactone in liver failure, which was more common than its use in CHF in our cohort. Exclusion of these 2 medications would have improved the error rate in the HTN/BPH and HTN/CHF alerts. When triggered, alerts led to a problem being added to the problem list 57.5% of the time. HTN had the highest yield of 67.1%. Diagnostic groups with more indications had lower yields in general, with HTN/CHF being the worst at 40.6%. HTN was associated with the fewest inaccurate problems being placed on the list (2.8%), whereas HTN/BPH had the most (12.4%). Overall, an inaccurate problem was added to the list in 4.8% of the alerts. Full accuracy was defined by chart reviewer agreement that all problems were successfully added to the problem list, plus instances where no problem addition was needed. The full accuracy rate within alerts varied significantly from 48.4% for HTN/BPH to 76.3% for HTN alone, with a weighted overall full accuracy rate of 63.1%.
3.4.
Multivariate analyses
Table 4 shows the results of the multivariate analyses involving the performance measures as dependent variables. Compared to the hypertension alone alert (HTN), the other three alerts resulted in significantly lower overall accuracy after controlling for physician type and setting. Non-hypertension alone alerts had significantly higher odds of having no problem added to the list, and were mixed in terms of an inaccurate problem being added. Physician type also was significant. When attending physicians received an alert, they had an odds ratio of nearly 2 for fully accurate additions to the problem list relative to other types of physicians. In addition, attending physicians were less likely to add an inaccurate problem to the list. We also evaluated whether there was a difference in accuracy and yield depending upon the venue where the alert fired. There was a trend toward significance for clinicians being more likely to place an inaccurate problem when the alert fired in the in-patient setting.
4.
Discussion
The prompts studied are different than most other alerts typically in use in CPOE systems. The most basic alerts are drug-allergy and drug-drug alerts. More advanced systems may employ drug-lab alerts. These alerts exploit the relationship between a drug and its indication and can be labeled drug-indication, or drug-problem list alerts. Using the relationship between drugs and their indications has the potential to improve both diagnostic documentation as well as medication use. Our study showed that alerts increased the correct addition of problems to the problem list with only infrequent incorrect
Please cite this article in press as: S. Falck, et al., A trial of indication based prescribing of antihypertensive medications during computerized order entry to improve problem list documentation, Int. J. Med. Inform. (2013), http://dx.doi.org/10.1016/j.ijmedinf.2013.07.003
ARTICLE IN PRESS
IJB-3019; No. of Pages 8
5
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 1 3 ) xxx–xxx
Table 3 – Alert validity and problem addition accuracy by alert type. (N)
Sample size
Alert error rate
Any correct problem placed
All correct problems placed
Fully accurate
HTN HTN/CHF HTN/CHF/Nephropathy HTN/BPH
16,202 1288 11,212 7264
249 251 250 250
12.0% 36.3% 16.8% 26.4%
67.1% 40.6% 52.4% 46.8%
67.1% 26.3% 39.6% 30.4%
76.3% 54.2% 54.6% 48.4%
Total
35,966
1000
17.3%a
57.5%a
49.7%a
63.1%a
a
Inaccurate problem placed 2.8% 9.2% 2.4% 12.4% 4.8%a
The total rates are weighted rates are calculated based on the observed frequency within the evaluated samples weighted by the actual frequency of each category out of the total 35,966 alerts. For example, the total weighted alert error rate is (16,202 * 12% + 1288 * 36.3% + 11,212 * 16.8% + 7264 * 26.4%)/35,966 = 17.3%.
additions. Clinicians were more likely to add simpler indications such as hypertension than more complicated ones such as hypertension/CHF. The accuracy varied significantly across the four alert categories with the simplest problem, HTN, most likely to be completely accurate, whereas alerts for medications with multiple indications had lower accuracy. Inaccuracy generally increased as the number of indications of the medications increased, and at lower levels of training. For HTN alone, inaccurate problems were added only 2.8% of the time. Orders for the three classes of medications with multiple indications had inaccuracy rates of 2.4%, 9.2%, and 12.4%, with a combined weighted inaccuracy rate of 4.8% These findings of improved yield and accuracy for orders of medications with fewer indications are consistent with and add more examples to our previous studies [13,14]. When we previously looked at medications that mostly had one indication [13], the yield for placing a problem was 76%, similar to the present study, and the accuracy was 95%, also similar to our present data. However, for medications which are used infrequently for evidence based indications but have considerable off-label and non-evidence based prescribing [14], the yield dropped significantly to between 22% and 64%, and the accuracy was even worse than in our present study varying from a 9% accurate addition rate for proton pump inhibitors (PPI) to 24% for intravenous immune globulin (IVIG) in the in-patient setting.
Our finding that attending clinicians performed better than trainees needs to be better understood. It may be that indication based prescribing is not appropriate for all levels of trainees. The fact that performance tended to be worse in the inpatient setting also has ramifications for future versions of this type of system. These findings of differences in yield and accuracy of problem placement need to be merged to balance the yield and accurate placement against inaccurate problem additions. There are other ways in which problem list maintenance could be automated within an EMR. One approach would be to use billing diagnoses as a source for problem list entries. This could be implemented as a semi-automated process, e.g., display a prompt when clinicians add a billing diagnosis to allow selection of appropriate entries. A fully-automated transfer of billing codes to the problem list would increase the yield of diagnoses, but would also be likely to fill the problem list with many transient, encounter-based codes that may not be relevant. Another potential method for problem list maintenance is through the placement of problems when diagnostic codes are entered for laboratory orders, as mandated in the ambulatory setting by the federal government [19]. We have begun preliminary work in this area, which appears to be a fruitful area for further study. Currently our institution only uses interruptive alerts or direct addition of problems to the problem list.
Table 4 – Effect of indication group, clinical setting, and type of clinician on the accuracy of patient’s problem list additions. Accurate problem list
Inaccurate problem list
Fully accurate problem list Inaccurate problem added to list Problem not added to list Odds ratio a
HTN-CHF HTN-CHF-Nepha HTN-BPHa Alert fired at an inpatient settingb Alert addressed by an attending physicianc
0.394 0.367 0.281 0.794* 1.901
95% CI
Odds ratio
0.268–0.581 0.249–0.541 0.191–0.414 0.592–1.066 1.298–2.784
3.074 0.843NS 5.052 1.047NS 0.128
95% CI 1.290–7.325 0.278–2.552 2.171–11.761 0.586–1.870 0.030–0.543
Odds ratio 2.089 2.889 2.481 1.262NS 0.703**
95% CI 1.398–3.123 1.941–4.299 1.665–3.699 0.933–1.706 0.478–1.034
NS: effect not statistically significant. Compared to HTN. b Compared to outpatient settings. c Compared to others (residents, nurse practitioners). ∗ p = 0.124. ∗∗ p = 0.074. a
Please cite this article in press as: S. Falck, et al., A trial of indication based prescribing of antihypertensive medications during computerized order entry to improve problem list documentation, Int. J. Med. Inform. (2013), http://dx.doi.org/10.1016/j.ijmedinf.2013.07.003
IJB-3019; No. of Pages 8
6
ARTICLE IN PRESS i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 1 3 ) xxx–xxx
A promising technology is the use of natural language processing to determine problems based on free text clinical documentation [20–22]. Much work has been done establishing that clinical documentation can be mined for coded content, but there are precision errors and it is likely that fullyautomating problem list updates without clinician review would produce unacceptably high inaccuracy rates. However, prompt- driven alerts generated by natural language analysis could be an efficient method of improving problem list additions for clinicians. Wright et al, recently reported a CDS to add problems which was triggered upon saving notes or signing dictations [15]. For diabetes and hypertension, Wright showed acceptance rates of 39% and 42.5%, respectively. The use of indication-based prescribing in our prior work showed a rate of 79% [13] for diabetes while our current study showed an overall rate of 57.5% for hypertension. These acceptance rate differences may be due to alert design (home grown vs. vendor), trigger (note completion vs. medication ordering), or clinician differences (diverse vs. ambulatory attending primary care physicians). Analysis of all intervention methodologies is important to help produce the next generation of CDS based problem list improvements. A recent study by Eguale et al. [23] showed an impressive rate of 98.5% in documenting indications during medication use, higher than both our prior [13] and present study. The higher rate may be related to a variety of differences between their study and ours, including the study location, Canada, and the fact that they only used attending primary care doctors. In addition, the selection of an indication was mandatory and the gold standard problem was based on clinician recall rather than expert chart review One of the potential benefits of our design is that the clinician is presented a specific or small set of problems at the time of medication order. A second and potentially critical advantage is that alerts based on a smaller set of problems could help prevent wrong medication errors or wrong chart errors if the clinician realizes that the choices offered by the alert do not correspond to those for the patient [24,25]. Though a larger set of problems are presented, the system reported by Eguale may potentially have this benefit also. We are presently pursuing this topic in further research. The indication prompts studied have previously been shown to be able to improve problem list documentation [13], intercept wrong chart medication errors [24] and intercept drug name confusion errors [25]. It is also known that poor selection of medication can produce prompts that do not produce high fidelity additions to the problem list [14]. Our present data and analysis demonstrate that in terms of problem list additions, these prompts work better for attending physicians than housestaff when ordering single indication medications. There was a strong trend toward improved performance in the ambulatory setting versus inpatient.
4.1.
prescriptions as triggers for this CDS. The purpose of using refills was to improve the yield of problems entered, but when the trigger was a refill, the exact patient condition at the time of the first prescription was not always available to help determine the indication. This study was conducted at a single medical institution which has a significant specialty medicine presence and a relatively low-income urban city patient population. This may not reflect patient and disease profiles at other institutions. For example, during the review it was identified that spironolactone alerts were more frequent for patients with cirrhosis than for CHF. Cirrhosis was not a pre-established alert prompt, which contributed to a higher error rate for this indication. UIHHSS uses a single EMR, and it is possible that how alerts were implemented in Cerner® Powerchart® could have had an impact on the results. Similar alert implementations using other EMR systems may have other interfaces producing different alert performance.
5.
Conclusions
Indication based prescribing of antihypertensive medications triggered by CPOE led to a reasonable yield of accurate problem list additions, 57.5%, with a low rate of inaccurate additions 4.8%. Yield and accuracy were highest for medications with a single indication. Both yield and accuracy were lower for prescribed medications that had multiple indicated problems, but inaccuracy rates were not fully correlated with the number of indication choices. Attending physicians produced more frequent and accurate problem list additions than less experienced clinicians. The accuracy of alerts selected in an out-patient setting slightly exceeded those placed in the in-patient setting. This work demonstrated trends in the accuracy and yield of problem list additions based on clinician and medication factors which should help in design of future problem list prompts as well as with future studies.
Authors’ contributions Suzanne Falck contributed substantially to the conceptualization, design, data analysis, interpretation and writing of the manuscript and gave final approval. Sruthi Adimadhyam assisted with the data analysis and writing of the manuscript and gave final approval. David O. Meltzer contributed to the conceptualization and funding of the project as well as the interpretation and writing of the manuscript and gave final approval. Surrey M. Walton assisted with the study design and data analysis, as well as the interpretation and writing of the manuscript and gave final approval. William L. Galanter led the conceptualization, design, data analysis, interpretation and writing of the manuscript and gave final approval.
Study limitations
One limitation of this study was the difficulty, in some instances, of establishing a gold standard to evaluate the ordering physicians’ alert selections. It is very likely that this was in part due to the use of both refills and initial
Conflicts of interest Dr. Galanter has a research grant from Abbott Laboratories, no other authors have any conflicts to disclose.
Please cite this article in press as: S. Falck, et al., A trial of indication based prescribing of antihypertensive medications during computerized order entry to improve problem list documentation, Int. J. Med. Inform. (2013), http://dx.doi.org/10.1016/j.ijmedinf.2013.07.003
ARTICLE IN PRESS
IJB-3019; No. of Pages 8
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 1 3 ) xxx–xxx
Summary points What was already known • Problem lists have several potentially valuable uses but are often are incomplete or inaccurate • The completeness and accuracy of electronic problem lists can be improved using clinical decision support (CDS) to semi-automate the creation of problem list entries via indication based prescribing. What this study added to our knowledge • Indication based prescribing of antihypertensive medications triggered by CPOE led to accurate problem list additions 58%, with a low rate of inaccurate additions, 5%. • Attending physicians produced more frequent and accurate problem list additions than less experienced clinicians, and the accuracy of alerts selected in an outpatient setting slightly exceeded those placed in the in-patient setting.
Acknowledgements With special thanks to Lisa Canonge, Marla Lax, R.N, Amy Looi, R.N. and Jennifer Welch CT (ASCP) for assistance in the CDS alert development. John Falck for editorial assistance. This study was approved by the University of Illinois Hospital and Health Sciences System Institutional Review Board. The study was in part funded by Grant number 1-U18 HSO16967-01 from AHRQ. The funding source had no role in the production of this manuscript.
references
[1] L.L. Weed, Medical records that guide and teach, N. Engl. J. Med. 278 (1968) 593–600. [2] D.S. Benson, W. Van Osdol, P. Townes, Quality ambulatory care: the role of the diagnostic and medication summary lists, Qual. Rev. Bull. 14 (6) (1988) 192–197. [3] M.E. Johnston, K.B. Langton, R.B. Haynes, A. Mathieu, Effects of computer-based clinical decision support systems on clinician performance and patient outcome: a critical appraisal of research, Ann. Intern. Med. 120 (2) (1994) 135–142. [4] P.C. Tang, M.P. LaRosa, S.M. Gorden, Use of computer-based records, completeness of documentation, and appropriateness of documented clinical decisions, J. Am. Med. Inform. Assoc. 6 (1999) 245–251. [5] U. Sarkar, J.T. Carter, T.A. Omachi, A.R. Vidyarthi, R. Cucina, S. Bokser, et al., SynopSIS: integrating physician sign-out with the electronic medical record, J. Hosp. Med. 2 (5) (2007) 336–342. [6] G. Frank, S.T. Lawless, T.H. Steinberg, Improving physician communication through an automated, integrated sign-out
7
system, J. Healthc. Inf. Manag. 19 (4) (2005) 68–74. [7] J.A. Bernstein, D.L. Imler, P. Sharek, C.A. Longhurst, Improved physician work flow after integrating sign-out notes into the electronic medical record, Jt. Comm. J. Qual. Patient Saf. 36 (February (2)) (2010) 72–78. [8] L.L. Weed, Medical Records, Medical Education, and Patient Care, Yearbook Medical Publishers, Chicago, 1969. [9] Comprehensive Accreditation Manual for Hospitals, The Joint Commission on Accreditation of Healthcare Organizations, Oakbrook Terrace, IL, 1996. [10] D. Blumenthal, M. Tavenner, The “meaningful use” regulation for electronic health records, N. Engl. J. Med. 363 (August (6)) (2010) 501–504 (Epub 2010 July 13). [11] H.C. Szeto, R.K. Coleman, P. Gholami, B.B. Hoffman, M.K. Goldstein, Accuracy of computerized outpatient diagnoses in a Veterans Affairs general medicine clinic, Am. J. Manag. Care 8 (2002) 37–43. [12] S.M. Meystre, P.J. Haug, Randomized controlled trial of an automated problem list with improved sensitivity, Int. J. Med. Inform. 77 (September (9)) (2008) 602–612. [13] W.L. Galanter, D.B. Hier, C. Jao, D. Sarne, Computerized physician order entry of medications and clinical decision support can improve problem list documentation compliance, Int. J. Med. Inform. 79 (May (5)) (2010) 332–338. [14] S.M. Walton, W. Galanter, H. Rosencranz, D. Meltzer, R.S. Stafford, F. Tiryaki, D. Sarne, A trial of inpatient indication based prescribing during computerized order entry with medications commonly used off-label, Appl. Clin. Inform. 21 (2011) 94–103. [15] A. Wright, J. Pang, J.C. Feblowitz, et al., Improving completeness of electronic problem lists through clinical decision support: a randomized, controlled trial, J. Am. Med. Inform. Assoc. 19 (4) (2012) 555–561. [16] V. Aram, A. Chobanian, G. Bakris, H. Black, et al., The seventh report of the Joint National Committee on the prevention, detection, and evaluation and treatment of high blood pressure. The National High Blood Pressure Education Program Coordinating Committee, Hypertension 42 (2003) 1206. [17] ICD - ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification, 2013, http://www.cdc.gov/nchs/about/otheract/icd9/abticd9.htm (accessed 17.12.07). [18] SNOMED® . Systematized Nomenclature of Medicine, 2013, http://www.cap.org/apps/cap.portal? nfpb=true& pageLabel= snomed page (accessed 17.12.07). [19] R.A. Raschke, B. Gollihare, T.A. Wunderlich, J.R. Guidry, A.I. Leibowitz, J.C. Peirce, L. Lemelson, M.A. Heisler, C. Susong, A computer alert system to prevent injury from adverse drug events: development and evaluation in a community teaching hospital, JAMA 280 (October (15)) (1998) 1317–1320. [20] D.M. Wolman, A.L. Kalfoglou, L. LeRoy, Medicare Laboratory Payment Policy, 2013. [21] S. Pakhomov, H. Hemingway, S. Jacobsen, R. Rodeheffer, V. Roger, Natural language processing or ICD- 9 diagnostic codes: which is better at identifying patients with stable angina pectoris? Circulation 113 (May (21)) (2006) e815. [22] I. Solti, B. Aaronson, G. Fletcher, M. Solti, J.H. Gennari, M. Cooper, T. Payne, Building an automated problem list based on natural language processing: lessons learned in the early phase of development, AMIA Annu. Symp. Proc. 6 (November) (2008) 687–691. [23] T. Eguale, N. Winslade, J. Hanley, et al., Enhancing pharmacosurveillance with systematic collection of
Please cite this article in press as: S. Falck, et al., A trial of indication based prescribing of antihypertensive medications during computerized order entry to improve problem list documentation, Int. J. Med. Inform. (2013), http://dx.doi.org/10.1016/j.ijmedinf.2013.07.003
IJB-3019; No. of Pages 8
8
ARTICLE IN PRESS i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 1 3 ) xxx–xxx
treatment indication in electronic prescribing, Drug Saf. 33 (7) (2010) 559–567. [24] W. Galanter, S. Falck, M. Burns, M. Laragh, B. Lambert, Indication-based prescribing prevents wrong-patient medication errors in computerized provider order
entry, J. Am. Med. Inform. Assoc. 20 (May (3)) (2013) 477–481. [25] B. Lambert, HIT and Medication Safety, in: 15th Annual NPSF Patient Safety Congress, Presentation, New Orleans, May, 2013.
Please cite this article in press as: S. Falck, et al., A trial of indication based prescribing of antihypertensive medications during computerized order entry to improve problem list documentation, Int. J. Med. Inform. (2013), http://dx.doi.org/10.1016/j.ijmedinf.2013.07.003
ARTICLE IN PRESS i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 1 3 ) xxx–xxx
journal homepage: www.ijmijournal.com
A trial of indication based prescribing of antihypertensive medications during computerized order entry to improve problem list documentation Suzanne Falck a,∗ , Sruthi Adimadhyam b , David O. Meltzer c , Surrey M. Walton b , William L. Galanter a,d a
Department of Medicine, Section of General Internal Medicine, University of Illinois Hospital and Health Sciences System (UIHHSS), United States b Department of Pharmacy Administration, College of Pharmacy, UIHHSS, United States c Section of Hospital Medicine, University of Chicago, United States d Department of Pharmacy Practice, College of Pharmacy, UIHHSS, United States
a r t i c l e
i n f o
a b s t r a c t
Article history:
Background: Maintenance of problem lists in electronic medical records is required for the
Received 5 September 2012
meaningful use incentive and by the Joint Commission. Linking indication with prescribed
Received in revised form
medications using computerized physician order entry (CPOE) can improve problem list
20 May 2013
documentation. Prescribing of antihypertensive medications is an excellent target for inter-
Accepted 1 July 2013
ventions to improve indication-based prescribing because antihypertensive medications often have multiple indications and are frequently prescribed.
Keywords:
Objective: To measure the accuracy and completeness of electronic problem list additions
Clinical decision support
using indication-based prescribing of antihypertensives.
CDS
Design: Clinical decision support (CDS) was implemented so that orders of antihypertensives
EMR
prompted ordering physicians to select from problem list additions indicated by that med-
Problem list
ication. An observational analysis of 1000 alerts was performed to determine the accuracy
CPOE
of physicians’ selections.
Indication based
Results: At least one accurate problem was placed 57.5% of the time. Inaccurate problems
Hypertension
were placed 4.8% of the time. Accuracy was lower in medications with multiple indications and the likelihood of omitted problems was higher compared to medications whose only indication was hypertension. Attending physicians outperformed other clinicians. There was somewhat lower accuracy for inpatients compared to outpatients. Conclusion: CDS using indication-based prescribing of antihypertensives produced accurate problem placement roughly two-thirds of time with fewer than 5% inaccurate problems placed. Performance of alerts was sensitive to the number of potential indications of the medication and attendings vs. other clinicians prescribing. Indication-based prescribing during CPOE can be used for problem list maintenance, but requires optimization. © 2013 Elsevier Ireland Ltd. All rights reserved.
∗ Corresponding author at: University of Illinois Hospital and Health Sciences System, Department of Medicine, Section of General Internal Medicine (M/C 718), 840 S. Wood Street, Chicago, IL 60612, United States. Tel.: +1 312 413 5576; fax: +1 312 413 1343. E-mail address: [email protected] (S. Falck). 1386-5056/$ – see front matter © 2013 Elsevier Ireland Ltd. All rights reserved. http://dx.doi.org/10.1016/j.ijmedinf.2013.07.003
Please cite this article in press as: S. Falck, et al., A trial of indication based prescribing of antihypertensive medications during computerized order entry to improve problem list documentation, Int. J. Med. Inform. (2013), http://dx.doi.org/10.1016/j.ijmedinf.2013.07.003
IJB-3019; No. of Pages 8
2
ARTICLE IN PRESS i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 1 3 ) xxx–xxx
1.
Introduction
The problem list is an important tool for patient care, helping physicians organize notes and coordinate patient rounds [1]. Problem lists add value to several clinically relevant activities including patient registries, billing support, data mining research and identifying subjects for clinical trials [2–7]. As originally conceived by Weed [1,8], the problem list, as part of the electronic medical record (EMR), has great potential as a linkable index to facilitate better clinical decisions. The Joint Commission requirements for problem lists [9] currently do not specify if problem lists must be in paper or electronic format, but electronic problem lists of “structured data” are required by Medicare and Medicaid to qualify for incentives for the meaningful use of electronic health records [10]. In either form, all of these uses depend on problem list completeness and accuracy. In addition, for the meaningful use incentive the problem list must be “structured data”, so a system that automatically allows the clinician to add problems in an encoded manner makes meeting this criteria easier. Presently, however, problem lists often are incomplete or inaccurate. For example, a study of the Veteran’s Administrations EMR [11] found that only 49% of patients with hypertension had the diagnosis on the problem list. Further, a study from Intermountain Healthcare reported that their problem lists are “usually incomplete and inaccurate, and are often totally unused” [12]. The completeness and accuracy of electronic problem lists can be improved using clinical decision support (CDS) to semiautomate the creation of problem list entries via indication based prescribing [13,14], or, as recently reported by Wright, using pop-up screens independent from orders [15]. The idea for problem list CDS based on indication based prescribing is that every medication would be linked to a specific indication and supported by evidence to ensure optimal prescribing. However, many medications are used for multiple indications, and not all prescribing is evidence-based. Despite these complications, problem list completeness and accuracy may be improved if medications prescribed through a computerized physician order entry system (CPOE) are used to alert physicians to add potential entries to the patient’s electronic problem list, where alerts prompt the common indications of the prescribed medication. We have previously published results using CPOE and CDS to generate indication-based alerts for diabetes, HIV, hypothyroidism, hyperlipidemia, asthma, COPD, TIA’s and off-label use of medications such as IVIG. The accuracy and yield of problem list additions in our previous research depended on the number of indications for a medication [13,14]. For medications with very few indications, such as metformin, the yield of problem list entries and the accuracy of selected alert generated entries were high [13]. For medications with multiple indications, both labeled and not, such as intravenous immune globulin, both the yield and accuracy were lower [14]. This study analyses antihypertensive medications, which also have a wide range of indications. Over 50 million Americans are affected by hypertension, making it the most frequent primary problem for which patients are seen in clinics in the United States [15]. This
disease commonly is treated with pharmacotherapy, typically with more than one medication per patient [15]. However, antihypertensive medications are diverse, with multiple mechanisms of action that also can be used for congestive heart failure, nephropathy, benign prostatic hypertrophy, and disease processes such as migraine prevention and cirrhosis with esophageal varices. In this study we examined the yield and accuracy of problem list entries selected by physicians using CPOE generated indication alerts for antihypertensive medications. We examined variations in alert accuracy depending on the number of indications for the medication, physician characteristics, and whether the clinical setting was inpatient or outpatient in order to better understand the appropriate uses of this new type of CDS.
2.
Methods
2.1.
EMR, CPOE, CDS environment
The University of Illinois Hospital and Health Sciences System (UIHHSS) has a 450-bed teaching hospital and a large multi-specialty ambulatory clinic utilizing a commercial EMR (Millennium® , Cerner Corporation, Kansas City, MO) as the primary repository for problem lists, clinical notes, test results, medication lists and orders. The EMR is used by all specialties, allowing any clinician to update patient records and problem lists either as free text or using the common discrete coded nomenclatures (ICD-9 CM [16] or SNOMED® [17]). All medication orders are placed by CPOE using a commercially available CDS (Discern Expert® , Cerner Corporation) [13,14,18].
2.2.
Indication based alerts
Similar indication alerts have been described previously [13,14]. For this implementation, antihypertensive medications were categorized by potential indications: HTN (hypertension), CHF (congestive heart failure), BPH (benign prostatic hypertrophy), and NEPH (nephropathy) (see example in Fig. 1 below). The choices presented to the ordering physician varied by medication category, the association linking the medication to indications was done in-house by physicians and pharmacists. Minoxidil and methyldopa, for example, each offered HTN as the only pre-identified indication. ACE inhibitors prompted choices of HTN, CHF or NEPH, any combination of which could be selected. If the clinician chose to enter free text, the free text was not automatically added to the problem list. The CDS was triggered when a clinician initiated a new order, a refill, or modified an order for any designated antihypertensive medication. If an indication supporting the use of the medication was already active on the patient’s problem list, then no indication alert was shown to the ordering clinician. However, if there was no supporting indication on the problem list, then an alert was presented to the ordering clinician. Depending on the medication, the alert displayed one or more problems as outlined in Table 1. The clinician could select one or more of the offered indications, make a free text
Please cite this article in press as: S. Falck, et al., A trial of indication based prescribing of antihypertensive medications during computerized order entry to improve problem list documentation, Int. J. Med. Inform. (2013), http://dx.doi.org/10.1016/j.ijmedinf.2013.07.003
IJB-3019; No. of Pages 8
ARTICLE IN PRESS i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 1 3 ) xxx–xxx
3
Fig. 1 – Example alert.
Table 1 – Medication classes and offered indications. Medication classes
Offered indications
Beta blockers (except carvedilol and metoprolol succinate), calcium channel blockers, thiazide diuretics, methyldopa, nitroprusside, combination drugs, minoxidil, anti-adrenergic central acting.
• HTN • Other (free text)
Carvedilol, hydralazine, metoprolol succinate, spironolactone, epleronone
• HTN • CHF • Other (free text)
Peripheral alpha blockers
• HTN • BPH • Other (free text)
ACE (angiotensinconvertingenzyme inhibitors), ARB (angiotensin II receptor blockers)
• HTN • CHF • NEPH • Other (free text)
entry, or choose not to enter a problem. When selected, offered indications were automatically added to the patient’s problem list. Similar alerts previously had been used at UIHHSS, so no additional physician training was performed.
2.3.
Observational trial and evaluation
The alerts were implemented in March 2007 and were studied for three years. To evaluate the accuracy of the ordering physicians’ selections, two experienced internists reviewed charts in a sample of 1000 alerts selected randomly to create approximately equal sized samples of all four alert categories. The reviewers independently determined the problems the patient had unaware of the choices made by the ordering clinicians or the other reviewer. If the first two reviewers did not agree a third reviewer evaluated the case. The problem agreed upon by two of the reviewers was considered the ‘gold standard’, and used to evaluate ordering physician alert choices for accuracy. In no instance did the third reviewer select problems that did not match one of the initial reviewers. The following rates were calculated for each alert type; Alert error rate: The percentage of alerts where prompts did not include at least one correct problem as determined by the reviewers. This is a measure of design errors in not offering all indications that a medication may be used for and is not a function of clinician response to the alerts. Any correct problem placed: The percentage of alerts where clinicians selected one or more correct problems from the prompts. All Correct problems placed: The percentage of alerts where clinicians selected all the correct problems. Fully accurate: The percentage of alerts where clinicians produced fully accurate problem list additions. This included instances when no problem list addition was warranted and none was placed. Inaccurate problems placed: The percentage of alerts where at least one inaccurate problem was added to the problem list.
Please cite this article in press as: S. Falck, et al., A trial of indication based prescribing of antihypertensive medications during computerized order entry to improve problem list documentation, Int. J. Med. Inform. (2013), http://dx.doi.org/10.1016/j.ijmedinf.2013.07.003
ARTICLE IN PRESS
IJB-3019; No. of Pages 8
4
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 1 3 ) xxx–xxx
Table 2 – Alert distribution by problem category, location & clinician type. Percent of alerts
Type of alert
Number of alerts (N)
HTN HTN/CHF HTN/CHF/Nephropathy HTN/BPH
16,202 1288 11,212 7264
45.0% 3.6% 31.2% 20.2%
Location Inpatient Outpatient ER Unknown
17,444 11,617 5107 1798
48.5% 32.3% 14.2% 5.0%
Receiving clinician Attendings Residents Other
7769 25,248 2949
21.6% 70.2% 8.2%
Total alerts
35,966
These performance rates were examined across the medication groups, clinician types (residents, attending & others) and clinical venues (inpatient versus outpatient) using standard multivariate logistic regression. A P-value of 0.05 was used to determine statistical significance.
3.
Results
3.1.
Distribution of alerts
The total number of alerts during the 3 year study period was 35,966 and these fired on 22,653 patients. The distribution of alerts by type, location and receiving clinician is shown in Table 2. The most common alert type was HTN (45%), and the least frequent alert was HTN/CHF (3.6%). Roughly half the alerts fired in the inpatient setting, with outpatient as the next most frequent, and then ED. Residents received 70% of the alerts and attendings 22%.
3.2.
Validity of alerts
We noted many cases in which two reviewers disagreed on problems from chart review, and relied on a third reviewer 59–88% of the time depending on the type of alert. For example, in some charts it was not clear if well controlled blood pressure was due to an antihypertensive medication or if the medication may have been chosen for a different indication, such as an ACE inhibitor for proteinuria or CHF, a beta-blocker for headache prophylaxis, or an alpha blocker for benign prostatic hypertrophy. Often these medications were started prior to initiation of the alert system, or prior to beginning care at our health system, thus limiting clinical data availability, particularly clinicians’ notes. These issues limited the reviewers’ ability to determine the actual indications.
3.3.
Alert performance
The various performance measures are described in Table 3. The alert error rate was higher than expected with an average of 17.3%, ranging from 12.0% to 36.3% depending on the
alert’s diagnostic group (Table 3). This rate reflects the inability to offer all possible problems in the prompted alerts, as well as omission of some common problems. Two problems which occurred more frequently than anticipated were the use of tamsulosin in patients with lower urinary tract symptoms (LUTS), with or without urolithiasis, but without HTN or BPH, and the use of spironolactone in liver failure, which was more common than its use in CHF in our cohort. Exclusion of these 2 medications would have improved the error rate in the HTN/BPH and HTN/CHF alerts. When triggered, alerts led to a problem being added to the problem list 57.5% of the time. HTN had the highest yield of 67.1%. Diagnostic groups with more indications had lower yields in general, with HTN/CHF being the worst at 40.6%. HTN was associated with the fewest inaccurate problems being placed on the list (2.8%), whereas HTN/BPH had the most (12.4%). Overall, an inaccurate problem was added to the list in 4.8% of the alerts. Full accuracy was defined by chart reviewer agreement that all problems were successfully added to the problem list, plus instances where no problem addition was needed. The full accuracy rate within alerts varied significantly from 48.4% for HTN/BPH to 76.3% for HTN alone, with a weighted overall full accuracy rate of 63.1%.
3.4.
Multivariate analyses
Table 4 shows the results of the multivariate analyses involving the performance measures as dependent variables. Compared to the hypertension alone alert (HTN), the other three alerts resulted in significantly lower overall accuracy after controlling for physician type and setting. Non-hypertension alone alerts had significantly higher odds of having no problem added to the list, and were mixed in terms of an inaccurate problem being added. Physician type also was significant. When attending physicians received an alert, they had an odds ratio of nearly 2 for fully accurate additions to the problem list relative to other types of physicians. In addition, attending physicians were less likely to add an inaccurate problem to the list. We also evaluated whether there was a difference in accuracy and yield depending upon the venue where the alert fired. There was a trend toward significance for clinicians being more likely to place an inaccurate problem when the alert fired in the in-patient setting.
4.
Discussion
The prompts studied are different than most other alerts typically in use in CPOE systems. The most basic alerts are drug-allergy and drug-drug alerts. More advanced systems may employ drug-lab alerts. These alerts exploit the relationship between a drug and its indication and can be labeled drug-indication, or drug-problem list alerts. Using the relationship between drugs and their indications has the potential to improve both diagnostic documentation as well as medication use. Our study showed that alerts increased the correct addition of problems to the problem list with only infrequent incorrect
Please cite this article in press as: S. Falck, et al., A trial of indication based prescribing of antihypertensive medications during computerized order entry to improve problem list documentation, Int. J. Med. Inform. (2013), http://dx.doi.org/10.1016/j.ijmedinf.2013.07.003
ARTICLE IN PRESS
IJB-3019; No. of Pages 8
5
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 1 3 ) xxx–xxx
Table 3 – Alert validity and problem addition accuracy by alert type. (N)
Sample size
Alert error rate
Any correct problem placed
All correct problems placed
Fully accurate
HTN HTN/CHF HTN/CHF/Nephropathy HTN/BPH
16,202 1288 11,212 7264
249 251 250 250
12.0% 36.3% 16.8% 26.4%
67.1% 40.6% 52.4% 46.8%
67.1% 26.3% 39.6% 30.4%
76.3% 54.2% 54.6% 48.4%
Total
35,966
1000
17.3%a
57.5%a
49.7%a
63.1%a
a
Inaccurate problem placed 2.8% 9.2% 2.4% 12.4% 4.8%a
The total rates are weighted rates are calculated based on the observed frequency within the evaluated samples weighted by the actual frequency of each category out of the total 35,966 alerts. For example, the total weighted alert error rate is (16,202 * 12% + 1288 * 36.3% + 11,212 * 16.8% + 7264 * 26.4%)/35,966 = 17.3%.
additions. Clinicians were more likely to add simpler indications such as hypertension than more complicated ones such as hypertension/CHF. The accuracy varied significantly across the four alert categories with the simplest problem, HTN, most likely to be completely accurate, whereas alerts for medications with multiple indications had lower accuracy. Inaccuracy generally increased as the number of indications of the medications increased, and at lower levels of training. For HTN alone, inaccurate problems were added only 2.8% of the time. Orders for the three classes of medications with multiple indications had inaccuracy rates of 2.4%, 9.2%, and 12.4%, with a combined weighted inaccuracy rate of 4.8% These findings of improved yield and accuracy for orders of medications with fewer indications are consistent with and add more examples to our previous studies [13,14]. When we previously looked at medications that mostly had one indication [13], the yield for placing a problem was 76%, similar to the present study, and the accuracy was 95%, also similar to our present data. However, for medications which are used infrequently for evidence based indications but have considerable off-label and non-evidence based prescribing [14], the yield dropped significantly to between 22% and 64%, and the accuracy was even worse than in our present study varying from a 9% accurate addition rate for proton pump inhibitors (PPI) to 24% for intravenous immune globulin (IVIG) in the in-patient setting.
Our finding that attending clinicians performed better than trainees needs to be better understood. It may be that indication based prescribing is not appropriate for all levels of trainees. The fact that performance tended to be worse in the inpatient setting also has ramifications for future versions of this type of system. These findings of differences in yield and accuracy of problem placement need to be merged to balance the yield and accurate placement against inaccurate problem additions. There are other ways in which problem list maintenance could be automated within an EMR. One approach would be to use billing diagnoses as a source for problem list entries. This could be implemented as a semi-automated process, e.g., display a prompt when clinicians add a billing diagnosis to allow selection of appropriate entries. A fully-automated transfer of billing codes to the problem list would increase the yield of diagnoses, but would also be likely to fill the problem list with many transient, encounter-based codes that may not be relevant. Another potential method for problem list maintenance is through the placement of problems when diagnostic codes are entered for laboratory orders, as mandated in the ambulatory setting by the federal government [19]. We have begun preliminary work in this area, which appears to be a fruitful area for further study. Currently our institution only uses interruptive alerts or direct addition of problems to the problem list.
Table 4 – Effect of indication group, clinical setting, and type of clinician on the accuracy of patient’s problem list additions. Accurate problem list
Inaccurate problem list
Fully accurate problem list Inaccurate problem added to list Problem not added to list Odds ratio a
HTN-CHF HTN-CHF-Nepha HTN-BPHa Alert fired at an inpatient settingb Alert addressed by an attending physicianc
0.394 0.367 0.281 0.794* 1.901
95% CI
Odds ratio
0.268–0.581 0.249–0.541 0.191–0.414 0.592–1.066 1.298–2.784
3.074 0.843NS 5.052 1.047NS 0.128
95% CI 1.290–7.325 0.278–2.552 2.171–11.761 0.586–1.870 0.030–0.543
Odds ratio 2.089 2.889 2.481 1.262NS 0.703**
95% CI 1.398–3.123 1.941–4.299 1.665–3.699 0.933–1.706 0.478–1.034
NS: effect not statistically significant. Compared to HTN. b Compared to outpatient settings. c Compared to others (residents, nurse practitioners). ∗ p = 0.124. ∗∗ p = 0.074. a
Please cite this article in press as: S. Falck, et al., A trial of indication based prescribing of antihypertensive medications during computerized order entry to improve problem list documentation, Int. J. Med. Inform. (2013), http://dx.doi.org/10.1016/j.ijmedinf.2013.07.003
IJB-3019; No. of Pages 8
6
ARTICLE IN PRESS i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 1 3 ) xxx–xxx
A promising technology is the use of natural language processing to determine problems based on free text clinical documentation [20–22]. Much work has been done establishing that clinical documentation can be mined for coded content, but there are precision errors and it is likely that fullyautomating problem list updates without clinician review would produce unacceptably high inaccuracy rates. However, prompt- driven alerts generated by natural language analysis could be an efficient method of improving problem list additions for clinicians. Wright et al, recently reported a CDS to add problems which was triggered upon saving notes or signing dictations [15]. For diabetes and hypertension, Wright showed acceptance rates of 39% and 42.5%, respectively. The use of indication-based prescribing in our prior work showed a rate of 79% [13] for diabetes while our current study showed an overall rate of 57.5% for hypertension. These acceptance rate differences may be due to alert design (home grown vs. vendor), trigger (note completion vs. medication ordering), or clinician differences (diverse vs. ambulatory attending primary care physicians). Analysis of all intervention methodologies is important to help produce the next generation of CDS based problem list improvements. A recent study by Eguale et al. [23] showed an impressive rate of 98.5% in documenting indications during medication use, higher than both our prior [13] and present study. The higher rate may be related to a variety of differences between their study and ours, including the study location, Canada, and the fact that they only used attending primary care doctors. In addition, the selection of an indication was mandatory and the gold standard problem was based on clinician recall rather than expert chart review One of the potential benefits of our design is that the clinician is presented a specific or small set of problems at the time of medication order. A second and potentially critical advantage is that alerts based on a smaller set of problems could help prevent wrong medication errors or wrong chart errors if the clinician realizes that the choices offered by the alert do not correspond to those for the patient [24,25]. Though a larger set of problems are presented, the system reported by Eguale may potentially have this benefit also. We are presently pursuing this topic in further research. The indication prompts studied have previously been shown to be able to improve problem list documentation [13], intercept wrong chart medication errors [24] and intercept drug name confusion errors [25]. It is also known that poor selection of medication can produce prompts that do not produce high fidelity additions to the problem list [14]. Our present data and analysis demonstrate that in terms of problem list additions, these prompts work better for attending physicians than housestaff when ordering single indication medications. There was a strong trend toward improved performance in the ambulatory setting versus inpatient.
4.1.
prescriptions as triggers for this CDS. The purpose of using refills was to improve the yield of problems entered, but when the trigger was a refill, the exact patient condition at the time of the first prescription was not always available to help determine the indication. This study was conducted at a single medical institution which has a significant specialty medicine presence and a relatively low-income urban city patient population. This may not reflect patient and disease profiles at other institutions. For example, during the review it was identified that spironolactone alerts were more frequent for patients with cirrhosis than for CHF. Cirrhosis was not a pre-established alert prompt, which contributed to a higher error rate for this indication. UIHHSS uses a single EMR, and it is possible that how alerts were implemented in Cerner® Powerchart® could have had an impact on the results. Similar alert implementations using other EMR systems may have other interfaces producing different alert performance.
5.
Conclusions
Indication based prescribing of antihypertensive medications triggered by CPOE led to a reasonable yield of accurate problem list additions, 57.5%, with a low rate of inaccurate additions 4.8%. Yield and accuracy were highest for medications with a single indication. Both yield and accuracy were lower for prescribed medications that had multiple indicated problems, but inaccuracy rates were not fully correlated with the number of indication choices. Attending physicians produced more frequent and accurate problem list additions than less experienced clinicians. The accuracy of alerts selected in an out-patient setting slightly exceeded those placed in the in-patient setting. This work demonstrated trends in the accuracy and yield of problem list additions based on clinician and medication factors which should help in design of future problem list prompts as well as with future studies.
Authors’ contributions Suzanne Falck contributed substantially to the conceptualization, design, data analysis, interpretation and writing of the manuscript and gave final approval. Sruthi Adimadhyam assisted with the data analysis and writing of the manuscript and gave final approval. David O. Meltzer contributed to the conceptualization and funding of the project as well as the interpretation and writing of the manuscript and gave final approval. Surrey M. Walton assisted with the study design and data analysis, as well as the interpretation and writing of the manuscript and gave final approval. William L. Galanter led the conceptualization, design, data analysis, interpretation and writing of the manuscript and gave final approval.
Study limitations
One limitation of this study was the difficulty, in some instances, of establishing a gold standard to evaluate the ordering physicians’ alert selections. It is very likely that this was in part due to the use of both refills and initial
Conflicts of interest Dr. Galanter has a research grant from Abbott Laboratories, no other authors have any conflicts to disclose.
Please cite this article in press as: S. Falck, et al., A trial of indication based prescribing of antihypertensive medications during computerized order entry to improve problem list documentation, Int. J. Med. Inform. (2013), http://dx.doi.org/10.1016/j.ijmedinf.2013.07.003
ARTICLE IN PRESS
IJB-3019; No. of Pages 8
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 1 3 ) xxx–xxx
Summary points What was already known • Problem lists have several potentially valuable uses but are often are incomplete or inaccurate • The completeness and accuracy of electronic problem lists can be improved using clinical decision support (CDS) to semi-automate the creation of problem list entries via indication based prescribing. What this study added to our knowledge • Indication based prescribing of antihypertensive medications triggered by CPOE led to accurate problem list additions 58%, with a low rate of inaccurate additions, 5%. • Attending physicians produced more frequent and accurate problem list additions than less experienced clinicians, and the accuracy of alerts selected in an outpatient setting slightly exceeded those placed in the in-patient setting.
Acknowledgements With special thanks to Lisa Canonge, Marla Lax, R.N, Amy Looi, R.N. and Jennifer Welch CT (ASCP) for assistance in the CDS alert development. John Falck for editorial assistance. This study was approved by the University of Illinois Hospital and Health Sciences System Institutional Review Board. The study was in part funded by Grant number 1-U18 HSO16967-01 from AHRQ. The funding source had no role in the production of this manuscript.
references
[1] L.L. Weed, Medical records that guide and teach, N. Engl. J. Med. 278 (1968) 593–600. [2] D.S. Benson, W. Van Osdol, P. Townes, Quality ambulatory care: the role of the diagnostic and medication summary lists, Qual. Rev. Bull. 14 (6) (1988) 192–197. [3] M.E. Johnston, K.B. Langton, R.B. Haynes, A. Mathieu, Effects of computer-based clinical decision support systems on clinician performance and patient outcome: a critical appraisal of research, Ann. Intern. Med. 120 (2) (1994) 135–142. [4] P.C. Tang, M.P. LaRosa, S.M. Gorden, Use of computer-based records, completeness of documentation, and appropriateness of documented clinical decisions, J. Am. Med. Inform. Assoc. 6 (1999) 245–251. [5] U. Sarkar, J.T. Carter, T.A. Omachi, A.R. Vidyarthi, R. Cucina, S. Bokser, et al., SynopSIS: integrating physician sign-out with the electronic medical record, J. Hosp. Med. 2 (5) (2007) 336–342. [6] G. Frank, S.T. Lawless, T.H. Steinberg, Improving physician communication through an automated, integrated sign-out
7
system, J. Healthc. Inf. Manag. 19 (4) (2005) 68–74. [7] J.A. Bernstein, D.L. Imler, P. Sharek, C.A. Longhurst, Improved physician work flow after integrating sign-out notes into the electronic medical record, Jt. Comm. J. Qual. Patient Saf. 36 (February (2)) (2010) 72–78. [8] L.L. Weed, Medical Records, Medical Education, and Patient Care, Yearbook Medical Publishers, Chicago, 1969. [9] Comprehensive Accreditation Manual for Hospitals, The Joint Commission on Accreditation of Healthcare Organizations, Oakbrook Terrace, IL, 1996. [10] D. Blumenthal, M. Tavenner, The “meaningful use” regulation for electronic health records, N. Engl. J. Med. 363 (August (6)) (2010) 501–504 (Epub 2010 July 13). [11] H.C. Szeto, R.K. Coleman, P. Gholami, B.B. Hoffman, M.K. Goldstein, Accuracy of computerized outpatient diagnoses in a Veterans Affairs general medicine clinic, Am. J. Manag. Care 8 (2002) 37–43. [12] S.M. Meystre, P.J. Haug, Randomized controlled trial of an automated problem list with improved sensitivity, Int. J. Med. Inform. 77 (September (9)) (2008) 602–612. [13] W.L. Galanter, D.B. Hier, C. Jao, D. Sarne, Computerized physician order entry of medications and clinical decision support can improve problem list documentation compliance, Int. J. Med. Inform. 79 (May (5)) (2010) 332–338. [14] S.M. Walton, W. Galanter, H. Rosencranz, D. Meltzer, R.S. Stafford, F. Tiryaki, D. Sarne, A trial of inpatient indication based prescribing during computerized order entry with medications commonly used off-label, Appl. Clin. Inform. 21 (2011) 94–103. [15] A. Wright, J. Pang, J.C. Feblowitz, et al., Improving completeness of electronic problem lists through clinical decision support: a randomized, controlled trial, J. Am. Med. Inform. Assoc. 19 (4) (2012) 555–561. [16] V. Aram, A. Chobanian, G. Bakris, H. Black, et al., The seventh report of the Joint National Committee on the prevention, detection, and evaluation and treatment of high blood pressure. The National High Blood Pressure Education Program Coordinating Committee, Hypertension 42 (2003) 1206. [17] ICD - ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification, 2013, http://www.cdc.gov/nchs/about/otheract/icd9/abticd9.htm (accessed 17.12.07). [18] SNOMED® . Systematized Nomenclature of Medicine, 2013, http://www.cap.org/apps/cap.portal? nfpb=true& pageLabel= snomed page (accessed 17.12.07). [19] R.A. Raschke, B. Gollihare, T.A. Wunderlich, J.R. Guidry, A.I. Leibowitz, J.C. Peirce, L. Lemelson, M.A. Heisler, C. Susong, A computer alert system to prevent injury from adverse drug events: development and evaluation in a community teaching hospital, JAMA 280 (October (15)) (1998) 1317–1320. [20] D.M. Wolman, A.L. Kalfoglou, L. LeRoy, Medicare Laboratory Payment Policy, 2013. [21] S. Pakhomov, H. Hemingway, S. Jacobsen, R. Rodeheffer, V. Roger, Natural language processing or ICD- 9 diagnostic codes: which is better at identifying patients with stable angina pectoris? Circulation 113 (May (21)) (2006) e815. [22] I. Solti, B. Aaronson, G. Fletcher, M. Solti, J.H. Gennari, M. Cooper, T. Payne, Building an automated problem list based on natural language processing: lessons learned in the early phase of development, AMIA Annu. Symp. Proc. 6 (November) (2008) 687–691. [23] T. Eguale, N. Winslade, J. Hanley, et al., Enhancing pharmacosurveillance with systematic collection of
Please cite this article in press as: S. Falck, et al., A trial of indication based prescribing of antihypertensive medications during computerized order entry to improve problem list documentation, Int. J. Med. Inform. (2013), http://dx.doi.org/10.1016/j.ijmedinf.2013.07.003
IJB-3019; No. of Pages 8
8
ARTICLE IN PRESS i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 1 3 ) xxx–xxx
treatment indication in electronic prescribing, Drug Saf. 33 (7) (2010) 559–567. [24] W. Galanter, S. Falck, M. Burns, M. Laragh, B. Lambert, Indication-based prescribing prevents wrong-patient medication errors in computerized provider order
entry, J. Am. Med. Inform. Assoc. 20 (May (3)) (2013) 477–481. [25] B. Lambert, HIT and Medication Safety, in: 15th Annual NPSF Patient Safety Congress, Presentation, New Orleans, May, 2013.
Please cite this article in press as: S. Falck, et al., A trial of indication based prescribing of antihypertensive medications during computerized order entry to improve problem list documentation, Int. J. Med. Inform. (2013), http://dx.doi.org/10.1016/j.ijmedinf.2013.07.003