- Email: [email protected]
Evaluation of electronic discharge summaries: A comparison of documentation in electronic and handwritten discharge summaries
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 613–620
journal homepage: www.intl.elsevierhealth.com/journals/ijmi
Evaluation of electronic discharge summaries: A comparison of documentation in electronic and handwritten discharge summaries Joanne L. Callen a,∗ , Melanie Alderton b,1 , Jean McIntosh a,2 a b
Faculty of Health Sciences, The University of Sydney, PO Box 170, Lidcombe, NSW 1825, Australia Balmain Hospital, 29 Booth Street, Balmain, NSW 2041, Australia
a r t i c l e
i n f o
a b s t r a c t
Article history:
Background: Hospital discharge summaries have traditionally been paper-based (handwrit-
Received 11 April 2007
ten or dictated), and deficiencies have often been reported. On the increase is the utilisation
Received in revised form
of electronic summaries, which are considered of higher quality than paper-based sum-
4 September 2007
maries. However, comparisons between electronic and paper-based summaries regarding
Accepted 26 December 2007
documentation deficiencies have rarely been made and there have been none in recent years. Objectives: (1) To study the hospital discharge summaries, which were either handwritten
Keywords:
or electronic, of a population of inpatients, with regard to documentation of information
Electronic discharge summary
required for ongoing care; and (2) to compare the electronic with the handwritten summaries
Handwritten discharge summary
concerning documentation of this information.
Hospital discharge
Methods: The discharge summaries of 245 inpatients were examined for documentation
Discharge communication
of the items: discharge date; additional diagnoses; summary of the patient’s progress in
Australia
hospital; investigations; discharge medications; and follow-up (instructions to the patient’s general practitioner). One hundred and fifty-one (62%) discharge summaries were electronically created and 94 (38%) were handwritten. Odds ratios (ORs) with their confidence intervals (CI) were estimated to show strength of association between the electronic summary and documentation of individual study items. Results: Across all items studied, the electronic summaries contained a higher number of errors and/or omissions than the handwritten ones (OR 1.74, 95% CI 1.26–2.39, p < 0.05). Electronic summaries more commonly documented a summary of the patient’s progress in hospital (OR 18.3, 95% CI 3.33–100, p < 0.05) and less commonly recorded date of discharge and additional diagnoses (respective ORs 0.17 (95% CI 0.09–0.31, p < 0.05) and 0.33 (95% CI 0.15–0.89, p < 0.05). Conclusion: It is not necessarily the case that electronic discharge summaries are of higher quality than handwritten ones, but free text items such as summary of the patient’s progress may less likely be omitted in electronic summaries. It is unknown what factors contributed to incompleteness in creating the electronic discharge summaries investigated in this study. Possible causes for deficiencies include: insufficient training; insufficient education of,
∗
Corresponding author. Tel.: +61 2 93519494; fax: +61 2 93519672. E-mail addresses: [email protected] (J.L. Callen), [email protected] (M. Alderton), [email protected] (J. McIntosh). 1 Tel.: +61 2 93952145; fax: +61 2 93952148. 2 Tel.: +61 2 93519494; fax: +61 2 93519672. 1386-5056/$ – see front matter. Crown Copyright © 2008 Published by Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.ijmedinf.2007.12.002
614
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 613–620
and thus realisation by, doctors regarding the importance of accurate, complete discharge summaries; inadequate computer literacy; inadequate user interaction design, and insufficient integration into routine work processes. Research into these factors is recommended. This study suggests that not enough care is taken by doctors when creating discharge summaries, and that this is independent of the type of method used. The importance of the discharge summary as a chief means of transferring patient information from the hospital to the primary care provider needs to be strongly emphasised. Crown Copyright © 2008 Published by Elsevier Ireland Ltd. All rights reserved.
1.
Introduction
The hospital discharge summary is an essential document for communicating with general practitioners (GPs) as it provides clinical and administrative information necessary for the continuity of care of patients discharged from hospital [1–3]. A prerequisite for the delivery of quality ongoing care is the provision of an adequate summary of the patient’s hospital stay and details of any aftercare required. Data cited as being of prime importance include: admission and discharge diagnosis [4–6]; additional diagnoses [7]; physical examination findings and laboratory results [5,6]; investigations [4]; procedures [4,6,7]; complications [6]; management and outcomes [4,7]; hospital treatments [1]; drug allergies [5]; discharge medications [1,4–6]; medical problems at discharge [6]; instructions for ongoing management [7–9]; follow-up details [1,5–7]; and admission and discharge dates [4]. Traditionally, the hospital discharge summary has been created manually, either handwritten or dictated. Over time there have been numerous reports of errors and omissions in manually-created summaries [3,5,9–14]. In the attempt to improve quality, structured formats were introduced more than two decades ago as an alternative to the narrative format [15]. GPs appear to prefer a structured format [16,17]. In a further endeavour to improve quality, computer-generated (electronic) summaries have been developed [4,10,15,18–21]. Over the years, in creating electronic summaries, data entry has been carried out in different ways; these include the entering of data into the computer during hospitalisation and at discharge by medical staff [7,10,20] or the copying of data into the computer by non-medical staff from summaries written by doctors at discharge [4,12,15]. Recent advances in technology have enabled integration of administrative and clinical hospital information systems, to provide patient data for the electronic discharge summary [21,22]. Comparisons of handwritten/dictated and electronic summaries in relation to completeness have been reported on [10,15], and GPs have been canvassed as to their preference of type of summary [4,5,8,18]. GPs have tended to favour electronic summaries in respect of comprehensiveness and clarity [18] and information on continuity of care [4,8,18]. However, as with handwritten/dictated summaries, electronic summaries may also contain serious errors and omissions [7]. The two comparisons of handwritten/dictated and electronic summaries regarding completeness [10,15] were made several years ago, during earlier stages in the development of electronic discharge summary systems. The doctors creating the summaries were aware their work was being observed. No research comparing discharge summary methods has been published in recent years.
In the present study the discharge summaries of a population of recent inpatients was examined. The summaries had been either handwritten or created electronically. The aim of the study was to test whether, today, the electronic discharge summaries are superior to handwritten summaries regarding information on continuity of care, as GPs felt to be the case years ago. This was done by comparing electronic with handwritten summaries for the completeness of a number of items of information necessary to assist GPs deliver quality ongoing care. This research was carried out retrospectively, and the doctors who created the summaries were unaware their work was to be studied.
2.
Methods
2.1.
Study design and sample
The study sample consisted of the discharge summaries (handwritten or electronic) of patients hospitalised for at least two days between 1 April and 30 June 2005 in a 78-bed public hospital in Sydney, Australia. This hospital was an acute care facility for elderly patients and disabled patients of any age and a rehabilitation facility for patients of any age. All patients had been discharged to their place of residence into the care of a GP. The study was conducted by one investigator (MA) between August and November 2005 using a data collection tool designed by the first and second authors (JC and MA). Implementation of an electronic discharge summary system had been commenced in the study hospital in October 2004. The generic template of the electronic discharge summary had been recently developed at a large teaching hospital in Sydney [23] by a multi-disciplinary committee consisting of doctors, nurses, and administrative and health information management staff. The design of the template was based on the standard format produced by the New South Wales (Australia) Department of Health, and was developed by the teaching hospital’s Information Systems Department. The electronic discharge summary was trialled at a smaller Sydney teaching hospital and then rolled out to other hospitals including the study hospital. Several modifications had been made to the discharge summary based on feedback from clinicians at the hospital where it was trialled. The house doctors (interns, residents, registrars) who worked at the study hospital were on secondment from the teaching hospital where the discharge summary had been developed. It was intended that all doctors at this teaching hospital receive, at Orientation (to their work at the hospital), half an hour of training in the use of the electronic discharge summary system. This was a component of their training in clinical information systems
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 613–620
Fig. 1 – Screen shots of electronic discharge summary.
615
616
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 613–620
conducted by the hospital’s Information Systems Department. However training in this electronic system depended on availability of trainers and a shortage would cause a delay in a doctor’s training. Doctors were sometimes seconded to the study hospital before they received training. However, at Orientation, all doctors were given a lecture by a GP on the importance and relevance of the discharge summary per se, as an instrument of transferring information from hospital to the GP about a patient’s ongoing care. A repeat lecture was given six months later. At the study hospital, doctors who had received training in creating electronic discharge summaries were required to use this method; it was only permissible for them to create a handwritten one in the event of computer unavailability or system/technical difficulties such as the server being down. Doctors who had not been trained were not permitted to create electronic discharge summaries. The electronic summary could be started early during hospital stay and continued prospectively, but using or not using this particular function of the electronic summary system was the choice of the individual doctor. The electronic summary was accessed by the treating doctor with a secure log-in, via a computer terminal. The patient’s demographics and GP details automatically populated the electronic summary from the administrative hospital information system and results of investigations were copy/pasted into the discharge summary from the hospital clinical information system. The electronic summary was structured such that there was a separate field for each item of information with just one heading pertaining to the item, and space under the heading to enter free text where relevant (Fig. 1). Discharge medications were initially handwritten onto a pharmacy discharge script from which they were copied into the ‘medications on discharge’ field. The handwritten summary was similarly structured and consisted of a single-page form, carbon-copied in triplicate, with a detachable pharmacy discharge script. It was intended that handwritten summaries be completed by the time of discharge or at least as soon afterwards as possible. On discharge, patients were given a five-day supply of discharge medications. Therefore it was necessary to complete the pharmacy
discharge script of both the electronic and handwritten summaries before actual discharge, in time for the patient to be given, before leaving the ward, the discharge medications supplied by the hospital. The electronic summary was printed out and posted to the GP as was the handwritten summary. The data items reviewed in the discharge summaries and the criteria for scoring them are presented in Table 1. Presence of documentation of items was scored as ‘yes’, ‘no’, or ‘not applicable’ (N/A). The importance of data being entered into its allotted field is emphasised. In order that the GP can organise ongoing care for the patient precisely and efficiently it is imperative that the information in discharge summaries be presented clearly and in an orderly fashion. The GP may miss important information if it is not in its allotted field and is inappropriately placed, inserted amongst other data about the patient.
2.2.
Statistical analysis
The data were entered into Statistical Package for Social Sciences (SPSS) version 11.0 [24]. The 2 × 2 contingency table technique for calculating odds ratios (OR) and their 95% confidence intervals (CI) was used to estimate strength of associations between the electronic summary and documentation of the study items. Woolf’s procedure with the Haldane-Anscombe correction, as described and recommended by Lawson [25], was used in this analysis. Standard 2 analysis was used to test for differences between electronic versus handwritten summaries regarding frequency distributions of documentation deficiencies. The accepted alpha level for all significance testing was 0.05. Instances where there was no documentation of an item because it was not applicable for that patient were excluded in all statistical analyses.
3.
Results
During the study period, 272 patients were discharged whose discharge summaries were eligible for audit. Of the 272 patients, 27 (10%) were inpatients at time of audit or had been
Table 1 – Data items that were reviewed in the electronic and handwritten discharge summaries and the criteria for scoring them Data item Discharge date Additional (other) diagnoses
Summary of patient progress (including treatment) Investigations and results
Follow-up requirements
Discharge medications
Criteria for scoring This was counted as not documented if not in the allotted field, even if noted elsewhere. This was checked against the patient’s record for accuracy If present but none was listed in the allotted field, they were counted as not documented even if all were noted elsewhere. However, if at least some were listed in the allotted field and the remainder were listed elsewhere, they were counted as documented. No check was made against the patient’s record for completeness or accuracy This was counted as documented if there was an entry in the field. No check was made against the patient’s record for completeness or accuracy If this field was empty, the patient’s record was examined to check whether there had indeed been no investigations. If none, an N/A score was given. If investigations were documented in the discharge summary no check was made against the patient’s record for or completeness or accuracy If this field was empty the patient’s record was examined to check whether there were indeed no requirements of the GP concerning ongoing care. If none, an N/A score was given. If follow-up requirements were documented in the discharge summary, no check was made for completeness or accuracy All medications were checked for completeness and accuracy. If any discrepancy was found between medications in the discharge summary, the patient’s record and the pharmacy script, a senior clinician was asked to ascertain what was correct for that patient
617
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 613–620
Table 2 – Comparison of electronic and handwritten discharge summaries regarding documentation of information Data items Electronic
Handwritten (%)
n
(%)
87 64
(57.6) (42.4)
17 77
(18.1) (81.9)
1.0 0.17
Reference 0.09–0.31
27 124
(17.9) (82.1)
6 88
(6.4) (93.6)
1.0 0.33
Reference 0.15–0.89
1 150
(0.7) (99.3)
14 79 1
(14.9) (84.0) (1.1)
Investigations and results No Yes N/A
3 146 2
(2.0) (96.7) (1.3)
6 85 3
(6.4) (90.4) (3.2)
1.0 3.18
Reference 0.84–12.0
Follow-up No Yes N/A
17 131 3
(11.3) (86.8) (2.4)
10 82 2
(10.6) (87.2) (2.4)
1.0 0.96
Reference 0.42–2.16
19 132
(12.6) (87.4)
6 88
(6.4) (93.6)
1.0 0.50
Reference 0.20–1.26
Additional (other) diagnoses No Yes Summary of progress No Yes N/A
Information present and correct Discharge medications No Yes
b
95% CIb
n Information present Discharge date No Yes
a
ORa
Type of discharge summary
1.0 18.3
Reference 3.33–100
OR: odds ratio. CI: confidence interval.
transferred to another hospital with their records during the interval between the index discharge and the audit. Thus, 245 summaries were available for study. Of these, 177 (72.2%) had been created for medical patients and 68 (27.8%) for rehabilitation patients; 151 of the 245 summaries (62%) were electronic and 94 (38%) were handwritten. Table 2 shows the number of electronic and handwritten discharge summaries deficient and not deficient in the documentation of each data item of interest. The ORs show the strength of the associations between an electronic summary being created (as against a handwritten one) and documentation of the items. It is seen by the ORs that there was, respectively, a six-fold and three-fold lesser odds of the discharge date and the additional diagnoses being recorded in the electronic than the handwritten summaries, and that there was an 18-fold greater odds of there being documentation of a summary of patient progress in the electronic summaries. The CIs of the three ORs did not include 1.0, which shows these associations were statistically significant. In respect of the other three items, it is seen that there was a three-fold greater odds of there being a record of investigations/results in the electronic summaries, but only half the odds of there being correct documentation of discharge medications; these differences were not statistically significant. It can be seen in Table 2 that two items were rarely omitted in electronic summaries; these were summary of patient progress and investigations/results (these items were omit-
ted in only 2% or less of electronic summaries). The most frequently omitted item, by far, in electronic summaries was the discharge date (not documented in 57.6%). In handwritten summaries, omitted/incorrect documentation of a discharge medication was the least common deficiency (found in 6.4%) while the most common deficiency was omission of a summary of patient progress (found in 15%). Table 2 shows that across the six items, there were 901 documentations in the 151 electronic summaries and 558 in the 94 handwritten summaries. Documentation deficiencies in the electronic and handwritten summaries amounted to, respectively, 154 of the 901 (17.1%) and 59 of the 558 (10.75%) (OR 1.74, 95% CI 1.26–2.39), showing that deficiencies were significantly more common in the electronic summaries. Table 3 shows the frequency distribution of deficiencies in the electronic and handwritten summaries. No summary had more than three deficiencies. Significantly fewer electronic than handwritten summaries were free of deficiencies (2 = 20.9, 1 d.f., p < 0.0001), and it is seen that proportionately more electronic than handwritten summaries had one, two, or three deficiencies. The difference in the distribution of deficiencies in the two types of summary was highly significant (2 = 21.00, 2 d.f., p < 0.0001). Because so few summaries had three deficiencies (expected values were <5), the two–deficiency and three-deficiency categories were combined for 2 analysis (hence 2 d.f.). Omission of the discharge date was the deficiency in 60 of the 87 (69%) electronic summaries that had just one deficiency.
618
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 613–620
Table 3 – Comparison of electronic and handwritten discharge summaries by frequency of documentation deficiencies Number of deficiencies
Type of discharge summary Electronic (n = 151) n (%)
None One Two Three
Handwritten (n = 94) n (%)
33 (21.9) 87 (57.6) 26 (17.2) 5 (3.3)
Nineteen doctors had been involved in creating the discharge summaries. Table 4 shows the number of electronic and handwritten summaries created by each doctor and the number of deficiencies in the total number of summaries created by each doctor. It is seen that 10 of the 19 doctors had created the electronic summaries. Two of these doctors (A and B) had contributed between them 72.9% of all electronic summaries, and these contained 76% of all electronic summary deficiencies. These two doctors were responsible for 70 of the 87 omissions of the discharge date, 22 of the 27 omissions of additional diagnoses, 12 of the 19 medication errors and 10 of the 17 omissions of follow-up arrangements. Of the 62 electronic summaries created by Doctor A, 61 lacked the discharge date. It is seen in Table 4 that the 94 handwritten summaries had been created by 16 doctors of whom six (A, B, C, H, I and Q) contributed 71 (75.5%). These 71 summaries contained 76.7% of all handwritten summary deficiencies. Of Doctor D’s seven summaries, six lacked a summary of patient progress (but all six of Doctor D’s electronic summaries contained a progress summary) and of Doctor I’s nine summaries, three lacked a progress summary. The remaining five discharge summaries without a summary of patient progress had been written by five different doctors. It can be calcu-
47 (50.0) 36 (38.3) 10 (10.6) 1 (1.1)
lated from Table 4 that the two doctors who had created most electronic summaries and contributed the bulk of the deficiencies (A and B) averaged only fractionally more errors per summary than the five who had created the least number of electronic summaries (D, G, M, P and R) (1.06 versus 0.93). Likewise, the two who had most frequently created handwritten summaries averaged a similar number of errors per summary as the 10 who had most infrequently created them (both 0.56). Among doctors who created electronic summaries the one who contributed most of the deficiencies (A) averaged fewer deficiencies per summary (1.22) than two others (D (1.5) and E (1.75)).
4.
Discussion
The findings of this study show that an appreciable number of discharge summaries contained omissions and errors, but that these were more commonly present in the electronic than in the handwritten summaries. The value of reviewing discharge summaries has been demonstrated in this study, this research having shown that critical errors and omissions continue in the communication of clinical
Table 4 – Number of electronic and handwritten discharge summaries created by each of the 19 doctors and number of deficiencies in the total number of summaries per doctor Doctor identification A B C D E F G H I J K L M N O P Q R S Total
Electronic summaries created n (% of total)
Deficiencies found in electronic summaries n (% of total)
Handwritten summaries created n (% of total)
Deficiencies found in handwritten summaries n (% of total)
62 (41%) 48 (31.8%) 8 (5.3%) 6 (4.0%) 8 (5.3%) 10 (6.6%) 2 (1.3%) 0 0 0 0 0 1 (0.7%) 0 0 5 (3.3%) 0 1 (0.7%) 0
76 (49.4%) 41 (26.6%) 6 (3.9%) 9 (5.8%) 14 (9.1%) 3 (2.0%) 2 (1.3%) – – – – – 2 (1.3%) – – 0 – 1 (0.7%) –
17 (18.1%) 0 20 (21.3%) 7 (7.5%) 1 (1.1%) 0 4 (4.3%) 9 (9.8%) 9 (9.8%) 2 (2.1%) 2 (2.1%) 1 (1.1%) 2 (2.1%) 5 (5.3%) 4 (4.3%) 0 9 (9.8%) 1 (1.1%) 1 (1.1%)
7 (11.9%) – 14 (23.7%) 9 (15.3%) 1 (1.7%) – 2 (3.4%) 4 (6.8%) 8 (13.6%) 2 (3.4%) 0 0 1(1.7%) 6 (10.2%) 1 (1.7%) – 4 (6.8%) 0 0
151 (100%)
154 (100%)
94 (100%)
59 (100%)
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 613–620
information from the hospital to the GP, despite advances in computerisation of the discharge summary. The observation that a substantial amount of deficient documentation was linked to a small number of the doctors creating the summaries suggests that reasons for the deficiency variations between the electronic and handwritten discharge summaries pertained directly to the particular doctors involved. In the electronic summary, the fields for the discharge date and additional diagnoses were at the beginning of the document and were perhaps many screens up front from where final discharge data were entered. Repeated failure to refer back to the beginning of the summary for data entry could explain the extent of omitting the discharge date and additional diagnoses in electronic summaries. The higher rate of error (albeit non-significant) in recording discharge medications in electronic summaries could be due to repeat documentation of medications in the electronic summary creation (transcription from handwritten script to computer) as against once only documentation in the handwritten summary. Transcribing has been shown to cause incorrect medication documentation [26]. Our finding that among the 94 handwritten records, only six, that is, 6.3%, contained a medication error compares favourably with 17.5% in another Australian survey [3] and 49.5% in a New Zealand study [27]. Copying the discharge medications from the handwritten pharmacy script into the electronic summary involved extra work. However, this constituted the only difference between the two discharge summary methods in the amount of work expended. The importance of recording allergy data in discharge summaries is acknowledged. However, the study hospital’s handwritten summary form had no field for allergy details, and therefore, unlike in the electronic summary, there was no reminder to record allergy information. Because of this difference in format, it was considered inappropriate to compare the electronic and handwritten summaries on this item.
4.1.
Study limitations and strengths
A limitation of the present survey is that the bulk of the deficiencies found were due to a small number of the doctors who created the summaries; thus the results are not generalisable. A strength of this study is that it was a survey of a real-world situation. The doctors involved were unaware their work was to be observed, our study having been conducted well after the summaries had been created. Observing a real-world situation with doctors unaware was not the case with other studies in which electronically-created and handwritten summaries were compared [10,15,18,20]. Of three previous studies which compared the quality of electronic and paper-based summaries, two reported the electronic was superior in quality [10,18] and one reported a higher level of completeness in the electronic summary [15]. There is consensus there are important advantages of electronic over paper-based summaries that do not involve actual content. These include: greater likelihood of creation [4,10,15], quicker generation [4,8,10,15,18,20,28], and more timely receipt by the GP [4,15,18,20].
619
Summary points What was known before the study: • Many studies have claimed that paper-based (such as handwritten) discharge summaries contain errors and omissions, and that those with a narrative format are more likely to be deficient than those which have structure. • Electronically-created summaries are considered to have a potential for high quality, that is, for accuracy and completeness. • Comparisons between paper-based and electronic summaries in randomised trial and experimental situations have shown electronic summaries to be more complete. There are no published studies of comparisons made in real world situations, with the doctors who created the summaries unaware their work was to be studied. What the present study has added to the body of knowledge: • This first study of a comparison between structured (as against narrative) paper-based (handwritten) and structured electronic discharge summaries, all created in a real world situation, shows it is not necessarily the case that electronic summaries are of higher quality than handwritten ones. In the early stages of electronic summary implementation, electronic summaries may contain many more deficiencies than handwritten summaries. • Free text items (such as a summary of the patient’s progress in hospital) may more likely be documented in electronic than paper-based discharge summaries.
5.
Conclusion
The results of our study show it is not necessarily the case that electronic discharge summaries are of higher quality as regards accuracy and completeness than handwritten ones, and electronic summaries may be more deficient in the early stages of electronic implementation. However, free text items such as summary of the patient’s progress may more likely be documented in electronic summaries. It is unknown what factors contributed to incompleteness in creating the electronic discharge summaries investigated in this study. Possible causes for deficiencies include: insufficient training; insufficient education of, and thus realisation by, doctors regarding the importance of accurate, complete discharge summaries; inadequate computer literacy; unfamiliarity with creating discharge summaries electronically; inadequate user interaction design; insufficient integration into routine work processes. These factors require further study. Our survey reiterates that insufficient care is being taken when creating discharge summaries. We have shown that this is independent of the method used (manual or elec-
620
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 613–620
tronic). It is apparent it must be strongly emphasised among doctors creating discharge summaries that this document is crucial in transferring sufficient patient information from hospital to the GP to enable the provision of quality ongoing care.
[14]
[15]
Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at doi:10.1016/j.ijmedinf.2007.12.002.
[16]
references
[17]
[1] P. Bolton, M. Mira, P. Kennedy, M.M. Lahra, The quality of communication between hospitals and general practitioners: An assessment, J. Qual. Clin. Pract. 18 (4) (1998) 241–247. [2] P. Bolton, A quality assurance activity to improve discharge communication with general practice, J. Qual. Clin. Pract. 21 (3) (2001) 69–70. [3] S. Wilson, W. Ruscoe, M. Chapman, R. Miller, General practitioner–hospital communications: a review of discharge summaries, J. Qual. Clin. Pract. 21 (4) (2001) 104–108. [4] W.M. Castleden, M.C. Stacey, P.E. Norman, M.M. Lawrence-Brown, J.G. Brooks, General practitioners’ attitudes to computer-created surgical discharge letters, Med. J. Aust. 157 (6) (1992) 380–382. [5] C. van Walraven, A.L. Weinberg, Quality assessment of a discharge summary system, Can. Med. Assoc. J. 152 (9) (1995) 1437–1442. [6] C. van Walraven, E. Rokosh, What is necessary for high-quality discharge summaries? Am. J. Med. Qual. 14 (4) (1999) 160–169. [7] J.O. Jansen, I.C. Grant, Communication with general practitioners after accident and emergency attendance: computer created letters are often deficient, Emerg. Med. J. 20 (3) (2003) 256–257 (http://emj.bmj.com/cgi/reprint/20/3/256, accessed March 12, 2007). [8] J.E. Brazy, D.L. Langkamp, N.D. Brazy, R.F. De Luna, Do primary care physicians prefer dictated or computer-generated discharge summaries? Am. J. Dis. Child. 147 (9) (1993) 986–988. [9] A.N. Raval, G.E. Marchiori, J.M. Arnold, Improving the continuity of care following discharge of patients hospitalized with heart failure: is the discharge summary adequate? Can. J. Cardiol. 19 (4) (2003) 365–370. [10] T. Lissauer, C.M. Paterson, A. Simons, R.W. Beard, Evaluation of computer generated neonatal discharge summaries, Arch. Dis. Child. 66 (4) (1991) 433–436. [11] J. Pears, V. Alexander, G.F. Alexander, N.R. Waugh, Audit of the quality of hospital discharge data, Health Bull. (Edinb.) 50 (5) (1992) 356–361. [12] E.M. Macaulay, G.G. Cooper, J. Engeset, A.R. Naylor, Prospective audit of discharge summary errors, Br. J. Surg. 83 (6) (1996) 788–790. [13] J.M. Paterson, R.L. Allega, Improving communication between hospital and community physicians. Feasibility study of a handwritten, faxed hospital discharge summary.
[18]
[19]
[20]
[21]
[22]
[23]
[24]
[25]
[26]
[27]
[28]
Discharge Summary Study Group, Can. Fam. Physician 45 (1999) 2893–2899. D.S. Foster, C. Paterson, G. Fairfield, Evaluation of immediate discharge documents – room for improvement? Scott. Med. J. 47 (4) (2002) 77–79. C. van Walraven, A. Laupacis, R. Seth, G. Wells, Dictated versus database-generated discharge summaries: a randomized clinical trial, Can. Med. Assoc. J. 160 (3) (1999) 319–326. C. van Walraven, S.M. Duke, A.L. Weinberg, P. Wells, Standardized or narrative discharge summaries. Which do family physicians prefer? Can. Fam. Physician 44 (1998) 62–69. S.E. Hook, G.C. Banister, C. Topliss, J. Webb, Letters and notes in orthopaedic surgery, Ann. R. Coll. Surg. Engl. 88 (3) (2006) 292–296. R.A. Archbold, K. Laji, A. Suliman, K. Ranjadayalan, H. Hemingway, A.D. Timmis, Evaluation of a computer-generated discharge summary for patients with acute coronary syndromes, Br. J. Gen. Pract. 48 (429) (1998) 1163–1164. R. Crosswhite, S.H. Beckham, P. Gray, P.R. Hawkins, J. Hughes, Using a multidisciplinary automated discharge summary process to improve information management across the system, Am. J. Manage. Care 3 (3) (1997) 473–479. J. Kirby, B. Barker, D.J. Fernando, M. Jose, C. Curtis, A. Goodchild, C. Dickens, E. Olla, R. Cooke, I. Idris, G.A. Thomson, A prospective case control study of the benefits of electronic discharge summaries, J. Telemed. Telecare 12 (1) (2006) 20–21. T. Schabetsberger, E. Ammenwerth, S. Andreatta, G. Gratl, R. Haux, G. Lechleitner, K. Schindelwig, C. Stark, R. Volg, I. Wilhelmy, F. Wozak, From a paper-based transmission of discharge summaries to electronic communication in health care regions, Int. J. Med. Inform. 75 (3) (2006) 209–215. Global Health, Announcement to the Australian Stock Exchange, Sydney’s North Shore GPs get electronic hospital discharge referrals for their patients, April 5, 2004 (http://www.global-health.com/NorthernSydneyASX.pdf, accessed March 12, 2007). The Royal Australian College of General Practitioners, Examples of good practice in hospital-GP communication, Submission No. 66 to House of Representatives Standing Committee on Health and Ageing: Parliament House, Canberra, ACT, August 3, 2005 (http://www.aph.gov.au/ house/committee/haa/healthfunding/subs/sub066.pdf, accessed March 12, 2007). S.J. Coakes, L.G. Steed, SPSS: Analysis without Anguish: Version 11. 00 for Windows, John Wiley & Sons Inc., Australia, 2003. R. Lawson, Small sample confidence intervals for the odds ratio, Commun. Statist.: Simul. Comput. 33 (4) (2004) 1095–1113. M. Frank-Stromborg, A. Christensen, C. Elmhurst, Nurse documentation: not done or worse, done the wrong way – part 2, Oncol. Nurs. Forum 28 (5) (2001) 841–846. T.E. McMillan, W. Allan, P.N. Black, Accuracy of information on medicines in hospital discharge summaries, Intern. Med. J. 36 (4) (2006) 221–225. R.P. Smith, G.B. Holzman, The application of a computer data base system to the generation of hospital discharge summaries, Obstet. Gynecol. 73 (5 Pt1) (1989) 803–807.
journal homepage: www.intl.elsevierhealth.com/journals/ijmi
Evaluation of electronic discharge summaries: A comparison of documentation in electronic and handwritten discharge summaries Joanne L. Callen a,∗ , Melanie Alderton b,1 , Jean McIntosh a,2 a b
Faculty of Health Sciences, The University of Sydney, PO Box 170, Lidcombe, NSW 1825, Australia Balmain Hospital, 29 Booth Street, Balmain, NSW 2041, Australia
a r t i c l e
i n f o
a b s t r a c t
Article history:
Background: Hospital discharge summaries have traditionally been paper-based (handwrit-
Received 11 April 2007
ten or dictated), and deficiencies have often been reported. On the increase is the utilisation
Received in revised form
of electronic summaries, which are considered of higher quality than paper-based sum-
4 September 2007
maries. However, comparisons between electronic and paper-based summaries regarding
Accepted 26 December 2007
documentation deficiencies have rarely been made and there have been none in recent years. Objectives: (1) To study the hospital discharge summaries, which were either handwritten
Keywords:
or electronic, of a population of inpatients, with regard to documentation of information
Electronic discharge summary
required for ongoing care; and (2) to compare the electronic with the handwritten summaries
Handwritten discharge summary
concerning documentation of this information.
Hospital discharge
Methods: The discharge summaries of 245 inpatients were examined for documentation
Discharge communication
of the items: discharge date; additional diagnoses; summary of the patient’s progress in
Australia
hospital; investigations; discharge medications; and follow-up (instructions to the patient’s general practitioner). One hundred and fifty-one (62%) discharge summaries were electronically created and 94 (38%) were handwritten. Odds ratios (ORs) with their confidence intervals (CI) were estimated to show strength of association between the electronic summary and documentation of individual study items. Results: Across all items studied, the electronic summaries contained a higher number of errors and/or omissions than the handwritten ones (OR 1.74, 95% CI 1.26–2.39, p < 0.05). Electronic summaries more commonly documented a summary of the patient’s progress in hospital (OR 18.3, 95% CI 3.33–100, p < 0.05) and less commonly recorded date of discharge and additional diagnoses (respective ORs 0.17 (95% CI 0.09–0.31, p < 0.05) and 0.33 (95% CI 0.15–0.89, p < 0.05). Conclusion: It is not necessarily the case that electronic discharge summaries are of higher quality than handwritten ones, but free text items such as summary of the patient’s progress may less likely be omitted in electronic summaries. It is unknown what factors contributed to incompleteness in creating the electronic discharge summaries investigated in this study. Possible causes for deficiencies include: insufficient training; insufficient education of,
∗
Corresponding author. Tel.: +61 2 93519494; fax: +61 2 93519672. E-mail addresses: [email protected] (J.L. Callen), [email protected] (M. Alderton), [email protected] (J. McIntosh). 1 Tel.: +61 2 93952145; fax: +61 2 93952148. 2 Tel.: +61 2 93519494; fax: +61 2 93519672. 1386-5056/$ – see front matter. Crown Copyright © 2008 Published by Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.ijmedinf.2007.12.002
614
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 613–620
and thus realisation by, doctors regarding the importance of accurate, complete discharge summaries; inadequate computer literacy; inadequate user interaction design, and insufficient integration into routine work processes. Research into these factors is recommended. This study suggests that not enough care is taken by doctors when creating discharge summaries, and that this is independent of the type of method used. The importance of the discharge summary as a chief means of transferring patient information from the hospital to the primary care provider needs to be strongly emphasised. Crown Copyright © 2008 Published by Elsevier Ireland Ltd. All rights reserved.
1.
Introduction
The hospital discharge summary is an essential document for communicating with general practitioners (GPs) as it provides clinical and administrative information necessary for the continuity of care of patients discharged from hospital [1–3]. A prerequisite for the delivery of quality ongoing care is the provision of an adequate summary of the patient’s hospital stay and details of any aftercare required. Data cited as being of prime importance include: admission and discharge diagnosis [4–6]; additional diagnoses [7]; physical examination findings and laboratory results [5,6]; investigations [4]; procedures [4,6,7]; complications [6]; management and outcomes [4,7]; hospital treatments [1]; drug allergies [5]; discharge medications [1,4–6]; medical problems at discharge [6]; instructions for ongoing management [7–9]; follow-up details [1,5–7]; and admission and discharge dates [4]. Traditionally, the hospital discharge summary has been created manually, either handwritten or dictated. Over time there have been numerous reports of errors and omissions in manually-created summaries [3,5,9–14]. In the attempt to improve quality, structured formats were introduced more than two decades ago as an alternative to the narrative format [15]. GPs appear to prefer a structured format [16,17]. In a further endeavour to improve quality, computer-generated (electronic) summaries have been developed [4,10,15,18–21]. Over the years, in creating electronic summaries, data entry has been carried out in different ways; these include the entering of data into the computer during hospitalisation and at discharge by medical staff [7,10,20] or the copying of data into the computer by non-medical staff from summaries written by doctors at discharge [4,12,15]. Recent advances in technology have enabled integration of administrative and clinical hospital information systems, to provide patient data for the electronic discharge summary [21,22]. Comparisons of handwritten/dictated and electronic summaries in relation to completeness have been reported on [10,15], and GPs have been canvassed as to their preference of type of summary [4,5,8,18]. GPs have tended to favour electronic summaries in respect of comprehensiveness and clarity [18] and information on continuity of care [4,8,18]. However, as with handwritten/dictated summaries, electronic summaries may also contain serious errors and omissions [7]. The two comparisons of handwritten/dictated and electronic summaries regarding completeness [10,15] were made several years ago, during earlier stages in the development of electronic discharge summary systems. The doctors creating the summaries were aware their work was being observed. No research comparing discharge summary methods has been published in recent years.
In the present study the discharge summaries of a population of recent inpatients was examined. The summaries had been either handwritten or created electronically. The aim of the study was to test whether, today, the electronic discharge summaries are superior to handwritten summaries regarding information on continuity of care, as GPs felt to be the case years ago. This was done by comparing electronic with handwritten summaries for the completeness of a number of items of information necessary to assist GPs deliver quality ongoing care. This research was carried out retrospectively, and the doctors who created the summaries were unaware their work was to be studied.
2.
Methods
2.1.
Study design and sample
The study sample consisted of the discharge summaries (handwritten or electronic) of patients hospitalised for at least two days between 1 April and 30 June 2005 in a 78-bed public hospital in Sydney, Australia. This hospital was an acute care facility for elderly patients and disabled patients of any age and a rehabilitation facility for patients of any age. All patients had been discharged to their place of residence into the care of a GP. The study was conducted by one investigator (MA) between August and November 2005 using a data collection tool designed by the first and second authors (JC and MA). Implementation of an electronic discharge summary system had been commenced in the study hospital in October 2004. The generic template of the electronic discharge summary had been recently developed at a large teaching hospital in Sydney [23] by a multi-disciplinary committee consisting of doctors, nurses, and administrative and health information management staff. The design of the template was based on the standard format produced by the New South Wales (Australia) Department of Health, and was developed by the teaching hospital’s Information Systems Department. The electronic discharge summary was trialled at a smaller Sydney teaching hospital and then rolled out to other hospitals including the study hospital. Several modifications had been made to the discharge summary based on feedback from clinicians at the hospital where it was trialled. The house doctors (interns, residents, registrars) who worked at the study hospital were on secondment from the teaching hospital where the discharge summary had been developed. It was intended that all doctors at this teaching hospital receive, at Orientation (to their work at the hospital), half an hour of training in the use of the electronic discharge summary system. This was a component of their training in clinical information systems
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 613–620
Fig. 1 – Screen shots of electronic discharge summary.
615
616
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 613–620
conducted by the hospital’s Information Systems Department. However training in this electronic system depended on availability of trainers and a shortage would cause a delay in a doctor’s training. Doctors were sometimes seconded to the study hospital before they received training. However, at Orientation, all doctors were given a lecture by a GP on the importance and relevance of the discharge summary per se, as an instrument of transferring information from hospital to the GP about a patient’s ongoing care. A repeat lecture was given six months later. At the study hospital, doctors who had received training in creating electronic discharge summaries were required to use this method; it was only permissible for them to create a handwritten one in the event of computer unavailability or system/technical difficulties such as the server being down. Doctors who had not been trained were not permitted to create electronic discharge summaries. The electronic summary could be started early during hospital stay and continued prospectively, but using or not using this particular function of the electronic summary system was the choice of the individual doctor. The electronic summary was accessed by the treating doctor with a secure log-in, via a computer terminal. The patient’s demographics and GP details automatically populated the electronic summary from the administrative hospital information system and results of investigations were copy/pasted into the discharge summary from the hospital clinical information system. The electronic summary was structured such that there was a separate field for each item of information with just one heading pertaining to the item, and space under the heading to enter free text where relevant (Fig. 1). Discharge medications were initially handwritten onto a pharmacy discharge script from which they were copied into the ‘medications on discharge’ field. The handwritten summary was similarly structured and consisted of a single-page form, carbon-copied in triplicate, with a detachable pharmacy discharge script. It was intended that handwritten summaries be completed by the time of discharge or at least as soon afterwards as possible. On discharge, patients were given a five-day supply of discharge medications. Therefore it was necessary to complete the pharmacy
discharge script of both the electronic and handwritten summaries before actual discharge, in time for the patient to be given, before leaving the ward, the discharge medications supplied by the hospital. The electronic summary was printed out and posted to the GP as was the handwritten summary. The data items reviewed in the discharge summaries and the criteria for scoring them are presented in Table 1. Presence of documentation of items was scored as ‘yes’, ‘no’, or ‘not applicable’ (N/A). The importance of data being entered into its allotted field is emphasised. In order that the GP can organise ongoing care for the patient precisely and efficiently it is imperative that the information in discharge summaries be presented clearly and in an orderly fashion. The GP may miss important information if it is not in its allotted field and is inappropriately placed, inserted amongst other data about the patient.
2.2.
Statistical analysis
The data were entered into Statistical Package for Social Sciences (SPSS) version 11.0 [24]. The 2 × 2 contingency table technique for calculating odds ratios (OR) and their 95% confidence intervals (CI) was used to estimate strength of associations between the electronic summary and documentation of the study items. Woolf’s procedure with the Haldane-Anscombe correction, as described and recommended by Lawson [25], was used in this analysis. Standard 2 analysis was used to test for differences between electronic versus handwritten summaries regarding frequency distributions of documentation deficiencies. The accepted alpha level for all significance testing was 0.05. Instances where there was no documentation of an item because it was not applicable for that patient were excluded in all statistical analyses.
3.
Results
During the study period, 272 patients were discharged whose discharge summaries were eligible for audit. Of the 272 patients, 27 (10%) were inpatients at time of audit or had been
Table 1 – Data items that were reviewed in the electronic and handwritten discharge summaries and the criteria for scoring them Data item Discharge date Additional (other) diagnoses
Summary of patient progress (including treatment) Investigations and results
Follow-up requirements
Discharge medications
Criteria for scoring This was counted as not documented if not in the allotted field, even if noted elsewhere. This was checked against the patient’s record for accuracy If present but none was listed in the allotted field, they were counted as not documented even if all were noted elsewhere. However, if at least some were listed in the allotted field and the remainder were listed elsewhere, they were counted as documented. No check was made against the patient’s record for completeness or accuracy This was counted as documented if there was an entry in the field. No check was made against the patient’s record for completeness or accuracy If this field was empty, the patient’s record was examined to check whether there had indeed been no investigations. If none, an N/A score was given. If investigations were documented in the discharge summary no check was made against the patient’s record for or completeness or accuracy If this field was empty the patient’s record was examined to check whether there were indeed no requirements of the GP concerning ongoing care. If none, an N/A score was given. If follow-up requirements were documented in the discharge summary, no check was made for completeness or accuracy All medications were checked for completeness and accuracy. If any discrepancy was found between medications in the discharge summary, the patient’s record and the pharmacy script, a senior clinician was asked to ascertain what was correct for that patient
617
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 613–620
Table 2 – Comparison of electronic and handwritten discharge summaries regarding documentation of information Data items Electronic
Handwritten (%)
n
(%)
87 64
(57.6) (42.4)
17 77
(18.1) (81.9)
1.0 0.17
Reference 0.09–0.31
27 124
(17.9) (82.1)
6 88
(6.4) (93.6)
1.0 0.33
Reference 0.15–0.89
1 150
(0.7) (99.3)
14 79 1
(14.9) (84.0) (1.1)
Investigations and results No Yes N/A
3 146 2
(2.0) (96.7) (1.3)
6 85 3
(6.4) (90.4) (3.2)
1.0 3.18
Reference 0.84–12.0
Follow-up No Yes N/A
17 131 3
(11.3) (86.8) (2.4)
10 82 2
(10.6) (87.2) (2.4)
1.0 0.96
Reference 0.42–2.16
19 132
(12.6) (87.4)
6 88
(6.4) (93.6)
1.0 0.50
Reference 0.20–1.26
Additional (other) diagnoses No Yes Summary of progress No Yes N/A
Information present and correct Discharge medications No Yes
b
95% CIb
n Information present Discharge date No Yes
a
ORa
Type of discharge summary
1.0 18.3
Reference 3.33–100
OR: odds ratio. CI: confidence interval.
transferred to another hospital with their records during the interval between the index discharge and the audit. Thus, 245 summaries were available for study. Of these, 177 (72.2%) had been created for medical patients and 68 (27.8%) for rehabilitation patients; 151 of the 245 summaries (62%) were electronic and 94 (38%) were handwritten. Table 2 shows the number of electronic and handwritten discharge summaries deficient and not deficient in the documentation of each data item of interest. The ORs show the strength of the associations between an electronic summary being created (as against a handwritten one) and documentation of the items. It is seen by the ORs that there was, respectively, a six-fold and three-fold lesser odds of the discharge date and the additional diagnoses being recorded in the electronic than the handwritten summaries, and that there was an 18-fold greater odds of there being documentation of a summary of patient progress in the electronic summaries. The CIs of the three ORs did not include 1.0, which shows these associations were statistically significant. In respect of the other three items, it is seen that there was a three-fold greater odds of there being a record of investigations/results in the electronic summaries, but only half the odds of there being correct documentation of discharge medications; these differences were not statistically significant. It can be seen in Table 2 that two items were rarely omitted in electronic summaries; these were summary of patient progress and investigations/results (these items were omit-
ted in only 2% or less of electronic summaries). The most frequently omitted item, by far, in electronic summaries was the discharge date (not documented in 57.6%). In handwritten summaries, omitted/incorrect documentation of a discharge medication was the least common deficiency (found in 6.4%) while the most common deficiency was omission of a summary of patient progress (found in 15%). Table 2 shows that across the six items, there were 901 documentations in the 151 electronic summaries and 558 in the 94 handwritten summaries. Documentation deficiencies in the electronic and handwritten summaries amounted to, respectively, 154 of the 901 (17.1%) and 59 of the 558 (10.75%) (OR 1.74, 95% CI 1.26–2.39), showing that deficiencies were significantly more common in the electronic summaries. Table 3 shows the frequency distribution of deficiencies in the electronic and handwritten summaries. No summary had more than three deficiencies. Significantly fewer electronic than handwritten summaries were free of deficiencies (2 = 20.9, 1 d.f., p < 0.0001), and it is seen that proportionately more electronic than handwritten summaries had one, two, or three deficiencies. The difference in the distribution of deficiencies in the two types of summary was highly significant (2 = 21.00, 2 d.f., p < 0.0001). Because so few summaries had three deficiencies (expected values were <5), the two–deficiency and three-deficiency categories were combined for 2 analysis (hence 2 d.f.). Omission of the discharge date was the deficiency in 60 of the 87 (69%) electronic summaries that had just one deficiency.
618
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 613–620
Table 3 – Comparison of electronic and handwritten discharge summaries by frequency of documentation deficiencies Number of deficiencies
Type of discharge summary Electronic (n = 151) n (%)
None One Two Three
Handwritten (n = 94) n (%)
33 (21.9) 87 (57.6) 26 (17.2) 5 (3.3)
Nineteen doctors had been involved in creating the discharge summaries. Table 4 shows the number of electronic and handwritten summaries created by each doctor and the number of deficiencies in the total number of summaries created by each doctor. It is seen that 10 of the 19 doctors had created the electronic summaries. Two of these doctors (A and B) had contributed between them 72.9% of all electronic summaries, and these contained 76% of all electronic summary deficiencies. These two doctors were responsible for 70 of the 87 omissions of the discharge date, 22 of the 27 omissions of additional diagnoses, 12 of the 19 medication errors and 10 of the 17 omissions of follow-up arrangements. Of the 62 electronic summaries created by Doctor A, 61 lacked the discharge date. It is seen in Table 4 that the 94 handwritten summaries had been created by 16 doctors of whom six (A, B, C, H, I and Q) contributed 71 (75.5%). These 71 summaries contained 76.7% of all handwritten summary deficiencies. Of Doctor D’s seven summaries, six lacked a summary of patient progress (but all six of Doctor D’s electronic summaries contained a progress summary) and of Doctor I’s nine summaries, three lacked a progress summary. The remaining five discharge summaries without a summary of patient progress had been written by five different doctors. It can be calcu-
47 (50.0) 36 (38.3) 10 (10.6) 1 (1.1)
lated from Table 4 that the two doctors who had created most electronic summaries and contributed the bulk of the deficiencies (A and B) averaged only fractionally more errors per summary than the five who had created the least number of electronic summaries (D, G, M, P and R) (1.06 versus 0.93). Likewise, the two who had most frequently created handwritten summaries averaged a similar number of errors per summary as the 10 who had most infrequently created them (both 0.56). Among doctors who created electronic summaries the one who contributed most of the deficiencies (A) averaged fewer deficiencies per summary (1.22) than two others (D (1.5) and E (1.75)).
4.
Discussion
The findings of this study show that an appreciable number of discharge summaries contained omissions and errors, but that these were more commonly present in the electronic than in the handwritten summaries. The value of reviewing discharge summaries has been demonstrated in this study, this research having shown that critical errors and omissions continue in the communication of clinical
Table 4 – Number of electronic and handwritten discharge summaries created by each of the 19 doctors and number of deficiencies in the total number of summaries per doctor Doctor identification A B C D E F G H I J K L M N O P Q R S Total
Electronic summaries created n (% of total)
Deficiencies found in electronic summaries n (% of total)
Handwritten summaries created n (% of total)
Deficiencies found in handwritten summaries n (% of total)
62 (41%) 48 (31.8%) 8 (5.3%) 6 (4.0%) 8 (5.3%) 10 (6.6%) 2 (1.3%) 0 0 0 0 0 1 (0.7%) 0 0 5 (3.3%) 0 1 (0.7%) 0
76 (49.4%) 41 (26.6%) 6 (3.9%) 9 (5.8%) 14 (9.1%) 3 (2.0%) 2 (1.3%) – – – – – 2 (1.3%) – – 0 – 1 (0.7%) –
17 (18.1%) 0 20 (21.3%) 7 (7.5%) 1 (1.1%) 0 4 (4.3%) 9 (9.8%) 9 (9.8%) 2 (2.1%) 2 (2.1%) 1 (1.1%) 2 (2.1%) 5 (5.3%) 4 (4.3%) 0 9 (9.8%) 1 (1.1%) 1 (1.1%)
7 (11.9%) – 14 (23.7%) 9 (15.3%) 1 (1.7%) – 2 (3.4%) 4 (6.8%) 8 (13.6%) 2 (3.4%) 0 0 1(1.7%) 6 (10.2%) 1 (1.7%) – 4 (6.8%) 0 0
151 (100%)
154 (100%)
94 (100%)
59 (100%)
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 613–620
information from the hospital to the GP, despite advances in computerisation of the discharge summary. The observation that a substantial amount of deficient documentation was linked to a small number of the doctors creating the summaries suggests that reasons for the deficiency variations between the electronic and handwritten discharge summaries pertained directly to the particular doctors involved. In the electronic summary, the fields for the discharge date and additional diagnoses were at the beginning of the document and were perhaps many screens up front from where final discharge data were entered. Repeated failure to refer back to the beginning of the summary for data entry could explain the extent of omitting the discharge date and additional diagnoses in electronic summaries. The higher rate of error (albeit non-significant) in recording discharge medications in electronic summaries could be due to repeat documentation of medications in the electronic summary creation (transcription from handwritten script to computer) as against once only documentation in the handwritten summary. Transcribing has been shown to cause incorrect medication documentation [26]. Our finding that among the 94 handwritten records, only six, that is, 6.3%, contained a medication error compares favourably with 17.5% in another Australian survey [3] and 49.5% in a New Zealand study [27]. Copying the discharge medications from the handwritten pharmacy script into the electronic summary involved extra work. However, this constituted the only difference between the two discharge summary methods in the amount of work expended. The importance of recording allergy data in discharge summaries is acknowledged. However, the study hospital’s handwritten summary form had no field for allergy details, and therefore, unlike in the electronic summary, there was no reminder to record allergy information. Because of this difference in format, it was considered inappropriate to compare the electronic and handwritten summaries on this item.
4.1.
Study limitations and strengths
A limitation of the present survey is that the bulk of the deficiencies found were due to a small number of the doctors who created the summaries; thus the results are not generalisable. A strength of this study is that it was a survey of a real-world situation. The doctors involved were unaware their work was to be observed, our study having been conducted well after the summaries had been created. Observing a real-world situation with doctors unaware was not the case with other studies in which electronically-created and handwritten summaries were compared [10,15,18,20]. Of three previous studies which compared the quality of electronic and paper-based summaries, two reported the electronic was superior in quality [10,18] and one reported a higher level of completeness in the electronic summary [15]. There is consensus there are important advantages of electronic over paper-based summaries that do not involve actual content. These include: greater likelihood of creation [4,10,15], quicker generation [4,8,10,15,18,20,28], and more timely receipt by the GP [4,15,18,20].
619
Summary points What was known before the study: • Many studies have claimed that paper-based (such as handwritten) discharge summaries contain errors and omissions, and that those with a narrative format are more likely to be deficient than those which have structure. • Electronically-created summaries are considered to have a potential for high quality, that is, for accuracy and completeness. • Comparisons between paper-based and electronic summaries in randomised trial and experimental situations have shown electronic summaries to be more complete. There are no published studies of comparisons made in real world situations, with the doctors who created the summaries unaware their work was to be studied. What the present study has added to the body of knowledge: • This first study of a comparison between structured (as against narrative) paper-based (handwritten) and structured electronic discharge summaries, all created in a real world situation, shows it is not necessarily the case that electronic summaries are of higher quality than handwritten ones. In the early stages of electronic summary implementation, electronic summaries may contain many more deficiencies than handwritten summaries. • Free text items (such as a summary of the patient’s progress in hospital) may more likely be documented in electronic than paper-based discharge summaries.
5.
Conclusion
The results of our study show it is not necessarily the case that electronic discharge summaries are of higher quality as regards accuracy and completeness than handwritten ones, and electronic summaries may be more deficient in the early stages of electronic implementation. However, free text items such as summary of the patient’s progress may more likely be documented in electronic summaries. It is unknown what factors contributed to incompleteness in creating the electronic discharge summaries investigated in this study. Possible causes for deficiencies include: insufficient training; insufficient education of, and thus realisation by, doctors regarding the importance of accurate, complete discharge summaries; inadequate computer literacy; unfamiliarity with creating discharge summaries electronically; inadequate user interaction design; insufficient integration into routine work processes. These factors require further study. Our survey reiterates that insufficient care is being taken when creating discharge summaries. We have shown that this is independent of the method used (manual or elec-
620
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 613–620
tronic). It is apparent it must be strongly emphasised among doctors creating discharge summaries that this document is crucial in transferring sufficient patient information from hospital to the GP to enable the provision of quality ongoing care.
[14]
[15]
Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at doi:10.1016/j.ijmedinf.2007.12.002.
[16]
references
[17]
[1] P. Bolton, M. Mira, P. Kennedy, M.M. Lahra, The quality of communication between hospitals and general practitioners: An assessment, J. Qual. Clin. Pract. 18 (4) (1998) 241–247. [2] P. Bolton, A quality assurance activity to improve discharge communication with general practice, J. Qual. Clin. Pract. 21 (3) (2001) 69–70. [3] S. Wilson, W. Ruscoe, M. Chapman, R. Miller, General practitioner–hospital communications: a review of discharge summaries, J. Qual. Clin. Pract. 21 (4) (2001) 104–108. [4] W.M. Castleden, M.C. Stacey, P.E. Norman, M.M. Lawrence-Brown, J.G. Brooks, General practitioners’ attitudes to computer-created surgical discharge letters, Med. J. Aust. 157 (6) (1992) 380–382. [5] C. van Walraven, A.L. Weinberg, Quality assessment of a discharge summary system, Can. Med. Assoc. J. 152 (9) (1995) 1437–1442. [6] C. van Walraven, E. Rokosh, What is necessary for high-quality discharge summaries? Am. J. Med. Qual. 14 (4) (1999) 160–169. [7] J.O. Jansen, I.C. Grant, Communication with general practitioners after accident and emergency attendance: computer created letters are often deficient, Emerg. Med. J. 20 (3) (2003) 256–257 (http://emj.bmj.com/cgi/reprint/20/3/256, accessed March 12, 2007). [8] J.E. Brazy, D.L. Langkamp, N.D. Brazy, R.F. De Luna, Do primary care physicians prefer dictated or computer-generated discharge summaries? Am. J. Dis. Child. 147 (9) (1993) 986–988. [9] A.N. Raval, G.E. Marchiori, J.M. Arnold, Improving the continuity of care following discharge of patients hospitalized with heart failure: is the discharge summary adequate? Can. J. Cardiol. 19 (4) (2003) 365–370. [10] T. Lissauer, C.M. Paterson, A. Simons, R.W. Beard, Evaluation of computer generated neonatal discharge summaries, Arch. Dis. Child. 66 (4) (1991) 433–436. [11] J. Pears, V. Alexander, G.F. Alexander, N.R. Waugh, Audit of the quality of hospital discharge data, Health Bull. (Edinb.) 50 (5) (1992) 356–361. [12] E.M. Macaulay, G.G. Cooper, J. Engeset, A.R. Naylor, Prospective audit of discharge summary errors, Br. J. Surg. 83 (6) (1996) 788–790. [13] J.M. Paterson, R.L. Allega, Improving communication between hospital and community physicians. Feasibility study of a handwritten, faxed hospital discharge summary.
[18]
[19]
[20]
[21]
[22]
[23]
[24]
[25]
[26]
[27]
[28]
Discharge Summary Study Group, Can. Fam. Physician 45 (1999) 2893–2899. D.S. Foster, C. Paterson, G. Fairfield, Evaluation of immediate discharge documents – room for improvement? Scott. Med. J. 47 (4) (2002) 77–79. C. van Walraven, A. Laupacis, R. Seth, G. Wells, Dictated versus database-generated discharge summaries: a randomized clinical trial, Can. Med. Assoc. J. 160 (3) (1999) 319–326. C. van Walraven, S.M. Duke, A.L. Weinberg, P. Wells, Standardized or narrative discharge summaries. Which do family physicians prefer? Can. Fam. Physician 44 (1998) 62–69. S.E. Hook, G.C. Banister, C. Topliss, J. Webb, Letters and notes in orthopaedic surgery, Ann. R. Coll. Surg. Engl. 88 (3) (2006) 292–296. R.A. Archbold, K. Laji, A. Suliman, K. Ranjadayalan, H. Hemingway, A.D. Timmis, Evaluation of a computer-generated discharge summary for patients with acute coronary syndromes, Br. J. Gen. Pract. 48 (429) (1998) 1163–1164. R. Crosswhite, S.H. Beckham, P. Gray, P.R. Hawkins, J. Hughes, Using a multidisciplinary automated discharge summary process to improve information management across the system, Am. J. Manage. Care 3 (3) (1997) 473–479. J. Kirby, B. Barker, D.J. Fernando, M. Jose, C. Curtis, A. Goodchild, C. Dickens, E. Olla, R. Cooke, I. Idris, G.A. Thomson, A prospective case control study of the benefits of electronic discharge summaries, J. Telemed. Telecare 12 (1) (2006) 20–21. T. Schabetsberger, E. Ammenwerth, S. Andreatta, G. Gratl, R. Haux, G. Lechleitner, K. Schindelwig, C. Stark, R. Volg, I. Wilhelmy, F. Wozak, From a paper-based transmission of discharge summaries to electronic communication in health care regions, Int. J. Med. Inform. 75 (3) (2006) 209–215. Global Health, Announcement to the Australian Stock Exchange, Sydney’s North Shore GPs get electronic hospital discharge referrals for their patients, April 5, 2004 (http://www.global-health.com/NorthernSydneyASX.pdf, accessed March 12, 2007). The Royal Australian College of General Practitioners, Examples of good practice in hospital-GP communication, Submission No. 66 to House of Representatives Standing Committee on Health and Ageing: Parliament House, Canberra, ACT, August 3, 2005 (http://www.aph.gov.au/ house/committee/haa/healthfunding/subs/sub066.pdf, accessed March 12, 2007). S.J. Coakes, L.G. Steed, SPSS: Analysis without Anguish: Version 11. 00 for Windows, John Wiley & Sons Inc., Australia, 2003. R. Lawson, Small sample confidence intervals for the odds ratio, Commun. Statist.: Simul. Comput. 33 (4) (2004) 1095–1113. M. Frank-Stromborg, A. Christensen, C. Elmhurst, Nurse documentation: not done or worse, done the wrong way – part 2, Oncol. Nurs. Forum 28 (5) (2001) 841–846. T.E. McMillan, W. Allan, P.N. Black, Accuracy of information on medicines in hospital discharge summaries, Intern. Med. J. 36 (4) (2006) 221–225. R.P. Smith, G.B. Holzman, The application of a computer data base system to the generation of hospital discharge summaries, Obstet. Gynecol. 73 (5 Pt1) (1989) 803–807.