- Email: [email protected]
Technology for trauma: testing the validity of a smartphone app for pre-hospital clinicians
ARTICLE IN PRESS International Emergency Nursing ■■ (2014) ■■–■■
Contents lists available at ScienceDirect
International Emergency Nursing j o u r n a l h o m e p a g e : w w w. e l s e v i e r. c o m / l o c a t e / a a e n
Technology for trauma: testing the validity of a smartphone app for pre-hospital clinicians Eleanor S. Freshwater BA (Hons) PGCert a,b,*, Robert Crouch RN, PhD, FRCN a,b a b
Emergency Department, University Hospital Southampton, Tremona Road, Southampton, Hampshire, UK, SO16 6YD Faculty of Health Sciences, University of Southampton, 104 Burgess Road, Southampton, SO17 1BJ
A R T I C L E
I N F O
Article history: Received 4 February 2014 Received in revised form 17 April 2014 Accepted 22 April 2014 Keywords: Pre-hospital emergency care Emergency medical services Paramedics Emergency medicine Trauma
A B S T R A C T
Introduction: With the introduction of regional trauma networks in England, ambulance clinicians have been required to make triage decisions relating to severity of injury, and appropriate destination for the patient, which may require ‘bypassing’ the nearest Emergency Department. A ‘Trauma Unit Bypass Tool’ is utilised in this process. The Major Trauma Triage tool smartphone application (App) is a digital representation of a tool, available for clinicians to use on their smartphone. Prior to disseminating the application, validity and performance against the existing paper-based tool was explored. Methods: A case-based study using clinical scenarios was conducted. Scenarios, with appropriate triage decisions, were agreed by an expert panel. Ambulance clinicians were assigned to either the paperbased tool or smartphone app group and asked to make a triage decision using the available information. The positive predictive value (PPV) of each tool was calculated. Results: The PPV of the paper tool was 0.76 and 0.86 for the smartphone app. User comments were mainly positive for both tools with no negative comments relating to the smartphone app. Conclusion: The smartphone app version of the Trauma Unit Bypass Tool performs at least as well as the paper version and can be utilised safely by pre-hospital clinicians in supporting triage decisions relating to potential major trauma. © 2014 Elsevier Ltd. All rights reserved.
1. Background Major Trauma Networks began in London and went live across England in April 2012 which require patients with major trauma to be transported to dedicated major trauma centres (Department of Health, 2012). Trauma networks were developed in response to numerous reports that identified sub-optimal care and the need to improve survival from major trauma (Findlay et al., 2007; Fisher et al., 2010). The current system utilises a ‘hub and spoke’ model with Major Trauma Centres (MTCs) supported by Trauma Units (TUs) and Local Emergency Hospitals (LEHs). MTCs are able to offer enhanced care with input from multiple specialties for the most seriously injured patients. Internationally, similar networks have shown improved outcomes (Cameron et al., 2008; Celso et al., 2006; Lansink and Leenen, 2007; MacKenzie et al., 2006; Sampalis et al., 1999) Early work from England reports that 20% more patients are now surviving since the introduction of Major Trauma Networks (NHS England, 2013). In order to ensure that the most seriously injured patients are treated at an MTC, accurate identification of those suffering from major trauma in the pre-hospital phase is essential. Patients that
* Corresponding author. Tel.: +07816 880465. E-mail address: [email protected]
do not require specialist care should be transported to the closest appropriate hospital, avoiding overloading of MTCs and the subsequent use of resources. Accurate pre-hospital triage is vital to the success of Major Trauma Networks. Major Trauma Triage criteria and decision support tools have been used in the USA since the 1990s (Henry et al., 1996; Newgard et al., 2011; Purtill et al., 2008) to assist Emergency Medical Services (EMS) decision-making. Similar tools have been developed in the UK by expert panels; many have not been formally evaluated to ascertain their validity and reliability. Pre-hospital clinicians in the Wessex region are provided with the Trauma Unit Bypass (TUB) tool, a decision tree, combining anatomical injuries and physiological signs to guide hospital destination selection (Fig. 1). It is presented as an algorithm in pocketsized and A4 printed form. Clinical staff received education about trauma networks and use of the TUB tool prior to launch. 1.1. Development of the smartphone application Working in practice it became clear that some EMS providers were storing a version of the tool on smartphones and tablets to use in the field. In addition, it was recognised that many clinicians access other decision-support tools and information in the form of smartphone applications, such as the British National Formulary (BNF) and NICE guidelines in clinical practice. Observing this practice led
http://dx.doi.org/10.1016/j.ienj.2014.04.003 1755-599X/© 2014 Elsevier Ltd. All rights reserved.
Please cite this article in press as: Eleanor S. Freshwater, Robert Crouch, Technology for trauma: testing the validity of a smartphone app for pre-hospital clinicians, International Emergency Nursing (2014), doi: 10.1016/j.ienj.2014.04.003
ARTICLE IN PRESS 2
E.S. Freshwater and R. Crouch/International Emergency Nursing ■■ (2014) ■■–■■
Fig. 1. Wessex Trauma Unit Bypass Tool.
to discussions about development of a digital representation of the Wessex Trauma Network (WTN) TUB tool. A software development company (Volatile State Ltd) were approached and agreed to develop the application initially on the Apple® iOS iPhone® platform. The application, named the Major Trauma Triage Tool, consists of three elements: the TUB tool, recommendations for hospital destination, based on location, and collection and presentation of clinical handover information in an ATMIST format (SWASFT, 2013). Given the change in format of the tool, from a paper-based algorithm to an interactive digital representation in the form of an app, it was necessary to compare the utilisation of the formats in clinical decision-making and triage.
1.2. Aim To compare ambulance staff decision outcomes using the TUB paper-based version with the smartphone app. 2. Methods 2.1. Design A clinical scenario-based testing of triage decision-making of prehospital clinicians using either the paper-based TUB tool or smartphone app.
Please cite this article in press as: Eleanor S. Freshwater, Robert Crouch, Technology for trauma: testing the validity of a smartphone app for pre-hospital clinicians, International Emergency Nursing (2014), doi: 10.1016/j.ienj.2014.04.003
ARTICLE IN PRESS E.S. Freshwater and R. Crouch/International Emergency Nursing ■■ (2014) ■■–■■
3
Table 1 Comparison of sample groups.
CASE 1 Incident A car has collided with a bus with a combined speed of 50mph. The driver of the car is trapped by his lower legs and is combative.
Observations Pulse
105 felt at the wrist
Systolic BP
100 mmHg
Resp rate
20
GCS
12 (E3, V4, M5)
Primary survey Clear airway Chest sounds clear with equal chest wall movement No external bleeding seen.
Trained on TUB Tool Used TUB in clinical practice Smartphone user iPhone user ECA Technician Paramedic
Paper group (n = 25)
App group (n = 25)
23 18 22 15 3 6 16
24 20 21 15 5 4 16
the ‘app’ group. Those that were randomised to the ‘app’ group were provided with an iPhone® or iPad® on which to use the app. The questionnaire and scenarios were explained to each group together. They were then asked to use their allocated tool to recommend a hospital (TU or MTC) for each case, based on the information provided. The answers for each case were compared against those noted by the panel and graded as “matched” if they were the same and ‘not matched’ if they did not concur. 2.4. Data analysis
Pelvis appears intact although you suspect a left ankle fracture.
ASSESS
With the information provided, please indicate your destination hospital choice
Trauma Unit
Major Trauma Centre
Additional comments:
Fig. 2. Example clinical case.
Demographic information relating to their clinical grade or role was collected from each participant: their experience of using the TUB tool in practice and smartphone use was also recorded. No participant identifiable data were collected. The administration of the scenarios took place in four locations on five occasions. Data were entered into a spreadsheet and analysed using Microsoft Excel. The positive predictive value for each tool was calculated for the different scenarios and overall. This value described the ability of each tool to correctly identify those patients that were deemed to have suffered major trauma, as defined by the Wessex Trauma Network. Free text comments were collected relating to the ease of use of each tool, together with the perceived benefits and disadvantages to clinicians. 2.5. Definitions
2.2. Case study scenarios Using real clinical cases as a basis, six clinical scenarios were developed, and these included information on the mechanism of injury, patient injuries and vital signs. To determine ecological validity of the cases, they were reviewed by a multidisciplinary panel of experienced clinicians, including Emergency Department (ED) doctors, senior nurses and paramedics (two representatives from each profession). For each scenario, a standardised outcome, in terms of destination, was agreed on by the panel through this process. See Fig. 2 for an example. 2.3. Sample Following organisational approval to conduct the study, permission to recruit ambulance staff during regular team training sessions was granted by the Ambulance Trusts’ Clinical Review Group. Staff were invited to participate during these sessions; it was not mandatory and none declined. The purpose of the study was explained and verbal agreement to participate sought. This approach ensured a range of skill grades, to include Ambulance Technician, Paramedic and Emergency Care Assistants (ECAs), and experience amongst the sample (Table 1). Using an alternate numbering technique around the room, participants were assigned to one of two groups; the ‘paper’ group and
If the expert panel agreed the destination was a Trauma Unit, an answer not matching this would lead to a patient being transferred to a specialist MTC – this would be classified as overtriage. If the expert panel agreed the destination was a Major Trauma Centre, an answer not matching this would lead to a decision to transfer to a Trauma Unit – this would be classified as undertriage. 3. Results A total of 50 participants took part in the study; 25 used the paper tool and 25 used the smartphone app. The majority, 64% (n = 32), were paramedics, with the remainder ambulance technicians (n = 10) and emergency care assistants (n = 8). Training on the TUB Tool had been received by 47 of the 50 participants (94%). Most had used it between 1 and 5 times and 12 had not used it in clinical practice (Fig. 3). Of those participants that own a smartphone (n = 43), 67% (n = 30) use Apple® iOS, with the remainder being users of the Android® operating system. The paper tool had a positive predictive value (PPV) of 0.76, with a range of 0.44 to 1.00 in each of the cases. The smartphone version on the TUB tool had a PPV of 0.86 with a range of 0.80–0.92. In three of the six scenarios the paper version outperformed the app, the app was more accurate, in that it had a higher PPV, in two scenarios and equal in one. However, in cases where the app showed
Please cite this article in press as: Eleanor S. Freshwater, Robert Crouch, Technology for trauma: testing the validity of a smartphone app for pre-hospital clinicians, International Emergency Nursing (2014), doi: 10.1016/j.ienj.2014.04.003
ARTICLE IN PRESS E.S. Freshwater and R. Crouch/International Emergency Nursing ■■ (2014) ■■–■■
4
35
No. participants
30 25 20 15 10 5 0 0 1-5 6-10 >10 No. times TUB tool used previously by participant in clinical practice Fig. 3. Participant experience of using the TUB tool in practice.
a higher PPV, the paper tool was significantly lower (0.56 and 0.44), whereas when the paper tool performed better, the app showed a closer result (0.80, 0.84 and 0.88). Both of the cases where the paper tool showed a low PPV, the agreed result was triage to a major trauma centre, this would have resulted in significant undertriage where the paper tool was used (Table 2). Free text responses on ease of use were grouped into ‘positive’ or ‘negative’ (Fig. 4). Sixty-eight per cent (n = 17) of comments on the paper tool were positive, with 5 negative comments relating to layout and appearance of the tool and 3 respondents making neutral or no comment. There were no negative comments relating to the
smartphone version of the tool, with 92% making positive comments and the remaining 2 participants making no or neutral comment. Many of the comments relating to the smartphone app were regarding the decrease in time taken to make a decision in comparison to the paper app, which was felt to be a positive benefit in a time-critical scenario. Participants made general comments about the tool that they had used. Some were concerned that there may be infection control issues around using a piece of technology, particularly whilst wearing gloves. Other comments related to the battery life of the mobile telephone and the possibility of it not always working.
Table 2 Results. Case
1 2 3 4 5 6 Total
Agreed destination
TU MTC MTC TU TU MTC
Paper
App
Result of incorrect answer
Not-matched
Matched
PPV
Not-matched
Matched
PPV
4 9 0 5 2 14 34
21 14 25 20 23 11 114
0.84 0.56 1.00 0.80 0.92 0.44 0.76
5 2 4 5 3 2 21
20 23 21 20 22 23 129
0.80 0.92 0.84 0.80 0.88 0.92 0.86
Overtriage Undertriage Undertriage Overtriage Overtriage Undertriage
25
No. comments
20 Paper
15
App
10 5 0 Positive
Negative
Neutral/Nil
Classification of comments Fig. 4. Comments relating to ease of use of the paper and app tools.
Please cite this article in press as: Eleanor S. Freshwater, Robert Crouch, Technology for trauma: testing the validity of a smartphone app for pre-hospital clinicians, International Emergency Nursing (2014), doi: 10.1016/j.ienj.2014.04.003
ARTICLE IN PRESS E.S. Freshwater and R. Crouch/International Emergency Nursing ■■ (2014) ■■–■■
4. Discussion This study demonstrated a PPV of 0.86 for the new smartphone decision support tool, with the original paper-based tool showing a PPV of 0.76 in the same scenarios. The purpose of this study was to test the digital representation of the major trauma triage tool against the paper version. Exploring the sensitivity and specificity of the TUB tool in predicting major trauma was not studied as part of this evaluation. The findings suggest that the smartphone version of the tool performed at least as well as the paper-based tool. One of the benefits of utilising an electronic triage tool available as an app is improved version control. If a change is made to any of the triage criteria, the app can be amended and an update made available in a short time frame. The most commonly used operating systems now offer functionality to automatically update the smartphone. This ensures that clinicians will always be using the most up-to-date tool, avoiding confusion at the scene of an incident. In contrast, updating a paper-based tool, particularly where clinicians are geographically spread and often lone-working, presents challenges. It relies on each clinician being made aware of changes, and requires a formal system of version control. Morris et al. (2013) tested two smartphone applications for use in calculating fluid resuscitation in burns patients and found that they were much quicker to provide a result than the calculator previously available, albeit associated with a decrease in ease of use. Initially, there were some organisational concerns relating to information governance and storage of information on staff personal mobile devices. The app does not obtain any patient identifiable data, and the developers and designers worked with the relevant stakeholders to ensure that the app did not breach any conditions of confidentiality. Several papers describe the testing of applications developed to be used directly by patients, assessing the design and usability of the interface (Ben-Zeev et al., 2013; Brouillette et al., 2013; Robinson et al., 2013), whilst others focus on the technical performance against existing hardware (Mellone et al., 2012; Petersen et al., 2013). Suk Nam et al. (2014) describe the design and implementation of a stroke screening app which suggests hospital destination, but they have not provided data on its effectiveness in practice. The regulation of applications used to guide clinical decisions is suggested by Visvanathan et al. (2012) in order to safeguard patients by ensuring information contained in them is authenticated. Shortly after the launch of the iPhone®, Burdette et al. (2008) identified the potential for the use of smartphone technology in clinical practice, in particular with reference to the integration of apps with web browsers and mapping functions. In this fast moving area of technology, enhancements continue to be made with faster processing speeds, greater memory and more numerous applications available. As smartphones become an increasing part of clinicians and patients’ lives, it is likely that the use of apps in clinical practice will become more prolific. In order to assure quality and safety, it is essential that robust testing is carried out and is available publicly for review. The app was launched via the iTunes® store on 7 October 2013 and had been downloaded 1351 times by 31 December 2013. As the Wessex Trauma Network TUB tool is also used in the Thames Valley, Peninsula and Severn Trauma Networks, it has now been able to be rolled out across the south central and south-west regions of England. It is not possible at this time to produce a generic trauma triage app for the whole of England as there is variance in the triage tool used in other areas. A Scandinavian study (Hagiwara et al., 2013) identified that the physical format of pre-hospital guidelines was the main barrier to usage and compliance in ambulance staff, with many clinicians constructing their own paper and electronic versions to improve ease
5
of use. The development of well-designed, stable, rapid and accessible smartphone apps has the potential to improve concordance with guideline usage and associated improvements in patient safety. The Android® version of the Major Trauma Triage Tool app has also recently been launched to widen availability to clinicians. There have been positive reviews of the app on iTunes and interest from local media, raising the profile of the work of the ambulance services and major trauma networks. Scope exists to expand the app in other areas where specialist treatment centres receive patients, such as hyper-acute stroke and primary percutaneous coronary intervention, providing enhanced decision-support for clinicians managing patients with serious medical conditions. It is possible that as electronic ambulance patient care records are utilised further within UK ambulance services that the functionality contained in the app becomes part of an integrated system of monitoring, recording and transmitting data relating to patient condition. This would allow enhanced pre-alerting of hospitals, in order that they can ensure appropriate specialists are available to manage severely injured patients as soon as they arrive in hospital. There are additional possibilities to enhance further versions of the Major Trauma Triage Tool app, such as data collection. This has been utilised successfully by other apps, in order to build a database that can provide information to assist in understanding epidemiology of disease (Michael and Geleta, 2013). There are potential applications in injury prevention and pre-hospital resourcing that could benefit from accurate information on location and severity of trauma incidents such as road traffic collisions and assaults. The issue of information governance is critical in order to ensure compliance with requirements for patient confidentiality. Novel means of achieving this, such as automatic deletion of images after a set time period, have been described by other clinical application developers (Takao et al., 2012). 4.1. Limitations 4.1.1. Data collection Certain demographic information was not collected from participants. It was not possible to make a comparison between the testing groups in terms of their age, gender, experience in the ambulance service or other factors that may influence their use of particular tools. 4.1.2. Ecological validity This was a scenario-based study, and not all information could be provided. Participants were advised that if information was not noted, to assume it was normal, many comments suggest they didn’t always comply with this. For example, they wrote ‘possible major pelvis fracture due to mechanism’ or ‘answered “no” to airway and haemorrhage question as unsure’. This means that it is not possible to determine whether a Trauma Unit was selected as it was nearer for a severely injured patient or if the patient was undertriaged to this type of facility. The purpose of this evaluation was to test the App against the paper tool to ensure it performs as well. Therefore, comparisons cannot be made with real incidents, as there would be many more factors to consider such as environment, human factors and time pressures. 4.1.3. Implementation Our study did not measure time to reach a decision on transport destination for trauma patients in comparison to the paperbased tool although this could be an area for additional investigation, particularly pertinent when managing time-critical patients.
Please cite this article in press as: Eleanor S. Freshwater, Robert Crouch, Technology for trauma: testing the validity of a smartphone app for pre-hospital clinicians, International Emergency Nursing (2014), doi: 10.1016/j.ienj.2014.04.003
ARTICLE IN PRESS E.S. Freshwater and R. Crouch/International Emergency Nursing ■■ (2014) ■■–■■
6
A limitation with smartphone technology is that certain functions rely on the signal provided by the carrier network. A lack of signal would prevent the global positioning system (GPS) element of the app being utilised. However, the triage tool itself and the means to record handover information are unaffected by this. 5. Conclusion The Smartphone app performed at least as well as the paperbased version of the Wessex Trauma Network Trauma Unit Bypass Tool. It was therefore considered safe to release a general version of the app. The app is now being used within the Network. Acknowledgements Wessex Trauma Network for their support in designing and implementing the app and use of their Trauma Unit Bypass Tool. Volatile State Ltd for the technical development of the Major Trauma Triage Tool app. Mark Ainsworth-Smith and the staff of South Central Ambulance Service NHS Foundation Trust for their assistance in testing the app. References Ben-Zeev, D., Kaiser, S.M., Brenner, C.J., Begale, M., Duffecy, J., Mohr, D.C., 2013. Development and usability testing of focus: A smartphone system for selfmanagement of schizophrenia. Psychiatric Rehabilitation Journal. 36 (4), 289–296. Brouillette, R.M., Foil, H., Fontenot, S., Correro, A., Allen, R., Martin, C.K., et al., 2013. Feasibility, reliability, and validity of a smartphone based application for the assessment of cognitive function in the elderly. PLoS ONE. 8 (6). Burdette, S.D., Herchline, T.E., Oehler, R., 2008. Surfing the web: Practicing medicine in a technological age: Using smartphones in clinical practice. Clinical Infectious Diseases. 47 (1), 117–122. Cameron, P.A., Gabbe, B.J., Cooper, D.J., Walker, T., Judson, R., McNeil, J., 2008. A statewide system of trauma care in Victoria: Effect on patient survival. The Medical Journal of Australia. 189 (10), 546–550. Celso, B., Tepas, J., Langland-Orban, B., Pracht, E., Papa, L., Lottenberg, L., et al., 2006. A systematic review and meta-analysis comparing outcome of severely injured patients treated in trauma centers following the establishment of trauma systems. The Journal of Trauma. 60 (2), 371–378. Department of Health (2012) Major trauma centres to open across England, Available from: (Accessed 14.01.03.). Findlay, G., Martin, I.C., Smith, M., Wayman, D., Carter, S., Mason, M. Trauma: Who cares? National confidential enquiry into patient outcome and death, 2007. Available from: (Accessed 13.10.13.).
Fisher, A., Ross, C., Henderson, C., Kirk, S., Feroze, M., Richmond, R., 2010. Major Trauma Care in England, National Audit Office, London. Hagiwara, M.A., Suserud, B.-O., Jonsson, A., Henricson, M., 2013. Exclusion of context knowledge in the development of prehospital guidelines: Results produced by realistic evaluation. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine. 21 (46). Henry, M.C., Alicandro, J.M., Hollander, J.E., Moldashel, J.G., Cassara, G., Thode, H.C., Jr., 1996. Evaluation of American College of Surgeons Trauma Triage Criteria in a suburban and rural setting. The American Journal of Emergency Medicine. 14 (2), 124–129. Lansink, K.W.W., Leenen, L.P.H., 2007. Do designated trauma systems improve outcome? Current Opinion in Critical Care. 13 (6), 686–690. MacKenzie, E.J., Rivara, F.P., Jurkovich, G.J., Nathens, A.B., Frey, K.P., Egleston, B.L., et al., 2006. A national evaluation of the effect of trauma-center care on mortality. The New England Journal of Medicine. 354 (4), 366–378. Mellone, S., Tacconi, C., Chiari, L., 2012. Gait & posture. Gait and Posture. 36 (1), 163–165. Michael, B.D., Geleta, D., 2013. Development of clickclinica: A novel smartphone application to generate real-time global disease surveillance and clinical practice data. BMC Medical Informatics and Decision Making. 13 (1). Morris, R., Javed, M., Bodger, O., Gorse, S.H., Williams, D., 2013. ScienceDirect. Burns: Journal of the International Society for Burn Injuries. 1–9. Newgard, C.D., Zive, D., Holmes, J.F., Bulger, E.M., Staudenmayer, K., Liao, M., et al., 2011. A Multisite assessment of the American College of Surgeons Committee on trauma field triage decision scheme for identifying seriously injured children and adults. Journal of the American College of Surgeons. 213 (6), 709–721. NHS England (2013) Independent review of major trauma networks reveals increase in patient survival rates. Available from: (Accessed 14.01.29.). Takao, H., Murayama, Y., Ishibashi, T., Karagiozov, K.L., Abe, T., 2012. A new support system using a mobile device (smartphone) for diagnostic image display and treatment of stroke. Stroke; a Journal of Cerebral Circulation. 43 (1), 236–239. Visvanathan, A., Hamilton, A., Brady, R.R.W., 2012. Smartphone apps in microbiology-is better regulation required? Clinical Microbiology and Infection. 18 (7), E218– E220.
Please cite this article in press as: Eleanor S. Freshwater, Robert Crouch, Technology for trauma: testing the validity of a smartphone app for pre-hospital clinicians, International Emergency Nursing (2014), doi: 10.1016/j.ienj.2014.04.003
Contents lists available at ScienceDirect
International Emergency Nursing j o u r n a l h o m e p a g e : w w w. e l s e v i e r. c o m / l o c a t e / a a e n
Technology for trauma: testing the validity of a smartphone app for pre-hospital clinicians Eleanor S. Freshwater BA (Hons) PGCert a,b,*, Robert Crouch RN, PhD, FRCN a,b a b
Emergency Department, University Hospital Southampton, Tremona Road, Southampton, Hampshire, UK, SO16 6YD Faculty of Health Sciences, University of Southampton, 104 Burgess Road, Southampton, SO17 1BJ
A R T I C L E
I N F O
Article history: Received 4 February 2014 Received in revised form 17 April 2014 Accepted 22 April 2014 Keywords: Pre-hospital emergency care Emergency medical services Paramedics Emergency medicine Trauma
A B S T R A C T
Introduction: With the introduction of regional trauma networks in England, ambulance clinicians have been required to make triage decisions relating to severity of injury, and appropriate destination for the patient, which may require ‘bypassing’ the nearest Emergency Department. A ‘Trauma Unit Bypass Tool’ is utilised in this process. The Major Trauma Triage tool smartphone application (App) is a digital representation of a tool, available for clinicians to use on their smartphone. Prior to disseminating the application, validity and performance against the existing paper-based tool was explored. Methods: A case-based study using clinical scenarios was conducted. Scenarios, with appropriate triage decisions, were agreed by an expert panel. Ambulance clinicians were assigned to either the paperbased tool or smartphone app group and asked to make a triage decision using the available information. The positive predictive value (PPV) of each tool was calculated. Results: The PPV of the paper tool was 0.76 and 0.86 for the smartphone app. User comments were mainly positive for both tools with no negative comments relating to the smartphone app. Conclusion: The smartphone app version of the Trauma Unit Bypass Tool performs at least as well as the paper version and can be utilised safely by pre-hospital clinicians in supporting triage decisions relating to potential major trauma. © 2014 Elsevier Ltd. All rights reserved.
1. Background Major Trauma Networks began in London and went live across England in April 2012 which require patients with major trauma to be transported to dedicated major trauma centres (Department of Health, 2012). Trauma networks were developed in response to numerous reports that identified sub-optimal care and the need to improve survival from major trauma (Findlay et al., 2007; Fisher et al., 2010). The current system utilises a ‘hub and spoke’ model with Major Trauma Centres (MTCs) supported by Trauma Units (TUs) and Local Emergency Hospitals (LEHs). MTCs are able to offer enhanced care with input from multiple specialties for the most seriously injured patients. Internationally, similar networks have shown improved outcomes (Cameron et al., 2008; Celso et al., 2006; Lansink and Leenen, 2007; MacKenzie et al., 2006; Sampalis et al., 1999) Early work from England reports that 20% more patients are now surviving since the introduction of Major Trauma Networks (NHS England, 2013). In order to ensure that the most seriously injured patients are treated at an MTC, accurate identification of those suffering from major trauma in the pre-hospital phase is essential. Patients that
* Corresponding author. Tel.: +07816 880465. E-mail address: [email protected]
do not require specialist care should be transported to the closest appropriate hospital, avoiding overloading of MTCs and the subsequent use of resources. Accurate pre-hospital triage is vital to the success of Major Trauma Networks. Major Trauma Triage criteria and decision support tools have been used in the USA since the 1990s (Henry et al., 1996; Newgard et al., 2011; Purtill et al., 2008) to assist Emergency Medical Services (EMS) decision-making. Similar tools have been developed in the UK by expert panels; many have not been formally evaluated to ascertain their validity and reliability. Pre-hospital clinicians in the Wessex region are provided with the Trauma Unit Bypass (TUB) tool, a decision tree, combining anatomical injuries and physiological signs to guide hospital destination selection (Fig. 1). It is presented as an algorithm in pocketsized and A4 printed form. Clinical staff received education about trauma networks and use of the TUB tool prior to launch. 1.1. Development of the smartphone application Working in practice it became clear that some EMS providers were storing a version of the tool on smartphones and tablets to use in the field. In addition, it was recognised that many clinicians access other decision-support tools and information in the form of smartphone applications, such as the British National Formulary (BNF) and NICE guidelines in clinical practice. Observing this practice led
http://dx.doi.org/10.1016/j.ienj.2014.04.003 1755-599X/© 2014 Elsevier Ltd. All rights reserved.
Please cite this article in press as: Eleanor S. Freshwater, Robert Crouch, Technology for trauma: testing the validity of a smartphone app for pre-hospital clinicians, International Emergency Nursing (2014), doi: 10.1016/j.ienj.2014.04.003
ARTICLE IN PRESS 2
E.S. Freshwater and R. Crouch/International Emergency Nursing ■■ (2014) ■■–■■
Fig. 1. Wessex Trauma Unit Bypass Tool.
to discussions about development of a digital representation of the Wessex Trauma Network (WTN) TUB tool. A software development company (Volatile State Ltd) were approached and agreed to develop the application initially on the Apple® iOS iPhone® platform. The application, named the Major Trauma Triage Tool, consists of three elements: the TUB tool, recommendations for hospital destination, based on location, and collection and presentation of clinical handover information in an ATMIST format (SWASFT, 2013). Given the change in format of the tool, from a paper-based algorithm to an interactive digital representation in the form of an app, it was necessary to compare the utilisation of the formats in clinical decision-making and triage.
1.2. Aim To compare ambulance staff decision outcomes using the TUB paper-based version with the smartphone app. 2. Methods 2.1. Design A clinical scenario-based testing of triage decision-making of prehospital clinicians using either the paper-based TUB tool or smartphone app.
Please cite this article in press as: Eleanor S. Freshwater, Robert Crouch, Technology for trauma: testing the validity of a smartphone app for pre-hospital clinicians, International Emergency Nursing (2014), doi: 10.1016/j.ienj.2014.04.003
ARTICLE IN PRESS E.S. Freshwater and R. Crouch/International Emergency Nursing ■■ (2014) ■■–■■
3
Table 1 Comparison of sample groups.
CASE 1 Incident A car has collided with a bus with a combined speed of 50mph. The driver of the car is trapped by his lower legs and is combative.
Observations Pulse
105 felt at the wrist
Systolic BP
100 mmHg
Resp rate
20
GCS
12 (E3, V4, M5)
Primary survey Clear airway Chest sounds clear with equal chest wall movement No external bleeding seen.
Trained on TUB Tool Used TUB in clinical practice Smartphone user iPhone user ECA Technician Paramedic
Paper group (n = 25)
App group (n = 25)
23 18 22 15 3 6 16
24 20 21 15 5 4 16
the ‘app’ group. Those that were randomised to the ‘app’ group were provided with an iPhone® or iPad® on which to use the app. The questionnaire and scenarios were explained to each group together. They were then asked to use their allocated tool to recommend a hospital (TU or MTC) for each case, based on the information provided. The answers for each case were compared against those noted by the panel and graded as “matched” if they were the same and ‘not matched’ if they did not concur. 2.4. Data analysis
Pelvis appears intact although you suspect a left ankle fracture.
ASSESS
With the information provided, please indicate your destination hospital choice
Trauma Unit
Major Trauma Centre
Additional comments:
Fig. 2. Example clinical case.
Demographic information relating to their clinical grade or role was collected from each participant: their experience of using the TUB tool in practice and smartphone use was also recorded. No participant identifiable data were collected. The administration of the scenarios took place in four locations on five occasions. Data were entered into a spreadsheet and analysed using Microsoft Excel. The positive predictive value for each tool was calculated for the different scenarios and overall. This value described the ability of each tool to correctly identify those patients that were deemed to have suffered major trauma, as defined by the Wessex Trauma Network. Free text comments were collected relating to the ease of use of each tool, together with the perceived benefits and disadvantages to clinicians. 2.5. Definitions
2.2. Case study scenarios Using real clinical cases as a basis, six clinical scenarios were developed, and these included information on the mechanism of injury, patient injuries and vital signs. To determine ecological validity of the cases, they were reviewed by a multidisciplinary panel of experienced clinicians, including Emergency Department (ED) doctors, senior nurses and paramedics (two representatives from each profession). For each scenario, a standardised outcome, in terms of destination, was agreed on by the panel through this process. See Fig. 2 for an example. 2.3. Sample Following organisational approval to conduct the study, permission to recruit ambulance staff during regular team training sessions was granted by the Ambulance Trusts’ Clinical Review Group. Staff were invited to participate during these sessions; it was not mandatory and none declined. The purpose of the study was explained and verbal agreement to participate sought. This approach ensured a range of skill grades, to include Ambulance Technician, Paramedic and Emergency Care Assistants (ECAs), and experience amongst the sample (Table 1). Using an alternate numbering technique around the room, participants were assigned to one of two groups; the ‘paper’ group and
If the expert panel agreed the destination was a Trauma Unit, an answer not matching this would lead to a patient being transferred to a specialist MTC – this would be classified as overtriage. If the expert panel agreed the destination was a Major Trauma Centre, an answer not matching this would lead to a decision to transfer to a Trauma Unit – this would be classified as undertriage. 3. Results A total of 50 participants took part in the study; 25 used the paper tool and 25 used the smartphone app. The majority, 64% (n = 32), were paramedics, with the remainder ambulance technicians (n = 10) and emergency care assistants (n = 8). Training on the TUB Tool had been received by 47 of the 50 participants (94%). Most had used it between 1 and 5 times and 12 had not used it in clinical practice (Fig. 3). Of those participants that own a smartphone (n = 43), 67% (n = 30) use Apple® iOS, with the remainder being users of the Android® operating system. The paper tool had a positive predictive value (PPV) of 0.76, with a range of 0.44 to 1.00 in each of the cases. The smartphone version on the TUB tool had a PPV of 0.86 with a range of 0.80–0.92. In three of the six scenarios the paper version outperformed the app, the app was more accurate, in that it had a higher PPV, in two scenarios and equal in one. However, in cases where the app showed
Please cite this article in press as: Eleanor S. Freshwater, Robert Crouch, Technology for trauma: testing the validity of a smartphone app for pre-hospital clinicians, International Emergency Nursing (2014), doi: 10.1016/j.ienj.2014.04.003
ARTICLE IN PRESS E.S. Freshwater and R. Crouch/International Emergency Nursing ■■ (2014) ■■–■■
4
35
No. participants
30 25 20 15 10 5 0 0 1-5 6-10 >10 No. times TUB tool used previously by participant in clinical practice Fig. 3. Participant experience of using the TUB tool in practice.
a higher PPV, the paper tool was significantly lower (0.56 and 0.44), whereas when the paper tool performed better, the app showed a closer result (0.80, 0.84 and 0.88). Both of the cases where the paper tool showed a low PPV, the agreed result was triage to a major trauma centre, this would have resulted in significant undertriage where the paper tool was used (Table 2). Free text responses on ease of use were grouped into ‘positive’ or ‘negative’ (Fig. 4). Sixty-eight per cent (n = 17) of comments on the paper tool were positive, with 5 negative comments relating to layout and appearance of the tool and 3 respondents making neutral or no comment. There were no negative comments relating to the
smartphone version of the tool, with 92% making positive comments and the remaining 2 participants making no or neutral comment. Many of the comments relating to the smartphone app were regarding the decrease in time taken to make a decision in comparison to the paper app, which was felt to be a positive benefit in a time-critical scenario. Participants made general comments about the tool that they had used. Some were concerned that there may be infection control issues around using a piece of technology, particularly whilst wearing gloves. Other comments related to the battery life of the mobile telephone and the possibility of it not always working.
Table 2 Results. Case
1 2 3 4 5 6 Total
Agreed destination
TU MTC MTC TU TU MTC
Paper
App
Result of incorrect answer
Not-matched
Matched
PPV
Not-matched
Matched
PPV
4 9 0 5 2 14 34
21 14 25 20 23 11 114
0.84 0.56 1.00 0.80 0.92 0.44 0.76
5 2 4 5 3 2 21
20 23 21 20 22 23 129
0.80 0.92 0.84 0.80 0.88 0.92 0.86
Overtriage Undertriage Undertriage Overtriage Overtriage Undertriage
25
No. comments
20 Paper
15
App
10 5 0 Positive
Negative
Neutral/Nil
Classification of comments Fig. 4. Comments relating to ease of use of the paper and app tools.
Please cite this article in press as: Eleanor S. Freshwater, Robert Crouch, Technology for trauma: testing the validity of a smartphone app for pre-hospital clinicians, International Emergency Nursing (2014), doi: 10.1016/j.ienj.2014.04.003
ARTICLE IN PRESS E.S. Freshwater and R. Crouch/International Emergency Nursing ■■ (2014) ■■–■■
4. Discussion This study demonstrated a PPV of 0.86 for the new smartphone decision support tool, with the original paper-based tool showing a PPV of 0.76 in the same scenarios. The purpose of this study was to test the digital representation of the major trauma triage tool against the paper version. Exploring the sensitivity and specificity of the TUB tool in predicting major trauma was not studied as part of this evaluation. The findings suggest that the smartphone version of the tool performed at least as well as the paper-based tool. One of the benefits of utilising an electronic triage tool available as an app is improved version control. If a change is made to any of the triage criteria, the app can be amended and an update made available in a short time frame. The most commonly used operating systems now offer functionality to automatically update the smartphone. This ensures that clinicians will always be using the most up-to-date tool, avoiding confusion at the scene of an incident. In contrast, updating a paper-based tool, particularly where clinicians are geographically spread and often lone-working, presents challenges. It relies on each clinician being made aware of changes, and requires a formal system of version control. Morris et al. (2013) tested two smartphone applications for use in calculating fluid resuscitation in burns patients and found that they were much quicker to provide a result than the calculator previously available, albeit associated with a decrease in ease of use. Initially, there were some organisational concerns relating to information governance and storage of information on staff personal mobile devices. The app does not obtain any patient identifiable data, and the developers and designers worked with the relevant stakeholders to ensure that the app did not breach any conditions of confidentiality. Several papers describe the testing of applications developed to be used directly by patients, assessing the design and usability of the interface (Ben-Zeev et al., 2013; Brouillette et al., 2013; Robinson et al., 2013), whilst others focus on the technical performance against existing hardware (Mellone et al., 2012; Petersen et al., 2013). Suk Nam et al. (2014) describe the design and implementation of a stroke screening app which suggests hospital destination, but they have not provided data on its effectiveness in practice. The regulation of applications used to guide clinical decisions is suggested by Visvanathan et al. (2012) in order to safeguard patients by ensuring information contained in them is authenticated. Shortly after the launch of the iPhone®, Burdette et al. (2008) identified the potential for the use of smartphone technology in clinical practice, in particular with reference to the integration of apps with web browsers and mapping functions. In this fast moving area of technology, enhancements continue to be made with faster processing speeds, greater memory and more numerous applications available. As smartphones become an increasing part of clinicians and patients’ lives, it is likely that the use of apps in clinical practice will become more prolific. In order to assure quality and safety, it is essential that robust testing is carried out and is available publicly for review. The app was launched via the iTunes® store on 7 October 2013 and had been downloaded 1351 times by 31 December 2013. As the Wessex Trauma Network TUB tool is also used in the Thames Valley, Peninsula and Severn Trauma Networks, it has now been able to be rolled out across the south central and south-west regions of England. It is not possible at this time to produce a generic trauma triage app for the whole of England as there is variance in the triage tool used in other areas. A Scandinavian study (Hagiwara et al., 2013) identified that the physical format of pre-hospital guidelines was the main barrier to usage and compliance in ambulance staff, with many clinicians constructing their own paper and electronic versions to improve ease
5
of use. The development of well-designed, stable, rapid and accessible smartphone apps has the potential to improve concordance with guideline usage and associated improvements in patient safety. The Android® version of the Major Trauma Triage Tool app has also recently been launched to widen availability to clinicians. There have been positive reviews of the app on iTunes and interest from local media, raising the profile of the work of the ambulance services and major trauma networks. Scope exists to expand the app in other areas where specialist treatment centres receive patients, such as hyper-acute stroke and primary percutaneous coronary intervention, providing enhanced decision-support for clinicians managing patients with serious medical conditions. It is possible that as electronic ambulance patient care records are utilised further within UK ambulance services that the functionality contained in the app becomes part of an integrated system of monitoring, recording and transmitting data relating to patient condition. This would allow enhanced pre-alerting of hospitals, in order that they can ensure appropriate specialists are available to manage severely injured patients as soon as they arrive in hospital. There are additional possibilities to enhance further versions of the Major Trauma Triage Tool app, such as data collection. This has been utilised successfully by other apps, in order to build a database that can provide information to assist in understanding epidemiology of disease (Michael and Geleta, 2013). There are potential applications in injury prevention and pre-hospital resourcing that could benefit from accurate information on location and severity of trauma incidents such as road traffic collisions and assaults. The issue of information governance is critical in order to ensure compliance with requirements for patient confidentiality. Novel means of achieving this, such as automatic deletion of images after a set time period, have been described by other clinical application developers (Takao et al., 2012). 4.1. Limitations 4.1.1. Data collection Certain demographic information was not collected from participants. It was not possible to make a comparison between the testing groups in terms of their age, gender, experience in the ambulance service or other factors that may influence their use of particular tools. 4.1.2. Ecological validity This was a scenario-based study, and not all information could be provided. Participants were advised that if information was not noted, to assume it was normal, many comments suggest they didn’t always comply with this. For example, they wrote ‘possible major pelvis fracture due to mechanism’ or ‘answered “no” to airway and haemorrhage question as unsure’. This means that it is not possible to determine whether a Trauma Unit was selected as it was nearer for a severely injured patient or if the patient was undertriaged to this type of facility. The purpose of this evaluation was to test the App against the paper tool to ensure it performs as well. Therefore, comparisons cannot be made with real incidents, as there would be many more factors to consider such as environment, human factors and time pressures. 4.1.3. Implementation Our study did not measure time to reach a decision on transport destination for trauma patients in comparison to the paperbased tool although this could be an area for additional investigation, particularly pertinent when managing time-critical patients.
Please cite this article in press as: Eleanor S. Freshwater, Robert Crouch, Technology for trauma: testing the validity of a smartphone app for pre-hospital clinicians, International Emergency Nursing (2014), doi: 10.1016/j.ienj.2014.04.003
ARTICLE IN PRESS E.S. Freshwater and R. Crouch/International Emergency Nursing ■■ (2014) ■■–■■
6
A limitation with smartphone technology is that certain functions rely on the signal provided by the carrier network. A lack of signal would prevent the global positioning system (GPS) element of the app being utilised. However, the triage tool itself and the means to record handover information are unaffected by this. 5. Conclusion The Smartphone app performed at least as well as the paperbased version of the Wessex Trauma Network Trauma Unit Bypass Tool. It was therefore considered safe to release a general version of the app. The app is now being used within the Network. Acknowledgements Wessex Trauma Network for their support in designing and implementing the app and use of their Trauma Unit Bypass Tool. Volatile State Ltd for the technical development of the Major Trauma Triage Tool app. Mark Ainsworth-Smith and the staff of South Central Ambulance Service NHS Foundation Trust for their assistance in testing the app. References Ben-Zeev, D., Kaiser, S.M., Brenner, C.J., Begale, M., Duffecy, J., Mohr, D.C., 2013. Development and usability testing of focus: A smartphone system for selfmanagement of schizophrenia. Psychiatric Rehabilitation Journal. 36 (4), 289–296. Brouillette, R.M., Foil, H., Fontenot, S., Correro, A., Allen, R., Martin, C.K., et al., 2013. Feasibility, reliability, and validity of a smartphone based application for the assessment of cognitive function in the elderly. PLoS ONE. 8 (6). Burdette, S.D., Herchline, T.E., Oehler, R., 2008. Surfing the web: Practicing medicine in a technological age: Using smartphones in clinical practice. Clinical Infectious Diseases. 47 (1), 117–122. Cameron, P.A., Gabbe, B.J., Cooper, D.J., Walker, T., Judson, R., McNeil, J., 2008. A statewide system of trauma care in Victoria: Effect on patient survival. The Medical Journal of Australia. 189 (10), 546–550. Celso, B., Tepas, J., Langland-Orban, B., Pracht, E., Papa, L., Lottenberg, L., et al., 2006. A systematic review and meta-analysis comparing outcome of severely injured patients treated in trauma centers following the establishment of trauma systems. The Journal of Trauma. 60 (2), 371–378. Department of Health (2012) Major trauma centres to open across England, Available from:
Fisher, A., Ross, C., Henderson, C., Kirk, S., Feroze, M., Richmond, R., 2010. Major Trauma Care in England, National Audit Office, London. Hagiwara, M.A., Suserud, B.-O., Jonsson, A., Henricson, M., 2013. Exclusion of context knowledge in the development of prehospital guidelines: Results produced by realistic evaluation. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine. 21 (46). Henry, M.C., Alicandro, J.M., Hollander, J.E., Moldashel, J.G., Cassara, G., Thode, H.C., Jr., 1996. Evaluation of American College of Surgeons Trauma Triage Criteria in a suburban and rural setting. The American Journal of Emergency Medicine. 14 (2), 124–129. Lansink, K.W.W., Leenen, L.P.H., 2007. Do designated trauma systems improve outcome? Current Opinion in Critical Care. 13 (6), 686–690. MacKenzie, E.J., Rivara, F.P., Jurkovich, G.J., Nathens, A.B., Frey, K.P., Egleston, B.L., et al., 2006. A national evaluation of the effect of trauma-center care on mortality. The New England Journal of Medicine. 354 (4), 366–378. Mellone, S., Tacconi, C., Chiari, L., 2012. Gait & posture. Gait and Posture. 36 (1), 163–165. Michael, B.D., Geleta, D., 2013. Development of clickclinica: A novel smartphone application to generate real-time global disease surveillance and clinical practice data. BMC Medical Informatics and Decision Making. 13 (1). Morris, R., Javed, M., Bodger, O., Gorse, S.H., Williams, D., 2013. ScienceDirect. Burns: Journal of the International Society for Burn Injuries. 1–9. Newgard, C.D., Zive, D., Holmes, J.F., Bulger, E.M., Staudenmayer, K., Liao, M., et al., 2011. A Multisite assessment of the American College of Surgeons Committee on trauma field triage decision scheme for identifying seriously injured children and adults. Journal of the American College of Surgeons. 213 (6), 709–721. NHS England (2013) Independent review of major trauma networks reveals increase in patient survival rates. Available from:
Please cite this article in press as: Eleanor S. Freshwater, Robert Crouch, Technology for trauma: testing the validity of a smartphone app for pre-hospital clinicians, International Emergency Nursing (2014), doi: 10.1016/j.ienj.2014.04.003