- Email: [email protected]
Assessing the accuracy of an automated atrial fibrillation detection algorithm using smartphone technology: The iREAD Study
Assessing the accuracy of an automated atrial fibrillation detection algorithm using smartphone technology: The iREAD Study Amila D. William, MD,* Majd Kanbour, MD,† Thomas Callahan, MD, FHRS,* Mandeep Bhargava, MD, FHRS,* Niraj Varma, MD, PhD, FHRS,* John Rickard, MD, FHRS,* Walid Saliba, MD, FHRS,* Kathy Wolski, MPH,‡ Ayman Hussein, MD, FHRS,* Bruce D. Lindsay, MD, FHRS,* Oussama M. Wazni, MD, FHRS,* Khaldoun G. Tarakji, MD, MPH, FHRS* From the *Department of Cardiovascular Medicine, Cleveland Clinic, Cleveland, Ohio, †Department of Cardiovascular Medicine, Marshall University, Huntington, West Virginia, and ‡Cleveland Clinic Coordinating Center for Clinical Research, Cleveland Clinic, Cleveland, Ohio. BACKGROUND The Kardia Mobile Cardiac Monitor (KMCM) detects atrial fibrillation (AF) via a handheld cardiac rhythm recorder and AF detection algorithm. The algorithm operates within predefined parameters to provide a “normal” or “possible atrial fibrillation detected” interpretation; outside of these parameters, an “unclassified” rhythm is reported. The system has been increasingly used, but its performance has not been independently tested. OBJECTIVE The objective of this study was to evaluate whether the KMCM system can accurately detect AF. METHODS A single-center, adjudicator-blinded case series of 52 consecutive patients with AF admitted for antiarrhythmic drug initiation were enrolled. Serial 12-lead electrocardiograms (ECGs) and nearly simultaneously acquired KMCM recordings were obtained. RESULTS There were 225 nearly simultaneously acquired KMCM and ECG recordings across 52 enrolled patients (mean age 68 years; 67% male). After exclusion of unclassified recordings, the KMCM automated algorithm interpretation had 96.6% sensitivity and 94.1%
Introduction
Because of the high prevalence of atrial fibrillation (AF) and significant health care cost of its management,1–4 ambulatory detection has become an area of focus in the cardiovascular application of mobile health technology. However, given the paroxysmal and frequently subclinical nature of AF, diagnosis may be elusive. AF detection and characterization
Dr Varma serves on the advisory board of and as a consultant to Medtronic and Abbott; he is on speakers bureau for Biotronik. Dr Tarakji serves on the advisory board of Medtronic and AliveCor. Other authors report no conflicts of interest. ClinicalTrials.gov ID: NCT02214069. Address reprint requests and correspondence: Dr Khaldoun G. Tarakji, Section of Cardiac Pacing and Electrophysiology, Heart and Vascular Institute, Cleveland Clinic, 9500 Euclid Avenue / J2-2, Cleveland, OH 44195. E-mail address: [email protected].
1547-5271/$-see front matter © 2018 Heart Rhythm Society. All rights reserved.
specificity for AF detection as compared with physicianinterpreted ECGs, with a k coefficient of 0.89. Physicianinterpreted KMCM recordings had 100% sensitivity and 89.2% specificity for AF detection as compared with physician-interpreted ECGs, with a k coefficient of 0.85. Sixty-two recordings (27.6%) were unclassified by the KMCM algorithm. In these instances, physician interpretation of KMCM recordings had 100% sensitivity and 79.5% specificity for AF detection as compared with 12-lead ECG interpretation, with a k coefficient of 0.71. CONCLUSION The KMCM system provides sensitive and specific AF detection relative to 12-lead ECGs when an automated interpretation is provided. Direct physician review of KMCM recordings can enhance diagnostic yield, especially for unclassified recordings. KEYWORDS Atrial fibrillation; Cardiac rhythm monitoring; Digital health; Mobile health; Smartphone (Heart Rhythm 2018;-:1–5) All rights reserved.
©
2018 Heart Rhythm Society.
have bearings on medical decision making regarding thromboembolism prophylaxis and suppressive therapy or rate control. In those patients who have undergone ablation therapy or who are receiving antiarrhythmic therapy, assessment of ambulatory AF is crucial to adjudicating therapeutic efficacy and guiding future management.5 The Kardia Mobile Cardiac Monitor (KMCM; AliveCor, Mountain View, CA) is a handheld, smartphone-coupled, 2-electrode cardiac rhythm recorder that enables patients to record a rhythm strip equivalent to lead I for 30 seconds. The electrocardiogram (ECG) rhythm strip can be shared with a physician for review via a Health Insurance Portability and Accountability Act of 1996–compliant Internet site. Compared with the traditional transtelephonic monitor, an earlier version of the KMCM without automated rhythm analysis was shown to be effective in monitoring patients after AF https://doi.org/10.1016/j.hrthm.2018.06.037
Heart Rhythm, Vol -, No -, - 2018
2 ablation with excellent accuracy of the physician-interpreted transmissions and high patient satisfaction.6 More recently, the KMCM was paired with an automated rhythm adjudication algorithm for the diagnosis of AF to provide simple and instantaneous AF detection. The algorithm labels a recording as “normal” or “possible atrial fibrillation detected” using a Random Forest machine learning model that uses a collection of approximately 50 specific parameters calculated from the rhythm strip, including RR interval statistics, morphological characteristics, signal quality, and frequency domain features. The model was trained on normal and abnormal rhythm strips classified by human overreaders. If the algorithm cannot label a recording at a sufficient confidence level or if the calculated heart rate is less than 50 or greater than 100 beats/min and regular, the recording is labeled “unclassified.” Short recordings less than 30 seconds in duration are excluded from the algorithm analysis and are labeled “unclassified.” While automated arrhythmia analysis such as that offered by the KMCM engenders potential for enhanced diagnostics and tailored therapeutics, its implementation into clinical practice is contingent on multiple factors, perhaps most importantly, the fidelity of the data acquisition and the veracity of ensuing analyses.7,8 The primary objectives of this study were to examine whether the commercially available KMCM with its AF detection algorithm can accurately differentiate sinus rhythm from AF as compared with nearly simultaneously acquired physician-interpreted 12-lead ECGs. The correlation between KMCM automated algorithm detection and physician-interpreted nearly simultaneously acquired 12lead ECGs and the correlation between physicianinterpreted KMCM recordings and physician-interpreted simultaneous 12-lead ECGs were assessed to characterize the quality of the KMCM automated AF detection algorithm and the quality of the KMCM recordings, respectively. Patient feedback on the KMCM was also assessed.
Methods Study design The iRead Study was a single-center, nonrandomized, and adjudicator-blinded study designed to evaluate the accuracy of the KMCM automated algorithm for the detection of AF. AliveCor provided the KMCMs coupled with a Wi-Fi– enabled smart device (iPod, Apple Inc., Cupertino, CA) for use in the study. AliveCor was not involved in the design, implementation, data analysis, or manuscript preparation of the study. The study was approved by the Cleveland Clinic’s Institutional Review Board.
Study participants Patients with a diagnosis of AF who were admitted for antiarrhythmic drug initiation (dofetilide or sotalol) were screened for enrollment. Inclusion criteria included male or female patients, aged 35–85 years with a history of paroxysmal or persistent AF, with baseline corrected QT interval less than 470 or 500 ms if the QRS duration was greater
than 120 ms. Patients with pacemakers or defibrillators were excluded. All enrolled patients provided written informed consent. Enrolled patients were provided with a KMCM paired with an iPod at the time of their admission for antiarrhythmic drug initiation. Dofetilide or sotalol were administered twice daily for 6 monitored doses during admission, with 12-lead ECG recordings performed 2 hours after each dose. Patients who were in AF after the fourth dose underwent electrical cardioversion. Patients were instructed to perform a 30-second recording corresponding to a lead I ECG rhythm strip by placing at least 1 finger from each hand on the electrodes immediately after each 12-lead ECG recording. The rhythm strip was automatically analyzed using the KMCM algorithm. The algorithm generates an interpretation of “normal,” “possible atrial fibrillation detected,” or “unclassified.” The recorded rhythm strips were then automatically transferred to AliveCor’s Health Insurance Portability and Accountability Act of 1996–compliant cloud server and were downloaded and printed for review. All 12-lead ECGs and KMCM recordings were independently reviewed by blinded electrophysiologists who classified the rhythm as sinus rhythm, AF, or noninterpretable.
Statistical analysis Sensitivity and specificity were calculated for KMCM automated interpretation compared with physician-interpreted 12-lead ECGs, for physician-interpreted KMCM transmission compared with physician-interpreted 12-lead ECGs, and for KMCM automated interpretation compared with physician-interpreted KMCM recordings. k coefficients for interobserver agreement were calculated. k coefficients greater than 0.8 were considered to represent excellent agreement. AF and atrial flutter were considered as a single disease state for all interpretations.
Results Fifty-two patients were enrolled in the study from August 18th 2015 through July 1st 2016. The clinical characteristics of the study population are summarized in Table 1. To test the accuracy of the KMCM automated algorithm for the detection of AF, KMCM automated rhythm interpretation and physician-interpreted 12-lead ECG readings were compared. There were 225 simultaneous 12-lead ECG and KMCM recordings. Of these, 62 recordings (27.6%) were “unclassified” by the KMCM algorithm and 2 ECGs were noninterpretable by the interpreting physicians. Of the remaining 161 interpretable simultaneous recordings, KMCM automated algorithm interpretation had 96.6% sensitivity and 94.1% specificity for the detection of AF as compared with physician-interpreted 12-lead ECGs, with a k coefficient of 0.89 (95% confidence interval 0.82–0.97) (Table 2). Of the 225 simultaneous recordings, 28.8% of ECG-determined AF was not detected by the KMCM algorithm; 91.3% of these were due to “unclassified” recordings by the KMCM algorithm.
William et al
iRead Study
3
Table 1 Demographic and baseline characteristics of the study population (N 5 52) Characteristic
Value
Age, average (min-max) (y) Sex Male Female Type of atrial fibrillation Paroxysmal Persistent Symptoms Palpitations Shortness of breath Lightheadedness Chest pain Fatigue Anticoagulation Coumadin Dabigatran Rivaroxaban Apixaban Antiarrhythmic drug Dofetilide Sotalol
68.1 (42.6–85.6) 35 (67.3) 17 (32.7) 11 (21.2) 41 (78.8) 22 (42.3) 34 (65.4) 9 (17.3) 3 (5.8) 27 (51.9) 20 (38.5) 3 (5.8) 9 (17.3) 20 (38.5) 41 (78.8) 11 (21.2)
Values are presented as n (%) unless indicated otherwise.
Discussion
To assess the quality of the KMCM rhythm recordings and transmission, physician-interpreted KMCM recordings and 12-lead ECGs were compared. Of the 225 simultaneous 12-lead ECG and KMCM recordings, 9 KMCM recordings and 2 ECGs were noninterpretable by the blinded physicians. Of the remaining 214 simultaneous recordings, physician interpretation of the KMCM recording had 100% sensitivity and 89.2% specificity for the detection of AF as compared with physician-interpreted 12-lead ECGs, with a k coefficient of 0.85 (95% confidence interval 0.78–0.92) (Table 3). To account for the quality of the process of manual recording by patients, the KMCM automated algorithm interpretation was compared with physician interpretation of the same KMCM recordings. As noted, 62 recordings were “unclassified” by the KMCM algorithm. Four of the remaining recordings were noninterpretable by the physicians. Of the remaining 159 recordings, KMCM automated algorithm interpretation had 92.4% sensitivity and 97.8% specificity for the detection of AF as compared with physician-interpreted
Table 2
KMCM recordings, with a k coefficient of 0.91 (95% confidence interval 0.84-0.97) (Table 4). Of the 62 algorithm “unclassified” KMCM recordings, 28 (45.2%) had heart rates less than 50 or greater than 100 beats/min, 17 (27.4%) had significant noise, and 6 (9.7%) were less than 30 seconds in duration. The cause underlying “unclassified” KMCM adjudication was not immediately evident in the remaining 11 of 62 recordings (17.7%). Of these 62 “unclassified” recordings, 5 were noninterpretable by the physicians. In the remaining 57 recordings, physician KMCM recording interpretation had 100% sensitivity and 79.5% specificity for the detection of AF as compared with 12-lead ECG interpretation, with a k coefficient of 0.71, a false-positive rate of 20.5%, and a falsenegative rate of 0% (Table 5). The majority of patients (93.6%) found the KMCM easy to use, and 59.6% noted that the use of the KMCM subjectively lessened AF diagnosis–related anxiety. Of the survey responders, 63.8% preferred continued use of the KMCM for AF detection.
The application of mobile health technology toward ambulatory cardiac rhythm monitoring is becoming germane to cardiovascular care delivery.7 AF is a widespread chronic and relapsing condition with an unmet need for disease tracking and diagnosis. Like other chronic conditions, such as diabetes or hypertension, monitoring of disease-related metrics is essential to assess the efficacy of therapies and to plan management. Contemporary innovation in biometric devices has aimed to expand and automate these diagnostic capabilities at the consumer level, extending ambulatory arrhythmia analysis beyond those patients prescribed short-term surface ECG recordings and those with implanted cardiac electronic devices. The KMCM is a commercially available portable cardiac rhythm monitoring device that can be paired with a smartphone and allows patient-initiated ambulatory cardiac rhythm recording for an extended period of time. In contrast, traditional options for monitoring AF suffer from limited duration in the case of Holter and event monitors and from requiring invasive procedures with limited longevity in the case of implantable loop recorders. The iTransmit study
KMCM automated algorithm interpretation compared with physician-interpreted 12-lead ECGs
KMCM automated algorithm interpretation Sinus AF Noninterpretable Total
Physician-interpreted 12-lead ECGs Sinus
AF
Noninterpretable
Total
96 6 41 143
2 57 21 80
1 1 0 2
99 64 62 225
Sensitivity, specificity, and k coefficient are calculated only for the simultaneous transmission with interpretation (shown in bold). The k coefficient is 0.89 (95% confidence interval 0.82–0.97) for numbers in bold. AF 5 atrial fibrillation; ECG 5 electrocardiogram; KMCM 5 Kardia Mobile Cardiac Monitor.
Heart Rhythm, Vol -, No -, - 2018
4 Table 3
Physician-interpreted KMCM recordings compared with simultaneous physician-interpreted 12-lead ECGs Physician-interpreted 12-lead ECGs
Physician-interpreted KMCM recordings
Sinus
AF
Noninterpretable
Total
Sinus AF Noninterpretable Total
124 15 4 143
0 75 5 80
0 2 0 2
124 92 9 225
Sensitivity, specificity, and k coefficient are calculated only for the simultaneous transmission with interpretation (shown in bold). The k coefficient is 0.85 (95% confidence interval 0.78–0.92) for numbers in bold. AF 5 atrial fibrillation; ECG 5 electrocardiogram; KMCM 5 Kardia Mobile Cardiac Monitor.
demonstrated that the KMCM recordings correlated well with the interpretation of simultaneous recordings using the traditional transtelephonic monitor in detecting AF after pulmonary vein isolation procedures,6 and KMCM has been shown to be beneficial in screening patients for AF in the community at large.9,10 An automated rhythm adjudication algorithm for the detection of AF based on machine learning has recently been added to the KMCM system. Accurate real-time adjudication has the potential of facilitating AF awareness and allows for direct patient feedback. The KMCM AF detection algorithm has been developed and tested by Kardia via comparison of single 12-lead ECGs and KMCM readings separated by less than 6 hours.11 This study is the first independent validation of the KMCM system in the clinical setting using nearly simultaneously acquired 12-lead ECGs. Assessment of the accuracy of the KMCM system in this study indicates, in instances in which the KMCM system is able to provide a “normal” or “possible atrial fibrillation detected” interpretation, excellent agreement with physician interpretation of nearly simultaneously acquired 12-lead ECGs. The false-negative detection rate was low (3.4%). In addition to providing an instantaneous rhythm interpretation, the KMCM system is able to transmit a recording to a secure server where the recordings can be directly reviewed. Direct physician interpretation of KMCM recordings had excellent agreement with nearly simultaneously acquired 12-lead ECGs. The false-negative detection rate was negligible. However, 27.6% of recordings (62 of 225) were “unclassified” by the automated detection algorithm and 28.8% of 12-lead ECG–adjudicated AF was not detected by the automated algorithm. On review, the majority (82.3%) of these
Table 4
fell outside the predefined bounds for algorithm application (eg, recordings ,30 seconds in duration, heart rate ,50 or .100 beats/min, or noise). Specifically for “unclassified” KMCM recordings, direct physician interpretation had good sensitivity and specificity compared with 12-lead ECG interpretation and captured all 12-lead ECG– adjudicated AF diagnoses with a zero false-negative detection rate. These findings suggest that when the KMCM automated algorithm provides an AF interpretation, clinicians should have a high clinical suspicion for the presence of true AF. Furthermore, a strong correlation to 12-lead ECGs can be expected in situations in which a clinician directly reviews KMCM recordings. This is of particular relevance when the automated algorithm yields an “unclassified” interpretation. A low false-negative detection rate was observed in a comparison of both KMCM automated diagnosis and physician KMCM recording interpretation with nearly simultaneously acquired 12-lead ECGs. Study participants generally found the KMCM easy to use. Furthermore, having access to the KMCM subjectively lessened AF diagnosis–related anxiety in the majority of patients. Given the predefined algorithm operating parameters and high rate of “unclassified” recordings with resultant missed AF instances, the KMCM algorithm is not suited to be a replacement for physician analysis. However, given its highly accurate performance when able to provide an interpretation, it holds potential as an adjunct to clinical decision making. The KMCM provides patients and clinicians with a noninvasive and longitudinal modality to assess the presence of AF. Using this technology, recordings are symptom
KMCM automated algorithm interpretation compared with physician interpretation of the same KMCM recordings
KMCM automated algorithm interpretation Sinus AF Noninterpretable Total
Physician interpretation of the KMCM recordings Sinus
AF
Noninterpretable
Total
91 2 31 124
5 61 26 92
3 1 5 9
99 64 62 225
Sensitivity, specificity, and k coefficient are calculated only for the simultaneous transmission with interpretation (shown in bold). The k coefficient is 0.91 (95% confidence interval 0.84–0.97) for numbers in bold. AF 5 atrial fibrillation; ECG 5 electrocardiogram; KMCM 5 Kardia Mobile Cardiac Monitor.
William et al
iRead Study
5
Table 5 Physician interpretation of the KMCM recordings not interpretable by the automated algorithm compared with physician-interpreted 12-lead ECGs Physician interpretation of the KMCM recordings not interpretable by the automated algorithm
Physician-interpreted 12-lead ECGs Sinus
AF
Noninterpretable
Total
Sinus AF Noninterpretable Total
31 8 2 41
0 18 3 21
0 0 0 0
31 26 5 62
Sensitivity, specificity, and k coefficient are calculated only for the simultaneous transmission with interpretation (shown in bold). The k coefficient is 0.71 (95% confidence interval 0.53–0.88) for numbers in bold. AF 5 atrial fibrillation; ECG 5 electrocardiogram; KMCM 5 Kardia Mobile Cardiac Monitor.
driven and are performed on an intermittent basis. This may be particularly appealing to patients with symptomatic paroxysmal AF or with paroxysmal undifferentiated palpitations. Patients can use an automated AF detection as a basis for pursuing additional medical follow-up, and clinicians may use it to develop treatment plans supported by more objective data rather than relying only on symptoms. While both the KMCM device and the AF detection algorithm are Food and Drug Administration approved, the regulation of mobile health tools and applications has largely been safety oriented, often without prospective and unbiased evidence-based assessment of efficacy.12 Given the trajectory of mobile health tools, analyses such as those presented in this study will likely come to be of increasing importance. For a wearable device to achieve its goals, there are many conditions that need to be fulfilled in addition to proper use and training in order to have data that could be processed by the machine.
Study limitations This was a single-center study with limited sample size. The study population in this case had a known history of AF with a burden sufficient to prompt admission for antiarrhythmic drug initiation. Algorithm performance would be expected to vary in a population with a lower AF burden. All recordings in this study were performed in patients admitted to the hospital. The quality of KMCM recordings may be more variable in patients in the ambulatory setting. Patients with cardiac implantable electronic devices were not included in this study, and further assessment of the KMCM system is needed in this population. Patients enrolled in this study had never used the device before. With more experience and frequent use, the quality of the rhythm transmissions may have improved and could affect the automated algorithm interpretation.
Conclusion KMCM automated analysis may be a useful adjunct to clinical decision making for the management of patients with AF. When the KMCM automated algorithm provides a rhythm interpretation, it is able to accurately detect AF with very good sensitivity and specificity and excellent interobserver
agreement as compared with 12-lead ECGs. However, many recordings were “unclassified” by the automated algorithm. Direct physician review of KMCM recordings has a strong correlation with that of nearly simultaneously acquired 12-lead ECGs for the detection AF, including instances in which the KMCM algorithm is unable to provide a diagnosis. Further studies are needed to assess the clinical outcomes and cost-effectiveness of using the KMCM in the long-term management of patients with AF.
References 1. Colilla S, Crow A, Petkun W, Singer DE, Simon T, Liu X. Estimates of current and future incidence and prevalence of atrial fibrillation in the U.S. adult population. Am J Cardiol 2013;112:1142–1147. 2. Mozaffarian D, Benjamin EJ, Go AS, et al. Heart disease and stroke statistics— 2015 update: a report from the American Heart Association. Circulation 2015; 131:e29–e322. 3. Chugh SS, Havmoeller R, Narayanan K, et al. Worldwide epidemiology of atrial fibrillation: a Global Burden of Disease 2010 Study. Circulation 2014; 129:837–847. 4. January CT, Wann LS, Alpert JS, et al. 2014 AHA/ACC/HRS guideline for the management of patients with atrial fibrillation: executive summary: a report of the American College of Cardiology/American Heart Association Task Force on practice guidelines and the Heart Rhythm Society. Circulation 2014; 130:2071–2104. 5. Calkins H, Hindricks G, Cappato R, et al. 2017 HRS/EHRA/ECAS/APHRS/ SOLAECE expert consensus statement on catheter and surgical ablation of atrial fibrillation. Heart Rhythm 2017;14:e275–e444. 6. Tarakji KG, Wazni OM, Callahan T, Kanj M, Hakim AH, Wolski K, Wilkoff BL, Saliba W, Lindsay BD. Using a novel wireless system for monitoring patients after the atrial fibrillation ablation procedure: the iTransmit study. Heart Rhythm 2015;12:554–559. 7. Turakhia MP, Desai SA, Harrington RA. The outlook of digital health for cardiovascular medicine: challenges but also extraordinary opportunities. JAMA Cardiol 2016;1:743–744. 8. Patel MS, Asch DA, Volpp KG. Wearable devices as facilitators, not drivers, of health behavior change. JAMA 2015;313:459–460. 9. Lowres N, Neubeck L, Salkeld G, et al. Feasibility and cost-effectiveness of stroke prevention through community screening for atrial fibrillation using iPhone ECG in pharmacies: the SEARCH-AF study. Thromb Haemost 2014; 111:1167–1176. 10. Halcox JPJ, Wareham K, Cardew A, et al. Assessment of remote heart rhythm sampling using the AliveCor heart monitor to screen for atrial fibrillation: the REHEARSE-AF Study. Circulation 2017;136:1784–1794. 11. Lau JK, Lowres N, Neubeck L, Brieger DB, Sy RW, Galloway CD, Albert DE, Freedman SB. iPhone ECG application for community screening to detect silent atrial fibrillation: a novel technology to prevent stroke. Int J Cardiol 2013; 165:193–194. 12. Powell AC, Landman AB, Bates DW. In search of a few good apps. JAMA 2014; 311:1851–1852.
Introduction
Because of the high prevalence of atrial fibrillation (AF) and significant health care cost of its management,1–4 ambulatory detection has become an area of focus in the cardiovascular application of mobile health technology. However, given the paroxysmal and frequently subclinical nature of AF, diagnosis may be elusive. AF detection and characterization
Dr Varma serves on the advisory board of and as a consultant to Medtronic and Abbott; he is on speakers bureau for Biotronik. Dr Tarakji serves on the advisory board of Medtronic and AliveCor. Other authors report no conflicts of interest. ClinicalTrials.gov ID: NCT02214069. Address reprint requests and correspondence: Dr Khaldoun G. Tarakji, Section of Cardiac Pacing and Electrophysiology, Heart and Vascular Institute, Cleveland Clinic, 9500 Euclid Avenue / J2-2, Cleveland, OH 44195. E-mail address: [email protected].
1547-5271/$-see front matter © 2018 Heart Rhythm Society. All rights reserved.
specificity for AF detection as compared with physicianinterpreted ECGs, with a k coefficient of 0.89. Physicianinterpreted KMCM recordings had 100% sensitivity and 89.2% specificity for AF detection as compared with physician-interpreted ECGs, with a k coefficient of 0.85. Sixty-two recordings (27.6%) were unclassified by the KMCM algorithm. In these instances, physician interpretation of KMCM recordings had 100% sensitivity and 79.5% specificity for AF detection as compared with 12-lead ECG interpretation, with a k coefficient of 0.71. CONCLUSION The KMCM system provides sensitive and specific AF detection relative to 12-lead ECGs when an automated interpretation is provided. Direct physician review of KMCM recordings can enhance diagnostic yield, especially for unclassified recordings. KEYWORDS Atrial fibrillation; Cardiac rhythm monitoring; Digital health; Mobile health; Smartphone (Heart Rhythm 2018;-:1–5) All rights reserved.
©
2018 Heart Rhythm Society.
have bearings on medical decision making regarding thromboembolism prophylaxis and suppressive therapy or rate control. In those patients who have undergone ablation therapy or who are receiving antiarrhythmic therapy, assessment of ambulatory AF is crucial to adjudicating therapeutic efficacy and guiding future management.5 The Kardia Mobile Cardiac Monitor (KMCM; AliveCor, Mountain View, CA) is a handheld, smartphone-coupled, 2-electrode cardiac rhythm recorder that enables patients to record a rhythm strip equivalent to lead I for 30 seconds. The electrocardiogram (ECG) rhythm strip can be shared with a physician for review via a Health Insurance Portability and Accountability Act of 1996–compliant Internet site. Compared with the traditional transtelephonic monitor, an earlier version of the KMCM without automated rhythm analysis was shown to be effective in monitoring patients after AF https://doi.org/10.1016/j.hrthm.2018.06.037
Heart Rhythm, Vol -, No -, - 2018
2 ablation with excellent accuracy of the physician-interpreted transmissions and high patient satisfaction.6 More recently, the KMCM was paired with an automated rhythm adjudication algorithm for the diagnosis of AF to provide simple and instantaneous AF detection. The algorithm labels a recording as “normal” or “possible atrial fibrillation detected” using a Random Forest machine learning model that uses a collection of approximately 50 specific parameters calculated from the rhythm strip, including RR interval statistics, morphological characteristics, signal quality, and frequency domain features. The model was trained on normal and abnormal rhythm strips classified by human overreaders. If the algorithm cannot label a recording at a sufficient confidence level or if the calculated heart rate is less than 50 or greater than 100 beats/min and regular, the recording is labeled “unclassified.” Short recordings less than 30 seconds in duration are excluded from the algorithm analysis and are labeled “unclassified.” While automated arrhythmia analysis such as that offered by the KMCM engenders potential for enhanced diagnostics and tailored therapeutics, its implementation into clinical practice is contingent on multiple factors, perhaps most importantly, the fidelity of the data acquisition and the veracity of ensuing analyses.7,8 The primary objectives of this study were to examine whether the commercially available KMCM with its AF detection algorithm can accurately differentiate sinus rhythm from AF as compared with nearly simultaneously acquired physician-interpreted 12-lead ECGs. The correlation between KMCM automated algorithm detection and physician-interpreted nearly simultaneously acquired 12lead ECGs and the correlation between physicianinterpreted KMCM recordings and physician-interpreted simultaneous 12-lead ECGs were assessed to characterize the quality of the KMCM automated AF detection algorithm and the quality of the KMCM recordings, respectively. Patient feedback on the KMCM was also assessed.
Methods Study design The iRead Study was a single-center, nonrandomized, and adjudicator-blinded study designed to evaluate the accuracy of the KMCM automated algorithm for the detection of AF. AliveCor provided the KMCMs coupled with a Wi-Fi– enabled smart device (iPod, Apple Inc., Cupertino, CA) for use in the study. AliveCor was not involved in the design, implementation, data analysis, or manuscript preparation of the study. The study was approved by the Cleveland Clinic’s Institutional Review Board.
Study participants Patients with a diagnosis of AF who were admitted for antiarrhythmic drug initiation (dofetilide or sotalol) were screened for enrollment. Inclusion criteria included male or female patients, aged 35–85 years with a history of paroxysmal or persistent AF, with baseline corrected QT interval less than 470 or 500 ms if the QRS duration was greater
than 120 ms. Patients with pacemakers or defibrillators were excluded. All enrolled patients provided written informed consent. Enrolled patients were provided with a KMCM paired with an iPod at the time of their admission for antiarrhythmic drug initiation. Dofetilide or sotalol were administered twice daily for 6 monitored doses during admission, with 12-lead ECG recordings performed 2 hours after each dose. Patients who were in AF after the fourth dose underwent electrical cardioversion. Patients were instructed to perform a 30-second recording corresponding to a lead I ECG rhythm strip by placing at least 1 finger from each hand on the electrodes immediately after each 12-lead ECG recording. The rhythm strip was automatically analyzed using the KMCM algorithm. The algorithm generates an interpretation of “normal,” “possible atrial fibrillation detected,” or “unclassified.” The recorded rhythm strips were then automatically transferred to AliveCor’s Health Insurance Portability and Accountability Act of 1996–compliant cloud server and were downloaded and printed for review. All 12-lead ECGs and KMCM recordings were independently reviewed by blinded electrophysiologists who classified the rhythm as sinus rhythm, AF, or noninterpretable.
Statistical analysis Sensitivity and specificity were calculated for KMCM automated interpretation compared with physician-interpreted 12-lead ECGs, for physician-interpreted KMCM transmission compared with physician-interpreted 12-lead ECGs, and for KMCM automated interpretation compared with physician-interpreted KMCM recordings. k coefficients for interobserver agreement were calculated. k coefficients greater than 0.8 were considered to represent excellent agreement. AF and atrial flutter were considered as a single disease state for all interpretations.
Results Fifty-two patients were enrolled in the study from August 18th 2015 through July 1st 2016. The clinical characteristics of the study population are summarized in Table 1. To test the accuracy of the KMCM automated algorithm for the detection of AF, KMCM automated rhythm interpretation and physician-interpreted 12-lead ECG readings were compared. There were 225 simultaneous 12-lead ECG and KMCM recordings. Of these, 62 recordings (27.6%) were “unclassified” by the KMCM algorithm and 2 ECGs were noninterpretable by the interpreting physicians. Of the remaining 161 interpretable simultaneous recordings, KMCM automated algorithm interpretation had 96.6% sensitivity and 94.1% specificity for the detection of AF as compared with physician-interpreted 12-lead ECGs, with a k coefficient of 0.89 (95% confidence interval 0.82–0.97) (Table 2). Of the 225 simultaneous recordings, 28.8% of ECG-determined AF was not detected by the KMCM algorithm; 91.3% of these were due to “unclassified” recordings by the KMCM algorithm.
William et al
iRead Study
3
Table 1 Demographic and baseline characteristics of the study population (N 5 52) Characteristic
Value
Age, average (min-max) (y) Sex Male Female Type of atrial fibrillation Paroxysmal Persistent Symptoms Palpitations Shortness of breath Lightheadedness Chest pain Fatigue Anticoagulation Coumadin Dabigatran Rivaroxaban Apixaban Antiarrhythmic drug Dofetilide Sotalol
68.1 (42.6–85.6) 35 (67.3) 17 (32.7) 11 (21.2) 41 (78.8) 22 (42.3) 34 (65.4) 9 (17.3) 3 (5.8) 27 (51.9) 20 (38.5) 3 (5.8) 9 (17.3) 20 (38.5) 41 (78.8) 11 (21.2)
Values are presented as n (%) unless indicated otherwise.
Discussion
To assess the quality of the KMCM rhythm recordings and transmission, physician-interpreted KMCM recordings and 12-lead ECGs were compared. Of the 225 simultaneous 12-lead ECG and KMCM recordings, 9 KMCM recordings and 2 ECGs were noninterpretable by the blinded physicians. Of the remaining 214 simultaneous recordings, physician interpretation of the KMCM recording had 100% sensitivity and 89.2% specificity for the detection of AF as compared with physician-interpreted 12-lead ECGs, with a k coefficient of 0.85 (95% confidence interval 0.78–0.92) (Table 3). To account for the quality of the process of manual recording by patients, the KMCM automated algorithm interpretation was compared with physician interpretation of the same KMCM recordings. As noted, 62 recordings were “unclassified” by the KMCM algorithm. Four of the remaining recordings were noninterpretable by the physicians. Of the remaining 159 recordings, KMCM automated algorithm interpretation had 92.4% sensitivity and 97.8% specificity for the detection of AF as compared with physician-interpreted
Table 2
KMCM recordings, with a k coefficient of 0.91 (95% confidence interval 0.84-0.97) (Table 4). Of the 62 algorithm “unclassified” KMCM recordings, 28 (45.2%) had heart rates less than 50 or greater than 100 beats/min, 17 (27.4%) had significant noise, and 6 (9.7%) were less than 30 seconds in duration. The cause underlying “unclassified” KMCM adjudication was not immediately evident in the remaining 11 of 62 recordings (17.7%). Of these 62 “unclassified” recordings, 5 were noninterpretable by the physicians. In the remaining 57 recordings, physician KMCM recording interpretation had 100% sensitivity and 79.5% specificity for the detection of AF as compared with 12-lead ECG interpretation, with a k coefficient of 0.71, a false-positive rate of 20.5%, and a falsenegative rate of 0% (Table 5). The majority of patients (93.6%) found the KMCM easy to use, and 59.6% noted that the use of the KMCM subjectively lessened AF diagnosis–related anxiety. Of the survey responders, 63.8% preferred continued use of the KMCM for AF detection.
The application of mobile health technology toward ambulatory cardiac rhythm monitoring is becoming germane to cardiovascular care delivery.7 AF is a widespread chronic and relapsing condition with an unmet need for disease tracking and diagnosis. Like other chronic conditions, such as diabetes or hypertension, monitoring of disease-related metrics is essential to assess the efficacy of therapies and to plan management. Contemporary innovation in biometric devices has aimed to expand and automate these diagnostic capabilities at the consumer level, extending ambulatory arrhythmia analysis beyond those patients prescribed short-term surface ECG recordings and those with implanted cardiac electronic devices. The KMCM is a commercially available portable cardiac rhythm monitoring device that can be paired with a smartphone and allows patient-initiated ambulatory cardiac rhythm recording for an extended period of time. In contrast, traditional options for monitoring AF suffer from limited duration in the case of Holter and event monitors and from requiring invasive procedures with limited longevity in the case of implantable loop recorders. The iTransmit study
KMCM automated algorithm interpretation compared with physician-interpreted 12-lead ECGs
KMCM automated algorithm interpretation Sinus AF Noninterpretable Total
Physician-interpreted 12-lead ECGs Sinus
AF
Noninterpretable
Total
96 6 41 143
2 57 21 80
1 1 0 2
99 64 62 225
Sensitivity, specificity, and k coefficient are calculated only for the simultaneous transmission with interpretation (shown in bold). The k coefficient is 0.89 (95% confidence interval 0.82–0.97) for numbers in bold. AF 5 atrial fibrillation; ECG 5 electrocardiogram; KMCM 5 Kardia Mobile Cardiac Monitor.
Heart Rhythm, Vol -, No -, - 2018
4 Table 3
Physician-interpreted KMCM recordings compared with simultaneous physician-interpreted 12-lead ECGs Physician-interpreted 12-lead ECGs
Physician-interpreted KMCM recordings
Sinus
AF
Noninterpretable
Total
Sinus AF Noninterpretable Total
124 15 4 143
0 75 5 80
0 2 0 2
124 92 9 225
Sensitivity, specificity, and k coefficient are calculated only for the simultaneous transmission with interpretation (shown in bold). The k coefficient is 0.85 (95% confidence interval 0.78–0.92) for numbers in bold. AF 5 atrial fibrillation; ECG 5 electrocardiogram; KMCM 5 Kardia Mobile Cardiac Monitor.
demonstrated that the KMCM recordings correlated well with the interpretation of simultaneous recordings using the traditional transtelephonic monitor in detecting AF after pulmonary vein isolation procedures,6 and KMCM has been shown to be beneficial in screening patients for AF in the community at large.9,10 An automated rhythm adjudication algorithm for the detection of AF based on machine learning has recently been added to the KMCM system. Accurate real-time adjudication has the potential of facilitating AF awareness and allows for direct patient feedback. The KMCM AF detection algorithm has been developed and tested by Kardia via comparison of single 12-lead ECGs and KMCM readings separated by less than 6 hours.11 This study is the first independent validation of the KMCM system in the clinical setting using nearly simultaneously acquired 12-lead ECGs. Assessment of the accuracy of the KMCM system in this study indicates, in instances in which the KMCM system is able to provide a “normal” or “possible atrial fibrillation detected” interpretation, excellent agreement with physician interpretation of nearly simultaneously acquired 12-lead ECGs. The false-negative detection rate was low (3.4%). In addition to providing an instantaneous rhythm interpretation, the KMCM system is able to transmit a recording to a secure server where the recordings can be directly reviewed. Direct physician interpretation of KMCM recordings had excellent agreement with nearly simultaneously acquired 12-lead ECGs. The false-negative detection rate was negligible. However, 27.6% of recordings (62 of 225) were “unclassified” by the automated detection algorithm and 28.8% of 12-lead ECG–adjudicated AF was not detected by the automated algorithm. On review, the majority (82.3%) of these
Table 4
fell outside the predefined bounds for algorithm application (eg, recordings ,30 seconds in duration, heart rate ,50 or .100 beats/min, or noise). Specifically for “unclassified” KMCM recordings, direct physician interpretation had good sensitivity and specificity compared with 12-lead ECG interpretation and captured all 12-lead ECG– adjudicated AF diagnoses with a zero false-negative detection rate. These findings suggest that when the KMCM automated algorithm provides an AF interpretation, clinicians should have a high clinical suspicion for the presence of true AF. Furthermore, a strong correlation to 12-lead ECGs can be expected in situations in which a clinician directly reviews KMCM recordings. This is of particular relevance when the automated algorithm yields an “unclassified” interpretation. A low false-negative detection rate was observed in a comparison of both KMCM automated diagnosis and physician KMCM recording interpretation with nearly simultaneously acquired 12-lead ECGs. Study participants generally found the KMCM easy to use. Furthermore, having access to the KMCM subjectively lessened AF diagnosis–related anxiety in the majority of patients. Given the predefined algorithm operating parameters and high rate of “unclassified” recordings with resultant missed AF instances, the KMCM algorithm is not suited to be a replacement for physician analysis. However, given its highly accurate performance when able to provide an interpretation, it holds potential as an adjunct to clinical decision making. The KMCM provides patients and clinicians with a noninvasive and longitudinal modality to assess the presence of AF. Using this technology, recordings are symptom
KMCM automated algorithm interpretation compared with physician interpretation of the same KMCM recordings
KMCM automated algorithm interpretation Sinus AF Noninterpretable Total
Physician interpretation of the KMCM recordings Sinus
AF
Noninterpretable
Total
91 2 31 124
5 61 26 92
3 1 5 9
99 64 62 225
Sensitivity, specificity, and k coefficient are calculated only for the simultaneous transmission with interpretation (shown in bold). The k coefficient is 0.91 (95% confidence interval 0.84–0.97) for numbers in bold. AF 5 atrial fibrillation; ECG 5 electrocardiogram; KMCM 5 Kardia Mobile Cardiac Monitor.
William et al
iRead Study
5
Table 5 Physician interpretation of the KMCM recordings not interpretable by the automated algorithm compared with physician-interpreted 12-lead ECGs Physician interpretation of the KMCM recordings not interpretable by the automated algorithm
Physician-interpreted 12-lead ECGs Sinus
AF
Noninterpretable
Total
Sinus AF Noninterpretable Total
31 8 2 41
0 18 3 21
0 0 0 0
31 26 5 62
Sensitivity, specificity, and k coefficient are calculated only for the simultaneous transmission with interpretation (shown in bold). The k coefficient is 0.71 (95% confidence interval 0.53–0.88) for numbers in bold. AF 5 atrial fibrillation; ECG 5 electrocardiogram; KMCM 5 Kardia Mobile Cardiac Monitor.
driven and are performed on an intermittent basis. This may be particularly appealing to patients with symptomatic paroxysmal AF or with paroxysmal undifferentiated palpitations. Patients can use an automated AF detection as a basis for pursuing additional medical follow-up, and clinicians may use it to develop treatment plans supported by more objective data rather than relying only on symptoms. While both the KMCM device and the AF detection algorithm are Food and Drug Administration approved, the regulation of mobile health tools and applications has largely been safety oriented, often without prospective and unbiased evidence-based assessment of efficacy.12 Given the trajectory of mobile health tools, analyses such as those presented in this study will likely come to be of increasing importance. For a wearable device to achieve its goals, there are many conditions that need to be fulfilled in addition to proper use and training in order to have data that could be processed by the machine.
Study limitations This was a single-center study with limited sample size. The study population in this case had a known history of AF with a burden sufficient to prompt admission for antiarrhythmic drug initiation. Algorithm performance would be expected to vary in a population with a lower AF burden. All recordings in this study were performed in patients admitted to the hospital. The quality of KMCM recordings may be more variable in patients in the ambulatory setting. Patients with cardiac implantable electronic devices were not included in this study, and further assessment of the KMCM system is needed in this population. Patients enrolled in this study had never used the device before. With more experience and frequent use, the quality of the rhythm transmissions may have improved and could affect the automated algorithm interpretation.
Conclusion KMCM automated analysis may be a useful adjunct to clinical decision making for the management of patients with AF. When the KMCM automated algorithm provides a rhythm interpretation, it is able to accurately detect AF with very good sensitivity and specificity and excellent interobserver
agreement as compared with 12-lead ECGs. However, many recordings were “unclassified” by the automated algorithm. Direct physician review of KMCM recordings has a strong correlation with that of nearly simultaneously acquired 12-lead ECGs for the detection AF, including instances in which the KMCM algorithm is unable to provide a diagnosis. Further studies are needed to assess the clinical outcomes and cost-effectiveness of using the KMCM in the long-term management of patients with AF.
References 1. Colilla S, Crow A, Petkun W, Singer DE, Simon T, Liu X. Estimates of current and future incidence and prevalence of atrial fibrillation in the U.S. adult population. Am J Cardiol 2013;112:1142–1147. 2. Mozaffarian D, Benjamin EJ, Go AS, et al. Heart disease and stroke statistics— 2015 update: a report from the American Heart Association. Circulation 2015; 131:e29–e322. 3. Chugh SS, Havmoeller R, Narayanan K, et al. Worldwide epidemiology of atrial fibrillation: a Global Burden of Disease 2010 Study. Circulation 2014; 129:837–847. 4. January CT, Wann LS, Alpert JS, et al. 2014 AHA/ACC/HRS guideline for the management of patients with atrial fibrillation: executive summary: a report of the American College of Cardiology/American Heart Association Task Force on practice guidelines and the Heart Rhythm Society. Circulation 2014; 130:2071–2104. 5. Calkins H, Hindricks G, Cappato R, et al. 2017 HRS/EHRA/ECAS/APHRS/ SOLAECE expert consensus statement on catheter and surgical ablation of atrial fibrillation. Heart Rhythm 2017;14:e275–e444. 6. Tarakji KG, Wazni OM, Callahan T, Kanj M, Hakim AH, Wolski K, Wilkoff BL, Saliba W, Lindsay BD. Using a novel wireless system for monitoring patients after the atrial fibrillation ablation procedure: the iTransmit study. Heart Rhythm 2015;12:554–559. 7. Turakhia MP, Desai SA, Harrington RA. The outlook of digital health for cardiovascular medicine: challenges but also extraordinary opportunities. JAMA Cardiol 2016;1:743–744. 8. Patel MS, Asch DA, Volpp KG. Wearable devices as facilitators, not drivers, of health behavior change. JAMA 2015;313:459–460. 9. Lowres N, Neubeck L, Salkeld G, et al. Feasibility and cost-effectiveness of stroke prevention through community screening for atrial fibrillation using iPhone ECG in pharmacies: the SEARCH-AF study. Thromb Haemost 2014; 111:1167–1176. 10. Halcox JPJ, Wareham K, Cardew A, et al. Assessment of remote heart rhythm sampling using the AliveCor heart monitor to screen for atrial fibrillation: the REHEARSE-AF Study. Circulation 2017;136:1784–1794. 11. Lau JK, Lowres N, Neubeck L, Brieger DB, Sy RW, Galloway CD, Albert DE, Freedman SB. iPhone ECG application for community screening to detect silent atrial fibrillation: a novel technology to prevent stroke. Int J Cardiol 2013; 165:193–194. 12. Powell AC, Landman AB, Bates DW. In search of a few good apps. JAMA 2014; 311:1851–1852.