- Email: [email protected]
Evaluation of Significant Coronary Artery Disease Based on CT Fractional Flow Reserve and Plaque Characteristics Using Random Forest Analysis in Machine Learning
ARTICLE IN PRESS
Original Investigation
Evaluation of Significant Coronary Artery Disease Based on CT Fractional Flow Reserve and Plaque Characteristics Using Random Forest Analysis in Machine Learning Tomohiro Kawasaki, MD, Masafumi Kidoh, MD, Teruhito Kido, MD, Daisuke Sueta, MD, Shinichiro Fujimoto, MD, Kanako Kunishima Kumamaru, MD, Teruyoshi Uetani, MD, Yuki Tanabe, MD, Toshio Ueda, RT, Daisuke Sakabe, RT, Seitaro Oda, MD, Tsuneo Yamashiro, MD, Kenichi Tsujita, MD, Shingo Kato, MD, Hideaki Yuki, MD, Daisuke Utsunomiya, MD
Rationale and Objectives: Fractional flow reserve (FFR) is an established technique for detecting lesion-specific ischemia but is invasive. Our objective was to investigate the effects of combined assessment of coronary CT angiography (CCTA) imaging features and CT-FFR on detecting lesion-specific ischemia by comparing with invasive FFR. Materials and Methods: Forty-seven patients who had 60 coronary vessels with 30% 90% stenosis were included. Six anatomic CCTA descriptors (Agatston score, stenosis severity, mean plaque CT attenuation value, noncalcified and calcified plaque volumes, remodeling index) and a functional descriptor (CT-FFR) were measured. Random forest was used to identify which descriptors were useful to identify ischemia-related lesion. Receiver-operating characteristic (ROC) curves were calculated for 2 models: i.e. Model-1 for anatomical CT descriptors and Model-2 for anatomical CT descriptors plus CT-FFR. Results: Stenosis severity (40.8 § 15.7% vs 57.6 § 14.1%), noncalcified plaque volume (190 § 100 vs 254.8 § 133.3), and remodeling index (1.04 § 0.12 vs 1.11 § 0.13) were significantly higher in ischemia-related lesions than nonischemia-related lesions. CT-FFR was 0.84 § 0.14 and 0.71 § 0.14, respectively, for ischemia-related and nonischemia-related lesions, and the difference was significant. The area under the ROC curve was 0.738 and 0.835 in Model-1 and Model-2, respectively. Reclassification of ischemic lesion risk was significantly improved after adding CT-FFR: net reclassification improvement was 0.297 and integrated discrimination improvement was 0.254. Conclusion: Combined assessment of anatomical CCTA features and functional CT-FFR was helpful for detecting lesion-specific ischemia. Key Words: Coronary artery disease; Coronary CT angiography; CT-derived fractional flow reserve; Machine learning; Plaque characteristics. © 2020 The Association of University Radiologists. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license. (http://creativecommons.org/licenses/by-nc-nd/4.0/) Abbreviations: AUC area under the receiver-operating characteristic curve, CAD coronary artery disease, CI confidence interval, CT computed tomography, CCTA coronary CT angiography, FFR fractional flow reserve, ROC receiver-operating characteristic
Acad Radiol 2020; &:1–9 From the Cardiovascular, Shin Koga Hospital, Kurume City, Fukuoka, Japan (T.K.); Diagnostic Radiology, Faculty of Life Sciences, Kumamoto University, Chuo-ku, Kumamoto, Japan (M.K., S.O., H.Y.); Department of Radiology, Ehime University, Shitsukawa, Toon, Ehime 791-0295, Japan (T.K., Y.T.); Department of Cardiovascular Medicine, Kumamoto University Hospital, Chuo-ku, Kumamoto City, Japan (D.S., K.T.); Cardiovascular Medicine, Juntendo University, Bunkyo-ku, Tokyo, Japan (S.F.); Department of Radiology, Juntendo University Graduate School of Medicine, Bunkyo-ku, Tokyo, Japan (K.K.K.); Cardiovascular Medicine, Ehime University, Toon, Ehime, Japan (T.U.); Central Radiology, Shin Koga Hospital, Kurume City, Fukuoka, Japan (T.U.); Central Radiology, Kumamoto University Hospital, Chuo-ku, Kumamoto, Japan (D.S.); Radiology, Graduate School of Medical Science, University of the Ryukyu, Nakagami-gun, Okinawa, Japan (T.Y.); Cardiology, Kanagawa Cardiovascular and Respiratory Center, Kanazawa-ku, Yokohama, Japan (S.K.); Diagnostic Radiology, Yokohama City University Graduate School of Medicine, Kanazawa-ku, Yokohama, Japan (D.U.). Received October 21, 2019; revised December 8, 2019; accepted December 17, 2019. Address correspondence to: T.K. e-mail: [email protected] © 2020 The Association of University Radiologists. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license. (http://creativecommons.org/licenses/by-nc-nd/4.0/) https://doi.org/10.1016/j.acra.2019.12.013
1
ARTICLE IN PRESS KAWASAKI ET AL
Academic Radiology, Vol &, No &&, && 2020
INTRODUCTION
Acquisition of CCTA Data
C
All patients underwent CCTA examination using a 320-row detector CT scanner (Aquilion One Vision or Aquilion One Genesis; Canon Medical Systems, Otawara, Japan). The CT protocol consisted of obtaining the calcium score followed by CCTA. The parameters used for CT scanning were as follows: prospective electrocardiogram-gatedaxial scans; 320 rows £ 0.5mm collimation; rotation time, 275 ms; tube voltage, 120 kV; and tube current, 350 650 mA (automatic exposure control). All patients underwent prospective ECG-gated CCTA with data acquisition over 70% 99% of the R-R interval. All patients received sublingual nitroglycerin. If the heart rate exceeded 65 bpm in the CT suite, 6 10 mg landiolol hydrochloride (Corebeta; Ono Pharmaceutical Co. Ltd., Osaka, Japan) was administered intravenously as an additional betablocker 5 minutes before CCTA scanning. For CCTA, contrast material with an iodine concentration of 370 mg/mL (Iopamiron 370; Bayer Health Care, Osaka, Japan) was administered via a 20-gauge catheter inserted into an antecubital vein using a double-head power injector. The start time of data acquisition was determined by a computer-assisted bolus-tracking program. The trigger threshold was set at 150 HU for the region of interest in the ascending aorta. Acquisition of the CT data was started 6 seconds after the trigger.
oronary artery disease (CAD) was still the leading cause of death globally in 2015 (1), and it is important to improve the prognosis in patients with CAD by appropriate diagnosis and treatment. Coronary computed tomography angiography (CCTA) is an established minimally invasive diagnostic technique and has high diagnostic accuracy for the evaluation of CAD (2). In 2016, the National Institute for Health and Care Excellence in the UK endorsed CCTA as the first-line investigation for patients with angina and no history of CAD (3). However, the functional assessment of lesion-specific ischemia is necessary to determine the appropriate treatment strategy in the patients with CAD, and anatomical stenosis on CCTA is not equal to the lesion-specific ischemia (4). Therefore, additional functional tests are recommended in patients with anatomically significant stenosis (50 %) on CCTA to identify the lesionspecific ischemia (5). Fractional flow reserve (FFR), which is measured by a coronary pressure wire during catheterization, is considered the reference standard for assessment of lesion-specific ischemia (6 8), while, it is an invasive time-consuming procedure that has a risk of complications. The recent advent of noninvasive FFR derived from CT (CT-FFR) has enabled both anatomical and functional evaluation of CAD and increased our ability to detect hemodynamically significant stenosis (9 12). However, a recent systematic review revealed wide limits of agreement between invasive FFR and CT-FFR, especially for CT-FFR values of 0.7 0.8, suggesting that the clinical ‘gray zone’ was larger for CT-FFR than for invasive FFR (13,14). Therefore, CT-FFR alone may not be enough for completely identifying lesion-specific ischemia, and additional information is needed to improve the diagnostic accuracy such as the characteristics and volume of coronary plaque on CCTA (15,16). The purpose of this study was to investigate the diagnostic accuracy of comprehensive assessment of anatomical and functional indices derived from CCTA for detecting lesion-specific ischemia assessed by invasive FFR. MATERIALS AND METHODS The study protocol was approved by Ethics Committee of XXXX. The need for informed consent was waived in view of the retrospective nature of the analysis and the anonymity of the data. Patient Population
We retrospectively examined a total of 47 consecutive patients (36 men, 11 women; mean age § standard deviation, 69.4 § 11.2 years, range = 47 87 years) between May 2016 and March 2018. All patients firstly underwent CCTA/CT-FFR, followed by invasive FFR. Each patient had 30% 90% stenosis of 1 major epicardial vessel on CCTA and underwent invasive measurement of FFR in that vessel after CCTA between May 2016 and March 2018. The interval between CCTA and invasive FFR measurement was 23.7 § 14.3 days. The invasive FFR was 0.8 (nonischemic) in 25 of the 60 vessels and <0.8 (ischemic) in 35. 2
CCTA Analysis
The CCTA data were analyzed using a dedicated workstation (Vitrea, Canon Medical Systems) by consensus between two experienced cardiovascular radiologists in the core CCTA laboratory at Shin Koga Hospital. Both radiologists were blinded to the results of invasive coronary angiography and FFR. One target lesion per vessel was assessed when invasive FFR was performed. The following anatomical CTA parameters were measured: the Agatston score of the vessel containing the target lesion, severity of stenosis, mean plaque CT attenuation value (HU), noncalcified and calcified plaque volumes, and remodeling index. The diameters of the nonaffected vessel segments immediately proximal and distal to the lesion were measured as the reference for determination of the diameter of a stenosis; the degree of stenosis was calculated as the diameter of a lesion of interest divided by the reference diameter. Coronary plaque volume and morphology were assessed by the labeling method using dedicated software (CCTA Analysis, Canon Medical Systems) (17). On-site CT-FFR Analysis
The on-site CT-FFR analysis (CT-FFR Research Version, Canon Medical Systems) was performed by consensus between an experienced postprocessing technician and a cardiovascular radiologist using the dedicated Vitrea workstation, who were blinded to all information of the clinical data, CCTA, invasive coronary angiography, and FFR measurement. The CT-FFR analysis included four-dimensional CT
ARTICLE IN PRESS Academic Radiology, Vol &, No &&, && 2020
INVESTIGATING THE EFFECTS OF COMBINED ASSESSMENT
image tracking and structural and fluid analysis by considering the deformation of vessels, with four-phase CT image series at 70%, 80%, 90%, and 99% of the R-R interval. At first, the vessel contour and the centerline of the coronary vessel were semi-automatically extracted on the workstation. Two operators consensually edit the vessel contour and centerline especially in the calcified segments. After the edit, the CT-FFR value was estimated using the following steps: (1) measurement of the shape and changes in the cross-sectional area of the coronary artery and aorta in four volumes reconstructed at different time-points during 70% 99% of the R-R interval extraction of luminal deformation data for the coronary arteries and aorta; (2) application of Bayesian hierarchical modeling and the Markov chain Monte Carlo method to determine the analysis conditions; (3) fluid simulation using a reduced order fluid model; and (4) determination of pressure and flow through the coronary artery tree and estimation of the CT-FFR value (11,12). In the process of CT-FFR analysis, the machine-learning technique was not employed. The CT-FFR value was measured at the distal end of the coronary vessel where the diameter was 2 mm according to the previous study (11), in which the on-site CT-FFR (Canon Medical Systems) was used. The time for edit of coronary arterial contour and centerline was 30 60 minutes, and the calculation time for CT-FFR was 20 30 minutes. Machine Learning and Statistical Analysis
In this investigation, we used the random forest, which is one of the most common machine learning algorithms because of its relatively good accuracy, robustness, and ease of use, and built two learning models: Model-1 (including anatomical CTA parameters, i.e., Agatston score, severity of stenosis, mean plaque CT attenuation value, noncalcified and calcified plaque volumes, and remodeling index) and Model-2 (the same anatomical CTA parameters plus the CT-FFR value). The anatomical CTA parameters were transferred to the random forest model as numbers. The forest consisted of many decision trees, where each tree was trained using randomly selected samples. Two thirds of the training samples were used to train the trees, and the remaining third was used for cross-validation to estimate how well the resulting random forest models performed. In the random forest algorithm, importance is calculated with the Gini importance index to calculate the value of each feature, and we ranked the importance of all descriptors in Model-2 to separate the patients into ischemic and nonischemic groups based on Gini importance (18). The machine learning analyses were performed using the Python-based software program (Orange version 3.23.1; https://orange.biolab.si). We adopted the random forest to evaluate the added value of CT-FFR compared to CCTA descriptors alone in this study. The comparison among the logistic regression, support vector machine, and random forest is shown in the Appendix Figure. The data are shown as the mean § standard deviation or as the median and interquartile range depending on the data distribution. The Agatston score, severity of stenosis, mean plaque CT attenuation value, noncalcified and calcified plaque volumes,
remodeling index, and CT-FFR values were compared between the nonischemic and ischemic groups using the Student’s t-test. The area under the receiver-operating characteristic (ROC) curve (AUC) was calculated for Models-1 and -2 to differentiate between nonischemic and ischemic lesions. The AUCs for Models-1 and -2 were compared using Delong’s test. The net reclassification improvement and integrated discrimination improvement values were calculated to assess the added value of CT-FFR for distinguishing between ischemic and nonischemic lesions. Two-tailed p-values < 0.05 were considered statistically significant. The statistical analyses were performed using opensource statistical software (Microsoft R Open, version 3.4.2; https://mran.microsoft.com/open).
RESULTS The results of the univariate analysis are summarized in Table 1. The severity of stenosis, noncalcified plaque volume, remodeling index, and CT-FFR value were significantly different between the ischemic and nonischemic groups. Model-1 (anatomical CTA parameters) and Model-2 (the same anatomical parameters plus CT-FFR) are compared in Figure 1. The AUC was 0.738 (95% confidence interval [CI] 0.609 0.843) for Model-1 and 0.835 (95% CI 0.717 0.919) for Model-2. The reclassification of the risk of the ischemiarelated lesion was significantly improved after adding CTFFR (Model-2) to the anatomical parameters (Model-1). The net reclassification improvement was 0.297 (95% CI 0.031 0.564; p = 0.03) and the integrated discrimination improvement was 0.254 (95% CI 0.127 0.382; p < 0.001). Figure 2 shows the Gini importance index of the descriptors in the random forest. The parameter best able to differentiate an ischemic lesion from a nonischemic lesion was the CT-FFR value (0.496) followed by the severity of stenosis (0.262), remodeling index (0.115), calcified plaque volume (0.048), mean plaque CT attenuation value (0.046), noncalcified plaque volume (0.033), and Agatston score (<0.001). Representative cases are shown in Figures 3,4 and Appendix Figure. TABLE 1. Basic Characteristics of the Study Population According to Whether Ischemia was Present No Ischemia (n = 25) Agatston score Stenosis severity Plaque CT attenuation value Plaque volume (noncalcified) Plaque volume (calcified) Remodeling index CT-FFR value
Ischemia (n = 35)
p Value
37 (1.5 148.5) 105 (9 258) 40.8 § 15.7 57.6 § 14.1 70.1 § 58.5 55.1 § 27.9
0.59 <0.01 0.19
167 (119 256) 223 (177 295.7)
<0.05
4.1 (0 25)
23 (4.7 57.8)
1.04 § 0.12 0.84 § 0.14
1.11 § 0.13 0.71 § 0.14
0.20 <0.05 <0.01
CT, computed tomography; FFR, fractional flow reserve.
3
ARTICLE IN PRESS KAWASAKI ET AL
Figure 1. Receiver-operating characteristic curve analysis of machine learning (random forest) for Model 1 (anatomical CTA parameters) and Model 2 (anatomical CTA parameters plus CTFFR). CT, computed tomography; CTA, CT angiography; CT-FFR, CT-derived fractional flow reserve.
DISCUSSION The two main findings of this study were that (1) comprehensive assessment of multiple CT-derived indices, including severity of stenosis, plaque density, plaque volume, coronary artery calcium score, remodeling index, and CT-FFR based on a machine learning approach showed high diagnostic accuracy for detecting hemodynamically significant CAD, and (2) the CT-FFR had a more important incremental diagnostic value than the conventional anatomical indices. The anatomical severity of coronary artery stenosis is not necessarily equal to the hemodynamical severity of CAD. Meijboom
Academic Radiology, Vol &, No &&, && 2020
et al showed that there was agreement between significant stenosis (a reduction in diameter of 50%) on quantitative coronary angiography and positive FFR (<0.80) on only 65% of occasions (4). Hence, in addition to coronary stenosis severity, other indices have been necessary for accurate prediction of hemodynamically significant CAD. The coronary artery calcium score is known to be associated with the severity of CAD (19). The characteristics of coronary artery plaque seen on CCTA are also considered indicators of hemodynamically significant CAD (15,16). Indeed, the anatomical index of coronary artery plaque remodeling was important for the detection of hemodynamically significant CAD in the present study. However, plaque CT number and Agatston score had relatively small influence. In this study, we used the labeling method for the plaque characterization and quantification. Previous ex vivo and in vivo studies have reported that an opacified coronary lumen substantially increases the plaque attenuation because of partial volume effects, beam hardening, and plaque vascularity (20,21). Therefore, a method for accurate characterization and volumetric quantification of coronary plaques was required. The labeling method applied in the present study was developed for 320-row CT coronary angiography to overcome the limitations of CT attenuation value-based measurements and is a novel way of assessing coronary artery plaque (17). This method considers relative (as opposed to absolute) CT attenuation in addition to image noise and the three-dimensional continuity of volumes and classifies plaque components into fibrous tissue, necrotic core, and calcified areas. A previous study that compared this labeling method with CT attenuation value-based measurement of non-calcified plaques demonstrated superior correlation
Figure 2. Gini importance indices of seven descriptors for identification of ischemic lesions calculated using a random forest. CT-FFR was most important, followed by severity of stenosis and the remodeling index. CT-FFR, CT-derived fractional flow reserve.
4
ARTICLE IN PRESS Academic Radiology, Vol &, No &&, && 2020
INVESTIGATING THE EFFECTS OF COMBINED ASSESSMENT
Figure 3. Coronary CT angiogram (A) of a 70s woman shows partially calcified plaque in the proximal left anterior descending artery with intermediate stenosis (arrow). On the curved multiplanar reformation image in the CT-FFR display (B), the operators defined the measuring site (arrowheads [2 mm in vessel diameter]) distal to the stenosis (arrow). Her CT-FFR, stenosis severity, and remodeling index values were 0.72, 80%, and 1.11, respectively. Machine learning (random forest) classified the coronary vessel as “positive” (ischemic lesion). Coronary angiogram (C) shows moderate stenosis (arrow) with invasive FFR of 0.72. The labeling method identifies areas of low attenuation by comparing Hounsfield Unit measurements relative to surrounding areas, while differentiating low attenuation areas from image noise. Then, the algorithm excludes areas within the lesion with a low probability of clinically significant morphology. Residual low attenuation areas are considered as necrotic core, and remaining components not classified into either low-density areas or calcium are considered fibrous tissue. The labeling method can correctly identify the necrotic core area (red), calcified plaque (yellow), fibrous area (blue) and lumen area (green) (D). FFR, fractional flow reserve. (Color version of figure is available online.)
with virtual histology intravascular ultrasound as the reference standard. This is because of the decreased influence of the various factors associated with contrast material and scanning protocols (17). We considered the labeling method to be appropriate for the assessment of coronary plaque volume and morphology. Our results showed that the remodeling was a more important factor than plaque characteristics, i.e., CT number and Agatston score. We consider that the focal accumulation of plaque, which caused higher remodeling indices and stenosis severity, was associated with hemodynamically CAD (invasive FFR < 0.8). Furthermore, our study results demonstrated that added index of CT-FFR value increased the diagnostic performance. The diagnostic accuracy of anatomical indices measured on CCTA was moderate (AUC = 0.738 for Model-1) for detecting
hemodynamically significant CAD in spite of comprehensive assessment of multiple indices. However, the functional index of CT-FFR increased diagnostic value over the anatomical indices measured on CCTA (AUC = 0.835 for Model-2). CT-FFR has been reported to have high diagnostic accuracy for detecting hemodynamically significant CAD assessed by invasive FFR (22) and to be an effective gatekeeper for invasive coronary angiography in clinical settings (23). The on-site CT-FFR used in the present study relied on fluid-structure interaction for calculation of patient-specific CT-FFR (11), and a previous study indicated a close correlation between CT-FFR and invasive FFR that was highly reproducible (12). As a result, it was reasonable that the functional index of CT-FFR was more important than other anatomical indices, including anatomical stenosis, plaque characteristics, and coronary artery calcification. However, 5
ARTICLE IN PRESS KAWASAKI ET AL
Academic Radiology, Vol &, No &&, && 2020
Figure 4. Coronary CT angiogram (A) of a 60s woman shows partially calcified plaque in the proximal left anterior descending artery with intermediate stenosis (arrow). The CT-FFR display (B) shows the stenosis (arrow). Her CT-FFR, stenosis severity, and remodeling index values were 0.84, 60%, and 1.00. respectively. Machine learning (random forest) classified the coronary vessel as “negative” (nonischemic lesion). Coronary angiogram (C) shows moderate stenosis (arrow) with invasive FFR of 0.89. FFR, fractional flow reserve.
the diagnostic accuracy of CT-FFR varies markedly across the spectrum of CAD severity, and is particularly poor at invasive FFR values of 0.7 to 0.8, suggesting a larger clinical ‘grey zone’ for CT-FFR compared to invasive FFR (13). We believe that our results indicate that comprehensive assessment of multiple indices using the machine learning approach (random forest) has the potential to overcome the limitations of assessment based on a single index. Additionally, the anatomical and functional CT indices might play complementary roles. We did not compare the diagnostic accuracy of on-site CT-FFR with that of off-site CT-FFR developed by HeartFlow (Redwood, CA) (8 10). However, a single-center prospective study by Ko et al reported comparative results of on-site CT-FFR in detecting invasive FFR <0.8 (11). Although further validation is required in a larger prospective study, our present findings demonstrate that machine learning analysis of anatomical CCTA indices and CTFFR has the potential to allow for accurate evaluation of lesionspecific ischemia in a more cost-effective manner. Machine learning is a technique which has ability to learn using artificial intelligence (AI). Machine learning has been applied for cardiovascular disease in the fields of image 6
interpretation, diagnostic support and outcome prediction (24). Ruijsink et al reported that machine learning could automatically analyze of cardiac function from cine cardiovascular magnetic resonance (25). Zheng et al reported that machine learning could automatically perform aortic valve segmentation intraoperatively in real time from images taken using the C-arm during Transcatheter Aortic Valve Implantation (26). Motwani et al reported that machine learning combining clinical and CCTA data significantly improved prediction to 5-year all-cause mortality compared with existing clinical or CCTA alone (27). In future, machine learning may be integrated on existing information technology infrastructure such as medical records, reporting system, and picture archiving and communication systems, and it will allow for making clinical routine more efficient. However, physician should have knowledge about the machine learning tools to determine the appropriate diagnosis and treatment strategy. Random forest is an accurate, easily interpretable, and computationally efficient algorithm. Random forest is composed of a collection of many decision trees. Each decision tree in the classifier is trained using a subset of the various input variables with two thirds of the original data. The
ARTICLE IN PRESS Academic Radiology, Vol &, No &&, && 2020
INVESTIGATING THE EFFECTS OF COMBINED ASSESSMENT
remaining one third is used to generate the out-of-bag error, which is an internal validation of the final model. The outcome of a random forest model is determined by the majority vote. Random forest works on the hypothesis that an aggregation of correctly predicted classes from a large ensemble of randomly generated individual decision trees achieves higher classification accuracy (28). Random forest is considered less vulnerable to overfitting the training dataset given the large number of trees built, making each tree an independent model. Random forest requires little preprocessing of data; the data need not be normalised; and the approach is resilient to outliers. Random forest can select the most important features when creating decision trees. In this study, application of the random forest-based machine learning approach showed that the three most important indices for identification of ischemic lesions were the CT-FFR, severity of stenosis, and remodeling index. We could not compare the diagnostic accuracy of CT-FFR alone with that of combined assessment of anatomical CTA plus CT-FFR (Model-2) because we had used a random forest algorithm that evaluates multiple indices. However, we believe that the combined assessment (Model-2) is superior to CT-FFR alone because the stenosis severity and the remodeling index had relatively large influence on the differentiation between ischemia and nonischemia lesion (Fig 2). We did not use a deep learning approach in this study. Deep learning is a subfield of machine learning concerned with algorithms inspired by the structure and function of the brain, which are known as artificial neural networks. Deep learning has attracted the attention of the academic medical community in recent years because of its performance (29). It has also shown promising results with regard to segmentation of cardiac structures and identification of CAD (30). Although deep learning generates features automatically, it requires large amounts of labeled data. As larger amounts of data are added, the performance of deep learning continues to increase. However, the lack of high numbers of samples makes training difficult and causes overfitting, resulting in decreased accuracy. Therefore, traditional machine learning performs better than deep learning when sample sizes are small (31). In this study, we could not collect large amounts of clinical data and apply traditional machine learning because on-site computed CT-FFR based on fluidstructure interactions is a work-in-progress algorithm and is not readily accessible by most research institutions. A multicenter study of deep learning using large-scale cardiac CT data should be performed in the future. This study had some limitations. First, it had a retrospective observational design and a small sample size; therefore, it is not necessarily representative of the general population. We adopted the random forest because of the tradeoff between the number of variables and population size. Although we applied cross-validation in the analysis, it would have been preferable to include training, validation, and test datasets. A prospective study is needed to establish the feasibility of comprehensive assessment of multiple CT-derived indices using random forests. Second, we excluded vessels with tandem
lesions to make it easier to assess the effects of each index. Third, the accuracy of CT-FFR depends on the experience of the observer and the image quality. We did not evaluate the influence of CT image quality on the CT-FFR measurements. To our experience, it is considered that CT-FFR value may be affected by the image quality factors, e.g. contrast enhancement, motion artifacts, vessel calcification because these factors make it difficult to delineate the contour of coronary vessels in the process of CT-FFR measurement. The machine-learning or deep-learning algorithm should be necessary to perform fully automatic CT-FFR measurement to improve the accuracy and to reduce inter-operator variability. However, a previous study reported that automated CT-FFR measurement software using structural and fluid analysis displayed good reproducibility even when the postprocessing was performed by an inexperienced observer given only brief training (32). Lastly, we did not compare the CCTA plus CT-FFR with other gold standard functional imaging i.e. myocardial perfusion positron emission tomography due to unavailability in our institutions. In conclusion, comprehensive assessment of anatomical and functional indices derived from CCTA data based on machine learning (i.e. random forests) provided higher diagnostic accuracy for detecting hemodynamically significant CAD than the anatomical CT indices alone. A functional index, the CT-FFR, was more useful than conventional anatomical indices. FUNDING This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. UNBLINDED ETHICS STATEMENT The study protocol was approved by Ethics Committee of Kumamoto Medical College (approval number #1294). The need for informed consent was waived in view of the retrospective nature of the analysis and the anonymity of the data. REFERENCES 1. Nowbar AN, Gitto M, Howard JP, et al. Mortality from ischemic heart disease. Circ Cardiovasc Qual Outcomes 2019; 12:e005375. 2. Budoff MJ, Dowe D, Jollis JG, et al. Diagnostic performance of 64-multidetector row coronary computed tomographic angiography for evaluation of coronary artery stenosis in individuals without known coronary artery disease: results from the prospective multicenter ACCURACY (Assessment by Coronary Computed Tomographic Angiography of Individuals Undergoing Invasive Coronary Angiography) trial. J Am Coll Cardiol 2008; 52:1724–1732. 3. National Institute for Health and Care Excellence. Chest pain of recent onset: assessment and diagnosis of recent onset chest pain or discomfort of suspected cardiac origin (update): clinical guideline 95. London, England: National Institute for Health and Care Excellence, 2016. 4. Meijboom WB, Van Mieghem CA, van Pelt N, et al. Comprehensive assessment of coronary artery stenoses: computed tomography coronary angiography versus conventional coronary angiography and correlation with fractional flow reserve in patients with stable angina. J Am Coll Cardiol 2008; 52:636–643.
7
ARTICLE IN PRESS Academic Radiology, Vol &, No &&, && 2020
KAWASAKI ET AL
5. Cury RC, Abbara S, Achenbach S, et al. CAD-RADS(TM) Coronary Artery Disease - Reporting and Data System. An expert consensus document of the Society of Cardiovascular Computed Tomography (SCCT), the American College of Radiology (ACR) and the North American Society for Cardiovascular Imaging (NASCI). Endorsed by the American College of Cardiology. J Cardiovasc Comput Tomogr 2016; 10:269–281. 6. Tonino PA, De Bruyne B, Pijls NH, et al. Fractional flow reserve versus angiography for guiding percutaneous coronary intervention. N Engl J Med 2009; 360:213–224. 7. De Bruyne B, Fearon WF, Pijls NH, et al. Fractional flow reserve-guided PCI for stable coronary artery disease. N Engl J Med 2014; 371:1208–1217. 8. Pijls NH, van Schaardenburgh P, Manoharan G, et al. Percutaneous coronary intervention of functionally nonsignificant stenosis: 5-year followup of the DEFER Study. J Am Coll Cardiol 2007; 49:2105–2111. 9. Norgaard BL, Leipsic J, Gaur S, et al. Diagnostic performance of noninvasive fractional flow reserve derived from coronary computed tomography angiography in suspected coronary artery disease: the NXT trial (Analysis of Coronary Blood Flow Using CT Angiography: Next Steps). J Am Coll Cardiol 2014; 63:1145–1155. 10. Min JK, Leipsic J, Pencina MJ, et al. Diagnostic accuracy of fractional flow reserve from anatomic CT angiography. JAMA 2012; 308:1237–1245. 11. Ko BS, Cameron JD, Munnur RK, et al. Noninvasive CT-derived FFR based on Structural and fluid analysis: a comparison with invasive FFR for detection of functionally significant stenosis. JACC Cardiovasc Imaging 2017; 10:663–673. 12. Fujimoto S, Kawasaki T, Kumamaru KK, et al. Diagnostic performance of on-site computed CT-fractional flow reserve based on fluid structure interactions: comparison with invasive fractional flow reserve and instantaneous wave-free ratio. Eur Heart J Cardiovasc Imaging 2019; 20:343–352. 13. Cook CM, Petraco R, Shun-Shin MJ, et al. Diagnostic accuracy of computed tomography-derived fractional flow reserve: a systematic review. JAMA Cardiol 2017; 2:803–810. 14. Villines TC. Can CT-derived FFR better inform clinical decision-making and improve outcomes in stable ischaemic heart disease? Eur Heart J 2018; 39:3712–3714. 15. Gaur S, Ovrehus KA, Dey D, et al. Coronary plaque quantification and fractional flow reserve by coronary computed tomography angiography identify ischaemia-causing lesions. Eur Heart J 2016; 37:1220–1227. 16. Ovrehus KA, Gaur S, Leipsic J, et al. CT-based total vessel plaque analyses improves prediction of hemodynamic significance lesions as assessed by fractional flow reserve in patients with stable angina pectoris. J Cardiovasc Comput Tomogr 2018; 12:344–349. 17. Fujimoto S, Kondo T, Kodama T, et al. A novel method for non-invasive plaque morphology analysis by coronary computed tomography angiography. Int J Cardiovasc Imaging 2014; 30:1373–1382. 18. Menze BH, Kelm BM, Masuch R, et al. A comparison of random forest and its Gini importance with standard chemometric methods for the feature selection and classification of spectral data. BMC Bioinformatics 2009; 10:213. 19. Budoff MJ, Young R, Burke G, et al. Ten-year association of coronary artery calcium with atherosclerotic cardiovascular disease (ASCVD)
8
20.
21.
22.
23.
24.
25.
26.
27.
28.
29. 30.
31.
32.
events: the multi-ethnic study of atherosclerosis (MESA). Eur Heart J 2018; 39:2401–2408. Cademartiri F, Mollet NR, Runza G, et al. Influence of intracoronary attenuation on coronary plaque measurements using multislice computed tomography: observations in an ex vivo model of coronary computed tomography angiography. Eur Radiol 2005; 15:1426–1431. Kidoh M, Utsunomiya D, Oda S, et al. Evaluation of the effect of intracoronary attenuation on coronary plaque measurements using a dualphase coronary CT angiography technique on a 320-row CT Scanner in vivo validation study. Acad Radiol 2016; 23:315–320. Celeng C, Leiner T, Maurovich-Horvat P, et al. Anatomical and functional computed tomography for diagnosing hemodynamically significant coronary artery disease: a meta-analysis. JACC Cardiovasc Imaging 2019; 12:1316–1325. Douglas PS, Pontone G, Hlatky MA, et al. Clinical outcomes of fractional flow reserve by computed tomographic angiography-guided diagnostic strategies vs. usual care in patients with suspected coronary artery disease: the prospective longitudinal trial of FFR(CT): outcome and resource impacts study. Eur Heart J 2015; 36:3359–3367. Dey D, Slomka PJ, Leeson P, et al. Artificial intelligence in cardiovascular imaging: JACC State-of-the-Art Review. J Am Coll Cardiol 2019; 73:1317–1335. Ruijsink B, Puyol-Anton E, Oksuz I, et al. Fully automated, qualitycontrolled cardiac analysis from CMR: validation and large-scale application to characterize cardiac function. JACC Cardiovasc Imaging 2019. Zheng Y, John M, Liao R, et al. Automatic aorta segmentation and valve landmark detection in C-arm CT for transcatheter aortic valve implantation. IEEE Trans Med Imaging 2012; 31:2307–2321. Motwani M, Dey D, Berman DS, et al. Machine learning for prediction of all-cause mortality in patients with suspected coronary artery disease: a 5-year multicentre prospective registry analysis. Eur Heart J 2017; 38:500–507. Ozigis MS, Kaduk JD, Jarvis CH, et al. Detection of oil pollution impacts on vegetation using multifrequency SAR, multispectral images with fuzzy forest and random forest methods. Environ Pollut 2020; 256:113360. Chartrand G, Cheng PM, Vorontsov E, et al. Deep learning: a primer for radiologists. Radiographics 2017; 37:2113–2131. van Hamersvelt RW, Zreik M, Voskuil M, et al. Deep learning analysis of left ventricular myocardium in CT angiographic intermediatedegree coronary stenosis improves the diagnostic accuracy for identification of functionally significant stenosis. Eur Radiol 2019; 29:2350–2359. Tang A, Tam R, Cadrin-Chenevert A, et al. Canadian Association of Radiologists White Paper on Artificial Intelligence in Radiology. Can Assoc Radiol J 2018; 69:120–135. Ri K, Kumamaru KK, Fujimoto S, et al. Noninvasive computed tomography-derived fractional flow reserve based on structural and fluid analysis: reproducibility of on-site determination by unexperienced observers. J Comput Assist Tomogr 2018; 42:256–262.
ARTICLE IN PRESS Academic Radiology, Vol &, No &&, && 2020
INVESTIGATING THE EFFECTS OF COMBINED ASSESSMENT
APPENDIX
Appendix Figure. The comparison among the logistic regression, support vector machine, and random forest in the ROC analysis. The AUC to differentiate between nonischemic and ischemic lesions was 0.698, 0.827, and 0.835 for the logistic regression, support vector machine, and random forest, respectively. Note: LR, logistic regression; RF, random forest; SVM, support vector machine.
9
Original Investigation
Evaluation of Significant Coronary Artery Disease Based on CT Fractional Flow Reserve and Plaque Characteristics Using Random Forest Analysis in Machine Learning Tomohiro Kawasaki, MD, Masafumi Kidoh, MD, Teruhito Kido, MD, Daisuke Sueta, MD, Shinichiro Fujimoto, MD, Kanako Kunishima Kumamaru, MD, Teruyoshi Uetani, MD, Yuki Tanabe, MD, Toshio Ueda, RT, Daisuke Sakabe, RT, Seitaro Oda, MD, Tsuneo Yamashiro, MD, Kenichi Tsujita, MD, Shingo Kato, MD, Hideaki Yuki, MD, Daisuke Utsunomiya, MD
Rationale and Objectives: Fractional flow reserve (FFR) is an established technique for detecting lesion-specific ischemia but is invasive. Our objective was to investigate the effects of combined assessment of coronary CT angiography (CCTA) imaging features and CT-FFR on detecting lesion-specific ischemia by comparing with invasive FFR. Materials and Methods: Forty-seven patients who had 60 coronary vessels with 30% 90% stenosis were included. Six anatomic CCTA descriptors (Agatston score, stenosis severity, mean plaque CT attenuation value, noncalcified and calcified plaque volumes, remodeling index) and a functional descriptor (CT-FFR) were measured. Random forest was used to identify which descriptors were useful to identify ischemia-related lesion. Receiver-operating characteristic (ROC) curves were calculated for 2 models: i.e. Model-1 for anatomical CT descriptors and Model-2 for anatomical CT descriptors plus CT-FFR. Results: Stenosis severity (40.8 § 15.7% vs 57.6 § 14.1%), noncalcified plaque volume (190 § 100 vs 254.8 § 133.3), and remodeling index (1.04 § 0.12 vs 1.11 § 0.13) were significantly higher in ischemia-related lesions than nonischemia-related lesions. CT-FFR was 0.84 § 0.14 and 0.71 § 0.14, respectively, for ischemia-related and nonischemia-related lesions, and the difference was significant. The area under the ROC curve was 0.738 and 0.835 in Model-1 and Model-2, respectively. Reclassification of ischemic lesion risk was significantly improved after adding CT-FFR: net reclassification improvement was 0.297 and integrated discrimination improvement was 0.254. Conclusion: Combined assessment of anatomical CCTA features and functional CT-FFR was helpful for detecting lesion-specific ischemia. Key Words: Coronary artery disease; Coronary CT angiography; CT-derived fractional flow reserve; Machine learning; Plaque characteristics. © 2020 The Association of University Radiologists. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license. (http://creativecommons.org/licenses/by-nc-nd/4.0/) Abbreviations: AUC area under the receiver-operating characteristic curve, CAD coronary artery disease, CI confidence interval, CT computed tomography, CCTA coronary CT angiography, FFR fractional flow reserve, ROC receiver-operating characteristic
Acad Radiol 2020; &:1–9 From the Cardiovascular, Shin Koga Hospital, Kurume City, Fukuoka, Japan (T.K.); Diagnostic Radiology, Faculty of Life Sciences, Kumamoto University, Chuo-ku, Kumamoto, Japan (M.K., S.O., H.Y.); Department of Radiology, Ehime University, Shitsukawa, Toon, Ehime 791-0295, Japan (T.K., Y.T.); Department of Cardiovascular Medicine, Kumamoto University Hospital, Chuo-ku, Kumamoto City, Japan (D.S., K.T.); Cardiovascular Medicine, Juntendo University, Bunkyo-ku, Tokyo, Japan (S.F.); Department of Radiology, Juntendo University Graduate School of Medicine, Bunkyo-ku, Tokyo, Japan (K.K.K.); Cardiovascular Medicine, Ehime University, Toon, Ehime, Japan (T.U.); Central Radiology, Shin Koga Hospital, Kurume City, Fukuoka, Japan (T.U.); Central Radiology, Kumamoto University Hospital, Chuo-ku, Kumamoto, Japan (D.S.); Radiology, Graduate School of Medical Science, University of the Ryukyu, Nakagami-gun, Okinawa, Japan (T.Y.); Cardiology, Kanagawa Cardiovascular and Respiratory Center, Kanazawa-ku, Yokohama, Japan (S.K.); Diagnostic Radiology, Yokohama City University Graduate School of Medicine, Kanazawa-ku, Yokohama, Japan (D.U.). Received October 21, 2019; revised December 8, 2019; accepted December 17, 2019. Address correspondence to: T.K. e-mail: [email protected] © 2020 The Association of University Radiologists. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license. (http://creativecommons.org/licenses/by-nc-nd/4.0/) https://doi.org/10.1016/j.acra.2019.12.013
1
ARTICLE IN PRESS KAWASAKI ET AL
Academic Radiology, Vol &, No &&, && 2020
INTRODUCTION
Acquisition of CCTA Data
C
All patients underwent CCTA examination using a 320-row detector CT scanner (Aquilion One Vision or Aquilion One Genesis; Canon Medical Systems, Otawara, Japan). The CT protocol consisted of obtaining the calcium score followed by CCTA. The parameters used for CT scanning were as follows: prospective electrocardiogram-gatedaxial scans; 320 rows £ 0.5mm collimation; rotation time, 275 ms; tube voltage, 120 kV; and tube current, 350 650 mA (automatic exposure control). All patients underwent prospective ECG-gated CCTA with data acquisition over 70% 99% of the R-R interval. All patients received sublingual nitroglycerin. If the heart rate exceeded 65 bpm in the CT suite, 6 10 mg landiolol hydrochloride (Corebeta; Ono Pharmaceutical Co. Ltd., Osaka, Japan) was administered intravenously as an additional betablocker 5 minutes before CCTA scanning. For CCTA, contrast material with an iodine concentration of 370 mg/mL (Iopamiron 370; Bayer Health Care, Osaka, Japan) was administered via a 20-gauge catheter inserted into an antecubital vein using a double-head power injector. The start time of data acquisition was determined by a computer-assisted bolus-tracking program. The trigger threshold was set at 150 HU for the region of interest in the ascending aorta. Acquisition of the CT data was started 6 seconds after the trigger.
oronary artery disease (CAD) was still the leading cause of death globally in 2015 (1), and it is important to improve the prognosis in patients with CAD by appropriate diagnosis and treatment. Coronary computed tomography angiography (CCTA) is an established minimally invasive diagnostic technique and has high diagnostic accuracy for the evaluation of CAD (2). In 2016, the National Institute for Health and Care Excellence in the UK endorsed CCTA as the first-line investigation for patients with angina and no history of CAD (3). However, the functional assessment of lesion-specific ischemia is necessary to determine the appropriate treatment strategy in the patients with CAD, and anatomical stenosis on CCTA is not equal to the lesion-specific ischemia (4). Therefore, additional functional tests are recommended in patients with anatomically significant stenosis (50 %) on CCTA to identify the lesionspecific ischemia (5). Fractional flow reserve (FFR), which is measured by a coronary pressure wire during catheterization, is considered the reference standard for assessment of lesion-specific ischemia (6 8), while, it is an invasive time-consuming procedure that has a risk of complications. The recent advent of noninvasive FFR derived from CT (CT-FFR) has enabled both anatomical and functional evaluation of CAD and increased our ability to detect hemodynamically significant stenosis (9 12). However, a recent systematic review revealed wide limits of agreement between invasive FFR and CT-FFR, especially for CT-FFR values of 0.7 0.8, suggesting that the clinical ‘gray zone’ was larger for CT-FFR than for invasive FFR (13,14). Therefore, CT-FFR alone may not be enough for completely identifying lesion-specific ischemia, and additional information is needed to improve the diagnostic accuracy such as the characteristics and volume of coronary plaque on CCTA (15,16). The purpose of this study was to investigate the diagnostic accuracy of comprehensive assessment of anatomical and functional indices derived from CCTA for detecting lesion-specific ischemia assessed by invasive FFR. MATERIALS AND METHODS The study protocol was approved by Ethics Committee of XXXX. The need for informed consent was waived in view of the retrospective nature of the analysis and the anonymity of the data. Patient Population
We retrospectively examined a total of 47 consecutive patients (36 men, 11 women; mean age § standard deviation, 69.4 § 11.2 years, range = 47 87 years) between May 2016 and March 2018. All patients firstly underwent CCTA/CT-FFR, followed by invasive FFR. Each patient had 30% 90% stenosis of 1 major epicardial vessel on CCTA and underwent invasive measurement of FFR in that vessel after CCTA between May 2016 and March 2018. The interval between CCTA and invasive FFR measurement was 23.7 § 14.3 days. The invasive FFR was 0.8 (nonischemic) in 25 of the 60 vessels and <0.8 (ischemic) in 35. 2
CCTA Analysis
The CCTA data were analyzed using a dedicated workstation (Vitrea, Canon Medical Systems) by consensus between two experienced cardiovascular radiologists in the core CCTA laboratory at Shin Koga Hospital. Both radiologists were blinded to the results of invasive coronary angiography and FFR. One target lesion per vessel was assessed when invasive FFR was performed. The following anatomical CTA parameters were measured: the Agatston score of the vessel containing the target lesion, severity of stenosis, mean plaque CT attenuation value (HU), noncalcified and calcified plaque volumes, and remodeling index. The diameters of the nonaffected vessel segments immediately proximal and distal to the lesion were measured as the reference for determination of the diameter of a stenosis; the degree of stenosis was calculated as the diameter of a lesion of interest divided by the reference diameter. Coronary plaque volume and morphology were assessed by the labeling method using dedicated software (CCTA Analysis, Canon Medical Systems) (17). On-site CT-FFR Analysis
The on-site CT-FFR analysis (CT-FFR Research Version, Canon Medical Systems) was performed by consensus between an experienced postprocessing technician and a cardiovascular radiologist using the dedicated Vitrea workstation, who were blinded to all information of the clinical data, CCTA, invasive coronary angiography, and FFR measurement. The CT-FFR analysis included four-dimensional CT
ARTICLE IN PRESS Academic Radiology, Vol &, No &&, && 2020
INVESTIGATING THE EFFECTS OF COMBINED ASSESSMENT
image tracking and structural and fluid analysis by considering the deformation of vessels, with four-phase CT image series at 70%, 80%, 90%, and 99% of the R-R interval. At first, the vessel contour and the centerline of the coronary vessel were semi-automatically extracted on the workstation. Two operators consensually edit the vessel contour and centerline especially in the calcified segments. After the edit, the CT-FFR value was estimated using the following steps: (1) measurement of the shape and changes in the cross-sectional area of the coronary artery and aorta in four volumes reconstructed at different time-points during 70% 99% of the R-R interval extraction of luminal deformation data for the coronary arteries and aorta; (2) application of Bayesian hierarchical modeling and the Markov chain Monte Carlo method to determine the analysis conditions; (3) fluid simulation using a reduced order fluid model; and (4) determination of pressure and flow through the coronary artery tree and estimation of the CT-FFR value (11,12). In the process of CT-FFR analysis, the machine-learning technique was not employed. The CT-FFR value was measured at the distal end of the coronary vessel where the diameter was 2 mm according to the previous study (11), in which the on-site CT-FFR (Canon Medical Systems) was used. The time for edit of coronary arterial contour and centerline was 30 60 minutes, and the calculation time for CT-FFR was 20 30 minutes. Machine Learning and Statistical Analysis
In this investigation, we used the random forest, which is one of the most common machine learning algorithms because of its relatively good accuracy, robustness, and ease of use, and built two learning models: Model-1 (including anatomical CTA parameters, i.e., Agatston score, severity of stenosis, mean plaque CT attenuation value, noncalcified and calcified plaque volumes, and remodeling index) and Model-2 (the same anatomical CTA parameters plus the CT-FFR value). The anatomical CTA parameters were transferred to the random forest model as numbers. The forest consisted of many decision trees, where each tree was trained using randomly selected samples. Two thirds of the training samples were used to train the trees, and the remaining third was used for cross-validation to estimate how well the resulting random forest models performed. In the random forest algorithm, importance is calculated with the Gini importance index to calculate the value of each feature, and we ranked the importance of all descriptors in Model-2 to separate the patients into ischemic and nonischemic groups based on Gini importance (18). The machine learning analyses were performed using the Python-based software program (Orange version 3.23.1; https://orange.biolab.si). We adopted the random forest to evaluate the added value of CT-FFR compared to CCTA descriptors alone in this study. The comparison among the logistic regression, support vector machine, and random forest is shown in the Appendix Figure. The data are shown as the mean § standard deviation or as the median and interquartile range depending on the data distribution. The Agatston score, severity of stenosis, mean plaque CT attenuation value, noncalcified and calcified plaque volumes,
remodeling index, and CT-FFR values were compared between the nonischemic and ischemic groups using the Student’s t-test. The area under the receiver-operating characteristic (ROC) curve (AUC) was calculated for Models-1 and -2 to differentiate between nonischemic and ischemic lesions. The AUCs for Models-1 and -2 were compared using Delong’s test. The net reclassification improvement and integrated discrimination improvement values were calculated to assess the added value of CT-FFR for distinguishing between ischemic and nonischemic lesions. Two-tailed p-values < 0.05 were considered statistically significant. The statistical analyses were performed using opensource statistical software (Microsoft R Open, version 3.4.2; https://mran.microsoft.com/open).
RESULTS The results of the univariate analysis are summarized in Table 1. The severity of stenosis, noncalcified plaque volume, remodeling index, and CT-FFR value were significantly different between the ischemic and nonischemic groups. Model-1 (anatomical CTA parameters) and Model-2 (the same anatomical parameters plus CT-FFR) are compared in Figure 1. The AUC was 0.738 (95% confidence interval [CI] 0.609 0.843) for Model-1 and 0.835 (95% CI 0.717 0.919) for Model-2. The reclassification of the risk of the ischemiarelated lesion was significantly improved after adding CTFFR (Model-2) to the anatomical parameters (Model-1). The net reclassification improvement was 0.297 (95% CI 0.031 0.564; p = 0.03) and the integrated discrimination improvement was 0.254 (95% CI 0.127 0.382; p < 0.001). Figure 2 shows the Gini importance index of the descriptors in the random forest. The parameter best able to differentiate an ischemic lesion from a nonischemic lesion was the CT-FFR value (0.496) followed by the severity of stenosis (0.262), remodeling index (0.115), calcified plaque volume (0.048), mean plaque CT attenuation value (0.046), noncalcified plaque volume (0.033), and Agatston score (<0.001). Representative cases are shown in Figures 3,4 and Appendix Figure. TABLE 1. Basic Characteristics of the Study Population According to Whether Ischemia was Present No Ischemia (n = 25) Agatston score Stenosis severity Plaque CT attenuation value Plaque volume (noncalcified) Plaque volume (calcified) Remodeling index CT-FFR value
Ischemia (n = 35)
p Value
37 (1.5 148.5) 105 (9 258) 40.8 § 15.7 57.6 § 14.1 70.1 § 58.5 55.1 § 27.9
0.59 <0.01 0.19
167 (119 256) 223 (177 295.7)
<0.05
4.1 (0 25)
23 (4.7 57.8)
1.04 § 0.12 0.84 § 0.14
1.11 § 0.13 0.71 § 0.14
0.20 <0.05 <0.01
CT, computed tomography; FFR, fractional flow reserve.
3
ARTICLE IN PRESS KAWASAKI ET AL
Figure 1. Receiver-operating characteristic curve analysis of machine learning (random forest) for Model 1 (anatomical CTA parameters) and Model 2 (anatomical CTA parameters plus CTFFR). CT, computed tomography; CTA, CT angiography; CT-FFR, CT-derived fractional flow reserve.
DISCUSSION The two main findings of this study were that (1) comprehensive assessment of multiple CT-derived indices, including severity of stenosis, plaque density, plaque volume, coronary artery calcium score, remodeling index, and CT-FFR based on a machine learning approach showed high diagnostic accuracy for detecting hemodynamically significant CAD, and (2) the CT-FFR had a more important incremental diagnostic value than the conventional anatomical indices. The anatomical severity of coronary artery stenosis is not necessarily equal to the hemodynamical severity of CAD. Meijboom
Academic Radiology, Vol &, No &&, && 2020
et al showed that there was agreement between significant stenosis (a reduction in diameter of 50%) on quantitative coronary angiography and positive FFR (<0.80) on only 65% of occasions (4). Hence, in addition to coronary stenosis severity, other indices have been necessary for accurate prediction of hemodynamically significant CAD. The coronary artery calcium score is known to be associated with the severity of CAD (19). The characteristics of coronary artery plaque seen on CCTA are also considered indicators of hemodynamically significant CAD (15,16). Indeed, the anatomical index of coronary artery plaque remodeling was important for the detection of hemodynamically significant CAD in the present study. However, plaque CT number and Agatston score had relatively small influence. In this study, we used the labeling method for the plaque characterization and quantification. Previous ex vivo and in vivo studies have reported that an opacified coronary lumen substantially increases the plaque attenuation because of partial volume effects, beam hardening, and plaque vascularity (20,21). Therefore, a method for accurate characterization and volumetric quantification of coronary plaques was required. The labeling method applied in the present study was developed for 320-row CT coronary angiography to overcome the limitations of CT attenuation value-based measurements and is a novel way of assessing coronary artery plaque (17). This method considers relative (as opposed to absolute) CT attenuation in addition to image noise and the three-dimensional continuity of volumes and classifies plaque components into fibrous tissue, necrotic core, and calcified areas. A previous study that compared this labeling method with CT attenuation value-based measurement of non-calcified plaques demonstrated superior correlation
Figure 2. Gini importance indices of seven descriptors for identification of ischemic lesions calculated using a random forest. CT-FFR was most important, followed by severity of stenosis and the remodeling index. CT-FFR, CT-derived fractional flow reserve.
4
ARTICLE IN PRESS Academic Radiology, Vol &, No &&, && 2020
INVESTIGATING THE EFFECTS OF COMBINED ASSESSMENT
Figure 3. Coronary CT angiogram (A) of a 70s woman shows partially calcified plaque in the proximal left anterior descending artery with intermediate stenosis (arrow). On the curved multiplanar reformation image in the CT-FFR display (B), the operators defined the measuring site (arrowheads [2 mm in vessel diameter]) distal to the stenosis (arrow). Her CT-FFR, stenosis severity, and remodeling index values were 0.72, 80%, and 1.11, respectively. Machine learning (random forest) classified the coronary vessel as “positive” (ischemic lesion). Coronary angiogram (C) shows moderate stenosis (arrow) with invasive FFR of 0.72. The labeling method identifies areas of low attenuation by comparing Hounsfield Unit measurements relative to surrounding areas, while differentiating low attenuation areas from image noise. Then, the algorithm excludes areas within the lesion with a low probability of clinically significant morphology. Residual low attenuation areas are considered as necrotic core, and remaining components not classified into either low-density areas or calcium are considered fibrous tissue. The labeling method can correctly identify the necrotic core area (red), calcified plaque (yellow), fibrous area (blue) and lumen area (green) (D). FFR, fractional flow reserve. (Color version of figure is available online.)
with virtual histology intravascular ultrasound as the reference standard. This is because of the decreased influence of the various factors associated with contrast material and scanning protocols (17). We considered the labeling method to be appropriate for the assessment of coronary plaque volume and morphology. Our results showed that the remodeling was a more important factor than plaque characteristics, i.e., CT number and Agatston score. We consider that the focal accumulation of plaque, which caused higher remodeling indices and stenosis severity, was associated with hemodynamically CAD (invasive FFR < 0.8). Furthermore, our study results demonstrated that added index of CT-FFR value increased the diagnostic performance. The diagnostic accuracy of anatomical indices measured on CCTA was moderate (AUC = 0.738 for Model-1) for detecting
hemodynamically significant CAD in spite of comprehensive assessment of multiple indices. However, the functional index of CT-FFR increased diagnostic value over the anatomical indices measured on CCTA (AUC = 0.835 for Model-2). CT-FFR has been reported to have high diagnostic accuracy for detecting hemodynamically significant CAD assessed by invasive FFR (22) and to be an effective gatekeeper for invasive coronary angiography in clinical settings (23). The on-site CT-FFR used in the present study relied on fluid-structure interaction for calculation of patient-specific CT-FFR (11), and a previous study indicated a close correlation between CT-FFR and invasive FFR that was highly reproducible (12). As a result, it was reasonable that the functional index of CT-FFR was more important than other anatomical indices, including anatomical stenosis, plaque characteristics, and coronary artery calcification. However, 5
ARTICLE IN PRESS KAWASAKI ET AL
Academic Radiology, Vol &, No &&, && 2020
Figure 4. Coronary CT angiogram (A) of a 60s woman shows partially calcified plaque in the proximal left anterior descending artery with intermediate stenosis (arrow). The CT-FFR display (B) shows the stenosis (arrow). Her CT-FFR, stenosis severity, and remodeling index values were 0.84, 60%, and 1.00. respectively. Machine learning (random forest) classified the coronary vessel as “negative” (nonischemic lesion). Coronary angiogram (C) shows moderate stenosis (arrow) with invasive FFR of 0.89. FFR, fractional flow reserve.
the diagnostic accuracy of CT-FFR varies markedly across the spectrum of CAD severity, and is particularly poor at invasive FFR values of 0.7 to 0.8, suggesting a larger clinical ‘grey zone’ for CT-FFR compared to invasive FFR (13). We believe that our results indicate that comprehensive assessment of multiple indices using the machine learning approach (random forest) has the potential to overcome the limitations of assessment based on a single index. Additionally, the anatomical and functional CT indices might play complementary roles. We did not compare the diagnostic accuracy of on-site CT-FFR with that of off-site CT-FFR developed by HeartFlow (Redwood, CA) (8 10). However, a single-center prospective study by Ko et al reported comparative results of on-site CT-FFR in detecting invasive FFR <0.8 (11). Although further validation is required in a larger prospective study, our present findings demonstrate that machine learning analysis of anatomical CCTA indices and CTFFR has the potential to allow for accurate evaluation of lesionspecific ischemia in a more cost-effective manner. Machine learning is a technique which has ability to learn using artificial intelligence (AI). Machine learning has been applied for cardiovascular disease in the fields of image 6
interpretation, diagnostic support and outcome prediction (24). Ruijsink et al reported that machine learning could automatically analyze of cardiac function from cine cardiovascular magnetic resonance (25). Zheng et al reported that machine learning could automatically perform aortic valve segmentation intraoperatively in real time from images taken using the C-arm during Transcatheter Aortic Valve Implantation (26). Motwani et al reported that machine learning combining clinical and CCTA data significantly improved prediction to 5-year all-cause mortality compared with existing clinical or CCTA alone (27). In future, machine learning may be integrated on existing information technology infrastructure such as medical records, reporting system, and picture archiving and communication systems, and it will allow for making clinical routine more efficient. However, physician should have knowledge about the machine learning tools to determine the appropriate diagnosis and treatment strategy. Random forest is an accurate, easily interpretable, and computationally efficient algorithm. Random forest is composed of a collection of many decision trees. Each decision tree in the classifier is trained using a subset of the various input variables with two thirds of the original data. The
ARTICLE IN PRESS Academic Radiology, Vol &, No &&, && 2020
INVESTIGATING THE EFFECTS OF COMBINED ASSESSMENT
remaining one third is used to generate the out-of-bag error, which is an internal validation of the final model. The outcome of a random forest model is determined by the majority vote. Random forest works on the hypothesis that an aggregation of correctly predicted classes from a large ensemble of randomly generated individual decision trees achieves higher classification accuracy (28). Random forest is considered less vulnerable to overfitting the training dataset given the large number of trees built, making each tree an independent model. Random forest requires little preprocessing of data; the data need not be normalised; and the approach is resilient to outliers. Random forest can select the most important features when creating decision trees. In this study, application of the random forest-based machine learning approach showed that the three most important indices for identification of ischemic lesions were the CT-FFR, severity of stenosis, and remodeling index. We could not compare the diagnostic accuracy of CT-FFR alone with that of combined assessment of anatomical CTA plus CT-FFR (Model-2) because we had used a random forest algorithm that evaluates multiple indices. However, we believe that the combined assessment (Model-2) is superior to CT-FFR alone because the stenosis severity and the remodeling index had relatively large influence on the differentiation between ischemia and nonischemia lesion (Fig 2). We did not use a deep learning approach in this study. Deep learning is a subfield of machine learning concerned with algorithms inspired by the structure and function of the brain, which are known as artificial neural networks. Deep learning has attracted the attention of the academic medical community in recent years because of its performance (29). It has also shown promising results with regard to segmentation of cardiac structures and identification of CAD (30). Although deep learning generates features automatically, it requires large amounts of labeled data. As larger amounts of data are added, the performance of deep learning continues to increase. However, the lack of high numbers of samples makes training difficult and causes overfitting, resulting in decreased accuracy. Therefore, traditional machine learning performs better than deep learning when sample sizes are small (31). In this study, we could not collect large amounts of clinical data and apply traditional machine learning because on-site computed CT-FFR based on fluidstructure interactions is a work-in-progress algorithm and is not readily accessible by most research institutions. A multicenter study of deep learning using large-scale cardiac CT data should be performed in the future. This study had some limitations. First, it had a retrospective observational design and a small sample size; therefore, it is not necessarily representative of the general population. We adopted the random forest because of the tradeoff between the number of variables and population size. Although we applied cross-validation in the analysis, it would have been preferable to include training, validation, and test datasets. A prospective study is needed to establish the feasibility of comprehensive assessment of multiple CT-derived indices using random forests. Second, we excluded vessels with tandem
lesions to make it easier to assess the effects of each index. Third, the accuracy of CT-FFR depends on the experience of the observer and the image quality. We did not evaluate the influence of CT image quality on the CT-FFR measurements. To our experience, it is considered that CT-FFR value may be affected by the image quality factors, e.g. contrast enhancement, motion artifacts, vessel calcification because these factors make it difficult to delineate the contour of coronary vessels in the process of CT-FFR measurement. The machine-learning or deep-learning algorithm should be necessary to perform fully automatic CT-FFR measurement to improve the accuracy and to reduce inter-operator variability. However, a previous study reported that automated CT-FFR measurement software using structural and fluid analysis displayed good reproducibility even when the postprocessing was performed by an inexperienced observer given only brief training (32). Lastly, we did not compare the CCTA plus CT-FFR with other gold standard functional imaging i.e. myocardial perfusion positron emission tomography due to unavailability in our institutions. In conclusion, comprehensive assessment of anatomical and functional indices derived from CCTA data based on machine learning (i.e. random forests) provided higher diagnostic accuracy for detecting hemodynamically significant CAD than the anatomical CT indices alone. A functional index, the CT-FFR, was more useful than conventional anatomical indices. FUNDING This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. UNBLINDED ETHICS STATEMENT The study protocol was approved by Ethics Committee of Kumamoto Medical College (approval number #1294). The need for informed consent was waived in view of the retrospective nature of the analysis and the anonymity of the data. REFERENCES 1. Nowbar AN, Gitto M, Howard JP, et al. Mortality from ischemic heart disease. Circ Cardiovasc Qual Outcomes 2019; 12:e005375. 2. Budoff MJ, Dowe D, Jollis JG, et al. Diagnostic performance of 64-multidetector row coronary computed tomographic angiography for evaluation of coronary artery stenosis in individuals without known coronary artery disease: results from the prospective multicenter ACCURACY (Assessment by Coronary Computed Tomographic Angiography of Individuals Undergoing Invasive Coronary Angiography) trial. J Am Coll Cardiol 2008; 52:1724–1732. 3. National Institute for Health and Care Excellence. Chest pain of recent onset: assessment and diagnosis of recent onset chest pain or discomfort of suspected cardiac origin (update): clinical guideline 95. London, England: National Institute for Health and Care Excellence, 2016. 4. Meijboom WB, Van Mieghem CA, van Pelt N, et al. Comprehensive assessment of coronary artery stenoses: computed tomography coronary angiography versus conventional coronary angiography and correlation with fractional flow reserve in patients with stable angina. J Am Coll Cardiol 2008; 52:636–643.
7
ARTICLE IN PRESS Academic Radiology, Vol &, No &&, && 2020
KAWASAKI ET AL
5. Cury RC, Abbara S, Achenbach S, et al. CAD-RADS(TM) Coronary Artery Disease - Reporting and Data System. An expert consensus document of the Society of Cardiovascular Computed Tomography (SCCT), the American College of Radiology (ACR) and the North American Society for Cardiovascular Imaging (NASCI). Endorsed by the American College of Cardiology. J Cardiovasc Comput Tomogr 2016; 10:269–281. 6. Tonino PA, De Bruyne B, Pijls NH, et al. Fractional flow reserve versus angiography for guiding percutaneous coronary intervention. N Engl J Med 2009; 360:213–224. 7. De Bruyne B, Fearon WF, Pijls NH, et al. Fractional flow reserve-guided PCI for stable coronary artery disease. N Engl J Med 2014; 371:1208–1217. 8. Pijls NH, van Schaardenburgh P, Manoharan G, et al. Percutaneous coronary intervention of functionally nonsignificant stenosis: 5-year followup of the DEFER Study. J Am Coll Cardiol 2007; 49:2105–2111. 9. Norgaard BL, Leipsic J, Gaur S, et al. Diagnostic performance of noninvasive fractional flow reserve derived from coronary computed tomography angiography in suspected coronary artery disease: the NXT trial (Analysis of Coronary Blood Flow Using CT Angiography: Next Steps). J Am Coll Cardiol 2014; 63:1145–1155. 10. Min JK, Leipsic J, Pencina MJ, et al. Diagnostic accuracy of fractional flow reserve from anatomic CT angiography. JAMA 2012; 308:1237–1245. 11. Ko BS, Cameron JD, Munnur RK, et al. Noninvasive CT-derived FFR based on Structural and fluid analysis: a comparison with invasive FFR for detection of functionally significant stenosis. JACC Cardiovasc Imaging 2017; 10:663–673. 12. Fujimoto S, Kawasaki T, Kumamaru KK, et al. Diagnostic performance of on-site computed CT-fractional flow reserve based on fluid structure interactions: comparison with invasive fractional flow reserve and instantaneous wave-free ratio. Eur Heart J Cardiovasc Imaging 2019; 20:343–352. 13. Cook CM, Petraco R, Shun-Shin MJ, et al. Diagnostic accuracy of computed tomography-derived fractional flow reserve: a systematic review. JAMA Cardiol 2017; 2:803–810. 14. Villines TC. Can CT-derived FFR better inform clinical decision-making and improve outcomes in stable ischaemic heart disease? Eur Heart J 2018; 39:3712–3714. 15. Gaur S, Ovrehus KA, Dey D, et al. Coronary plaque quantification and fractional flow reserve by coronary computed tomography angiography identify ischaemia-causing lesions. Eur Heart J 2016; 37:1220–1227. 16. Ovrehus KA, Gaur S, Leipsic J, et al. CT-based total vessel plaque analyses improves prediction of hemodynamic significance lesions as assessed by fractional flow reserve in patients with stable angina pectoris. J Cardiovasc Comput Tomogr 2018; 12:344–349. 17. Fujimoto S, Kondo T, Kodama T, et al. A novel method for non-invasive plaque morphology analysis by coronary computed tomography angiography. Int J Cardiovasc Imaging 2014; 30:1373–1382. 18. Menze BH, Kelm BM, Masuch R, et al. A comparison of random forest and its Gini importance with standard chemometric methods for the feature selection and classification of spectral data. BMC Bioinformatics 2009; 10:213. 19. Budoff MJ, Young R, Burke G, et al. Ten-year association of coronary artery calcium with atherosclerotic cardiovascular disease (ASCVD)
8
20.
21.
22.
23.
24.
25.
26.
27.
28.
29. 30.
31.
32.
events: the multi-ethnic study of atherosclerosis (MESA). Eur Heart J 2018; 39:2401–2408. Cademartiri F, Mollet NR, Runza G, et al. Influence of intracoronary attenuation on coronary plaque measurements using multislice computed tomography: observations in an ex vivo model of coronary computed tomography angiography. Eur Radiol 2005; 15:1426–1431. Kidoh M, Utsunomiya D, Oda S, et al. Evaluation of the effect of intracoronary attenuation on coronary plaque measurements using a dualphase coronary CT angiography technique on a 320-row CT Scanner in vivo validation study. Acad Radiol 2016; 23:315–320. Celeng C, Leiner T, Maurovich-Horvat P, et al. Anatomical and functional computed tomography for diagnosing hemodynamically significant coronary artery disease: a meta-analysis. JACC Cardiovasc Imaging 2019; 12:1316–1325. Douglas PS, Pontone G, Hlatky MA, et al. Clinical outcomes of fractional flow reserve by computed tomographic angiography-guided diagnostic strategies vs. usual care in patients with suspected coronary artery disease: the prospective longitudinal trial of FFR(CT): outcome and resource impacts study. Eur Heart J 2015; 36:3359–3367. Dey D, Slomka PJ, Leeson P, et al. Artificial intelligence in cardiovascular imaging: JACC State-of-the-Art Review. J Am Coll Cardiol 2019; 73:1317–1335. Ruijsink B, Puyol-Anton E, Oksuz I, et al. Fully automated, qualitycontrolled cardiac analysis from CMR: validation and large-scale application to characterize cardiac function. JACC Cardiovasc Imaging 2019. Zheng Y, John M, Liao R, et al. Automatic aorta segmentation and valve landmark detection in C-arm CT for transcatheter aortic valve implantation. IEEE Trans Med Imaging 2012; 31:2307–2321. Motwani M, Dey D, Berman DS, et al. Machine learning for prediction of all-cause mortality in patients with suspected coronary artery disease: a 5-year multicentre prospective registry analysis. Eur Heart J 2017; 38:500–507. Ozigis MS, Kaduk JD, Jarvis CH, et al. Detection of oil pollution impacts on vegetation using multifrequency SAR, multispectral images with fuzzy forest and random forest methods. Environ Pollut 2020; 256:113360. Chartrand G, Cheng PM, Vorontsov E, et al. Deep learning: a primer for radiologists. Radiographics 2017; 37:2113–2131. van Hamersvelt RW, Zreik M, Voskuil M, et al. Deep learning analysis of left ventricular myocardium in CT angiographic intermediatedegree coronary stenosis improves the diagnostic accuracy for identification of functionally significant stenosis. Eur Radiol 2019; 29:2350–2359. Tang A, Tam R, Cadrin-Chenevert A, et al. Canadian Association of Radiologists White Paper on Artificial Intelligence in Radiology. Can Assoc Radiol J 2018; 69:120–135. Ri K, Kumamaru KK, Fujimoto S, et al. Noninvasive computed tomography-derived fractional flow reserve based on structural and fluid analysis: reproducibility of on-site determination by unexperienced observers. J Comput Assist Tomogr 2018; 42:256–262.
ARTICLE IN PRESS Academic Radiology, Vol &, No &&, && 2020
INVESTIGATING THE EFFECTS OF COMBINED ASSESSMENT
APPENDIX
Appendix Figure. The comparison among the logistic regression, support vector machine, and random forest in the ROC analysis. The AUC to differentiate between nonischemic and ischemic lesions was 0.698, 0.827, and 0.835 for the logistic regression, support vector machine, and random forest, respectively. Note: LR, logistic regression; RF, random forest; SVM, support vector machine.
9