- Email: [email protected]
Accuracy of Left Ventricular Volumes and Ejection Fraction Measurements by Contemporary Three-Dimensional Echocardiography with Semi- and Fully Automated Software: Systematic Review and Meta-Analysis of 1,881 Subjects
THREE-DIMENSIONAL ECHOCARDIOGRAPHY ACCURACY
Accuracy of Left Ventricular Volumes and Ejection Fraction Measurements by Contemporary Three-Dimensional Echocardiography with Semi- and Fully Automated Software: Systematic Review and Meta-Analysis of 1,881 Subjects Tetsuji Kitano, MD, Yosuke Nabeshima, MD, Yutaka Otsuji, MD, PhD, Kazuaki Negishi, MD, PhD, and Masaaki Takeuchi, MD, PhD, FASE, FESC, FJCC, Kitakyushu, Japan; and Hobart, Tasmania and Kingswood, Australia
Background: Although left ventricular (LV) quantification software for transthoracic three-dimensional echocardiography (3DE) have been steadily advancing, the measurement accuracy of semiautomated and fully automated software has not been systematically investigated. Methods: We performed a systematic review and meta-analysis following a search of the PubMed, Embase, and Scopus databases for studies that directly compared LV volumes and ejection fraction (EF) using 3DE with ultrasound machines manufactured by one of the four manufacturers with either semiautomated (n = 4) or fully automated software (n = 2) and cardiac magnetic resonance (CMR) during the period from January 1, 2011, to July 23, 2018. A random effects model was used to determine the bias and 95% CI of LV end-diastolic volume (EDV), end-systolic volume (ESV), and EF. A metaregression and subgroup analysis were performed to investigate intersoftware variabilities. Results: Of a total of 38 studies (1,881 subjects), the pooled bias and 95% CI for EDV, ESV, and EF of semiautomated software were –39.3 mL (–49.2 to –29.5 mL, P < .001), –19.6 mL (–25.9 to –13.3 mL, P < .001), and –0.6% (–2.0% to 0.7%, P = .360), respectively. The corresponding values for fully automated software were –14.5 mL (–25.7 to –3.3 mL, P = .011), –6.3 mL (–11.4 to –1.2 mL, P = .016), and –1.1% (–3.5% to 1.3%, P = .356) compared with CMR. The EDV and ESV biases for 3DE and CMR became significantly smaller and less heterogeneous when fully automated software was used. A metaregression analysis revealed that EDV bias became larger with an increase in EDV when semiautomated software was used but not when fully automated software was used. Conclusions: Although 3DE still underestimates LV volumes, EF showed similar values to CMR for both types of software. The adoption of fully automated software could further improve measurement accuracy with fewer observer variabilities. (J Am Soc Echocardiogr 2019;32:1105-15.) Keywords: Three-dimensional echocardiography, Left ventricle, Ejection fraction, Meta-analysis
From the Second Department of Internal Medicine, University of Occupational and Environmental Health, School of Medicine (T.K., Y.N., Y.O.); Department of Laboratory and Transfusion Medicine, University of Occupational and Environmental Health Hospital (M.T.), Kitakyushu, Japan; Menzies Institute for Medical Research, University of Tasmania, Hobart, Tasmania (K.N.); Nepean Clinical School, University of Sydney, Kingswood (K.N.), Australia. Conflicts of Interest: None. Reprint requests: Masaaki Takeuchi, MD, PhD, FASE, FESC, FJCC, Department of Laboratory and Transfusion Medicine, University of Occupational and Environmental Health Hospital, 1-1 Iseigaoka, Yahatanishi-ku, Kitakyushu 807-8556, Japan (E-mail: [email protected]). 0894-7317/$36.00 Copyright 2019 by the American Society of Echocardiography. https://doi.org/10.1016/j.echo.2019.04.417
Transthoracic three-dimensional echocardiography (3DE) has gained increasing popularity for quantifying left and right heart geometry and function.1 Although its advantages are more obvious for irregularly shaped heart chambers such as the right ventricle, many studies have applied 3DE for the measurements of left ventricular (LV) volumes and ejection fraction (EF) because every ultrasound company provides LV quantification software, and these parameters remain most important for clinical decision-making. During the last decade, two meta-analyses have been conducted on the measurement accuracy of 3DE for LV volumes and EF compared with cardiac magnetic resonance (CMR),2,3 and the studies have consistently demonstrated that 3DE significantly underestimates LV volumes. However, due to rapid advancement of ultrasound equipment/transducer technology and analytical software, some 1105
1106 Kitano et al
ultrasound vendors have now provided fully automated 2D = Two-dimensional 3DE left heart chamber quantification software.4,5 3DE = Three-dimensional When not performing additional echocardiography manual editing, the software CMR = Cardiac magnetic provides consistent LV volumes resonance and EF values from the same 3DE data sets every time. It is EDV = End-diastolic volume also free of any observer EF = Ejection fraction variabilities that are a potential ESV = End-systolic volume source of error. If these software programs prove to be more LV = Left ventricular accurate and reliable for PRISMA = Preferred measuring LV volumes and EF, Reporting Items for the technology can be adopted Systematic reviews and Metain daily clinical practice. Analysis Although LV quantification software packages for transthoracic 3DE have been steadily advancing, the measurement accuracy of semiautomated and fully automated software has not been systematically investigated. Accordingly, we performed this systematic review and meta-analysis to (1) compare the LV volumes and EF biases measured by CMR with semiautomated and fully automated 3DE quantification software and (2) investigate whether image quality, LV size, and analytical software would affect measurement biases. Abbreviations
METHODS Search Strategy We followed the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analysis) guidelines when conducting this systematic review and meta-analysis.6 Two authors (T.K. and Y.N.) used three electronic databases (PubMed, Embase, and Scopus) to systematically search for studies that described a comparison of LV volumes (LV end-diastolic volume [EDV] and LV end-systolic volume [ESV]) and EF using transthoracic 3DE and CMR on July 23, 2018. The key terms used were ‘‘three-dimensional echocardiography’’, ‘‘3D echocardiography’’, ‘‘cardiovascular magnetic resonance’’, and ‘‘cardiac magnetic resonance’’. We selected only human studies. We searched only articles written in English. The search strategies used are listed in Supplemental Table 1 (available at www.onlinejase.com). Since the first study using fully automated 3DE LV quantification software was published in 2011, and given the hypothesis that more recent imaging studies are more accurate and precise, we searched for articles that investigated the use of either semiautomated software or fully automated software during the period from January 1, 2011, to July 23, 2018. The study was prospectively registered with the PROSPERO database of systematic reviews (Accuracy and feasibility of automatic LV analysis software using three-dimensional transthoracic echocardiography compared to CMR imaging analysis; CRD42018106287). Study Selection From the search results, we included studies if they described results of comparisons of EDV, ESV, and EF using 3DE and CMR in adult patients. We included only studies that used either semiautomated or fully automated 3DE quantification software because the manual tracing method used in multiple two-dimensional (2D) images extracted from 3DE data sets still uses geometric assumption, and
Journal of the American Society of Echocardiography September 2019
manual tracing of the whole LV endocardium may have led to potential observer variabilities. Semiautomated software was defined as software that generated LV endocardial contours in the 3D space after manual initialization of some specific anatomical landmarks (both sides of the mitral annulus or center of the mitral annulus and the apex), followed by automatic LV border detection throughout one cardiac cycle using 3DE speckle-tracking analysis or a vendorspecific algorithm, resulting in the creation of an LV volume curve. Fully automated software was defined as software that automatically generated LV casts, followed by time domain LV volume curve analysis without any manual input. We excluded studies that only included children, only evaluated cardiac chambers other than the left ventricle, only used manual tracing of 2D images extracted from 3DE data sets at end diastole and end systole, lacked important information (i.e., vendor and software), or were published in abstract form. Data Extraction The quantitative data collected were mean 6 SD of EDV, ESV, and EF using both 3DE and CMR on the same subjects in each study. If there was no description, but there were graphs showing the Pearson’s correlation of EDV, ESV, and EF between 3DE and CMR, data points were digitized using software (WebPlotDigitizer, ver. 4.1, https://automeris.io/WebPlotDigitizer).7 We extracted all data, from which we finally calculated the mean 6 SD of LV volumes and EF in each modality. In addition to fundamental parameters such as age and gender, we collected information regarding the method of patient selection, number of patients excluded for the analysis, specific type of software used for the analysis, and the name of the transducer for acquiring the data sets. When the studies used several different software programs in the same population, we used all data individually. When the studies described the results using different 3DE acquisition settings, we selected the results that used the highest volume rate setting because we assumed it was the most accurate. When the articles presented the results obtained from several different examiners, we used only the results from the most experienced examiner. Statistical Analysis Continuous data were expressed as mean 6 SD or median and interquartile interval according to the data distribution. Categorical variables were presented as absolute numbers or percentages. The mean bias and its 95% CI of EDV, ESV, and EF for 3DE and CMR were computed using random effects models and were presented as forest plots. Funnel plots were constructed, and the Egger test was used to assess potential publication bias.8 Heterogeneity between subgroups was assessed using the Cochran Q test and the inconsistency factor (I2). Meta-regressions were performed to examine possible factors for bias. The quality of the studies was assessed using the quality assessment tools of Downs and Black.9 A two-sided P value < .05 was considered statistically significant except for in the Egger test, where P < .1 was used. All statistical analyses were performed using commercial software (JMP version 13.1.0, SAS Institute, Cary, NC; R version 3.4.3 and EZR version 1.36, The R Foundation for Statistical Computing, Vienna, Austria).
RESULTS Figure 1 shows the PRISMA flow chart illustrating the selection process used in this study. Among a total of 3,053 titles searched from three databases (PubMed [n = 479], Embase [n = 1,140], and
Kitano et al 1107
Journal of the American Society of Echocardiography Volume 32 Number 9
HIGHLIGHTS A meta-analysis for 3DE and cardiac magnetic resonance. 3DE still significantly underestimated LV volumes but not EF. Underestimation was more obvious when semiautomated software was used. Biases were smaller and less heterogeneous using fully automated software. There were different trends of biases among six 3DE LV quantification programs. Scopus [n = 1,434]), we excluded 2,979 titles due to duplication in 824, publication before 2011 in 598, inappropriate title or abstract in 1,505, and abstract only (not a full article) in 52, leaving 74 titles for further full-text assessment. We excluded 41 articles that met our exclusion criteria after reading the full text, and 33 articles with a total of 1,725 subjects were selected for this meta-analysis.4,5,10-40 Four of these articles reported results on different software programs,23,31,37,39 resulting in 38 distinct studies with data from 1,881 subjects. Table 1 depicts the characteristics of included studies. There were 27 studies (n = 1,259) that used four semiautomated software packages (3DQ Advance, Philips Healthcare, Andover, MA; 4D Auto LVQ, GE Healthcare, Horten, Norway; 3D wall motion tracking, Toshiba Medical Systems, Otawara, Japan; and 4D LV analysis, TomTec Imaging Systems, Unterschleissheim, Germany). A total of 11 studies (n = 622) had used two fully automated software packages (eSie LVA, Siemens AG, Munich, Germany; HeartModel, Philips Healthcare). All studies defined study objectives and described the main findings. Reasons for exclusion and the summary of study quality analysis are found in Supplemental Tables 2 and 3 (available at www.onlinejase.com). Among 38 studies, the mean 6 SD values of LV volumes and/or EF were described in 19 studies. The other 19 studies only showed correlation graphs of LV volumes and/or EF for 3DE and CMR (n = 10) or only showed correlation graphs and had missing data (n = 9). Thus, we calculated mean 6 SD values from extracted data using digitized software in 19 studies (907 subjects). 3DE and CMR Measurements Figures 2-4 present forest plots of the mean differences between 3DE and CMR in EDV, ESV, and EF using semiautomated software and fully automated software. The pooled bias and 95% CI of EDV, ESV, and EF of semiautomated software were –39.3 mL (–49.2 to –29.5 mL, P < .001), –19.6 mL (–25.9 to –13.3 mL, P < .001), and –0.6% (–2.0% to 0.7%, P = .360), respectively. The corresponding values for fully automated software were –14.5 mL (–25.7 to –3.3 mL, P = .011), –6.3 mL (–11.4 to –1.2 mL, P = .016), and –1.1% (–3.5% to 1.3%, P = .356), respectively. Significant heterogeneities were found for EDV, ESV, and EF among the studies (I2: 79%, 65%, and 60%, respectively) using semiautomated software. Corresponding values using fully automated software were 66%, 0%, and 60%, respectively. Thus, we performed a subsequent metaregression analysis. First, we investigated the accuracy of 3DE measurements using semiautomated and fully automated software. The metaregression analysis revealed that there were significant differences in mean differences of both EDV (P = .002) and ESV (P = .009) between semiautomated software and fully automated software (Figure 5A and
5B). However, no significant differences were noted in mean differences of EF between semiautomated software and fully automated software (Figure 5C, P = .670). Second, we investigated the effect of LV size on the bias between 3DE and CMR using semiautomated and fully automated software. The metaregression analysis revealed that EDV bias became larger according to an increase in EDV when semiautomated software was used (Figure 6A: coefficient, –0.333, P = .005) but not when fully automated software was used (Figure 6B: coefficient, 0.012, P = .483). Third, to determine the effect of transducer size on the measurement differences between 3DE and CMR, we selected 338 subjects who had undergone 3DE examinations with a large footprint transducer (X4 or X3-1, Philips Healthcare, n = 268 from six articles)11,16-18,26,30 or a smaller foot print transducer (X5-1, n = 90 from three articles).23,37,39 All analyses were performed on the same semiautomated software (3DQ Advance, Philips Healthcare). The metaregression analysis revealed that there were no significant differences in the mean EDV, ESV, and EF biases measured with the two transducers (EDV, P = .625; ESV, P = .349; EF, P = .522). Finally, we investigated the effect of different software programs on the accuracy of 3DE measurements (Supplemental Figures 1, 2, and 3, available at www.onlinejase.com), where significant differences in bias for different software were demonstrated. Specifically, a fully automated software program (eSie LVA) showed no statistically significant differences in EDV compared with CMR (Supplemental Figure 1D, available at www. onlinejase.com). Neither of the fully automated software packages showed significant differences in ESV against CMR (Supplemental Figure 2B and 2D, available at www.onlinejase.com). All semiautomated software showed significant underestimation of both EDV (Supplemental Figure 1A, 1C, 1E, and 1F, available at www. onlinejase.com) and ESV (Supplemental Figure 2A, 2C, 2E, and 2F, available at www.onlinejase.com) compared with CMR. There were significant differences in EF between CMR and 3DE using GE and TomTec semiautomated software packages (Supplemental Figure 3C and 3F, available at www.onlinejase. com).
DISCUSSION The major findings in this systematic review and meta-analysis can be summarized as follows: (1) 3DE with either semiautomated or fully automated software still significantly underestimated LV volumes compared with CMR; (2) however, both types of software showed no EF bias difference (–0.6% and –1.1%) between the two modalities; (3) the EDV and ESV bias for 3DE and CMR became significantly smaller and less heterogeneous when fully automated software was used; (4) the EDV bias became larger according to an increase in EDV when semiautomated software was used but not when fully automated software was used; and (5) there were different bias trends among the six 3DE LV quantification programs. Previous Studies To the best of our knowledge, there are only two published metaanalyses regarding the measurement accuracy of 3DE for LV volumes and EF compared with CMR.2,3 Table 2 summarizes the main
1108 Kitano et al
Journal of the American Society of Echocardiography September 2019
Figure 1 PRISMA flow chart of study selection. The systematic review and meta-analysis were conducted according to guidelines recommended by PRISMA.
findings in the previous meta-analysis and the current analysis. Shimada and Shiota2 performed a meta-analysis to assess the bias of 3DE in evaluating LV volumes and EF compared with CMR and to investigate factors affecting that bias. The authors searched the medical literature from 1996 to 2010, and their cohort consisted of 3,055 subjects from 95 studies. Although 3DE significantly underestimated EDV (–9.9 mL) and ESV (–4.7 mL), the EF bias was not significant (–0.13%). The metaregression analysis revealed that the use of semiautomated software and matrix array transducers was associated with less underestimation of LV volumes. Dorosz and colleagues3 performed a systematic review of comparative studies of 3DE and CMR published from January 1, 1990, to September 14, 2011, and they collected 23 articles including 1,638 subjects. Approximately half of the articles used software incorporating manual tracing on several 2D cutting planes extracted from 3D data sets, and the other half used semiautomated software. The pooled biases for 3DE compared with CMR were –19.1 mL, –10.1 mL, and –0.6% for EDV, ESV, and EF, respectively. Among the four studies that directly compared software incorporating manual tracing and semiautomated software, pooled biases for the semiautomated software compared to CMR were –22.7 mL, –12.1 mL, and –0.4%, respectively. The corresponding values from the software that used a manual tracing method were –12.6 mL, -8.1 mL, and 1.0%, respectively. Current Study Among 33 articles published from January 1, 2011, to July 23, 2018, that were used for this meta-analysis,4,5,10-40 we found that the
authors used six different software packages including four semiautomated and two fully automated 3DE software packages from four ultrasound system vendors and one vendor-independent software company. Even though the software used for analysis was either semiautomated or fully automated, 3DE significantly underestimated both EDV (semiautomated, –39.3 mL; fully automated, –14.5 mL) and ESV (semiautomated, –19.6 mL; fully automated, –6.3 mL) compared with CMR, but no EF bias (–0.6% and –1.1%) was observed. The results agreed with previous studies.2,3 It is intriguing to note that the bias of LV volumes using semiautomated software in the current analysis was larger than that of previous analyses. This finding is in agreement with the subgroup analysis of Dorosz et al3 This meta-analysis also verified that EDV and ESV biases on 3DE and CMR became significantly smaller and less heterogeneous with fully automated software than with semiautomated software. LV size could have affected the bias of LV volumes on 3DE and CMR (i.e., larger LV volumes = larger bias). Interestingly, the EDV bias became larger according to an increase in EDV when semiautomated software was used, but this trend was not observed when fully automated software was used. This finding might be partly related to different biases of LV volumes against CMR between semiautomated software and fully automated software observed in this study. Since very few studies have described the effect of image quality on the accuracy of 3DE-derived LV volumes and EF,26 we performed a subgroup analysis to determine the effect of 3DE transducers on measurement accuracy to assess the hypothesis that smaller sized 3DE transducers might be associated with better image quality, resulting in a smaller mean bias in the two modalities. Nevertheless, we did
Kitano et al 1109
Journal of the American Society of Echocardiography Volume 32 Number 9
Table 1 Fundamental characteristics in 33 articles Age, mean 6 SD
Volume rate
Vendor
91 (NS)
54 6 14
13.1
Siemens
4Z1c
eSie LVA
52 (37)
62 6 10
NS
Philips
X3-1
3DQ Advance
2011
28 (22)
51 6 16
36
GE
4V
4D Auto LVQ
Armstrong13
2012
114 (47)
38 6 6
NS
GE
NS
4D Auto LVQ
Greupner14
2012
36 (31)
62 6 11
39
GE
3V
4D LV analysis
Kleijn15
2012
45 (35)
39 6 15
20
Toshiba
PST-25SX
3D wall motion tracking
Miller16
2012
60 (41)
61 6 12
22.5
Philips
X3-1
3DQ Advance
Moceri17
2012
24 (17)
58 6 15
NS
Philips
X3-1
3DQ Advance
Thavendiranathan4
2012
67 (42)
45 6 15*
32
Siemens
4Z1c
eSie LVA
Inaba18
2013
23 (19)
55 6 17
NS
Philips
X3-1
3DQ Advance
Shibayama19
2013
41 (30)
63 6 11
22
Siemens
4Z1c
eSie LVA
Thorstensen20
2013
58 (44)
57 6 14
26
GE
3V
4D LV analysis
Zhang21
2013
61 (38)
62 6 14
NS
Siemens
NS
eSie LVA
Aurich22
2014
47 (21)
51 6 19
30-50
GE
4V
4D Auto LVQ
Driessen23
2014
30 (19)
50 6 16
NS
Philips
X5-1
3DQ Advance
Driessen23
2014
27 (19)
50 6 16
NS
3D wall motion tracking
Hoffmann24
2014
63 (49)
64 6 10
17-20
Kawamura25
2014
64 (38)
55 6 12
Saloux26
2014
41 (29)
58 6 15
10
Philips
X3-1
3DQ Advance
Aly27
2016
120 (92)
62 6 13*
23
Toshiba
PST-25SX
3D wall motion tracking
Bech-Hanssen28
2016
83 (70)
NS
NS
GE
3V
4D Auto LVQ
Mizukoshi29
2016
57 (29)
64 6 13
NS
Philips
X5-1
4D LV analysis
Tsang5
2016
65 (31)
50 6 17
16
Philips
X5-1
HeartModel
Squeri30
2016
66 (42)
NS
16
Philips
X3-1
3DQ Advance
Yang31
2016
34 (27)
64 6 12
19.6
Philips
X5-1
HeartModel
Yang31
2016
34 (27)
64 6 12
19.8
Philips
X5-1
4D LV analysis
Levy32
2017
54 (40)
63 6 13
19.6
Philips
X5-1
HeartModel
Lilli
2017
23 (20)
43 6 10
15.7-18.8
Toshiba
PST-25SX
3D wall motion tracking
Spartera34
2017
50 (35)
52 6 19
34
Philips
X3-1
4D LV analysis
Tamborini35
2017
84 (NS)
59 6 15
NS
Philips
X5-1
HeartModel
Barletta36
2018
20 (NS)
NS
NS
Philips
X5-1
HeartModel
Castel37
2018
25 (14)
55 6 20
41
GE
M5S-D
4D Auto LVQ
Castel37
2018
25 (14)
55 6 20
32
Philips
X5-1
3DQ Advance
Levy38
2018
53 (37)
64 6 12
20
Philips
X5-1
HeartModel
Muraru39
2018
35 (22)
44
NS
Philips
X5-1
3DQ Advance
Muraru39
2018
35 (22)
44
NS
Siemens
4Z1c
eSie LVA
Muraru39
2018
35 (22)
44
NS
GE
4V
4D Auto LVQ
Stoebe40
2018
32 (NS)
NS
NS
GE
6VT
4D LV analysis
First author (reference)
Year
Chang10
2011
Marsan11
2011
Reant12
33
Number (men)
30
Probe
Software
Toshiba
PST-25SX
Philips
NS
4D LV analysis
Toshiba
NS
3D wall motion tracking
GE, General Electric; NS, not specified. *Since actual values in the whole subjects were not presented in the manuscript, the values were imputed by mean value, SD, and number of the subjects, which were described in two separate groups.
not find any differences in the EDV and ESV measurement bias for the two different sized 3DE transducers. The proposed reason for large interinstitutional differences in bias of LV volumes, especially when semiautomated software is used, is that interindividual or interinstitutional differences for the manual determination of the LV endocardial border affects the LV volume measurements to a greater degree. A previous study clearly showed
large interinstitutional differences in mean EDV and ESV biases for 3DE and CMR among five internationally recognized cardiovascular institutions where 3DE had been extensively used, and differences as small as 1 mm in tracing the border produced 11% differences in volume measurements.41 The standardization of LV border tracing protocols among individuals/institutions is necessary to overcome this problem.
1110 Kitano et al
Journal of the American Society of Echocardiography September 2019
Figure 2 Forest plots of mean difference in EDV between 3DE and CMR analyzed separately with either semiautomated software (A) or fully automated software (B).
To the best of our knowledge, this is the first study to investigate the accuracy of LV volume and EF measurements according to various 3DE LV quantification software packages. We observed that EDV and ESV biases were quite different among the six subgroup studies based on the use of different software packages. Specifically, fully automated software programs showed less EDV and ESV biases with less heterogeneity compared with semiautomated software programs. Both fully automated software packages used 3DE data sets with one-beat acquisition for the analysis, facilitating expansion of their adoption to patients with irregular heartbeats. The software detects the endocardial surface using a knowledge-based workflow with a 3D discriminative model to match relevant image features of the
given LV shape to the database. However, the current performance of the LV endocardial surface detection algorithm is still not perfect, because additional manual endocardial border editing was frequently performed in the majority of studies, and some studies found that LV volumes measured with fully automated software were smaller than the corresponding volumes obtained after manual endocardial border editing.5 Although our results are promising, further refinement of the fully automated software algorithms and expansion of this technology to other ultrasound companies is needed for their routine clinical adoption. The differences in biases among 3DE LV quantification programs may have implications for serial LV volume determination, for example, in following patients with asymptomatic significant valvular
Journal of the American Society of Echocardiography Volume 32 Number 9
Kitano et al 1111
Figure 3 Forest plots of mean difference in ESV between 3DE and CMR analyzed separately with either semiautomated software (A) or fully automated software (B).
regurgitation. Our results suggest that the same LV quantification software should be used when LV volumes are being followed longitudinally in an individual patient. Study Limitations There are several limitations that should be acknowledged. First, as with all meta-analyses, this work was limited by the variations in the original studies. The observed results could be influenced by publication bias, although funnel plot analysis of EDV and EF showed no significant bias. Second, in half of the studies (19 studies, 907 subjects), true values for mean 6 SD were not provided. Therefore, we used digitized software to estimate the mean 6 SD values using extracted data plots from the graphs showing the Pearson’s correlation of EDV, ESV, and EF for 3DE and CMR. However, we did not find any differ-
ences between the estimated values and true values when we used digitized software in the 16 articles where both true values and graphs were presented (Supplemental Table 4, available at www.onlinejase. com). Even though we performed a meta-analysis without extracted data, there was still a significant difference in EDV bias for semiautomated software and fully automated software. However, we believe the exclusion of these articles may have produced selection bias. Third, four studies reported measurement values separately using different software, and we treated these substudies as individual data sources, and that may have produced some bias. Fourth, there were no uniform border settings applied in the article that used fully automated software,5,31,32,35,36,38 and this might have produced measurement variabilities. Finally, for the subgroup analysis of specific software, we grouped several generations of software as
1112 Kitano et al
Journal of the American Society of Echocardiography September 2019
Figure 4 Forest plots of mean difference in EF between 3DE and CMR analyzed separately with either semiautomated software (A) or fully automated software (B).
one software because many articles did not mention the version of the software.
with less measurement variabilities compared with semiautomated software, which reflects the improved accuracy and reliability of LV volume measurements with the latest software packages.
CONCLUSION In this meta-analysis including articles published after the advent of fully automated software, 3DE still showed a significant underestimation of LV volumes compared with CMR. Nevertheless, fully automated LV quantification software provided smaller LV volumes bias
SUPPLEMENTARY MATERIAL Supplementary data to this article can be found online at https://doi. org/10.1016/j.echo.2019.04.417.
Kitano et al 1113
Journal of the American Society of Echocardiography Volume 32 Number 9
Figure 5 Meta-regression analysis in mean difference of EDV (A), ESV (B), and EF (C) using fully automated software and semiautomated software. The black square of variable size indicates outliers. Full-auto, Fully automated software; Semi-auto, semiautomated software.
Figure 6 Meta-regression analysis of mean differences in EDV using 3DE and CMR according to LV EDV measured by CMR using semiautomated software (A) and fully automated software (B). Values from each study are presented as mean (black square) 6 SD. coef, coefficient.
Table 2 Main findings from previous meta-analysis and current study First author
Publication year
Search period
Type of software
No. of No. of studies samples Bias of EDV [95% CI]
Shimada
2011
1996-2010 Manual or semiautomated
95
3055
Dorosz
2012
1990-2011 Manual or semiautomated
23
1638
This study
2019
2011-18
Semiautomated
27
1259
–39.3 [–49.2, –29.5] –19.6 [–25.9, –13.3] –0.6 [–2.0, 0.7]
Fully automated
11
622
REFERENCES 1. Lang RM, Badano LP, Tsang W, Adams DH, Agricola E, Buck T, et al. EAE/ ASE recommendations for image acquisition and display using threedimensional echocardiography. J Am Soc Echocardiogr 2012;25:3-46. 2. Shimada YJ, Shiota T. A meta-analysis and investigation for the source of bias of left ventricular volumes and function by three-dimensional echocardiography in comparison with magnetic resonance imaging. Am J Cardiol 2011;107:126-38. 3. Dorosz JL, Lezotte DC, Weitzenkamp DA, Allen LA, Salcedo EE. Performance of 3-dimensional echocardiography in measuring left ventricular
Bias of ESV [95% CI]
Bias of EF [95% CI]
–9.9 [–11.8, –8.0]
–4.7 [–5.6, –3.7]
–0.13 [–0.45, 0.18]
–19.1
–10.1
–0.6
–14.5 [–25.7, –3.3]
-6.3 [–11.4, –1.2]
–1.1 [–3.5, 1.3]
volumes and ejection fraction: A systematic review and meta-analysis. J Am Coll Cardiol 2012;59:1799-808. 4. Thavendiranathan P, Liu S, Verhaert D, Calleja A, Nitinunu A, Van Houten T, et al. Feasibility, accuracy, and reproducibility of realtime full-volume 3D transthoracic echocardiography to measure LV volumes and systolic function. J Am Coll Cardiol Imaging 2012;5: 239-51. 5. Tsang W, Salgo IS, Medvedofsky D, Takeuchi M, Prater D, Weinert L, et al. Transthoracic 3D echocardiographic left heart chamber quantification using an automated adaptive analytics algorithm. J Am Coll Cardiol Imaging 2016;9:769-82.
1114 Kitano et al
6. Moher D, Liberati A, Tetzlaff J, Altman DG, Group TP. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement. PLOS Med 2009;6:e1000097. 7. Burda B, O’Connor E, Webber E, Redmond N, Perdue L. Estimating data from figures with a Web-based program: considerations for a systematic review. Res Synth Meth 2017;8:258-62. 8. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. Br Med J 1997;315:629-34. 9. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and nonrandomised studies of health care interventions. J Epidemiol Comm Health 1998;52:377-84. 10. Chang S-A, Lee S-C, Kim E-Y, Hahm S-H, Jang SY, Park S-J, et al. Feasibility of single-beat full-volume capture real-time three-dimensional echocardiography and auto-contouring algorithm for quantification of left ventricular volume: validation with cardiac magnetic resonance imaging. J Am Soc Echocardiogr 2011;24:853-9. 11. Marsan NA, Westenberg JJM, Roes SD, van Bommel RJ, Delgado V, van der Geest RJ, et al. Three-dimensional echocardiography for the preoperative assessment of patients with left ventricular aneurysm. Ann Thorac Surg 2011;91:113-21. 12. Reant P, Barbot L, Montaudon M, Landelle M, Arsac F, Dijos M, et al. Robustness of a new three-dimensional echocardiographic algorithm for left ventricular volume and ejection fraction quantification: experts vs. novices. Eur J Echocardiogr 2011;12:895-903. 13. Armstrong GT, Plana JC, Zhang N, Srivastava D, Green DM, Ness KK, et al. Screening adult survivors of childhood cancer for cardiomyopathy: comparison of echocardiography and cardiac magnetic resonance imaging. J Clin Oncol 2012;30:2876-84. 14. Greupner J, Zimmermann E, Grohmann A, D€ ubel H-P, Althoff TF, Althoff T, et al. Head-to-head comparison of left ventricular function assessment with 64-row computed tomography, biplane left cineventriculography, and both 2- and 3-dimensional transthoracic echocardiography: comparison with magnetic resonance imaging as the reference standard. J Am Coll Cardiol 2012;59:1897-907. 15. Kleijn SA, Brouwer WP, Aly MFA, R€ ussel IK, de Roest GJ, Beek AM, et al. Comparison between three-dimensional speckle-tracking echocardiography and cardiac magnetic resonance imaging for quantification of left ventricular volumes and function. Eur Heart J Cardiovasc Imaging 2012;13: 834-9. 16. Miller CA, Pearce K, Jordan P, Argyle R, Clark D, Stout M, et al. Comparison of real-time three-dimensional echocardiography with cardiovascular magnetic resonance for left ventricular volumetric assessment in unselected patients. Eur Heart J Cardiovasc Imaging 2012;13:187-95. 17. Moceri P, Doyen D, Bertora D, Cerboni P, Ferrari E, Gibelin P. Real time three-dimensional echocardiographic assessment of left ventricular function in heart failure patients: underestimation of left ventricular volume increases with the degree of dilatation. Echocardiography 2012;29:970-7. 18. Inaba T, Yao A, Nakao T, Hatano M, Maki H, Imamura T, et al. Volumetric and functional assessment of ventricles in pulmonary hypertension on 3dimensional echocardiography. Circ J 2013;77:198-206. 19. Shibayama K, Watanabe H, Iguchi N, Sasaki S, Mahara K, Umemura J, et al. Evaluation of automated measurement of left ventricular volume by novel real-time 3-dimensional echocardiographic system: validation with cardiac magnetic resonance imaging and 2-dimensional echocardiography. J Cardiol 2013;61:281-8. 20. Thorstensen A, Dalen H, Hala P, Kiss G, D’hooge J, Torp H, et al. Threedimensional echocardiography in the evaluation of global and regional function in patients with recent myocardial infarction: a comparison with magnetic resonance imaging. Echocardiography 2013;30:682-92. 21. Zhang QB, Sun JP, Gao RF, Lee AP-W, Feng YL, Liu XR, et al. Novel singlebeat full-volume capture real-time three-dimensional echocardiography and auto-contouring algorithm for quantification of left ventricular volume: validation with cardiac magnetic resonance imaging. Int J Cardiol 2013;168:2946-8. 22. Aurich M, Andre F, Keller M, Greiner S, Hess A, Buss SJ, et al. Assessment of left ventricular volumes with echocardiography and cardiac magnetic
Journal of the American Society of Echocardiography September 2019
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
resonance imaging: real-life evaluation of standard versus new semiautomatic methods. J Am Soc Echocardiogr 2014;27:1017-24. Driessen MMP, Kort E, Cramer MJM, Doevendans PA, Angevaare MJ, Leiner T, et al. Assessment of LVejection fraction using real-time 3D echocardiography in daily practice: direct comparison of the volumetric and speckle tracking methodologies to CMR. Neth Heart J 2014;22:383-90. Hoffmann R, Barletta G, von Bardeleben S, Vanoverschelde J-L, Kasprzak J, Greis C, et al. Analysis of left ventricular volumes and function: A multicenter comparison of cardiac magnetic resonance imaging, cine ventriculography, and unenhanced and contrast-enhanced two-dimensional and three-dimensional echocardiography. J Am Soc Echocardiogr 2014;27:292-301. Kawamura R, Seo Y, Ishizu T, Atsumi A, Yamamoto M, MachinoOhtsuka T, et al. Feasibility of left ventricular volume measurements by three-dimensional speckle tracking echocardiography depends on image quality and degree of left ventricular enlargement: validation study with cardiac magnetic resonance imaging. J Cardiol 2014;63:230-8. Saloux E, Labombarda F, Pellissier A, Anthune B, Dugue AE, Provost N, et al. Diagnostic value of three-dimensional contrast-enhanced echocardiography for left ventricular volume and ejection fraction measurement in patients with poor acoustic windows: A comparison of echocardiography and magnetic resonance imaging. J Am Soc Echocardiogr 2014;27: 1029-40. Aly MFA, Kleijn SA, Menken-Negroiu RF, Robbers LF, Beek AM, Kamp O. Three-dimensional speckle tracking echocardiography and cardiac magnetic resonance for left ventricular chamber quantification and identification of myocardial transmural scar. Neth Heart J 2016;24:600-8. Bech-Hanssen O, Polte CL, Lagerstrand KM, Johnsson AA, Fadel BM, Gao SA. Left ventricular volumes by echocardiography in chronic aortic and mitral regurgitations. Scand Cardiovasc J 2016;50:154-61. Mizukoshi K, Takeuchi M, Nagata Y, Addetia K, Lang RM, Akashi YJ, et al. Normal values of left ventricular mass index assessed by transthoracic threedimensional echocardiography. J Am Soc Echocardiogr 2016;29:51-61. Squeri A, Censi S, Reverberi C, Gaibazzi N, Baldelli M, Binno SM, et al. Three-dimensional echocardiography in various types of heart disease: a comparison study of magnetic resonance imaging and 64-slice computed tomography in a real-world population. J Echocardiogr 2016;15:18-26. Yang L-T, Nagata Y, Otani K, Kado Y, Otsuji Y, Takeuchi M. Feasibility of one-beat real-time full-volume three-dimensional echocardiography for assessing left ventricular volumes and deformation parameters. J Am Soc Echocardiogr 2016;29:853-60.e2. Levy F, Schouver E-D, Iacuzio L, Civaia F, Rusek S, Dommerc C, et al. Performance of new automated transthoracic three-dimensional echocardiographic software for left ventricular volumes and function assessment in routine clinical practice: Comparison with 3 Tesla cardiac magnetic resonance. Arch Cardiovasc Dis 2017;110:580-9. Lilli A, Tessa C, Diciotti S, Croisille P, Clarysse P, Del Meglio J, et al. Simultaneous strain-volume analysis by three-dimensional echocardiography. J Cardiovasc Med 2017;18:223-9. Spartera M, Damascelli A, Mozes F, De Cobelli F, La Canna G. Threedimensional speckle tracking longitudinal strain is related to myocardial fibrosis determined by late-gadolinium enhancement. Int J Cardiovasc Imaging 2017;33:1351-60. Tamborini G, Piazzese C, Lang RM, Muratori M, Chiorino E, Mapelli M, et al. Feasibility and accuracy of automated software for transthoracic three-dimensional left ventricular volume and function analysis: comparisons with two-dimensional echocardiography, three-dimensional transthoracic manual method, and cardiac magnetic resonance imaging. J Am Soc Echocardiogr 2017;30:1049-58. Barletta V, Hinojar R, Carbonell A, Gonzalez-Gomez A, Fabiani I, Di Bello V, et al. Three-dimensional full automated software in the evaluation of the left ventricle function: from theory to clinical practice. Int J Cardiovasc Imaging 2018;34:1205-13. Castel A-L, Toledano M, Tribouilloy C, Delelis F, Mailliet A, Marotte N, et al. Assessment of left ventricular size and function by 3-dimensional transthoracic echocardiography: impact of the echocardiography platform and analysis software. Am Heart J 2018;202:127-36.
Journal of the American Society of Echocardiography Volume 32 Number 9
38. Levy F, Marechaux S, Iacuzio L, Schouver E-D, Castel A-L, Toledano M, et al. Quantitative assessment of primary mitral regurgitation using left ventricular volumes obtained with new automated three-dimensional transthoracic echocardiographic software: a comparison with 3-Tesla cardiac magnetic resonance. Arch Cardiovasc Dis 2018;111:507-17. 39. Muraru D, Cecchetto A, Cucchini U, Zhou X, Lang RM, Romeo G, et al. Intervendor consistency and accuracy of left ventricular volume measure-
Kitano et al 1115
ments using three-dimensional echocardiography. J Am Soc Echocardiogr 2018;31:158-68.e1. 40. Stoebe S, Metze M, Jurisch D, Tayal B, Solty K, Laufs U, et al. Analysis of chronic aortic regurgitation by 2D and 3D echocardiography and cardiac MRI. Echo Res Pract 2018;5:51-62. 41. Mor-Avi V, Jenkins C, K€ uhl HP, Nesser H-J, Marwick T, Franke A, et al. Real-time 3-dimensional echocardiographic quantification of left ventricular volumes. J Am Coll Cardiol Imaging 2008;1:413-23.
1115.e1 Kitano et al
Journal of the American Society of Echocardiography September 2019
APPENDIX
Supplemental Figure 1 Forest plots of mean difference in EDV for 3DE and CMR according to six different software packages. (A) 3DQ Advance (Philips Healthcare); (B) HeartModel (Philips Healthcare); (C) 4D Auto LVQ (GE Healthcare); (D) eSie LVA (Siemens AG); (E) 3D wall motion tracking (Toshiba Medical Systems); (F) 4D LV analysis (TomTec Imaging Systems).
Supplemental Figure 2 Forest plots of mean difference in ESV for 3DE and CMR according to six different software packages. Letters (A)-(F) refer to the same software packages as in Supplemental Figure 1.
Journal of the American Society of Echocardiography Volume 32 Number 9
Kitano et al 1115.e2
Supplemental Figure 3 Forest plots of mean difference in EF for 3DE and CMR according to six different software packages. Letters (A)-(F) refer to the same software packages as in Supplemental Figure 1.
Supplemental Table 1 Search strategies EMBASE Search 3D echo and CMR Search date on July 23, 2018 EMBASE: 1. ‘/three dimensional echocardiography/or exp three dimensional imaging/’ (90,227 articles) 2. ‘exp cardiovascular magnetic resonance/’ (with all subheadings) (26,795 articles) 3. no. 1 AND no. 2 Results: 1,140 articles Scopus search 3D echo and CMR Search date on July 23, 2018 Scopus: 1. (TITLE-ABS-KEY(three AND dimensional AND echocardiography)) (14,467 articles) 2. (TITLE-ABS-KEY(cardiac AND magnetic AND resonance)) (41,099 articles) 3. no. 1 AND no. 2 Filters: English, Humans, Human Results: 1,434 articles PubMed search 3D echo and CMR Search date on July 23, 2018 PubMed: Keywords: Search ((3d echocardiography) AND (cardiac magnetic resonance)) Filters: English, Humans, Adults Results: 479 articles Records after duplicates removed n = 2,253
1115.e3 Kitano et al
Journal of the American Society of Echocardiography September 2019
Supplemental Table 2 Reasons for full-text exclusion First author
Year
Reason for exclusion
D’Andrea
2011
Not including CMR data
Caselli
2012
Not including CMR data
Coon
2012
Not semi- or fully automated software, 3DQ, Philips
Hayat
2012
Not including LV volume and LVEF data, only strain data
La Gerche
2012
Not including CMR data
Muraru
2012
No available data
Saha
2012
Not including CMR data
Chang
2013
Not including LV volume and LVEF data, only LV mass data
Grabka
2013
No available data
Kusunose
2013
Not including LV volume and LVEF data, only LV mass data
Pacileo
2013
Not including LV volume and LVEF data, only LV mass data
Abate
2014
No available data
Caiani
2014
No available data
Goleski
2014
No available data
Hoffmann
2014
No available data
Kaku
2014
Not including LV volume and LVEF data, only strain data
Meris
2014
Not including CMR data, only 2DE and 3DE
Wu
2014
No available data
Ylanen
2014
Not absolute data, only index data
Choi
2015
Not including LV volume and LVEF data, only regurgitation volume
Heggemann
2015
Not including LV volume and LVEF data, only strain data
Jeong
2015
Not including 3DE data, only CMR data
Kim
2015
Not including LV data, only RV data
Leng
2015
Not including 3DE data, only CMR data
Onishi
2015
Not including 3DE data
Polte
2015
No available data
Avegliano
2016
Not including LV volume and LVEF data, only LV mass data
Fredriksson
2016
Not including LV data, only RV data
Kara
2016
Not including 3DE data, only 2DE data
Obokata
2016
Not including LV volume and LVEF data, only strain data
Soslow
2016
Data including children
Toro-Salazar
2016
Data including children
Trotier
2016
No available data
Zhou
2016
No available data
Nacif
2017
Not including 3DE data, only 2DE data
Quick
2017
Not including 3DE data, only CMR data
Satriano
2017
Not including LV volume and LVEF data, only strain data
Wissmann
2017
No available data
Barba
2018
Not including 4DE data, only CMR and CT data
Carminati
2018
No available data
Parsaee
2018
Not including CMR data, only 2DE and 3DE
Kitano et al 1115.e4
Journal of the American Society of Echocardiography Volume 32 Number 9
Supplemental Table 3 Summary of studies quality analysis Reporting First author (reference)
Internal validity
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
Chang
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Marsan11
Yes
Yes
Yes
Yes
Yes
NR
Yes
Yes
Yes
Reant
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Armstrong13
Yes
Yes
Yes
Yes
Yes
NR
Yes
Yes
No
Greupner14
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Kleijn15
Yes
Yes
No
Yes
Yes
NR
Yes
Yes
No
Miller16
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Moceri17
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Thavendiranathan4
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Inaba18
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Shibayama19
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Thorstensen
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Zhang21
Yes
No
No
Yes
Yes
NR
No
No
Yes
Aurich22
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Driessen23
Yes
Yes
No
Yes
Yes
NR
Yes
Yes
Yes
Hoffmann24
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Kawamura25
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Saloux26
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Aly27
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Bech-Hanssen
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Mizukoshi29
Yes
Yes
Yes
Yes
Yes
NR
Yes
Yes
Yes
Tsang5
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Squeri30
Yes
Yes
Yes
No
Yes
Yes
Yes
Yes
Yes
Yang31
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Levy32
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Lilli33
Yes
Yes
No
No
Yes
Yes
Yes
Yes
No
Spartera34
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Tamborini35
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Barletta
Yes
Yes
Yes
Yes
Yes
NR
Yes
Yes
No
Castel37
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Levy38
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Muraru39
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Stoebe40
Yes
Yes
Yes
Yes
Yes
NR
Yes
Yes
No
10
12
20
28
36
(1) Purpose described; (2) outcome described; (3) characteristics described; (4) confound described; (5) main findings described; (6) blinded to CMR data; (7) echo imaging protocol; (8) CMR imaging protocol; (9) reproducibility analysis performed. NR, Not reported.
1115.e5 Kitano et al
Journal of the American Society of Echocardiography September 2019
Supplemental Table 4 Reliability analysis for the use of WebPlotDigitizer 3DE Mean
SD
n
True
Estimated
Absolute difference
% Var
P value
ICC
True
Estimated
Absolute difference
% Var
P value
ICC
EDV, mL
12
161.4
160.5
0.9
1.2
.525
0.995
57.1
57.8
0.7
3.4
.642
0.909
ESV, mL
12
90.1
90.5
0.4
1.7
.593
0.998
50.2
50.9
0.8
2.6
.509
0.962
EF, %
14
47.7
47.6
0.2
1.8
.654
0.990
12.4
12.7
0.3
6.5
.314
0.971
Absolute difference
% Var
P value
ICC
Variable
CMR Mean
SD
n
True
Estimated
Absolute difference
% Var
P value
ICC
True
EDV, mL
12
189.2
188.4
0.8
0.8
.405
0.997
68.9
68.4
0.5
4.6
.821
0.874
ESV, mL
12
106.6
106.1
0.5
1.6
.683
0.995
63.2
63.7
0.5
1.4
.375
0.995
EF, %
14
49.0
49.0
0.0
1.9
.979
0.987
13.8
14.0
0.2
6.1
.563
0.973
Estimated
ICC, intraclass correlation; % var, percent variability which was defined as absolute difference between true value and estimated value divided by true value (%).
Accuracy of Left Ventricular Volumes and Ejection Fraction Measurements by Contemporary Three-Dimensional Echocardiography with Semi- and Fully Automated Software: Systematic Review and Meta-Analysis of 1,881 Subjects Tetsuji Kitano, MD, Yosuke Nabeshima, MD, Yutaka Otsuji, MD, PhD, Kazuaki Negishi, MD, PhD, and Masaaki Takeuchi, MD, PhD, FASE, FESC, FJCC, Kitakyushu, Japan; and Hobart, Tasmania and Kingswood, Australia
Background: Although left ventricular (LV) quantification software for transthoracic three-dimensional echocardiography (3DE) have been steadily advancing, the measurement accuracy of semiautomated and fully automated software has not been systematically investigated. Methods: We performed a systematic review and meta-analysis following a search of the PubMed, Embase, and Scopus databases for studies that directly compared LV volumes and ejection fraction (EF) using 3DE with ultrasound machines manufactured by one of the four manufacturers with either semiautomated (n = 4) or fully automated software (n = 2) and cardiac magnetic resonance (CMR) during the period from January 1, 2011, to July 23, 2018. A random effects model was used to determine the bias and 95% CI of LV end-diastolic volume (EDV), end-systolic volume (ESV), and EF. A metaregression and subgroup analysis were performed to investigate intersoftware variabilities. Results: Of a total of 38 studies (1,881 subjects), the pooled bias and 95% CI for EDV, ESV, and EF of semiautomated software were –39.3 mL (–49.2 to –29.5 mL, P < .001), –19.6 mL (–25.9 to –13.3 mL, P < .001), and –0.6% (–2.0% to 0.7%, P = .360), respectively. The corresponding values for fully automated software were –14.5 mL (–25.7 to –3.3 mL, P = .011), –6.3 mL (–11.4 to –1.2 mL, P = .016), and –1.1% (–3.5% to 1.3%, P = .356) compared with CMR. The EDV and ESV biases for 3DE and CMR became significantly smaller and less heterogeneous when fully automated software was used. A metaregression analysis revealed that EDV bias became larger with an increase in EDV when semiautomated software was used but not when fully automated software was used. Conclusions: Although 3DE still underestimates LV volumes, EF showed similar values to CMR for both types of software. The adoption of fully automated software could further improve measurement accuracy with fewer observer variabilities. (J Am Soc Echocardiogr 2019;32:1105-15.) Keywords: Three-dimensional echocardiography, Left ventricle, Ejection fraction, Meta-analysis
From the Second Department of Internal Medicine, University of Occupational and Environmental Health, School of Medicine (T.K., Y.N., Y.O.); Department of Laboratory and Transfusion Medicine, University of Occupational and Environmental Health Hospital (M.T.), Kitakyushu, Japan; Menzies Institute for Medical Research, University of Tasmania, Hobart, Tasmania (K.N.); Nepean Clinical School, University of Sydney, Kingswood (K.N.), Australia. Conflicts of Interest: None. Reprint requests: Masaaki Takeuchi, MD, PhD, FASE, FESC, FJCC, Department of Laboratory and Transfusion Medicine, University of Occupational and Environmental Health Hospital, 1-1 Iseigaoka, Yahatanishi-ku, Kitakyushu 807-8556, Japan (E-mail: [email protected]). 0894-7317/$36.00 Copyright 2019 by the American Society of Echocardiography. https://doi.org/10.1016/j.echo.2019.04.417
Transthoracic three-dimensional echocardiography (3DE) has gained increasing popularity for quantifying left and right heart geometry and function.1 Although its advantages are more obvious for irregularly shaped heart chambers such as the right ventricle, many studies have applied 3DE for the measurements of left ventricular (LV) volumes and ejection fraction (EF) because every ultrasound company provides LV quantification software, and these parameters remain most important for clinical decision-making. During the last decade, two meta-analyses have been conducted on the measurement accuracy of 3DE for LV volumes and EF compared with cardiac magnetic resonance (CMR),2,3 and the studies have consistently demonstrated that 3DE significantly underestimates LV volumes. However, due to rapid advancement of ultrasound equipment/transducer technology and analytical software, some 1105
1106 Kitano et al
ultrasound vendors have now provided fully automated 2D = Two-dimensional 3DE left heart chamber quantification software.4,5 3DE = Three-dimensional When not performing additional echocardiography manual editing, the software CMR = Cardiac magnetic provides consistent LV volumes resonance and EF values from the same 3DE data sets every time. It is EDV = End-diastolic volume also free of any observer EF = Ejection fraction variabilities that are a potential ESV = End-systolic volume source of error. If these software programs prove to be more LV = Left ventricular accurate and reliable for PRISMA = Preferred measuring LV volumes and EF, Reporting Items for the technology can be adopted Systematic reviews and Metain daily clinical practice. Analysis Although LV quantification software packages for transthoracic 3DE have been steadily advancing, the measurement accuracy of semiautomated and fully automated software has not been systematically investigated. Accordingly, we performed this systematic review and meta-analysis to (1) compare the LV volumes and EF biases measured by CMR with semiautomated and fully automated 3DE quantification software and (2) investigate whether image quality, LV size, and analytical software would affect measurement biases. Abbreviations
METHODS Search Strategy We followed the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analysis) guidelines when conducting this systematic review and meta-analysis.6 Two authors (T.K. and Y.N.) used three electronic databases (PubMed, Embase, and Scopus) to systematically search for studies that described a comparison of LV volumes (LV end-diastolic volume [EDV] and LV end-systolic volume [ESV]) and EF using transthoracic 3DE and CMR on July 23, 2018. The key terms used were ‘‘three-dimensional echocardiography’’, ‘‘3D echocardiography’’, ‘‘cardiovascular magnetic resonance’’, and ‘‘cardiac magnetic resonance’’. We selected only human studies. We searched only articles written in English. The search strategies used are listed in Supplemental Table 1 (available at www.onlinejase.com). Since the first study using fully automated 3DE LV quantification software was published in 2011, and given the hypothesis that more recent imaging studies are more accurate and precise, we searched for articles that investigated the use of either semiautomated software or fully automated software during the period from January 1, 2011, to July 23, 2018. The study was prospectively registered with the PROSPERO database of systematic reviews (Accuracy and feasibility of automatic LV analysis software using three-dimensional transthoracic echocardiography compared to CMR imaging analysis; CRD42018106287). Study Selection From the search results, we included studies if they described results of comparisons of EDV, ESV, and EF using 3DE and CMR in adult patients. We included only studies that used either semiautomated or fully automated 3DE quantification software because the manual tracing method used in multiple two-dimensional (2D) images extracted from 3DE data sets still uses geometric assumption, and
Journal of the American Society of Echocardiography September 2019
manual tracing of the whole LV endocardium may have led to potential observer variabilities. Semiautomated software was defined as software that generated LV endocardial contours in the 3D space after manual initialization of some specific anatomical landmarks (both sides of the mitral annulus or center of the mitral annulus and the apex), followed by automatic LV border detection throughout one cardiac cycle using 3DE speckle-tracking analysis or a vendorspecific algorithm, resulting in the creation of an LV volume curve. Fully automated software was defined as software that automatically generated LV casts, followed by time domain LV volume curve analysis without any manual input. We excluded studies that only included children, only evaluated cardiac chambers other than the left ventricle, only used manual tracing of 2D images extracted from 3DE data sets at end diastole and end systole, lacked important information (i.e., vendor and software), or were published in abstract form. Data Extraction The quantitative data collected were mean 6 SD of EDV, ESV, and EF using both 3DE and CMR on the same subjects in each study. If there was no description, but there were graphs showing the Pearson’s correlation of EDV, ESV, and EF between 3DE and CMR, data points were digitized using software (WebPlotDigitizer, ver. 4.1, https://automeris.io/WebPlotDigitizer).7 We extracted all data, from which we finally calculated the mean 6 SD of LV volumes and EF in each modality. In addition to fundamental parameters such as age and gender, we collected information regarding the method of patient selection, number of patients excluded for the analysis, specific type of software used for the analysis, and the name of the transducer for acquiring the data sets. When the studies used several different software programs in the same population, we used all data individually. When the studies described the results using different 3DE acquisition settings, we selected the results that used the highest volume rate setting because we assumed it was the most accurate. When the articles presented the results obtained from several different examiners, we used only the results from the most experienced examiner. Statistical Analysis Continuous data were expressed as mean 6 SD or median and interquartile interval according to the data distribution. Categorical variables were presented as absolute numbers or percentages. The mean bias and its 95% CI of EDV, ESV, and EF for 3DE and CMR were computed using random effects models and were presented as forest plots. Funnel plots were constructed, and the Egger test was used to assess potential publication bias.8 Heterogeneity between subgroups was assessed using the Cochran Q test and the inconsistency factor (I2). Meta-regressions were performed to examine possible factors for bias. The quality of the studies was assessed using the quality assessment tools of Downs and Black.9 A two-sided P value < .05 was considered statistically significant except for in the Egger test, where P < .1 was used. All statistical analyses were performed using commercial software (JMP version 13.1.0, SAS Institute, Cary, NC; R version 3.4.3 and EZR version 1.36, The R Foundation for Statistical Computing, Vienna, Austria).
RESULTS Figure 1 shows the PRISMA flow chart illustrating the selection process used in this study. Among a total of 3,053 titles searched from three databases (PubMed [n = 479], Embase [n = 1,140], and
Kitano et al 1107
Journal of the American Society of Echocardiography Volume 32 Number 9
HIGHLIGHTS A meta-analysis for 3DE and cardiac magnetic resonance. 3DE still significantly underestimated LV volumes but not EF. Underestimation was more obvious when semiautomated software was used. Biases were smaller and less heterogeneous using fully automated software. There were different trends of biases among six 3DE LV quantification programs. Scopus [n = 1,434]), we excluded 2,979 titles due to duplication in 824, publication before 2011 in 598, inappropriate title or abstract in 1,505, and abstract only (not a full article) in 52, leaving 74 titles for further full-text assessment. We excluded 41 articles that met our exclusion criteria after reading the full text, and 33 articles with a total of 1,725 subjects were selected for this meta-analysis.4,5,10-40 Four of these articles reported results on different software programs,23,31,37,39 resulting in 38 distinct studies with data from 1,881 subjects. Table 1 depicts the characteristics of included studies. There were 27 studies (n = 1,259) that used four semiautomated software packages (3DQ Advance, Philips Healthcare, Andover, MA; 4D Auto LVQ, GE Healthcare, Horten, Norway; 3D wall motion tracking, Toshiba Medical Systems, Otawara, Japan; and 4D LV analysis, TomTec Imaging Systems, Unterschleissheim, Germany). A total of 11 studies (n = 622) had used two fully automated software packages (eSie LVA, Siemens AG, Munich, Germany; HeartModel, Philips Healthcare). All studies defined study objectives and described the main findings. Reasons for exclusion and the summary of study quality analysis are found in Supplemental Tables 2 and 3 (available at www.onlinejase.com). Among 38 studies, the mean 6 SD values of LV volumes and/or EF were described in 19 studies. The other 19 studies only showed correlation graphs of LV volumes and/or EF for 3DE and CMR (n = 10) or only showed correlation graphs and had missing data (n = 9). Thus, we calculated mean 6 SD values from extracted data using digitized software in 19 studies (907 subjects). 3DE and CMR Measurements Figures 2-4 present forest plots of the mean differences between 3DE and CMR in EDV, ESV, and EF using semiautomated software and fully automated software. The pooled bias and 95% CI of EDV, ESV, and EF of semiautomated software were –39.3 mL (–49.2 to –29.5 mL, P < .001), –19.6 mL (–25.9 to –13.3 mL, P < .001), and –0.6% (–2.0% to 0.7%, P = .360), respectively. The corresponding values for fully automated software were –14.5 mL (–25.7 to –3.3 mL, P = .011), –6.3 mL (–11.4 to –1.2 mL, P = .016), and –1.1% (–3.5% to 1.3%, P = .356), respectively. Significant heterogeneities were found for EDV, ESV, and EF among the studies (I2: 79%, 65%, and 60%, respectively) using semiautomated software. Corresponding values using fully automated software were 66%, 0%, and 60%, respectively. Thus, we performed a subsequent metaregression analysis. First, we investigated the accuracy of 3DE measurements using semiautomated and fully automated software. The metaregression analysis revealed that there were significant differences in mean differences of both EDV (P = .002) and ESV (P = .009) between semiautomated software and fully automated software (Figure 5A and
5B). However, no significant differences were noted in mean differences of EF between semiautomated software and fully automated software (Figure 5C, P = .670). Second, we investigated the effect of LV size on the bias between 3DE and CMR using semiautomated and fully automated software. The metaregression analysis revealed that EDV bias became larger according to an increase in EDV when semiautomated software was used (Figure 6A: coefficient, –0.333, P = .005) but not when fully automated software was used (Figure 6B: coefficient, 0.012, P = .483). Third, to determine the effect of transducer size on the measurement differences between 3DE and CMR, we selected 338 subjects who had undergone 3DE examinations with a large footprint transducer (X4 or X3-1, Philips Healthcare, n = 268 from six articles)11,16-18,26,30 or a smaller foot print transducer (X5-1, n = 90 from three articles).23,37,39 All analyses were performed on the same semiautomated software (3DQ Advance, Philips Healthcare). The metaregression analysis revealed that there were no significant differences in the mean EDV, ESV, and EF biases measured with the two transducers (EDV, P = .625; ESV, P = .349; EF, P = .522). Finally, we investigated the effect of different software programs on the accuracy of 3DE measurements (Supplemental Figures 1, 2, and 3, available at www.onlinejase.com), where significant differences in bias for different software were demonstrated. Specifically, a fully automated software program (eSie LVA) showed no statistically significant differences in EDV compared with CMR (Supplemental Figure 1D, available at www. onlinejase.com). Neither of the fully automated software packages showed significant differences in ESV against CMR (Supplemental Figure 2B and 2D, available at www.onlinejase.com). All semiautomated software showed significant underestimation of both EDV (Supplemental Figure 1A, 1C, 1E, and 1F, available at www. onlinejase.com) and ESV (Supplemental Figure 2A, 2C, 2E, and 2F, available at www.onlinejase.com) compared with CMR. There were significant differences in EF between CMR and 3DE using GE and TomTec semiautomated software packages (Supplemental Figure 3C and 3F, available at www.onlinejase. com).
DISCUSSION The major findings in this systematic review and meta-analysis can be summarized as follows: (1) 3DE with either semiautomated or fully automated software still significantly underestimated LV volumes compared with CMR; (2) however, both types of software showed no EF bias difference (–0.6% and –1.1%) between the two modalities; (3) the EDV and ESV bias for 3DE and CMR became significantly smaller and less heterogeneous when fully automated software was used; (4) the EDV bias became larger according to an increase in EDV when semiautomated software was used but not when fully automated software was used; and (5) there were different bias trends among the six 3DE LV quantification programs. Previous Studies To the best of our knowledge, there are only two published metaanalyses regarding the measurement accuracy of 3DE for LV volumes and EF compared with CMR.2,3 Table 2 summarizes the main
1108 Kitano et al
Journal of the American Society of Echocardiography September 2019
Figure 1 PRISMA flow chart of study selection. The systematic review and meta-analysis were conducted according to guidelines recommended by PRISMA.
findings in the previous meta-analysis and the current analysis. Shimada and Shiota2 performed a meta-analysis to assess the bias of 3DE in evaluating LV volumes and EF compared with CMR and to investigate factors affecting that bias. The authors searched the medical literature from 1996 to 2010, and their cohort consisted of 3,055 subjects from 95 studies. Although 3DE significantly underestimated EDV (–9.9 mL) and ESV (–4.7 mL), the EF bias was not significant (–0.13%). The metaregression analysis revealed that the use of semiautomated software and matrix array transducers was associated with less underestimation of LV volumes. Dorosz and colleagues3 performed a systematic review of comparative studies of 3DE and CMR published from January 1, 1990, to September 14, 2011, and they collected 23 articles including 1,638 subjects. Approximately half of the articles used software incorporating manual tracing on several 2D cutting planes extracted from 3D data sets, and the other half used semiautomated software. The pooled biases for 3DE compared with CMR were –19.1 mL, –10.1 mL, and –0.6% for EDV, ESV, and EF, respectively. Among the four studies that directly compared software incorporating manual tracing and semiautomated software, pooled biases for the semiautomated software compared to CMR were –22.7 mL, –12.1 mL, and –0.4%, respectively. The corresponding values from the software that used a manual tracing method were –12.6 mL, -8.1 mL, and 1.0%, respectively. Current Study Among 33 articles published from January 1, 2011, to July 23, 2018, that were used for this meta-analysis,4,5,10-40 we found that the
authors used six different software packages including four semiautomated and two fully automated 3DE software packages from four ultrasound system vendors and one vendor-independent software company. Even though the software used for analysis was either semiautomated or fully automated, 3DE significantly underestimated both EDV (semiautomated, –39.3 mL; fully automated, –14.5 mL) and ESV (semiautomated, –19.6 mL; fully automated, –6.3 mL) compared with CMR, but no EF bias (–0.6% and –1.1%) was observed. The results agreed with previous studies.2,3 It is intriguing to note that the bias of LV volumes using semiautomated software in the current analysis was larger than that of previous analyses. This finding is in agreement with the subgroup analysis of Dorosz et al3 This meta-analysis also verified that EDV and ESV biases on 3DE and CMR became significantly smaller and less heterogeneous with fully automated software than with semiautomated software. LV size could have affected the bias of LV volumes on 3DE and CMR (i.e., larger LV volumes = larger bias). Interestingly, the EDV bias became larger according to an increase in EDV when semiautomated software was used, but this trend was not observed when fully automated software was used. This finding might be partly related to different biases of LV volumes against CMR between semiautomated software and fully automated software observed in this study. Since very few studies have described the effect of image quality on the accuracy of 3DE-derived LV volumes and EF,26 we performed a subgroup analysis to determine the effect of 3DE transducers on measurement accuracy to assess the hypothesis that smaller sized 3DE transducers might be associated with better image quality, resulting in a smaller mean bias in the two modalities. Nevertheless, we did
Kitano et al 1109
Journal of the American Society of Echocardiography Volume 32 Number 9
Table 1 Fundamental characteristics in 33 articles Age, mean 6 SD
Volume rate
Vendor
91 (NS)
54 6 14
13.1
Siemens
4Z1c
eSie LVA
52 (37)
62 6 10
NS
Philips
X3-1
3DQ Advance
2011
28 (22)
51 6 16
36
GE
4V
4D Auto LVQ
Armstrong13
2012
114 (47)
38 6 6
NS
GE
NS
4D Auto LVQ
Greupner14
2012
36 (31)
62 6 11
39
GE
3V
4D LV analysis
Kleijn15
2012
45 (35)
39 6 15
20
Toshiba
PST-25SX
3D wall motion tracking
Miller16
2012
60 (41)
61 6 12
22.5
Philips
X3-1
3DQ Advance
Moceri17
2012
24 (17)
58 6 15
NS
Philips
X3-1
3DQ Advance
Thavendiranathan4
2012
67 (42)
45 6 15*
32
Siemens
4Z1c
eSie LVA
Inaba18
2013
23 (19)
55 6 17
NS
Philips
X3-1
3DQ Advance
Shibayama19
2013
41 (30)
63 6 11
22
Siemens
4Z1c
eSie LVA
Thorstensen20
2013
58 (44)
57 6 14
26
GE
3V
4D LV analysis
Zhang21
2013
61 (38)
62 6 14
NS
Siemens
NS
eSie LVA
Aurich22
2014
47 (21)
51 6 19
30-50
GE
4V
4D Auto LVQ
Driessen23
2014
30 (19)
50 6 16
NS
Philips
X5-1
3DQ Advance
Driessen23
2014
27 (19)
50 6 16
NS
3D wall motion tracking
Hoffmann24
2014
63 (49)
64 6 10
17-20
Kawamura25
2014
64 (38)
55 6 12
Saloux26
2014
41 (29)
58 6 15
10
Philips
X3-1
3DQ Advance
Aly27
2016
120 (92)
62 6 13*
23
Toshiba
PST-25SX
3D wall motion tracking
Bech-Hanssen28
2016
83 (70)
NS
NS
GE
3V
4D Auto LVQ
Mizukoshi29
2016
57 (29)
64 6 13
NS
Philips
X5-1
4D LV analysis
Tsang5
2016
65 (31)
50 6 17
16
Philips
X5-1
HeartModel
Squeri30
2016
66 (42)
NS
16
Philips
X3-1
3DQ Advance
Yang31
2016
34 (27)
64 6 12
19.6
Philips
X5-1
HeartModel
Yang31
2016
34 (27)
64 6 12
19.8
Philips
X5-1
4D LV analysis
Levy32
2017
54 (40)
63 6 13
19.6
Philips
X5-1
HeartModel
Lilli
2017
23 (20)
43 6 10
15.7-18.8
Toshiba
PST-25SX
3D wall motion tracking
Spartera34
2017
50 (35)
52 6 19
34
Philips
X3-1
4D LV analysis
Tamborini35
2017
84 (NS)
59 6 15
NS
Philips
X5-1
HeartModel
Barletta36
2018
20 (NS)
NS
NS
Philips
X5-1
HeartModel
Castel37
2018
25 (14)
55 6 20
41
GE
M5S-D
4D Auto LVQ
Castel37
2018
25 (14)
55 6 20
32
Philips
X5-1
3DQ Advance
Levy38
2018
53 (37)
64 6 12
20
Philips
X5-1
HeartModel
Muraru39
2018
35 (22)
44
NS
Philips
X5-1
3DQ Advance
Muraru39
2018
35 (22)
44
NS
Siemens
4Z1c
eSie LVA
Muraru39
2018
35 (22)
44
NS
GE
4V
4D Auto LVQ
Stoebe40
2018
32 (NS)
NS
NS
GE
6VT
4D LV analysis
First author (reference)
Year
Chang10
2011
Marsan11
2011
Reant12
33
Number (men)
30
Probe
Software
Toshiba
PST-25SX
Philips
NS
4D LV analysis
Toshiba
NS
3D wall motion tracking
GE, General Electric; NS, not specified. *Since actual values in the whole subjects were not presented in the manuscript, the values were imputed by mean value, SD, and number of the subjects, which were described in two separate groups.
not find any differences in the EDV and ESV measurement bias for the two different sized 3DE transducers. The proposed reason for large interinstitutional differences in bias of LV volumes, especially when semiautomated software is used, is that interindividual or interinstitutional differences for the manual determination of the LV endocardial border affects the LV volume measurements to a greater degree. A previous study clearly showed
large interinstitutional differences in mean EDV and ESV biases for 3DE and CMR among five internationally recognized cardiovascular institutions where 3DE had been extensively used, and differences as small as 1 mm in tracing the border produced 11% differences in volume measurements.41 The standardization of LV border tracing protocols among individuals/institutions is necessary to overcome this problem.
1110 Kitano et al
Journal of the American Society of Echocardiography September 2019
Figure 2 Forest plots of mean difference in EDV between 3DE and CMR analyzed separately with either semiautomated software (A) or fully automated software (B).
To the best of our knowledge, this is the first study to investigate the accuracy of LV volume and EF measurements according to various 3DE LV quantification software packages. We observed that EDV and ESV biases were quite different among the six subgroup studies based on the use of different software packages. Specifically, fully automated software programs showed less EDV and ESV biases with less heterogeneity compared with semiautomated software programs. Both fully automated software packages used 3DE data sets with one-beat acquisition for the analysis, facilitating expansion of their adoption to patients with irregular heartbeats. The software detects the endocardial surface using a knowledge-based workflow with a 3D discriminative model to match relevant image features of the
given LV shape to the database. However, the current performance of the LV endocardial surface detection algorithm is still not perfect, because additional manual endocardial border editing was frequently performed in the majority of studies, and some studies found that LV volumes measured with fully automated software were smaller than the corresponding volumes obtained after manual endocardial border editing.5 Although our results are promising, further refinement of the fully automated software algorithms and expansion of this technology to other ultrasound companies is needed for their routine clinical adoption. The differences in biases among 3DE LV quantification programs may have implications for serial LV volume determination, for example, in following patients with asymptomatic significant valvular
Journal of the American Society of Echocardiography Volume 32 Number 9
Kitano et al 1111
Figure 3 Forest plots of mean difference in ESV between 3DE and CMR analyzed separately with either semiautomated software (A) or fully automated software (B).
regurgitation. Our results suggest that the same LV quantification software should be used when LV volumes are being followed longitudinally in an individual patient. Study Limitations There are several limitations that should be acknowledged. First, as with all meta-analyses, this work was limited by the variations in the original studies. The observed results could be influenced by publication bias, although funnel plot analysis of EDV and EF showed no significant bias. Second, in half of the studies (19 studies, 907 subjects), true values for mean 6 SD were not provided. Therefore, we used digitized software to estimate the mean 6 SD values using extracted data plots from the graphs showing the Pearson’s correlation of EDV, ESV, and EF for 3DE and CMR. However, we did not find any differ-
ences between the estimated values and true values when we used digitized software in the 16 articles where both true values and graphs were presented (Supplemental Table 4, available at www.onlinejase. com). Even though we performed a meta-analysis without extracted data, there was still a significant difference in EDV bias for semiautomated software and fully automated software. However, we believe the exclusion of these articles may have produced selection bias. Third, four studies reported measurement values separately using different software, and we treated these substudies as individual data sources, and that may have produced some bias. Fourth, there were no uniform border settings applied in the article that used fully automated software,5,31,32,35,36,38 and this might have produced measurement variabilities. Finally, for the subgroup analysis of specific software, we grouped several generations of software as
1112 Kitano et al
Journal of the American Society of Echocardiography September 2019
Figure 4 Forest plots of mean difference in EF between 3DE and CMR analyzed separately with either semiautomated software (A) or fully automated software (B).
one software because many articles did not mention the version of the software.
with less measurement variabilities compared with semiautomated software, which reflects the improved accuracy and reliability of LV volume measurements with the latest software packages.
CONCLUSION In this meta-analysis including articles published after the advent of fully automated software, 3DE still showed a significant underestimation of LV volumes compared with CMR. Nevertheless, fully automated LV quantification software provided smaller LV volumes bias
SUPPLEMENTARY MATERIAL Supplementary data to this article can be found online at https://doi. org/10.1016/j.echo.2019.04.417.
Kitano et al 1113
Journal of the American Society of Echocardiography Volume 32 Number 9
Figure 5 Meta-regression analysis in mean difference of EDV (A), ESV (B), and EF (C) using fully automated software and semiautomated software. The black square of variable size indicates outliers. Full-auto, Fully automated software; Semi-auto, semiautomated software.
Figure 6 Meta-regression analysis of mean differences in EDV using 3DE and CMR according to LV EDV measured by CMR using semiautomated software (A) and fully automated software (B). Values from each study are presented as mean (black square) 6 SD. coef, coefficient.
Table 2 Main findings from previous meta-analysis and current study First author
Publication year
Search period
Type of software
No. of No. of studies samples Bias of EDV [95% CI]
Shimada
2011
1996-2010 Manual or semiautomated
95
3055
Dorosz
2012
1990-2011 Manual or semiautomated
23
1638
This study
2019
2011-18
Semiautomated
27
1259
–39.3 [–49.2, –29.5] –19.6 [–25.9, –13.3] –0.6 [–2.0, 0.7]
Fully automated
11
622
REFERENCES 1. Lang RM, Badano LP, Tsang W, Adams DH, Agricola E, Buck T, et al. EAE/ ASE recommendations for image acquisition and display using threedimensional echocardiography. J Am Soc Echocardiogr 2012;25:3-46. 2. Shimada YJ, Shiota T. A meta-analysis and investigation for the source of bias of left ventricular volumes and function by three-dimensional echocardiography in comparison with magnetic resonance imaging. Am J Cardiol 2011;107:126-38. 3. Dorosz JL, Lezotte DC, Weitzenkamp DA, Allen LA, Salcedo EE. Performance of 3-dimensional echocardiography in measuring left ventricular
Bias of ESV [95% CI]
Bias of EF [95% CI]
–9.9 [–11.8, –8.0]
–4.7 [–5.6, –3.7]
–0.13 [–0.45, 0.18]
–19.1
–10.1
–0.6
–14.5 [–25.7, –3.3]
-6.3 [–11.4, –1.2]
–1.1 [–3.5, 1.3]
volumes and ejection fraction: A systematic review and meta-analysis. J Am Coll Cardiol 2012;59:1799-808. 4. Thavendiranathan P, Liu S, Verhaert D, Calleja A, Nitinunu A, Van Houten T, et al. Feasibility, accuracy, and reproducibility of realtime full-volume 3D transthoracic echocardiography to measure LV volumes and systolic function. J Am Coll Cardiol Imaging 2012;5: 239-51. 5. Tsang W, Salgo IS, Medvedofsky D, Takeuchi M, Prater D, Weinert L, et al. Transthoracic 3D echocardiographic left heart chamber quantification using an automated adaptive analytics algorithm. J Am Coll Cardiol Imaging 2016;9:769-82.
1114 Kitano et al
6. Moher D, Liberati A, Tetzlaff J, Altman DG, Group TP. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement. PLOS Med 2009;6:e1000097. 7. Burda B, O’Connor E, Webber E, Redmond N, Perdue L. Estimating data from figures with a Web-based program: considerations for a systematic review. Res Synth Meth 2017;8:258-62. 8. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. Br Med J 1997;315:629-34. 9. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and nonrandomised studies of health care interventions. J Epidemiol Comm Health 1998;52:377-84. 10. Chang S-A, Lee S-C, Kim E-Y, Hahm S-H, Jang SY, Park S-J, et al. Feasibility of single-beat full-volume capture real-time three-dimensional echocardiography and auto-contouring algorithm for quantification of left ventricular volume: validation with cardiac magnetic resonance imaging. J Am Soc Echocardiogr 2011;24:853-9. 11. Marsan NA, Westenberg JJM, Roes SD, van Bommel RJ, Delgado V, van der Geest RJ, et al. Three-dimensional echocardiography for the preoperative assessment of patients with left ventricular aneurysm. Ann Thorac Surg 2011;91:113-21. 12. Reant P, Barbot L, Montaudon M, Landelle M, Arsac F, Dijos M, et al. Robustness of a new three-dimensional echocardiographic algorithm for left ventricular volume and ejection fraction quantification: experts vs. novices. Eur J Echocardiogr 2011;12:895-903. 13. Armstrong GT, Plana JC, Zhang N, Srivastava D, Green DM, Ness KK, et al. Screening adult survivors of childhood cancer for cardiomyopathy: comparison of echocardiography and cardiac magnetic resonance imaging. J Clin Oncol 2012;30:2876-84. 14. Greupner J, Zimmermann E, Grohmann A, D€ ubel H-P, Althoff TF, Althoff T, et al. Head-to-head comparison of left ventricular function assessment with 64-row computed tomography, biplane left cineventriculography, and both 2- and 3-dimensional transthoracic echocardiography: comparison with magnetic resonance imaging as the reference standard. J Am Coll Cardiol 2012;59:1897-907. 15. Kleijn SA, Brouwer WP, Aly MFA, R€ ussel IK, de Roest GJ, Beek AM, et al. Comparison between three-dimensional speckle-tracking echocardiography and cardiac magnetic resonance imaging for quantification of left ventricular volumes and function. Eur Heart J Cardiovasc Imaging 2012;13: 834-9. 16. Miller CA, Pearce K, Jordan P, Argyle R, Clark D, Stout M, et al. Comparison of real-time three-dimensional echocardiography with cardiovascular magnetic resonance for left ventricular volumetric assessment in unselected patients. Eur Heart J Cardiovasc Imaging 2012;13:187-95. 17. Moceri P, Doyen D, Bertora D, Cerboni P, Ferrari E, Gibelin P. Real time three-dimensional echocardiographic assessment of left ventricular function in heart failure patients: underestimation of left ventricular volume increases with the degree of dilatation. Echocardiography 2012;29:970-7. 18. Inaba T, Yao A, Nakao T, Hatano M, Maki H, Imamura T, et al. Volumetric and functional assessment of ventricles in pulmonary hypertension on 3dimensional echocardiography. Circ J 2013;77:198-206. 19. Shibayama K, Watanabe H, Iguchi N, Sasaki S, Mahara K, Umemura J, et al. Evaluation of automated measurement of left ventricular volume by novel real-time 3-dimensional echocardiographic system: validation with cardiac magnetic resonance imaging and 2-dimensional echocardiography. J Cardiol 2013;61:281-8. 20. Thorstensen A, Dalen H, Hala P, Kiss G, D’hooge J, Torp H, et al. Threedimensional echocardiography in the evaluation of global and regional function in patients with recent myocardial infarction: a comparison with magnetic resonance imaging. Echocardiography 2013;30:682-92. 21. Zhang QB, Sun JP, Gao RF, Lee AP-W, Feng YL, Liu XR, et al. Novel singlebeat full-volume capture real-time three-dimensional echocardiography and auto-contouring algorithm for quantification of left ventricular volume: validation with cardiac magnetic resonance imaging. Int J Cardiol 2013;168:2946-8. 22. Aurich M, Andre F, Keller M, Greiner S, Hess A, Buss SJ, et al. Assessment of left ventricular volumes with echocardiography and cardiac magnetic
Journal of the American Society of Echocardiography September 2019
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
resonance imaging: real-life evaluation of standard versus new semiautomatic methods. J Am Soc Echocardiogr 2014;27:1017-24. Driessen MMP, Kort E, Cramer MJM, Doevendans PA, Angevaare MJ, Leiner T, et al. Assessment of LVejection fraction using real-time 3D echocardiography in daily practice: direct comparison of the volumetric and speckle tracking methodologies to CMR. Neth Heart J 2014;22:383-90. Hoffmann R, Barletta G, von Bardeleben S, Vanoverschelde J-L, Kasprzak J, Greis C, et al. Analysis of left ventricular volumes and function: A multicenter comparison of cardiac magnetic resonance imaging, cine ventriculography, and unenhanced and contrast-enhanced two-dimensional and three-dimensional echocardiography. J Am Soc Echocardiogr 2014;27:292-301. Kawamura R, Seo Y, Ishizu T, Atsumi A, Yamamoto M, MachinoOhtsuka T, et al. Feasibility of left ventricular volume measurements by three-dimensional speckle tracking echocardiography depends on image quality and degree of left ventricular enlargement: validation study with cardiac magnetic resonance imaging. J Cardiol 2014;63:230-8. Saloux E, Labombarda F, Pellissier A, Anthune B, Dugue AE, Provost N, et al. Diagnostic value of three-dimensional contrast-enhanced echocardiography for left ventricular volume and ejection fraction measurement in patients with poor acoustic windows: A comparison of echocardiography and magnetic resonance imaging. J Am Soc Echocardiogr 2014;27: 1029-40. Aly MFA, Kleijn SA, Menken-Negroiu RF, Robbers LF, Beek AM, Kamp O. Three-dimensional speckle tracking echocardiography and cardiac magnetic resonance for left ventricular chamber quantification and identification of myocardial transmural scar. Neth Heart J 2016;24:600-8. Bech-Hanssen O, Polte CL, Lagerstrand KM, Johnsson AA, Fadel BM, Gao SA. Left ventricular volumes by echocardiography in chronic aortic and mitral regurgitations. Scand Cardiovasc J 2016;50:154-61. Mizukoshi K, Takeuchi M, Nagata Y, Addetia K, Lang RM, Akashi YJ, et al. Normal values of left ventricular mass index assessed by transthoracic threedimensional echocardiography. J Am Soc Echocardiogr 2016;29:51-61. Squeri A, Censi S, Reverberi C, Gaibazzi N, Baldelli M, Binno SM, et al. Three-dimensional echocardiography in various types of heart disease: a comparison study of magnetic resonance imaging and 64-slice computed tomography in a real-world population. J Echocardiogr 2016;15:18-26. Yang L-T, Nagata Y, Otani K, Kado Y, Otsuji Y, Takeuchi M. Feasibility of one-beat real-time full-volume three-dimensional echocardiography for assessing left ventricular volumes and deformation parameters. J Am Soc Echocardiogr 2016;29:853-60.e2. Levy F, Schouver E-D, Iacuzio L, Civaia F, Rusek S, Dommerc C, et al. Performance of new automated transthoracic three-dimensional echocardiographic software for left ventricular volumes and function assessment in routine clinical practice: Comparison with 3 Tesla cardiac magnetic resonance. Arch Cardiovasc Dis 2017;110:580-9. Lilli A, Tessa C, Diciotti S, Croisille P, Clarysse P, Del Meglio J, et al. Simultaneous strain-volume analysis by three-dimensional echocardiography. J Cardiovasc Med 2017;18:223-9. Spartera M, Damascelli A, Mozes F, De Cobelli F, La Canna G. Threedimensional speckle tracking longitudinal strain is related to myocardial fibrosis determined by late-gadolinium enhancement. Int J Cardiovasc Imaging 2017;33:1351-60. Tamborini G, Piazzese C, Lang RM, Muratori M, Chiorino E, Mapelli M, et al. Feasibility and accuracy of automated software for transthoracic three-dimensional left ventricular volume and function analysis: comparisons with two-dimensional echocardiography, three-dimensional transthoracic manual method, and cardiac magnetic resonance imaging. J Am Soc Echocardiogr 2017;30:1049-58. Barletta V, Hinojar R, Carbonell A, Gonzalez-Gomez A, Fabiani I, Di Bello V, et al. Three-dimensional full automated software in the evaluation of the left ventricle function: from theory to clinical practice. Int J Cardiovasc Imaging 2018;34:1205-13. Castel A-L, Toledano M, Tribouilloy C, Delelis F, Mailliet A, Marotte N, et al. Assessment of left ventricular size and function by 3-dimensional transthoracic echocardiography: impact of the echocardiography platform and analysis software. Am Heart J 2018;202:127-36.
Journal of the American Society of Echocardiography Volume 32 Number 9
38. Levy F, Marechaux S, Iacuzio L, Schouver E-D, Castel A-L, Toledano M, et al. Quantitative assessment of primary mitral regurgitation using left ventricular volumes obtained with new automated three-dimensional transthoracic echocardiographic software: a comparison with 3-Tesla cardiac magnetic resonance. Arch Cardiovasc Dis 2018;111:507-17. 39. Muraru D, Cecchetto A, Cucchini U, Zhou X, Lang RM, Romeo G, et al. Intervendor consistency and accuracy of left ventricular volume measure-
Kitano et al 1115
ments using three-dimensional echocardiography. J Am Soc Echocardiogr 2018;31:158-68.e1. 40. Stoebe S, Metze M, Jurisch D, Tayal B, Solty K, Laufs U, et al. Analysis of chronic aortic regurgitation by 2D and 3D echocardiography and cardiac MRI. Echo Res Pract 2018;5:51-62. 41. Mor-Avi V, Jenkins C, K€ uhl HP, Nesser H-J, Marwick T, Franke A, et al. Real-time 3-dimensional echocardiographic quantification of left ventricular volumes. J Am Coll Cardiol Imaging 2008;1:413-23.
1115.e1 Kitano et al
Journal of the American Society of Echocardiography September 2019
APPENDIX
Supplemental Figure 1 Forest plots of mean difference in EDV for 3DE and CMR according to six different software packages. (A) 3DQ Advance (Philips Healthcare); (B) HeartModel (Philips Healthcare); (C) 4D Auto LVQ (GE Healthcare); (D) eSie LVA (Siemens AG); (E) 3D wall motion tracking (Toshiba Medical Systems); (F) 4D LV analysis (TomTec Imaging Systems).
Supplemental Figure 2 Forest plots of mean difference in ESV for 3DE and CMR according to six different software packages. Letters (A)-(F) refer to the same software packages as in Supplemental Figure 1.
Journal of the American Society of Echocardiography Volume 32 Number 9
Kitano et al 1115.e2
Supplemental Figure 3 Forest plots of mean difference in EF for 3DE and CMR according to six different software packages. Letters (A)-(F) refer to the same software packages as in Supplemental Figure 1.
Supplemental Table 1 Search strategies EMBASE Search 3D echo and CMR Search date on July 23, 2018 EMBASE: 1. ‘/three dimensional echocardiography/or exp three dimensional imaging/’ (90,227 articles) 2. ‘exp cardiovascular magnetic resonance/’ (with all subheadings) (26,795 articles) 3. no. 1 AND no. 2 Results: 1,140 articles Scopus search 3D echo and CMR Search date on July 23, 2018 Scopus: 1. (TITLE-ABS-KEY(three AND dimensional AND echocardiography)) (14,467 articles) 2. (TITLE-ABS-KEY(cardiac AND magnetic AND resonance)) (41,099 articles) 3. no. 1 AND no. 2 Filters: English, Humans, Human Results: 1,434 articles PubMed search 3D echo and CMR Search date on July 23, 2018 PubMed: Keywords: Search ((3d echocardiography) AND (cardiac magnetic resonance)) Filters: English, Humans, Adults Results: 479 articles Records after duplicates removed n = 2,253
1115.e3 Kitano et al
Journal of the American Society of Echocardiography September 2019
Supplemental Table 2 Reasons for full-text exclusion First author
Year
Reason for exclusion
D’Andrea
2011
Not including CMR data
Caselli
2012
Not including CMR data
Coon
2012
Not semi- or fully automated software, 3DQ, Philips
Hayat
2012
Not including LV volume and LVEF data, only strain data
La Gerche
2012
Not including CMR data
Muraru
2012
No available data
Saha
2012
Not including CMR data
Chang
2013
Not including LV volume and LVEF data, only LV mass data
Grabka
2013
No available data
Kusunose
2013
Not including LV volume and LVEF data, only LV mass data
Pacileo
2013
Not including LV volume and LVEF data, only LV mass data
Abate
2014
No available data
Caiani
2014
No available data
Goleski
2014
No available data
Hoffmann
2014
No available data
Kaku
2014
Not including LV volume and LVEF data, only strain data
Meris
2014
Not including CMR data, only 2DE and 3DE
Wu
2014
No available data
Ylanen
2014
Not absolute data, only index data
Choi
2015
Not including LV volume and LVEF data, only regurgitation volume
Heggemann
2015
Not including LV volume and LVEF data, only strain data
Jeong
2015
Not including 3DE data, only CMR data
Kim
2015
Not including LV data, only RV data
Leng
2015
Not including 3DE data, only CMR data
Onishi
2015
Not including 3DE data
Polte
2015
No available data
Avegliano
2016
Not including LV volume and LVEF data, only LV mass data
Fredriksson
2016
Not including LV data, only RV data
Kara
2016
Not including 3DE data, only 2DE data
Obokata
2016
Not including LV volume and LVEF data, only strain data
Soslow
2016
Data including children
Toro-Salazar
2016
Data including children
Trotier
2016
No available data
Zhou
2016
No available data
Nacif
2017
Not including 3DE data, only 2DE data
Quick
2017
Not including 3DE data, only CMR data
Satriano
2017
Not including LV volume and LVEF data, only strain data
Wissmann
2017
No available data
Barba
2018
Not including 4DE data, only CMR and CT data
Carminati
2018
No available data
Parsaee
2018
Not including CMR data, only 2DE and 3DE
Kitano et al 1115.e4
Journal of the American Society of Echocardiography Volume 32 Number 9
Supplemental Table 3 Summary of studies quality analysis Reporting First author (reference)
Internal validity
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
Chang
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Marsan11
Yes
Yes
Yes
Yes
Yes
NR
Yes
Yes
Yes
Reant
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Armstrong13
Yes
Yes
Yes
Yes
Yes
NR
Yes
Yes
No
Greupner14
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Kleijn15
Yes
Yes
No
Yes
Yes
NR
Yes
Yes
No
Miller16
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Moceri17
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Thavendiranathan4
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Inaba18
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Shibayama19
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Thorstensen
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Zhang21
Yes
No
No
Yes
Yes
NR
No
No
Yes
Aurich22
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Driessen23
Yes
Yes
No
Yes
Yes
NR
Yes
Yes
Yes
Hoffmann24
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Kawamura25
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Saloux26
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Aly27
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Bech-Hanssen
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Mizukoshi29
Yes
Yes
Yes
Yes
Yes
NR
Yes
Yes
Yes
Tsang5
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Squeri30
Yes
Yes
Yes
No
Yes
Yes
Yes
Yes
Yes
Yang31
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Levy32
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Lilli33
Yes
Yes
No
No
Yes
Yes
Yes
Yes
No
Spartera34
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Tamborini35
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Barletta
Yes
Yes
Yes
Yes
Yes
NR
Yes
Yes
No
Castel37
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Levy38
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Muraru39
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Stoebe40
Yes
Yes
Yes
Yes
Yes
NR
Yes
Yes
No
10
12
20
28
36
(1) Purpose described; (2) outcome described; (3) characteristics described; (4) confound described; (5) main findings described; (6) blinded to CMR data; (7) echo imaging protocol; (8) CMR imaging protocol; (9) reproducibility analysis performed. NR, Not reported.
1115.e5 Kitano et al
Journal of the American Society of Echocardiography September 2019
Supplemental Table 4 Reliability analysis for the use of WebPlotDigitizer 3DE Mean
SD
n
True
Estimated
Absolute difference
% Var
P value
ICC
True
Estimated
Absolute difference
% Var
P value
ICC
EDV, mL
12
161.4
160.5
0.9
1.2
.525
0.995
57.1
57.8
0.7
3.4
.642
0.909
ESV, mL
12
90.1
90.5
0.4
1.7
.593
0.998
50.2
50.9
0.8
2.6
.509
0.962
EF, %
14
47.7
47.6
0.2
1.8
.654
0.990
12.4
12.7
0.3
6.5
.314
0.971
Absolute difference
% Var
P value
ICC
Variable
CMR Mean
SD
n
True
Estimated
Absolute difference
% Var
P value
ICC
True
EDV, mL
12
189.2
188.4
0.8
0.8
.405
0.997
68.9
68.4
0.5
4.6
.821
0.874
ESV, mL
12
106.6
106.1
0.5
1.6
.683
0.995
63.2
63.7
0.5
1.4
.375
0.995
EF, %
14
49.0
49.0
0.0
1.9
.979
0.987
13.8
14.0
0.2
6.1
.563
0.973
Estimated
ICC, intraclass correlation; % var, percent variability which was defined as absolute difference between true value and estimated value divided by true value (%).