- Email: [email protected]
Carotid plaque morphometric assessment with three-dimensional ultrasound imaging
From the Society for Vascular Surgery
Carotid plaque morphometric assessment with three-dimensional ultrasound imaging Khalid AlMuhanna, MS,a,b Md Murad Hossain, MS,b Limin Zhao, MBBS,a,c Jonathan Fischell, BS,a Gregory Kowalewski, MS,a,c Moira Dux, PhD,c Siddhartha Sikdar, PhD,b and Brajesh K. Lal, MD,a,c Baltimore, Md; and Fairfax, Va Objective: As investigations into nonsurgical treatment for atherosclerosis expand, the measurement of plaque regression and progression has become an important end point to evaluate. Measurements of three-dimensional (3D) plaque volume are more reliable and sensitive to change than are traditional estimates of stenosis severity or cross-sectional area. 3D ultrasound (3D US) imaging may allow monitoring of plaque volume changes but has not been used routinely due to the cumbersome motorized units required to drive transducers. We investigated the variability, reliability, and the least amount of change detectable by 1D plaque measures, as well as 2D and 3D measures of plaque morphometry, that can be applied in a clinical environment. Methods: 3D US imaging was obtained in 10 patients with carotid stenosis. The lumen and outer wall boundaries were outlined in serial cross-sectional images 1 mm apart. Three observers manually segmented vessel wall volumes (VWVs), and the segmentation was repeated again 4 weeks later. This allowed measurement of interobserver and intraobserver variability of 6 pairs of observations. We measured Bland-Altman statistics, intraclass correlation coefficients, coefficient of variability, and the minimum detectable plaque change for each morphometric measure. Results: The mean VWV of carotid lesions in the study was 1276.8 mm3 (range, 620.6-1956.3 mm3). Bland-Altman plots demonstrated low interobserver and intraobserver variability. The interobserver variability of volume measurements as a function of mean volume was 14.8% and interobserver variability was 8.9%. Reliability was 87% as quantified by the interclass correlation and was 95% by the intraclass correlation. The least detectable change in VWV was 12.9% for interobserver variability and 4.5% for intraobserver variability for the three observers. Conclusions: Carotid plaque diameter measurements from B-mode images have high variability. Plaque burden, as estimated by VWV, can be measured reliably with a 3D US technique using a clinical scanner. The volumetric change, with 95% confidence, that must be observed to establish that a plaque has undergone growth or regression is w12.9% for different observers and 4.5% for the same observer performing the follow-up study. (J Vasc Surg 2015;61:690-7.)
Traditional methods of assessing the severity of carotid atherosclerosis use a Doppler velocity-based classification of stenosis categories in broad ranges. However, these ranges of stenoses are insensitive to small changes in the plaque burden. An alternative is direct measurement of diameter reduction by B-mode imaging. However, plaques progress along the length of an artery 2.4 times faster than they thicken.1 Therefore, methods that capture longitudinal and circumferential growth (ie, area From the Center for Vascular Diagnostics, Division of Vascular Surgery, University of Maryland School of Medicine, Baltimorea; the Department of Electrical and Computer Engineering, George Mason University, Fairfaxb; and the Vascular Service, Veterans Affairs Medical Center, Baltimore.c This work was supported by a Veterans Affairs (VA) Merit Review Grant Award to B.K.L. Author conflict of interest: none. Presented at the 2014 Vascular Annual Meeting of the Society for Vascular Surgery, Boston, Mass, June 4-7, 2014. Reprint requests: Brajesh K. Lal, MD, 22 S Greene St, S10-B00, Baltimore, MD 21201 (e-mail: [email protected]). The editors and reviewers of this article have no relevant financial relationships to disclose per the JVS policy that requires reviewers to decline review of any manuscript for which they may have a conflict of interest. 0741-5214 Copyright Ó 2015 by the Society for Vascular Surgery. Published by Elsevier Inc. http://dx.doi.org/10.1016/j.jvs.2014.10.003
690
and volume) are inherently more sensitive to a change in plaque burden (change in size of the plaque) than methods limited to thickness measurements (ie, diameter-reducing stenosis). These measures are important in clinical practice as emphasis increases for pharmacologic management of asymptomatic carotid atherosclerosis. The ability to accurately detect and quantify plaque change will become a critical determinant of treatment success or failure, which may predict a reduced or increased risk for stroke.2,3 Only three-dimensional (3D) plaque imaging can capture all critical dimensions of a plaque. Although computed tomography (CT) or magnetic resonance imaging (MRI) offer 3D imaging capabilities, they are expensive, often associated with nephrotoxic contrast agents (CT and MRI) and radiation exposure (CT), and are less suitable for longitudinal life-long surveillance of carotid plaques. Imaging with 3D ultrasound (US) can reduce the operator variability inherent in traditional US imaging and is economical and safe for serial testing. However, previously reported 3D US methods for carotid plaque imaging have involved cumbersome and specialized motorized units that move the transducer across the neck, required significant postprocessing time and experience, and have not received widespread clinical acceptance.4
JOURNAL OF VASCULAR SURGERY Volume 61, Number 3
In this study, we tested the reliability of 3D plaque imaging using a new commercially available 3D US transducer. We measured operator variability, reliability, and the least amount of change detectable by traditional 1D plaque measures, as well as nontraditional 2D and 3D measures of plaque morphometry. Our study forms the first comprehensive assessment of the same carotid plaques for 1D (diameter), 2D (longitudinal and cross-sectional area), and 3D (volume) measures. We also provide one of the first evaluations of the reliability of atherosclerotic carotid vessel wall volume (VWV) measurements as a measure of total plaque burden using a clinical 3D US transducer. METHODS The University of Maryland School of Medicine Institutional Review Board and the VA Medical Center Research and Development Committee, in Baltimore, Md, approved the protocol for this study. Patients. The study enrolled 10 consecutive patients with $50% asymptomatic carotid stenosis. Patients with an occlusion in the target artery were excluded. Demographics and risk factors recorded prospectively were ethnicity, diabetes mellitus, coronary artery disease, hypertension, hypercholesterolemia, smoking, and peripheral arterial occlusive disease. The patients were recruited from the Veteran Affairs Medical Center in Baltimore, Md, and provided informed consent. Clinical duplex US examination. Patients first underwent a standard carotid duplex US examination according to recommendations of the Inter-societal Accreditation Commission using a Sonix MDP system (Ultrasonix, Richmond, BC, Canada) and an L9-4/38 linear probe. The degree of stenosis in the carotid artery was estimated using Doppler velocities with appropriate angle correction according to standard techniques used by our group5-7 (Fig 1, A). We used consensus velocity criteria to define the degree of stenosis for the purposes of inclusion into the study.8 2D US imaging protocol. Standard clinical B-mode imaging techniques were used to define the least luminal diameter (LLD) and plaque area in longitudinal and cross-sectional views as reported previously by our group.6,9 As in clinical testing protocols, the sonographer was free to select the optimal insonation angle to obtain the best image of the plaque. A longitudinal image (Fig 1, B) was obtained first, and the transducer was then swept from the base of the neck to the angle of the mandible to identify and record the cross-sectional image where the tightest stenosis was visualized (Fig 1, C). The images were digitally recorded and analyzed off-line with a computer-assisted image-analysis program by independent observers blinded to clinical findings. The entire US examination was recorded on digital video for subsequent interpretation. 3D US imaging protocol. A standardized imaging protocol described by our group previously10 was used to obtain 3D images of the carotid artery with a 4DL14-5/38 transducer (Fig 1, D). This 3D probe consists of a
AlMuhanna et al 691
motorized linear array transducer that moves within a housing to capture a sequence of 2D image frames at different elevation angles, which can then be reconstructed into a 3D volume (Fig 1, E). The operator’s hand and the patient were held still to reduce movement artifacts. A complete volume was acquired in <1 second, which minimized movement artifact from cardiac and respiratory movements. Image analysis protocols. The acquired volumes consisted of raw unprocessed US data after envelope detection. The volumes were reconstructed using custom software developed using MATLAB (The MathWorks Inc, Natick, Mass). Brightness levels were adjusted using an automatic adaptive histogram-equalization method with a Rayleigh distribution11 to enhance visualization of the vessel boundary. After postprocessing, three observers analyzed the images from the 10 patients. Stradwin (Cambridge, United Kingdom) image analysis software was used for manually outlining the plaque in reconstructed 2D sagittal (cross-sectional) images.12 Each observer was blinded to measurements performed by the other individuals. The three observers repeated the segmentation on all 10 patients after an interval of at least 4 weeks between the two segmentations to minimize recall bias. This approach allowed calculation of interobserver and intraobserver variability of the protocol. Intrascan analysis was performed by scanning the same patient (for the initial volume), then resting the patient, and performing the scan again. A single observer performed the intrascan segmentation on three patient data sets. Analysis of 3D images. Each individual crosssectional image was segmented commencing from where the plaque started until where it ended, with an interslice distance of 1 mm. The plaque in most of the cohort became visible in the common carotid artery proximal to the carotid bulb and ended in the proximal or middle internal carotid artery. Two boundaries were manually traced: first, the lumen-intima boundary (LIB), which defined the plaque surface; and second, the outer wall boundary (OWB), the outer edge of the adventitia layer, which defined the outer surface of the plaque. For all situations, including those where the boundaries in cross-sectional slices were ill defined due to shadowing or echolucent plaques, the cross-sectional outlining was guided by longitudinal sectional slices of the volume to identify the relative location of the boundaries in that slice. Additional views obtained in color-flow and grayscale modes, as well as video of the 2D US examination, were also used as a guide to identify the boundaries. The region between these two boundaries was the VWV and is the most complete estimation of atherosclerosis within the carotid artery. Previous segmentation methods identified plaque as the region between the LIB and media adventitia boundary (MAB).13 However, MAB is often poorly visualized in US images, and the OWB, including the adventitial layer, is easier to visualize. We therefore used OWB to determine VWV rather than MAB. At the end of segmentation, the program
692 AlMuhanna et al
JOURNAL OF VASCULAR SURGERY March 2015
Fig 1. Duplex ultrasound (US) imaging for plaque morphometry. A, Standard Doppler measurements to localize the plaque and measure peak systolic and end-diastolic velocities to estimate the percentage of stenosis. B, Best longitudinal view as determined by the sonographer. The luminal (red line) and adventitial (yellow line) boundaries were outlined to obtain the plaque area in the longitudinal section and the least luminal diameter (LLD). C, Cross-sectional image at the region with tightest stenosis, with the luminal and adventitial boundaries outlined to measure the plaque cross-sectional area. D, The 4DL14-5/38 US transducer (Ultrasonix, Richmond, BC, Canada) used for volume acquisition. E, Different slices of the acquired three-dimensional (3D) volume. F, Reconstructed volume generated after segmentation of the vessel wall volume (VWV).
automatically reconstructed all of the selected regions of interest into a 3D format and calculated the VWV (Fig 1, F). In essence, the series of 2D images were stacked and interpolated into a 3D volume set, similar to what is done with serial CT scans or MRIs. Analysis of 2D images. The longitudinal plaque image acquired using 2D B-mode imaging at the optimal insonation angle was opened in Stradwin. The LIB and the OWB were outlined in each of the 10 images from
the 10 patients. This enabled measurement of the LLD and the plaque area in the longitudinal section (Fig 1, B). Similarly, the LIB and OWB were outlined in each of the 10 cross-sectional images obtained at the point of highest stenosis to measure the plaque area in cross-section (Fig 1, C). Statistical analysis. Statistical analysis was performed using MATLAB software. Categoric data are presented as percentages and continuous data as mean 6 standard
JOURNAL OF VASCULAR SURGERY Volume 61, Number 3
deviation. The reliability of the imaging protocol for estimating the LLD, plaque area in the longitudinal section, plaque area in cross-sectional view, and total VWV, was tested using the traditional Bland-Altman analysis to determine intraobserver variability.14 Because there were three observers to test interobserver variability, we used the multiobserver Bland-Altman analysis for this purpose.15 Each measurement in this analysis is compared with the mean of all three measurements made by each observer. We also calculated the intraclass correlation coefficient (ICC) as an additional measure of accuracy.16 We determined the sensitivity to change of each measurement to identify its utility as a surveillance tool in sequential imaging over time. We used root-mean square analysis to measure the minimum detectable change (MDC) that could be identified for each aspect of plaque dimension measured in this study.17 The MDC was calculated at a 95% confidence interval defined as MDC ¼ 1.96O(2)SEM, where SEM is the standard error of measurement. The SEM ¼ SDO(1 r), where SD is the standard deviation and r is the ICC. The coefficient of variability (CV) was also calculated, which is a measure of the relative variance in the measurements, and defined as CV ¼ SD/m, where m is the mean.18 An important limitation of comparing area or volume measurements of 3D objects (such as plaques) is that despite similar area or volume measurements by different observers, the shapes outlined by them may still vary significantly. For instance, a plaque may be measured by each of two observers to have a volume of 1000 mm3, and therefore, the protocol would score well in traditional reliability and accuracy testing. However, the two observers might have outlined two completely different looking plaques, and this variability would remain unidentified. We therefore performed a comparison of shapes of the plaque outlines produced by the different observers. These unique analyses of the 3D volume images included measurements of the Dice similarity coefficient (DSC), a measure of the percent overlap between two outlines (green region in Fig 2),19 and the modified Hausdorff distance (MHD), a measure of the average distance between corresponding points of two contours (black dashed lines in Fig 2).20 MHD and DSC were performed for each slice in each volume data set in the study. RESULTS Patient characteristics. The degree of carotid stenosis as determined by velocity measures was $50%. Demographic information of the patients in the study is described in Table I. Measurement of LLD. The LLD was measured from the optimum longitudinal section image obtained by the sonographer. There was significant variability in measuring LLD using traditional B-mode imaging related to variability of the angle at which a sonographer chose to insonate the artery. In the representative example shown in Fig 3, the first longitudinal sectional slice was taken along the anterior-posterior direction relative to the artery, and
AlMuhanna et al 693
Fig 2. Quantification of shape variations between and within observers. Dice similarity coefficient (DSC), defined as the percentage of overlapping area between outlines produced by two observers (highlighted in green) divided by the average area between the two contours. Modified Hausdorff distance (MHD), is the mean distance between corresponding points of two contours produced by two different observers (black dashed lines).
Table I. Demographic and clinical information Variables Age, years Male sex Caucasian Hypercholesterolemia Diabetes mellitus Coronary artery disease Hypertension Smoking (past or present) Peripheral arterial disease Right carotid stenosis Stenosis 50%-70% $70%
Mean 6 SD or percentage (n ¼ 10) 66 6 5 100 70 60 50 40 80 70 60 60 90 10
SD, Standard deviation.
the LLD measured in that view was 4.9 mm (Fig 3, B). The longitudinal sectional slice in Fig 3, C was obtained through the right-anterior-oblique view relative to the artery, and the LLD measured at that insonation angle was 6.6 mm. The longitudinal sectional slice in Fig 3, D, was obtained from the left-anterior-oblique view, and the diameter was 4.6-mm. For all 10 subjects, the average CV of the diameter at different angles was 13% (range, 6%28%). 2D imaging. Optimal longitudinal and cross-sectional images were obtained by the sonographer to capture the best sections showing the plaque in each patient. One longitudinal and one cross-sectional image that showed the largest amount of plaque were selected for each patient for analysis. Three observers outlined the area of the plaque in each of the 10 longitudinal-sectional images and in each of the 10 cross-sectional images. Table II reports the mean plaque area, SEM, and MDC, expressed as a percentage of the mean area, the average CV, and the interclass/intraclass correlation coefficients for observations. The multiobserver Bland-Altman plots for the interobserver analysis
JOURNAL OF VASCULAR SURGERY March 2015
694 AlMuhanna et al
Fig 3. Variations in least luminal diameter (LLD) measurement depend on the angle of insonation of the transducer. A, B-mode cross-sectional view of the region of tightest stenosis. The yellow solid lines indicate different angles of insonation. The corresponding longitudinal sectional images acquired along the respective insonation angles are shown for (B) anterior-posterior (AP), (C) right anterior oblique (RAO), and (D) left anterior oblique (LAO). In this example, the luminal diameter was (B) 4.9 mm when viewed in the AP projection, (C) 6.6 mm in RAO projection, and (D) 4.6 mm in the LAO projection.
Table II. Measures of reliability and minimum detectable change (MDC) for two-dimensional (2D) area and three-dimensional (3D) volume measurements of carotid plaques on B-mode ultrasound (US) imaging Morphometric measurement
Mean value
SEMinter (%)
SEMintra (%)
MDCinter (%)
MDCintra (%)
CVinter (%)
CVintra (%)
ICCinter
ICCintra
Plaque area, mm2 Longitudinal section view Cross-section view VWV, mm3
201.33 73.44 1277.6
6.9 3.5 4.7
0.9 0.7 1.6
19.0 9.6 12.9
2.4 1.9 4.5
11.0 11.8 14.8
3.2 4.6 8.9
0.63 0.91 0.87
0.92 0.97 0.95
CV, Coefficient of variation; ICCinter, interclass correlation coefficient; ICCintra, intraclass correlation coefficient; inter, interobserver; intra, intraobserver; SEM, standard error of mean; VWV, vessel wall volume.
are shown in Fig 4. The differences in area measurements between observers are #1.96 SDs, indicating 95% agreement between measurements. 3D imaging. Serial cross-sectional images were obtained using the 3D transducer for each of 10 patients. The plaque was outlined in each image with an interslice distance of 1 mm, and the data set was used to form volume reconstructions. The average number of cross-sectional images in which the plaque was outlined was 18 per patient (range, 10-24 slices), with 175 images manually segmented. This process was performed by three different observers and then repeated by them for a total of 1050
manually segmented images in this study. Table II reports the mean volume values of the plaque, the SEM, and MDC as a percentage of the mean volume, the average CV, and the ICC of observations for interobserver and intraobserver values. The interobserver and intraobserver analyses (Fig 5) demonstrated that the measurements were reliable, and there was >95% agreement between observations. Shape analysis of the volume measurements are shown as box-and-whisker plots in Fig 6. The DSC and MHD confirmed a high degree of shape concordance among the three observers (observer 1 vs 2, 2 vs 3, and 1 vs 3)
JOURNAL OF VASCULAR SURGERY Volume 61, Number 3
AlMuhanna et al 695
Fig 4. Reliability of the longitudinal plaque area measurement obtained from a standard clinical B-mode image. A, Multiobserver Bland-Altman plot for interobserver variability in vessel wall area (VWA), with each color representing an observer (O). B, Bland-Altman plot for intraobserver variability, with each square representing the difference between two measurements made by the same observer. The black solid lines are 1.96 the standard deviation.
Fig 5. Reliability of plaque volume measurement with the three-dimensional (3D) transducer. A, Multiobserver Bland-Altman plot for interobserver variability, with each color representing an observer (O). B, Bland-Altman plot for intraobserver variability, with each square representing the difference between two measurements made by the same observer. The black solid lines are 1.96 the standard deviation.
for each patient as well as within-observer measurements. There was 79.6% 6 4.3% overlap in plaque shape outlines made by different observers, whereas the overlap was 82.9% 6 4.5% when the three observers repeated the outlines. Similarly, the average difference between similar points on the outlines was 0.49 6 0.12 mm between outlines made by different observers and 0.40 6 0.08 mm between outlines made by the same observer. DISCUSSION In this study, we tested traditional duplex US imaging and 3D plaque imaging using a commercially available 3D US transducer to identify the variability, reliability, and MDC that each method can identify. We found that: 1. The LLD measurement using traditional B-mode longitudinal sectional imaging was highly variable and angle-dependent; 2. The plaque area measurement using traditional B-mode longitudinal sectional imaging could detect a change of 19% or more in plaque area; 3. The VWV measurement using the 3D transducer had low variability, high reliability, and the ability to
detect a change of 12.9% or more in VWV for different observers ($4.5% for the same observer); and 4. Advanced shape analysis confirmed the high reliability of volume measurements using 3D US. A well-known limitation of traditional 2D US is the inability to reproduce the same longitudinal view on repeated scanning. We quantified the extent to which LLD measurements could vary due to differences in transducer insonation angles. The CV of the LLD was large (13%), thereby confirming that this measurement is unreliable for clinical use. Although other studies have reported lower variability rates,21 this is likely because the effect of different angles of insonation was not included in the analysis. The plaque area increases during progression of atherosclerosis much faster than plaque thickness,22 and this measure has been used to assess plaque burden.1 Similar to our study (0.92), other authors have obtained high ICCs (0.94) for this measure.22 However, we observed that repeated measurements with another observer yielded a lower interclass correlation (0.63) and that plaque area
696 AlMuhanna et al
JOURNAL OF VASCULAR SURGERY March 2015
Fig 6. Reliability of shape measurement of plaque volume as evaluated by the Dice similarity coefficient (DSC) and the modified Hausdorff distance (MHD). The box-and-whisker plots include all images from each individual. The central line is the median; the bottom and top edges are the 25th and 75th percentiles, respectively, the whiskers represent the range, and outliers are plotted individually. Interobserver analysis shows the mean overlap between all possible combinations of the three observers (1 vs 2, 1 vs 3, and 2 vs 3) (A) for DSC and for (B) MHD.
must change by $19% more before it is detectable by US imaging. The cross-sectional area measurement proved to be more reliable (ICC, 0.91; intraclass correlation coefficient, 0.97). US imaging can detect smaller changes in area but is insensitive to changes in plaque burden occurring along the length of the carotid artery, an important limitation of the measure. Atherosclerosis grows along the arterial wall as well as in thickness23; therefore, 2D measures of area would miss growth in the third plane. In this study, we therefore defined plaque burden as the total 3D VWV. Two experimental methods have been used for 3D US imaging of carotid plaque: motorized units driving a 2D transducer along the neck and freehand scanning of sequential 2D images by using 3D position information from electromagnetic position sensors attached to a 2D transducer. These approaches have provided proof of concept that volumetric imaging is possible with US imaging. The reported observer variability for measuring VWV with this specialized experimental equipment has ranged from 6% to 20%.4,24 We have obtained similar results with a commercially available 3D US transducer (CV ¼ 14.8% for interobserver and 8.9% for intraobserver). Therefore, results obtained by the clinical transducer are equivalent to the experimental equipment. Furthermore, the transducer is user friendly and allows rapid image acquisition (<1 second), thereby minimizing cardiac and respiratory movement artifacts. This eliminates the need for electrocardiogram gating, a step that is necessary in the experimental protocols. Volume measurements were achieved with high reliability (interclass [0.87] and intraclass [0.95] correlation coefficients). These analyses show the MDC with our technique is 12.9% when two different observers reviewed the plaque; therefore, a volume change of $12.9% is identified with 95% confidence. The MDC in volume was 12.3% between repeated scans of the same patient and was 4.5% when the same observer repeated the analysis on the same scan. In our analysis, we defined the VWV (region between the adventitia the intima) as an assessment of plaque
burden. The MAB has also been used to delineate the plaque; however, the MAB is often not well defined throughout the circumference of the artery. This could underestimate plaque burden and increase variability of observations.25 For this reason, MRI-based plaque segmentations have typically included the adventitia.26 The inclusion of the adventitial layer increases the estimated VWV, improves reproducibility, and improves sensitivity to change of plaque burden. Comparison of volume outlines is best achieved with numeric volume measurement as well as shape analysis. Using DSC and MHD analyses, we obtained good inter-rater and intrarater agreement in the outlines produced by multiple observers for the same plaque as well as repeated observations made by the same person. These comparisons are critical to evaluating the accuracy of volumetric measurement techniques and must become an integral part of future studies. Among the limitations of this preliminary study was that we had a sample size of 10 men, and the applicability of these results may need to be confirmed in a larger cohort. We are, however, comfortable that the variability statistics generated in this intensive study with three different observers provide an accurate assessment of the utility and limitations of this methodology. Importantly, these results using a clinical transducer are comparable to those produced by more complex experimental systems and, therefore, indicate its potential clinical utility. Second, the volume measurements were not validated against a standard modality such as MR or CT angiography, and therefore, the accuracy of our volume measurements are not known. The aim of the present study was to measure the capability of an inexpensive and safe noninvasive modality to detect a change in plaque burden, and this was accomplished successfully. Third, the patient’s position may alter the appearance of the plaque and arterial geometry.27 We did not systematically assess this variability; however, the 3D acquisition of data accommodated for such variability.
JOURNAL OF VASCULAR SURGERY Volume 61, Number 3
Fourth, although we have demonstrated the sensitivity with which we can measure changes in VWV and that such measures have been proposed previously as useful measures of plaque progression, their clinical utility as an outcome measure has yet to be fully investigated. This will require a longitudinal study with a larger cohort. Additional potential measures of plaque vulnerability include plaque tissue characteristics and topography, which were beyond the scope of this analysis and will be addressed in a separate study. In keeping with general US imaging limitations, it is possible that the plaque measures may be less reliable in the very distal internal carotid artery or the very proximal common carotid artery due to image quality degradation. However, as is with most cases, all plaques in this series were midcervical in location, and therefore, we cannot comment on this issue. CONCLUSIONS We present a comprehensive assessment of the reliability and variability in the measurement of 1D (diameter), 2D (longitudinal and cross-sectional area), and 3D (volume) measures of atherosclerosis in the carotid artery using a commercial 3D US transducer, which could facilitate translation to a clinical environment. The imaging protocol presented is convenient, reliable, has low variability, and can be accomplished in a clinical vascular laboratory. It can identify a change in carotid plaque burden (VWV) of $12.9% when two different observers perform repeated measurements and a change of $4.5% when the same observer monitors the patient. AUTHOR CONTRIBUTIONS Conception and design: KM, BL Analysis and interpretation: KM, MH, LZ, JF, MD, SS, BL Data collection: KM, LZ, GK Writing the article: KM, SS, BL Critical revision of the article: KM, MH, LZ, JF, MD, SS, BL Final approval of the article: KM, MH, LZ, JF, MD, SS, BL Statistical analysis: KM, MH Obtained funding: BL Overall responsibility: BL REFERENCES 1. Spence JD, Eliasziw M, DiCicco M, Hackam DG, Galil R, Lohmann T. Carotid plaque area: a tool for targeting and evaluating vascular preventive therapy. Stroke 2002;33:2916-22. 2. Kakkos SK, Charalambous I, Sabetai MM, Griffin MB, Georgiou N, Geroulakos G, et al. Progression of carotid artery stenosis is associated with the occurrence of subsequent ipsilateral ischemic events and stroke: results from the ACSRS Study. J Vasc Surg 2013;57:26S-7S. 3. Balestrini S, Lupidi F, Balucani C, Altamura C, Vernieri F, Provinciali L, et al. One-year progression of moderate asymptomatic carotid stenosis predicts the risk of vascular events. Stroke 2013;44:792-4. 4. Landry A, Spence JD, Fenster A. Measurement of carotid plaque volume by 3-dimensional ultrasound. Stroke 2004;35:864-9. 5. Lal BK, Hobson RW 2nd, Goldstein J, Chakhtoura EY, Duran WN. Carotid artery stenting: is there a need to revise ultrasound velocity. J Vasc Surg 2004;39:58-66.
AlMuhanna et al 697
6. Lal BK, Hobson RW 2nd, Goldstein J, Geohagan M, Chakhtoura E, Pappas PJ, et al. In-stent recurrent stenosis after carotid artery stenting: life table analysis and clinical relevance. J Vasc Surg 2003;38:1162-8. 7. Hobson RW 2nd, Strandness DE. Carotid artery stenosis: what’s in the measurement? J Vasc Surg 1993;18:1069-70. 8. Grant EG, Benson CB, Moneta GL, Alexandrov AV, Baker JD, Bluth EI, et al. Carotid artery stenosis: gray-scale and Doppler US diagnosisdSociety of Radiologists in Ultrasound Consensus Conference. Radiology 2003;229:340-6. 9. Lal BK, Kaperonis EA, Cuadra S, Kapadia I, Hobson RW 2nd. Patterns of in-stent restenosis after carotid artery stenting: classification and implications for long-term outcome. J Vasc Surg 2007;46:833-40. 10. Zhao L, Prior SJ, Kampmann M, Sorkin JD, Caldwell K, Braganza M, et al. Measurement of thrombus resolution using three-dimensional ultrasound assessment of deep vein thrombosis volume. J Vasc Surg Venous Lymphat Disord 2014;2:140-7. 11. Zuiderveld K. Contrast limited adaptive histograph equalization. Graph Gems IV 1994:474-85. 12. Treece GM, Prager RW, Gee AH, Berman L. Surface interpolation from sparse cross-sections using region correspondence. IEEE Trans Med Imaging 2000;19:1106-14. 13. Egger M, Spence JD, Fenster A, Parraga G. Validation of 3D ultrasound vessel wall volume: an imaging phenotype of carotid atherosclerosis. Ultrasound Med Biol 2007;33:905-14. 14. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;8:307-10. 15. Jones M, Dobson A, O’Brian S. A graphical method for assessing agreement with the mean between multiple observers using continuous measures. Int J Epidemiol 2011;40:1308-13. 16. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979;86:420-8. 17. Stratford PW. Getting more from the literature: estimating the standard error of measurement from reliability studies. Physiother Can 2004;56:27-30. 18. Vidakovic B. Statistics for bioengineering sciences: with MATLAB and WinBUGS Support. In: Casella G, Fienberg S, Olkin I, editors. Springer texts in statistics. New York: Springer; 2011. 19. Dice LR. Measures of the amount of ecologic association between species. Ecology 1945;26:297-302. 20. Dubuisson MP, Jain AK. A modified Hausdorff distance for object matching. Pattern recognition, 1994 Vol 1dConference A Computer Vision and Image Processing. Proc 12th IAPR Int Conf, Jerusalem, Israel, Oct 9-13 1994; 1994;1:566e8. 21. Shaalan WE, Wahlgren CM, Desai T, Piano G, Skelly C, Bassiouny HS. Reappraisal of velocity criteria for carotid bulb/internal carotid artery stenosis utilizing high-resolution B-mode ultrasound validated with computed tomography angiography. J Vasc Surg 2008;48:104-12. 22. Barnett PA, Spence JD, Manuck SB, Jennings JR. Psychological stress and the progression of carotid artery disease. J Hypertens 1997;15: 49-55. 23. Makris GC, Lavida A, Griffin M, Geroulakos G, Nicolaides AN. Threedimensional ultrasound imaging for the evaluation of carotid atherosclerosis. Atherosclerosis 2011;219:377-83. 24. Delcker A, Diener HC. Quantification of atherosclerotic plaques in carotid arteries by three-dimensional ultrasound. Br J Radiol 1994;67: 672-8. 25. Egger M, Krasinski A, Rutt BK, Fenster A, Parraga G. Comparison of B-Mode ultrasound, 3-dimensional ultrasound, and magnetic resonance imaging measurements of carotid atherosclerosis. J Ultrasound Med 2008;27:1321-34. 26. Underhill HR, Kerwin WS, Hatsukami TS, Yuan C. Automated measurement of mean wall thickness in the common carotid artery by MRI: a comparison to intima-media thickness by B-mode ultrasound. Am J Physiol Heart Circ Physiol 2006;24:379-87. 27. Glor FP, Ariff B, Hughes AD, Verdonck PR, Barratt DC, Augst AD, et al. Influence of head position on carotid hemodynamics in young adults. Am J Physiol Heart Circ Physiol 2004;287:H1670-81.
Submitted Jun 17, 2014; accepted Oct 1, 2014.
Carotid plaque morphometric assessment with three-dimensional ultrasound imaging Khalid AlMuhanna, MS,a,b Md Murad Hossain, MS,b Limin Zhao, MBBS,a,c Jonathan Fischell, BS,a Gregory Kowalewski, MS,a,c Moira Dux, PhD,c Siddhartha Sikdar, PhD,b and Brajesh K. Lal, MD,a,c Baltimore, Md; and Fairfax, Va Objective: As investigations into nonsurgical treatment for atherosclerosis expand, the measurement of plaque regression and progression has become an important end point to evaluate. Measurements of three-dimensional (3D) plaque volume are more reliable and sensitive to change than are traditional estimates of stenosis severity or cross-sectional area. 3D ultrasound (3D US) imaging may allow monitoring of plaque volume changes but has not been used routinely due to the cumbersome motorized units required to drive transducers. We investigated the variability, reliability, and the least amount of change detectable by 1D plaque measures, as well as 2D and 3D measures of plaque morphometry, that can be applied in a clinical environment. Methods: 3D US imaging was obtained in 10 patients with carotid stenosis. The lumen and outer wall boundaries were outlined in serial cross-sectional images 1 mm apart. Three observers manually segmented vessel wall volumes (VWVs), and the segmentation was repeated again 4 weeks later. This allowed measurement of interobserver and intraobserver variability of 6 pairs of observations. We measured Bland-Altman statistics, intraclass correlation coefficients, coefficient of variability, and the minimum detectable plaque change for each morphometric measure. Results: The mean VWV of carotid lesions in the study was 1276.8 mm3 (range, 620.6-1956.3 mm3). Bland-Altman plots demonstrated low interobserver and intraobserver variability. The interobserver variability of volume measurements as a function of mean volume was 14.8% and interobserver variability was 8.9%. Reliability was 87% as quantified by the interclass correlation and was 95% by the intraclass correlation. The least detectable change in VWV was 12.9% for interobserver variability and 4.5% for intraobserver variability for the three observers. Conclusions: Carotid plaque diameter measurements from B-mode images have high variability. Plaque burden, as estimated by VWV, can be measured reliably with a 3D US technique using a clinical scanner. The volumetric change, with 95% confidence, that must be observed to establish that a plaque has undergone growth or regression is w12.9% for different observers and 4.5% for the same observer performing the follow-up study. (J Vasc Surg 2015;61:690-7.)
Traditional methods of assessing the severity of carotid atherosclerosis use a Doppler velocity-based classification of stenosis categories in broad ranges. However, these ranges of stenoses are insensitive to small changes in the plaque burden. An alternative is direct measurement of diameter reduction by B-mode imaging. However, plaques progress along the length of an artery 2.4 times faster than they thicken.1 Therefore, methods that capture longitudinal and circumferential growth (ie, area From the Center for Vascular Diagnostics, Division of Vascular Surgery, University of Maryland School of Medicine, Baltimorea; the Department of Electrical and Computer Engineering, George Mason University, Fairfaxb; and the Vascular Service, Veterans Affairs Medical Center, Baltimore.c This work was supported by a Veterans Affairs (VA) Merit Review Grant Award to B.K.L. Author conflict of interest: none. Presented at the 2014 Vascular Annual Meeting of the Society for Vascular Surgery, Boston, Mass, June 4-7, 2014. Reprint requests: Brajesh K. Lal, MD, 22 S Greene St, S10-B00, Baltimore, MD 21201 (e-mail: [email protected]). The editors and reviewers of this article have no relevant financial relationships to disclose per the JVS policy that requires reviewers to decline review of any manuscript for which they may have a conflict of interest. 0741-5214 Copyright Ó 2015 by the Society for Vascular Surgery. Published by Elsevier Inc. http://dx.doi.org/10.1016/j.jvs.2014.10.003
690
and volume) are inherently more sensitive to a change in plaque burden (change in size of the plaque) than methods limited to thickness measurements (ie, diameter-reducing stenosis). These measures are important in clinical practice as emphasis increases for pharmacologic management of asymptomatic carotid atherosclerosis. The ability to accurately detect and quantify plaque change will become a critical determinant of treatment success or failure, which may predict a reduced or increased risk for stroke.2,3 Only three-dimensional (3D) plaque imaging can capture all critical dimensions of a plaque. Although computed tomography (CT) or magnetic resonance imaging (MRI) offer 3D imaging capabilities, they are expensive, often associated with nephrotoxic contrast agents (CT and MRI) and radiation exposure (CT), and are less suitable for longitudinal life-long surveillance of carotid plaques. Imaging with 3D ultrasound (US) can reduce the operator variability inherent in traditional US imaging and is economical and safe for serial testing. However, previously reported 3D US methods for carotid plaque imaging have involved cumbersome and specialized motorized units that move the transducer across the neck, required significant postprocessing time and experience, and have not received widespread clinical acceptance.4
JOURNAL OF VASCULAR SURGERY Volume 61, Number 3
In this study, we tested the reliability of 3D plaque imaging using a new commercially available 3D US transducer. We measured operator variability, reliability, and the least amount of change detectable by traditional 1D plaque measures, as well as nontraditional 2D and 3D measures of plaque morphometry. Our study forms the first comprehensive assessment of the same carotid plaques for 1D (diameter), 2D (longitudinal and cross-sectional area), and 3D (volume) measures. We also provide one of the first evaluations of the reliability of atherosclerotic carotid vessel wall volume (VWV) measurements as a measure of total plaque burden using a clinical 3D US transducer. METHODS The University of Maryland School of Medicine Institutional Review Board and the VA Medical Center Research and Development Committee, in Baltimore, Md, approved the protocol for this study. Patients. The study enrolled 10 consecutive patients with $50% asymptomatic carotid stenosis. Patients with an occlusion in the target artery were excluded. Demographics and risk factors recorded prospectively were ethnicity, diabetes mellitus, coronary artery disease, hypertension, hypercholesterolemia, smoking, and peripheral arterial occlusive disease. The patients were recruited from the Veteran Affairs Medical Center in Baltimore, Md, and provided informed consent. Clinical duplex US examination. Patients first underwent a standard carotid duplex US examination according to recommendations of the Inter-societal Accreditation Commission using a Sonix MDP system (Ultrasonix, Richmond, BC, Canada) and an L9-4/38 linear probe. The degree of stenosis in the carotid artery was estimated using Doppler velocities with appropriate angle correction according to standard techniques used by our group5-7 (Fig 1, A). We used consensus velocity criteria to define the degree of stenosis for the purposes of inclusion into the study.8 2D US imaging protocol. Standard clinical B-mode imaging techniques were used to define the least luminal diameter (LLD) and plaque area in longitudinal and cross-sectional views as reported previously by our group.6,9 As in clinical testing protocols, the sonographer was free to select the optimal insonation angle to obtain the best image of the plaque. A longitudinal image (Fig 1, B) was obtained first, and the transducer was then swept from the base of the neck to the angle of the mandible to identify and record the cross-sectional image where the tightest stenosis was visualized (Fig 1, C). The images were digitally recorded and analyzed off-line with a computer-assisted image-analysis program by independent observers blinded to clinical findings. The entire US examination was recorded on digital video for subsequent interpretation. 3D US imaging protocol. A standardized imaging protocol described by our group previously10 was used to obtain 3D images of the carotid artery with a 4DL14-5/38 transducer (Fig 1, D). This 3D probe consists of a
AlMuhanna et al 691
motorized linear array transducer that moves within a housing to capture a sequence of 2D image frames at different elevation angles, which can then be reconstructed into a 3D volume (Fig 1, E). The operator’s hand and the patient were held still to reduce movement artifacts. A complete volume was acquired in <1 second, which minimized movement artifact from cardiac and respiratory movements. Image analysis protocols. The acquired volumes consisted of raw unprocessed US data after envelope detection. The volumes were reconstructed using custom software developed using MATLAB (The MathWorks Inc, Natick, Mass). Brightness levels were adjusted using an automatic adaptive histogram-equalization method with a Rayleigh distribution11 to enhance visualization of the vessel boundary. After postprocessing, three observers analyzed the images from the 10 patients. Stradwin (Cambridge, United Kingdom) image analysis software was used for manually outlining the plaque in reconstructed 2D sagittal (cross-sectional) images.12 Each observer was blinded to measurements performed by the other individuals. The three observers repeated the segmentation on all 10 patients after an interval of at least 4 weeks between the two segmentations to minimize recall bias. This approach allowed calculation of interobserver and intraobserver variability of the protocol. Intrascan analysis was performed by scanning the same patient (for the initial volume), then resting the patient, and performing the scan again. A single observer performed the intrascan segmentation on three patient data sets. Analysis of 3D images. Each individual crosssectional image was segmented commencing from where the plaque started until where it ended, with an interslice distance of 1 mm. The plaque in most of the cohort became visible in the common carotid artery proximal to the carotid bulb and ended in the proximal or middle internal carotid artery. Two boundaries were manually traced: first, the lumen-intima boundary (LIB), which defined the plaque surface; and second, the outer wall boundary (OWB), the outer edge of the adventitia layer, which defined the outer surface of the plaque. For all situations, including those where the boundaries in cross-sectional slices were ill defined due to shadowing or echolucent plaques, the cross-sectional outlining was guided by longitudinal sectional slices of the volume to identify the relative location of the boundaries in that slice. Additional views obtained in color-flow and grayscale modes, as well as video of the 2D US examination, were also used as a guide to identify the boundaries. The region between these two boundaries was the VWV and is the most complete estimation of atherosclerosis within the carotid artery. Previous segmentation methods identified plaque as the region between the LIB and media adventitia boundary (MAB).13 However, MAB is often poorly visualized in US images, and the OWB, including the adventitial layer, is easier to visualize. We therefore used OWB to determine VWV rather than MAB. At the end of segmentation, the program
692 AlMuhanna et al
JOURNAL OF VASCULAR SURGERY March 2015
Fig 1. Duplex ultrasound (US) imaging for plaque morphometry. A, Standard Doppler measurements to localize the plaque and measure peak systolic and end-diastolic velocities to estimate the percentage of stenosis. B, Best longitudinal view as determined by the sonographer. The luminal (red line) and adventitial (yellow line) boundaries were outlined to obtain the plaque area in the longitudinal section and the least luminal diameter (LLD). C, Cross-sectional image at the region with tightest stenosis, with the luminal and adventitial boundaries outlined to measure the plaque cross-sectional area. D, The 4DL14-5/38 US transducer (Ultrasonix, Richmond, BC, Canada) used for volume acquisition. E, Different slices of the acquired three-dimensional (3D) volume. F, Reconstructed volume generated after segmentation of the vessel wall volume (VWV).
automatically reconstructed all of the selected regions of interest into a 3D format and calculated the VWV (Fig 1, F). In essence, the series of 2D images were stacked and interpolated into a 3D volume set, similar to what is done with serial CT scans or MRIs. Analysis of 2D images. The longitudinal plaque image acquired using 2D B-mode imaging at the optimal insonation angle was opened in Stradwin. The LIB and the OWB were outlined in each of the 10 images from
the 10 patients. This enabled measurement of the LLD and the plaque area in the longitudinal section (Fig 1, B). Similarly, the LIB and OWB were outlined in each of the 10 cross-sectional images obtained at the point of highest stenosis to measure the plaque area in cross-section (Fig 1, C). Statistical analysis. Statistical analysis was performed using MATLAB software. Categoric data are presented as percentages and continuous data as mean 6 standard
JOURNAL OF VASCULAR SURGERY Volume 61, Number 3
deviation. The reliability of the imaging protocol for estimating the LLD, plaque area in the longitudinal section, plaque area in cross-sectional view, and total VWV, was tested using the traditional Bland-Altman analysis to determine intraobserver variability.14 Because there were three observers to test interobserver variability, we used the multiobserver Bland-Altman analysis for this purpose.15 Each measurement in this analysis is compared with the mean of all three measurements made by each observer. We also calculated the intraclass correlation coefficient (ICC) as an additional measure of accuracy.16 We determined the sensitivity to change of each measurement to identify its utility as a surveillance tool in sequential imaging over time. We used root-mean square analysis to measure the minimum detectable change (MDC) that could be identified for each aspect of plaque dimension measured in this study.17 The MDC was calculated at a 95% confidence interval defined as MDC ¼ 1.96O(2)SEM, where SEM is the standard error of measurement. The SEM ¼ SDO(1 r), where SD is the standard deviation and r is the ICC. The coefficient of variability (CV) was also calculated, which is a measure of the relative variance in the measurements, and defined as CV ¼ SD/m, where m is the mean.18 An important limitation of comparing area or volume measurements of 3D objects (such as plaques) is that despite similar area or volume measurements by different observers, the shapes outlined by them may still vary significantly. For instance, a plaque may be measured by each of two observers to have a volume of 1000 mm3, and therefore, the protocol would score well in traditional reliability and accuracy testing. However, the two observers might have outlined two completely different looking plaques, and this variability would remain unidentified. We therefore performed a comparison of shapes of the plaque outlines produced by the different observers. These unique analyses of the 3D volume images included measurements of the Dice similarity coefficient (DSC), a measure of the percent overlap between two outlines (green region in Fig 2),19 and the modified Hausdorff distance (MHD), a measure of the average distance between corresponding points of two contours (black dashed lines in Fig 2).20 MHD and DSC were performed for each slice in each volume data set in the study. RESULTS Patient characteristics. The degree of carotid stenosis as determined by velocity measures was $50%. Demographic information of the patients in the study is described in Table I. Measurement of LLD. The LLD was measured from the optimum longitudinal section image obtained by the sonographer. There was significant variability in measuring LLD using traditional B-mode imaging related to variability of the angle at which a sonographer chose to insonate the artery. In the representative example shown in Fig 3, the first longitudinal sectional slice was taken along the anterior-posterior direction relative to the artery, and
AlMuhanna et al 693
Fig 2. Quantification of shape variations between and within observers. Dice similarity coefficient (DSC), defined as the percentage of overlapping area between outlines produced by two observers (highlighted in green) divided by the average area between the two contours. Modified Hausdorff distance (MHD), is the mean distance between corresponding points of two contours produced by two different observers (black dashed lines).
Table I. Demographic and clinical information Variables Age, years Male sex Caucasian Hypercholesterolemia Diabetes mellitus Coronary artery disease Hypertension Smoking (past or present) Peripheral arterial disease Right carotid stenosis Stenosis 50%-70% $70%
Mean 6 SD or percentage (n ¼ 10) 66 6 5 100 70 60 50 40 80 70 60 60 90 10
SD, Standard deviation.
the LLD measured in that view was 4.9 mm (Fig 3, B). The longitudinal sectional slice in Fig 3, C was obtained through the right-anterior-oblique view relative to the artery, and the LLD measured at that insonation angle was 6.6 mm. The longitudinal sectional slice in Fig 3, D, was obtained from the left-anterior-oblique view, and the diameter was 4.6-mm. For all 10 subjects, the average CV of the diameter at different angles was 13% (range, 6%28%). 2D imaging. Optimal longitudinal and cross-sectional images were obtained by the sonographer to capture the best sections showing the plaque in each patient. One longitudinal and one cross-sectional image that showed the largest amount of plaque were selected for each patient for analysis. Three observers outlined the area of the plaque in each of the 10 longitudinal-sectional images and in each of the 10 cross-sectional images. Table II reports the mean plaque area, SEM, and MDC, expressed as a percentage of the mean area, the average CV, and the interclass/intraclass correlation coefficients for observations. The multiobserver Bland-Altman plots for the interobserver analysis
JOURNAL OF VASCULAR SURGERY March 2015
694 AlMuhanna et al
Fig 3. Variations in least luminal diameter (LLD) measurement depend on the angle of insonation of the transducer. A, B-mode cross-sectional view of the region of tightest stenosis. The yellow solid lines indicate different angles of insonation. The corresponding longitudinal sectional images acquired along the respective insonation angles are shown for (B) anterior-posterior (AP), (C) right anterior oblique (RAO), and (D) left anterior oblique (LAO). In this example, the luminal diameter was (B) 4.9 mm when viewed in the AP projection, (C) 6.6 mm in RAO projection, and (D) 4.6 mm in the LAO projection.
Table II. Measures of reliability and minimum detectable change (MDC) for two-dimensional (2D) area and three-dimensional (3D) volume measurements of carotid plaques on B-mode ultrasound (US) imaging Morphometric measurement
Mean value
SEMinter (%)
SEMintra (%)
MDCinter (%)
MDCintra (%)
CVinter (%)
CVintra (%)
ICCinter
ICCintra
Plaque area, mm2 Longitudinal section view Cross-section view VWV, mm3
201.33 73.44 1277.6
6.9 3.5 4.7
0.9 0.7 1.6
19.0 9.6 12.9
2.4 1.9 4.5
11.0 11.8 14.8
3.2 4.6 8.9
0.63 0.91 0.87
0.92 0.97 0.95
CV, Coefficient of variation; ICCinter, interclass correlation coefficient; ICCintra, intraclass correlation coefficient; inter, interobserver; intra, intraobserver; SEM, standard error of mean; VWV, vessel wall volume.
are shown in Fig 4. The differences in area measurements between observers are #1.96 SDs, indicating 95% agreement between measurements. 3D imaging. Serial cross-sectional images were obtained using the 3D transducer for each of 10 patients. The plaque was outlined in each image with an interslice distance of 1 mm, and the data set was used to form volume reconstructions. The average number of cross-sectional images in which the plaque was outlined was 18 per patient (range, 10-24 slices), with 175 images manually segmented. This process was performed by three different observers and then repeated by them for a total of 1050
manually segmented images in this study. Table II reports the mean volume values of the plaque, the SEM, and MDC as a percentage of the mean volume, the average CV, and the ICC of observations for interobserver and intraobserver values. The interobserver and intraobserver analyses (Fig 5) demonstrated that the measurements were reliable, and there was >95% agreement between observations. Shape analysis of the volume measurements are shown as box-and-whisker plots in Fig 6. The DSC and MHD confirmed a high degree of shape concordance among the three observers (observer 1 vs 2, 2 vs 3, and 1 vs 3)
JOURNAL OF VASCULAR SURGERY Volume 61, Number 3
AlMuhanna et al 695
Fig 4. Reliability of the longitudinal plaque area measurement obtained from a standard clinical B-mode image. A, Multiobserver Bland-Altman plot for interobserver variability in vessel wall area (VWA), with each color representing an observer (O). B, Bland-Altman plot for intraobserver variability, with each square representing the difference between two measurements made by the same observer. The black solid lines are 1.96 the standard deviation.
Fig 5. Reliability of plaque volume measurement with the three-dimensional (3D) transducer. A, Multiobserver Bland-Altman plot for interobserver variability, with each color representing an observer (O). B, Bland-Altman plot for intraobserver variability, with each square representing the difference between two measurements made by the same observer. The black solid lines are 1.96 the standard deviation.
for each patient as well as within-observer measurements. There was 79.6% 6 4.3% overlap in plaque shape outlines made by different observers, whereas the overlap was 82.9% 6 4.5% when the three observers repeated the outlines. Similarly, the average difference between similar points on the outlines was 0.49 6 0.12 mm between outlines made by different observers and 0.40 6 0.08 mm between outlines made by the same observer. DISCUSSION In this study, we tested traditional duplex US imaging and 3D plaque imaging using a commercially available 3D US transducer to identify the variability, reliability, and MDC that each method can identify. We found that: 1. The LLD measurement using traditional B-mode longitudinal sectional imaging was highly variable and angle-dependent; 2. The plaque area measurement using traditional B-mode longitudinal sectional imaging could detect a change of 19% or more in plaque area; 3. The VWV measurement using the 3D transducer had low variability, high reliability, and the ability to
detect a change of 12.9% or more in VWV for different observers ($4.5% for the same observer); and 4. Advanced shape analysis confirmed the high reliability of volume measurements using 3D US. A well-known limitation of traditional 2D US is the inability to reproduce the same longitudinal view on repeated scanning. We quantified the extent to which LLD measurements could vary due to differences in transducer insonation angles. The CV of the LLD was large (13%), thereby confirming that this measurement is unreliable for clinical use. Although other studies have reported lower variability rates,21 this is likely because the effect of different angles of insonation was not included in the analysis. The plaque area increases during progression of atherosclerosis much faster than plaque thickness,22 and this measure has been used to assess plaque burden.1 Similar to our study (0.92), other authors have obtained high ICCs (0.94) for this measure.22 However, we observed that repeated measurements with another observer yielded a lower interclass correlation (0.63) and that plaque area
696 AlMuhanna et al
JOURNAL OF VASCULAR SURGERY March 2015
Fig 6. Reliability of shape measurement of plaque volume as evaluated by the Dice similarity coefficient (DSC) and the modified Hausdorff distance (MHD). The box-and-whisker plots include all images from each individual. The central line is the median; the bottom and top edges are the 25th and 75th percentiles, respectively, the whiskers represent the range, and outliers are plotted individually. Interobserver analysis shows the mean overlap between all possible combinations of the three observers (1 vs 2, 1 vs 3, and 2 vs 3) (A) for DSC and for (B) MHD.
must change by $19% more before it is detectable by US imaging. The cross-sectional area measurement proved to be more reliable (ICC, 0.91; intraclass correlation coefficient, 0.97). US imaging can detect smaller changes in area but is insensitive to changes in plaque burden occurring along the length of the carotid artery, an important limitation of the measure. Atherosclerosis grows along the arterial wall as well as in thickness23; therefore, 2D measures of area would miss growth in the third plane. In this study, we therefore defined plaque burden as the total 3D VWV. Two experimental methods have been used for 3D US imaging of carotid plaque: motorized units driving a 2D transducer along the neck and freehand scanning of sequential 2D images by using 3D position information from electromagnetic position sensors attached to a 2D transducer. These approaches have provided proof of concept that volumetric imaging is possible with US imaging. The reported observer variability for measuring VWV with this specialized experimental equipment has ranged from 6% to 20%.4,24 We have obtained similar results with a commercially available 3D US transducer (CV ¼ 14.8% for interobserver and 8.9% for intraobserver). Therefore, results obtained by the clinical transducer are equivalent to the experimental equipment. Furthermore, the transducer is user friendly and allows rapid image acquisition (<1 second), thereby minimizing cardiac and respiratory movement artifacts. This eliminates the need for electrocardiogram gating, a step that is necessary in the experimental protocols. Volume measurements were achieved with high reliability (interclass [0.87] and intraclass [0.95] correlation coefficients). These analyses show the MDC with our technique is 12.9% when two different observers reviewed the plaque; therefore, a volume change of $12.9% is identified with 95% confidence. The MDC in volume was 12.3% between repeated scans of the same patient and was 4.5% when the same observer repeated the analysis on the same scan. In our analysis, we defined the VWV (region between the adventitia the intima) as an assessment of plaque
burden. The MAB has also been used to delineate the plaque; however, the MAB is often not well defined throughout the circumference of the artery. This could underestimate plaque burden and increase variability of observations.25 For this reason, MRI-based plaque segmentations have typically included the adventitia.26 The inclusion of the adventitial layer increases the estimated VWV, improves reproducibility, and improves sensitivity to change of plaque burden. Comparison of volume outlines is best achieved with numeric volume measurement as well as shape analysis. Using DSC and MHD analyses, we obtained good inter-rater and intrarater agreement in the outlines produced by multiple observers for the same plaque as well as repeated observations made by the same person. These comparisons are critical to evaluating the accuracy of volumetric measurement techniques and must become an integral part of future studies. Among the limitations of this preliminary study was that we had a sample size of 10 men, and the applicability of these results may need to be confirmed in a larger cohort. We are, however, comfortable that the variability statistics generated in this intensive study with three different observers provide an accurate assessment of the utility and limitations of this methodology. Importantly, these results using a clinical transducer are comparable to those produced by more complex experimental systems and, therefore, indicate its potential clinical utility. Second, the volume measurements were not validated against a standard modality such as MR or CT angiography, and therefore, the accuracy of our volume measurements are not known. The aim of the present study was to measure the capability of an inexpensive and safe noninvasive modality to detect a change in plaque burden, and this was accomplished successfully. Third, the patient’s position may alter the appearance of the plaque and arterial geometry.27 We did not systematically assess this variability; however, the 3D acquisition of data accommodated for such variability.
JOURNAL OF VASCULAR SURGERY Volume 61, Number 3
Fourth, although we have demonstrated the sensitivity with which we can measure changes in VWV and that such measures have been proposed previously as useful measures of plaque progression, their clinical utility as an outcome measure has yet to be fully investigated. This will require a longitudinal study with a larger cohort. Additional potential measures of plaque vulnerability include plaque tissue characteristics and topography, which were beyond the scope of this analysis and will be addressed in a separate study. In keeping with general US imaging limitations, it is possible that the plaque measures may be less reliable in the very distal internal carotid artery or the very proximal common carotid artery due to image quality degradation. However, as is with most cases, all plaques in this series were midcervical in location, and therefore, we cannot comment on this issue. CONCLUSIONS We present a comprehensive assessment of the reliability and variability in the measurement of 1D (diameter), 2D (longitudinal and cross-sectional area), and 3D (volume) measures of atherosclerosis in the carotid artery using a commercial 3D US transducer, which could facilitate translation to a clinical environment. The imaging protocol presented is convenient, reliable, has low variability, and can be accomplished in a clinical vascular laboratory. It can identify a change in carotid plaque burden (VWV) of $12.9% when two different observers perform repeated measurements and a change of $4.5% when the same observer monitors the patient. AUTHOR CONTRIBUTIONS Conception and design: KM, BL Analysis and interpretation: KM, MH, LZ, JF, MD, SS, BL Data collection: KM, LZ, GK Writing the article: KM, SS, BL Critical revision of the article: KM, MH, LZ, JF, MD, SS, BL Final approval of the article: KM, MH, LZ, JF, MD, SS, BL Statistical analysis: KM, MH Obtained funding: BL Overall responsibility: BL REFERENCES 1. Spence JD, Eliasziw M, DiCicco M, Hackam DG, Galil R, Lohmann T. Carotid plaque area: a tool for targeting and evaluating vascular preventive therapy. Stroke 2002;33:2916-22. 2. Kakkos SK, Charalambous I, Sabetai MM, Griffin MB, Georgiou N, Geroulakos G, et al. Progression of carotid artery stenosis is associated with the occurrence of subsequent ipsilateral ischemic events and stroke: results from the ACSRS Study. J Vasc Surg 2013;57:26S-7S. 3. Balestrini S, Lupidi F, Balucani C, Altamura C, Vernieri F, Provinciali L, et al. One-year progression of moderate asymptomatic carotid stenosis predicts the risk of vascular events. Stroke 2013;44:792-4. 4. Landry A, Spence JD, Fenster A. Measurement of carotid plaque volume by 3-dimensional ultrasound. Stroke 2004;35:864-9. 5. Lal BK, Hobson RW 2nd, Goldstein J, Chakhtoura EY, Duran WN. Carotid artery stenting: is there a need to revise ultrasound velocity. J Vasc Surg 2004;39:58-66.
AlMuhanna et al 697
6. Lal BK, Hobson RW 2nd, Goldstein J, Geohagan M, Chakhtoura E, Pappas PJ, et al. In-stent recurrent stenosis after carotid artery stenting: life table analysis and clinical relevance. J Vasc Surg 2003;38:1162-8. 7. Hobson RW 2nd, Strandness DE. Carotid artery stenosis: what’s in the measurement? J Vasc Surg 1993;18:1069-70. 8. Grant EG, Benson CB, Moneta GL, Alexandrov AV, Baker JD, Bluth EI, et al. Carotid artery stenosis: gray-scale and Doppler US diagnosisdSociety of Radiologists in Ultrasound Consensus Conference. Radiology 2003;229:340-6. 9. Lal BK, Kaperonis EA, Cuadra S, Kapadia I, Hobson RW 2nd. Patterns of in-stent restenosis after carotid artery stenting: classification and implications for long-term outcome. J Vasc Surg 2007;46:833-40. 10. Zhao L, Prior SJ, Kampmann M, Sorkin JD, Caldwell K, Braganza M, et al. Measurement of thrombus resolution using three-dimensional ultrasound assessment of deep vein thrombosis volume. J Vasc Surg Venous Lymphat Disord 2014;2:140-7. 11. Zuiderveld K. Contrast limited adaptive histograph equalization. Graph Gems IV 1994:474-85. 12. Treece GM, Prager RW, Gee AH, Berman L. Surface interpolation from sparse cross-sections using region correspondence. IEEE Trans Med Imaging 2000;19:1106-14. 13. Egger M, Spence JD, Fenster A, Parraga G. Validation of 3D ultrasound vessel wall volume: an imaging phenotype of carotid atherosclerosis. Ultrasound Med Biol 2007;33:905-14. 14. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;8:307-10. 15. Jones M, Dobson A, O’Brian S. A graphical method for assessing agreement with the mean between multiple observers using continuous measures. Int J Epidemiol 2011;40:1308-13. 16. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979;86:420-8. 17. Stratford PW. Getting more from the literature: estimating the standard error of measurement from reliability studies. Physiother Can 2004;56:27-30. 18. Vidakovic B. Statistics for bioengineering sciences: with MATLAB and WinBUGS Support. In: Casella G, Fienberg S, Olkin I, editors. Springer texts in statistics. New York: Springer; 2011. 19. Dice LR. Measures of the amount of ecologic association between species. Ecology 1945;26:297-302. 20. Dubuisson MP, Jain AK. A modified Hausdorff distance for object matching. Pattern recognition, 1994 Vol 1dConference A Computer Vision and Image Processing. Proc 12th IAPR Int Conf, Jerusalem, Israel, Oct 9-13 1994; 1994;1:566e8. 21. Shaalan WE, Wahlgren CM, Desai T, Piano G, Skelly C, Bassiouny HS. Reappraisal of velocity criteria for carotid bulb/internal carotid artery stenosis utilizing high-resolution B-mode ultrasound validated with computed tomography angiography. J Vasc Surg 2008;48:104-12. 22. Barnett PA, Spence JD, Manuck SB, Jennings JR. Psychological stress and the progression of carotid artery disease. J Hypertens 1997;15: 49-55. 23. Makris GC, Lavida A, Griffin M, Geroulakos G, Nicolaides AN. Threedimensional ultrasound imaging for the evaluation of carotid atherosclerosis. Atherosclerosis 2011;219:377-83. 24. Delcker A, Diener HC. Quantification of atherosclerotic plaques in carotid arteries by three-dimensional ultrasound. Br J Radiol 1994;67: 672-8. 25. Egger M, Krasinski A, Rutt BK, Fenster A, Parraga G. Comparison of B-Mode ultrasound, 3-dimensional ultrasound, and magnetic resonance imaging measurements of carotid atherosclerosis. J Ultrasound Med 2008;27:1321-34. 26. Underhill HR, Kerwin WS, Hatsukami TS, Yuan C. Automated measurement of mean wall thickness in the common carotid artery by MRI: a comparison to intima-media thickness by B-mode ultrasound. Am J Physiol Heart Circ Physiol 2006;24:379-87. 27. Glor FP, Ariff B, Hughes AD, Verdonck PR, Barratt DC, Augst AD, et al. Influence of head position on carotid hemodynamics in young adults. Am J Physiol Heart Circ Physiol 2004;287:H1670-81.
Submitted Jun 17, 2014; accepted Oct 1, 2014.