- Email: [email protected]
Computed tomography assessment of evolution of interstitial lung disease in systemic sclerosis: Comparison of two scoring systems
European Journal of Internal Medicine xxx (xxxx) xxx–xxx
Contents lists available at ScienceDirect
European Journal of Internal Medicine journal homepage: www.elsevier.com/locate/ejim
Original article
Computed tomography assessment of evolution of interstitial lung disease in systemic sclerosis: Comparison of two scoring systems ⁎
Fausto Salaffia, Marina Carottib, Marika Tardellaa, , Marco Di Carloa, Paolo Fraticellic, Colomba Fischettic, Andrea Giovagnonib, Armando Gabriellid a
Rheumatological Clinic, Ospedale Carlo Urbani Jesi, Università Politecnica delle Marche, Ancona, Italy Department of Radiology, Ospedali Riuniti, Università Politecnica delle Marche, Italy c Department of Internal Medicine, Ospedali Riuniti, Università Politecnica delle Marche, Ancona, Italy d Department of Clinical and Molecular Sciences, Università Politecnica delle Marche, Ancona, Italy b
A R T I C LE I N FO
A B S T R A C T
Keywords: High resolution computed tomography Computed-aided method Conventional visual reader-based score
Background: The aim of this study was to evaluate and compare the internal and external responsiveness of a computed-aided method (CaM) with a conventional visual reader-based score (CoVR) to measure interstitial lung disease (ILD) in patients with systemic sclerosis (SSc) on high resolution computed tomography (HRCT). Methods: Forty-five patients were evaluated in this retrospective cohort. HRCTs were collected at baseline and after 1 year. HRCT abnormalities were evaluated according to a CoVR (Warrick's method) and a quantitative CaM. Internal 1-year responsiveness was tested with a standardized mean response (SRM). Analyses of the receiver operating characteristic curves (ROCs) evaluated the sensitivity and specificity of the two methods to discriminate between clinically relevant progression and no relevant progression, using expert judgment as the gold standard (external responsiveness). Results: In one year, lung involvement was stable/improved in 17 of the 45 patients (37.8%) and worsened in 28 patients (62.2%). HRCT scores changed moderately over the follow-up period. Using SFM, CaM was significantly more responsive in detecting changes due to treatment than the CoVR method. Likewise, in the analysis of the ROC curve, CaM scores showed the highest performance (AUC ROC CaM vs. CoVR, 0.951 vs. 0.807; p = 0.0065). Conclusion: Quantitative analysis of CaM was more responsive than the CoVR method to accurately evaluate and monitor SSc-ILD progression or response to therapy.
1. Introduction Systemic sclerosis (SSc) is a connective tissue disease characterized by extensive skin fibrosis and abnormalities in large and small vessels. Although cutaneous manifestations are the most frequent sign of the disease, there may be involvement of internal organs, particularly the lungs, with a severe impact on prognosis [1,2]. Interstitial lung disease (ILD) is the most frequent manifestation of lung involvement, with a prevalence of 53% of cases with diffuse SSc and 35% of cases with limited SSc [3]. In daily clinical practice, the main tool for the detection and evaluation of SSc-ILDs is high resolution computed tomography (HRCT) [4,5]. In many studies, the high correlation between HRCT results and pulmonary function test (PFT) results has been demonstrated [6,7]. Several methods of semi-quantitative HRCT evaluation
were described and used to characterize and quantify the extent of pulmonary fibrosis [8,9]. Intra- and inter-reader changes and the need for detailed knowledge of pulmonary radiological anatomy limit the implementation and diffusion of these scoring systems. This impairs the possibility of using these methods in ILD practice. In recent years, a computed automatic evaluation of HRCT results has been developed, which has improved the objectivity, sensitivity and repeatability of quantitative changes in lung characteristics compared to traditional visual interpretation [10–12]. Previously, we demonstrated high agreement on semi-quantitative HRCT analysis performed by experienced radiologists and a significant association between descriptive parameters in both quantitative and semi-quantitative HRCT analysis [13]. We also demonstrated good validity, reliability and feasibility in the evaluation of SSc-ILD using a computed-aided method (CaM)
⁎
Corresponding author. E-mail addresses: fausto.salaffi@gmail.com (F. Salaffi), [email protected] (M. Carotti), [email protected] (M. Tardella), [email protected] (M. Di Carlo), [email protected] (P. Fraticelli), colomba.fi[email protected] (C. Fischetti), [email protected] (A. Giovagnoni), [email protected] (A. Gabrielli). https://doi.org/10.1016/j.ejim.2020.02.009 Received 2 November 2019; Received in revised form 29 January 2020; Accepted 11 February 2020 0953-6205/ © 2020 Published by Elsevier B.V. on behalf of European Federation of Internal Medicine.
Please cite this article as: Fausto Salaffi, et al., European Journal of Internal Medicine, https://doi.org/10.1016/j.ejim.2020.02.009
European Journal of Internal Medicine xxx (xxxx) xxx–xxx
F. Salaffi, et al.
intraobserver reliability, each reader examined all HRCTs twice, with an interval of at least four weeks. The Warrick et al. method was used to evaluate lung disease results [8]. The mean values of the two independent readers were used for the final control group. The intraclass correlation coefficient (ICC) of the radiologists' agreement level on total HRCT scores, calculated by the CoVR method, was 0.80 [14,15].
[14,15]. Although HRCT has been used to define the characteristics and extent of SSc-ILD and to predict results, experience with its use as a measure of results in therapeutic trials is limited [16–18]. In the present study, we examined data from serial HRCTs performed in a cohort of SSc patients treated with different protocols. The purpose of the study was to evaluate and compare the internal and external responsiveness of two different HRCT evaluation methods (CaM and conventional visual-reader based score [CoVR]) to detect response to one-year immunosuppressive therapy in SSc-ILD.
2.3. Computer-aided method (CaM) quantification process HRCT images were reconstructed and analyzed by OsiriX MD 7, a DICOM software viewer (OsiriX MD version 7, 64-bit format), on a Mac Mini (Intel Core 2 Duo desktop computer 2.8 GHz, 16 GB random access memory; Apple Computer, Cupertino, CA, USA) with Mac OSX 10.12.2 operating system. The program uses a semiautomatic thresholding technique to isolate the lungs from other tissues. A semiautomatic parenchymal pulmonary segmentation was performed in each section to obtain the analysis of all images. The calculated HRCT lung mapping analysis was performed using a radiodensity of −200 to −1024 HU for the pulmonary parenchyma (isolated from the mediastinal and thoracic wall) and −700 to −500 HU for ILD [21]. Minimal user intervention was required to exclude blood vessels and large hilum bronchi. The CT attenuation of the normal lung parenchyma is reported in a range from −800 to −900 HU, depending on inspiration or expiration, on the level of inspiration achieved for the scan and on the anatomical position, i.e. the ventral or dorsal portion. The attenuation area between −500 and −700 HU has been defined as the radiodensity value for ILD, which includes both the opacity of the ground glass and the reticular opacity. The radiodensity of −500 HU was chosen as the threshold between the consolidation and opacity of ground glass. Therefore, −700 HU was recognized as the default threshold value for normal lung portions. For each HRCT, two CaM measurements were performed with a total agreement between the first and the second calculation of the scores (95% of the agreement limits = 0–0, ICC = 1) [13].
2. Patients and methods 2.1. Study population and protocol This is a retrospective analysis carried out in patients attending the Rheumatological Clinic and the Medical Clinic, Università Politecnica delle Marche, Italy, from January 1, 2014 to June 30, 2017, stored in a dedicated database. We included in the study patients with SSc, defined by the classification criteria of the American College of Rheumatology/ European League against Rheumatism 2013 [19] and classified in limited and diffuse skin involvement (lcSSc and dcSSc, respectively) according to the extension of skin thickness [20]. The criteria for inclusion were: age over 18 years, presence of ILD demonstrated at HRCT for which specific immunosuppressive therapy was started, absence of known pleural and pulmonary pathology (e.g. chronic obstructive bronchopneumopathy, emphysema) other than ILD, absence of other diseases (e.g. chronic heart failure, hypoalbuminemia) or therapy that could influence the results of HRCT (potentially toxic drugs for the lungs). The study protocol focused the analysis on the comparison between the responsiveness of a CaM and a CoVR to measure the SSc-ILD and evaluate the changes in the score. We evaluated the results of HRCT chest investigations performed at baseline, when immunosuppressive therapy was started, and after 12 months. Patients who did not undergo HRCT after 12 months were excluded. The local Ethics Committee (Comitato Unico Regionale – ASUR Marche) approved the protocol, the patient information sheet and the consent form. The study was conducted in accordance with the Helsinki Declaration in its fifth edition (2000). The informed consent was signed by all participants.
2.4. Statistical analysis The data was entered into a Microsoft Excel database and analysed with MedCalc® version 18.6 (MedCalc Software, Ostend, Belgium). The values were expressed both as mean ± standard deviation (SD) and as median (interquartile range [IQR]). The Student's paired-t-test evaluated whether the scores of the CoVR method and the CaM changed over 1-year. Responsiveness was assessed in the 1-year longitudinal HRCT follow-up. Based on the suggestion of Husted et al. [22], internal and external responsiveness were measured. To analyze internal responsiveness over 1-year, the standardized response mean (SRM) was used. The SRM is calculated as the mean change score between baseline and 1-year divided by the SD of this difference [23]. Ninety-five percent confidence intervals (95% CIs) for the SRM were estimated through bootstrap resampling. Bootstrapping consists of resampling with replacement. We selected 1000 samples with replacement and calculated the SRM for each sample [24]. Receiver operating characteristic (ROC) curve analyses assessed the sensitivity and specificity of the two scoring methods to discriminate between clinically relevant and no relevant progression, using judgment of the experts as gold standard. For the ROC curves analyses, the external criterion has to be dichotomized. Patients were classified as “stable/improved” or “worsened” on the HRCT evaluation. The external criterion should be a well-accepted indication of change in the condition of the patient, clinicians should regard change in this standard as clinically meaningful [22]. The areas under the ROC curves (AUC) were calculated to determine the accuracy to discriminate “stable/improved” from “worsened” patients. A non-discriminating test has an AUC of 0.5 and a perfect discriminating test has an AUC of 1.0. The threshold level with the most discriminative utility is the best combination of sensitivity and specificity. ROC curves allow to choose the threshold that is the best compromise between sensitivity and specificity for each CaM
2.2. HRCT assessment and conventional visual reader-based score (CoVR) quantification Thin section volumetric CT examinations using a GE light Speed VCT power scanner were considered eligible for inclusion in the study. These scans were obtained in full inspiration with the patient in a supine position from the apex to the pulmonary base. The scanning parameters were: 120 kV, and 300 mAs, 0.8 s acquisition time, 1 mm slice thickness with 0.6 mm reconstructions and lowest possible field of view (FOV) covering both lungs. They were viewed with a window level of −600 Hounsfield units (HU) and a width of 1600 HU. Contrast media were not added to the HRCT evaluation. The HRCTs in question were evaluated independently by a radiologist (MC – consultant with 15 years of experience in the field of musculoskeletal radiology) and a rheumatologist (FS – trained in CT interpretation). The chronological order of the CT scans, but not the clinical information of the patients, was known to the readers. Readers were asked if they had observed any progression of the SSc-ILD. During the same observation session, if any progression was noticed, readers should state whether it was significant (i.e., to be considered a substantial progression in their relationship). The unanimous opinion of the two readers on the existence of a significant progression was used. A third reader (AGi – expert radiologist) examined the scans reported in a discordant way to obtain a final consensus decision. To estimate 2
European Journal of Internal Medicine xxx (xxxx) xxx–xxx
F. Salaffi, et al.
and CoVR criterion. The comparative accuracy of CoVR and CaM was determined by comparing the AUC using the Wilcoxon signed-rank test [25].
Table. 1 Statistics for the conventional visual reader-based score (CoVR) in the sample of “stable/improved” (n = 17) and “worsened” patients (n = 28). “Stable/improved” patients (n = 17) Lowest value Highest value Arithmetic mean 95% CI for the Arithmetic mean Median 95% CI for the median Variance Standard deviation Relative standard deviation Standard error of the mean Coefficient of Skewness
3. Results Forty-five patients with SSc (29 with lcSSc and 16 with dcSSc), 11 (24.4%) males and 34 (75.6%) females were included, with a mean ± SD age of 54.1 ± 11.0 years and a mean ± SD disease duration of 8.6 ± 4.7 years. Thirteen (28.9%) patients were positive for antitopoisomerase I antibodies, while 32 (71.1%) patients were positive for anticentromere antibodies. Treatment was performed according to different protocols: 17 (37.8%) patients were treated with rituximab (RTX) using two different regimens (1000 mg fortnightly x 2 or 375 mg/m²/week for 4 consecutive weeks) at baseline and after 6 months, associated with mycophenolate mofetil (MMF) 2000 mg/day, 16 (35.6%) patients received cyclophosphamide (CYC), titrated as tolerated at 2 mg/kg/day, 6 (13.3%) patients received a combination of oral imatinib (200 mg/day) and CYC, titrated as tolerated at 2 mg/kg/day and 6 (13.3%) patients received monotheraphy with MMF (2000 mg/day). All patients also received treatment with corticosteroids (≤10 mg/day prednisone equivalent), antiaggregant agents, calcium channel blockers, proton pump inhibitors and, if indicated, bisphosphonates and vitamin D supplement. After 12 months, 17 (37.8%) patients had stable or improved lung involvement (“stable/improved”), while 28 (62.2%) patients worsened (“worsened”) ILD. Longitudinal analysis of baseline and one-year differences in the two groups of patients, using the CoVR method, showed no significant differences (t-testing of paired samples) (Table 1); on the contrary, CaM found a statistically significant difference in either group (Table 2). Table 3 lists the SRM results for the two scoring methods, showing that CaM was more responsive in both the “stable/improved” and “worsened” groups compared to the performance of the CoVR method. A similar result emerges from the analysis of the ROC curves (Fig. 1). The AUC of CaM was 0.951 (standard error = 0.0287) with 95% CI from 0.841 to 0.993, while of CoVR it was 0.807 (standard error = 0.0644) with 95% CI from 0.662 to 0.909. The ability to evaluate changes in HRCT was better when using CaM than CoVR. The difference between the variations in the scores of the two methods was significant (differences between AUCs = 0.144, 95% C.I. 0.0402–0.248, p = 0.0065).
Paired samples t-test Mean difference Standard deviation 95% CI
4. Discussion
Test statistic t Two-tailed probability
Coefficient of Kurtosis Shapiro–Wilk test for Normal distribution Paired samples t-test Mean difference Standard deviation 95% CI Test statistic t Two-tailed probability “Worsened” patients (n = 28) Lowest value Highest value Arithmetic mean 95% CI for the Arithmetic mean Median 95% CI for the median Variance Standard deviation Relative standard deviation Standard error of the mean Coefficient of Skewness Coefficient of Kurtosis Shapiro-Wilk test for Normal distribution
Our study shows that, after 12 months, lung involvement improved or remained stable in 37.8% of patients and worsened in 62.2%. HRCT scores changed moderately in 12 months. CaM showed significantly better internal performance than CoVR. These results are in agreement with previous literature data [12,18]. Goldin et al. evaluated the variations of SSc-ILD extension on HRCT at baseline and after two years in 97 patients, divided into two groups: 47 subjects treated with CYC, 50 subjects with MMF. They used a CaM score able to quantify pulmonary fibrosis, ground glass, honeycombing and normal lung volumes, obtaining a total quantitative ILD score (QILD). They found an average QILD reduction of 2.51% in the whole group and no difference between the two groups. They also found a significant correlation between QILD and forced vital capacity, carbon monoxide diffusion capacity and dyspnea index. They concluded that changes in QILD score provide structural evidence for stabilization or improvement of ILD, suggesting its introduction in the follow-up of SScILD in clinical trials [26]. Recently, the same authors used a CaM to determine its ability to assess the likelihood of change from one ILD pattern to another at baseline and two years after therapy in the same group of SSc-ILD patients (47 and 50 patients in CYC and MMF arms, respectively). They considered four patterns: fibrotic, ground glass,
Baseline 8.00 30.00 16.58 12.97–20.20 15.00 12.00–20.91 49.38 7.02 0.42 (42.36%) 1.70 0.8839 (P = 0.1048) −0.6665 (P = 0.5628) W = 0.8534 reject Normality (P = 0.0121)
1-year changes 8.00 30.00 16.00 12.45–19.54 14.00 11.01–19.91 47.62 6.90 0.43 (43.13%) 1.67 0.9504 (P = 0.0833) −0.4807 (P = 0.7463) W = 0.8529 reject Normality (P = 0.0119)
−0.5882 0.8703 −1.0357 to −0.1408 −2087 P = 0.0632 Baseline 1-year changes 6.00 28.00 16.64 14.28–18.99 15.00 11.71–20.64 36.83 6.06 0.36 (36.47%) 1.14 0.1308 (P = 0.7531) −1.2048 (P = 0.0261) W = 0.9393 accept Normality (P = 0.1064)
7.00 29.00 17.35 15.01–19.70 15.50 14.35–21.00 36.53 6.04 0.34 (34.82%) 1.14 0.09353 (P = 0.8218) −1.1538 (P = 0.0398) W = 0.9450 accept Normality (P = 0.1483)
−0.7136 0.8176 −1.342 to −0.1503 −1987 P = 0.0724
honeycombing and normal pattern. They found an overall improvement from ground glass or fibrotic pattern to normal lung, significant in both groups of treatments (p < 0.001). They stated that the most important issue is the ability to reveal changes from one pattern to another, concluding that the evaluation of CaM must be validated and used in clinical practice [27]. An Italian group recently developed a computerized integrated index (CII) for ILD, which integrates mean lung attenuation, kurtosis and skewness. Eighty-three patients with SSc were studied, of whom 39 (47%) with ILD, comparing the CII with the HRCT visual score of Goh et al. [9] and, in addition, with laboratory data and lung function. They obtained a significant correlation (p < 0.001) between CII and visual score, concluding that CII discriminates between ILD and no ILD with excellent reproducibility (0.99) and strongly correlated with visual score. CII correlates with both lung function and immuno-inflammatory parameters even in early lung disease [28]. In the era of precision medicine, these results confirm the need to develop an easy and feasible computerized score to estimate the extent of ILD in the clinical setting, in order to customize the management of SSc-ILD, a field still debated in the medical literature 17,18,29,30]. 3
European Journal of Internal Medicine xxx (xxxx) xxx–xxx
F. Salaffi, et al.
Table. 2 Statistics for the computer-aided method (CaM) in the sample of “stable/improved” (n = 17) and “worsened” patients (n = 28). “Stable/improved” patients (n = 17) Lowest value Highest value Arithmetic mean 95% CI for the Arithmetic mean Median 95% CI for the median Variance Standard deviation Relative standard deviation Standard error of the mean Coefficient of Skewness Coefficient of Kurtosis Shapiro-Wilk test for Normal distribution Paired samples t-test Mean difference Standard deviation 95% CI Test statistic t Two-tailed probability “Worsened” patients (n = 28) Lowest value Highest value Arithmetic mean 95% CI for the Arithmetic mean Median 95% CI for the median Variance Standard deviation Relative standard deviation Standard error of the mean Coefficient of Skewness Coefficient of Kurtosis Shapiro-Wilk test for Normal distribution Paired samples t-test Mean difference Standard deviation 95% CI Test statistic t Two-tailed probability
Baseline
1-year changes
7.80 35.40 20.42 16.24–24.60 20.60 13.83–25.58 66.17 8.13 0.39 (39.83%) 1.97 0.2474 (P = 0.6345) −1.0030 (P = 0.2618) W = 0.9598 accept Normality (P = 0.6279)
6.60 33.40 17.12 12.99–21.26 13.10 11.91–25.40 64.77 8.04 0.46 (46.99%) 1.95 0.6013 (P = 0.2576) −0.7666 (P = 0.4665) W = 0.9174 accept Normality (P = 0.1335)
3.2941 2.7467 1.8819–4.7063 4.945 P = 0.0001 Baseline
1-year changes
6.90 37.50 16.73 14.26–19.19 15.55 13.58–19.49 40.33 6.35 0.37 (37.96%) 1.20 1.1917 (P = 0.0111) 2.9884 (P = 0.0187) W = 0.9262 reject Normality (P = 0.0493)
5.90 46.50 19.03 15.99–22.07 18.05 16.20–21.14 61.60 7.84 0.41 (41.23%) 1.48 1.4984 (P = 0.0024) 4.6150 (P = 0.0035) W = 0.8955 reject Normality (P = 0.0090)
Fig. 1. ROC curves illustrating the relationship between sensitivity and complement of specificity (100 – specificity) for differences in computer-aided method (CaM) and conventional visual reader based score (CoVR) between baseline and 1 year. The line running diagonally along the Figure from the bottom left to the top right defines an area of 0.5 and represents a tool not able to discriminate the different state of activity of the disease. CaM shows higher accuracy than the CoVR method to distinguish “stable/improved” from “worsened” (difference in AUC = 0.144. 95% C.I. 0.402–0.248, p = 0.0065). The expert's opinion is used as an external criterion.
computed evaluations of pulmonary fibrosis are easier and faster to perform, as it is fully processed by the machine, but above all it reduces errors by eliminating those produced by the personal interpretation of the reader. The CaM we used showed high agreement with visual HRCT analysis [13], good validity, reliability and feasibility in the evaluation of SSc-ILD [14,15]. Further merits are the simplicity and promptness of use even for a non-radiologist. It has a high sensitivity for small changes not estimated with a conventional method. These observations indicate that CaM analysis is useful for objective and reproducible measurements of ILD, improving the objectivity, sensitivity and repeatability of quantitative changes in lung features [11,18]. Our study has limitations. First, the group is small. To achieve more significant results it is necessary to enroll a larger group and compare it with a control group of untreated patients. Secondly, the CaM we use does not evaluate the pattern of fibrosis and the changes from one pattern to another. Moreover, to measure the construct of the ROC approach is necessary an anchor as gold standard, which in this case is the HRCT change of the severity of the fibrosis. Unfortunately, anchors used in the literature, such as expert opinion to estimate HRCT changes, have not been widely validated, so there is a strong criticism of their reliability and validity [32,33]. Finally, our work has evaluated only one aspect of the SSc, the ILD, and therefore our results only concern lung involvement, not considering a global evolution of the disease.
−2.3036 3.5671 −3.6867–−0.9204 −3.417 P = 0.0021
Table. 3 Standardized response mean (SRM) (with 95% CI) for the computer-aided method (CaM) and conventional visual reader-based score (CoVR) in both groups.
CaM “stable/improved” CaM “worsened” CoVR “stable/improved” CoVR “worsened”
Value
95% CIa
1.1993 0.8188 0.6059 0.4589
0.4919–1.8083 0.3658–1.0952 0.1584–1.2318 0.1717–0.8142
5. Conclusions The study confirms that CaM is more sensitive than CoVR, is quick and easy to use and has high reproducibility in estimating fibrosis. Further studies are needed to validate these results on larger patient groups in order to use this tool as a surrogate for the response to therapy.
a
BCa bootstrap confidence interval (1000 iterations; random number seed: 978).
The use of visual scoring systems in quantification of ILD is not widespread due to the complexity of execution and is performed only by dedicated radiologists with high experience, obtaining a low inter and intra-reader agreement [31, 32]. This moderate agreement of visual scoring systems degrades the clinical usefulness of the scores themselves, compromising their application in daily practice and clinical trials. The introduction of imaging techniques brings many advantages, helping to solve some of these difficulties [19,43]. Quantitative
Funding statement This work was not supported by any research grants. CRediT authorship contribution statement Fausto Salaffi: Conceptualization, Data curation, Formal analysis, 4
European Journal of Internal Medicine xxx (xxxx) xxx–xxx
F. Salaffi, et al.
Investigation, Methodology, Supervision, Writing - original draft. Marina Carotti: Investigation, Writing - review & editing. Marika Tardella: Investigation, Writing - original draft. Marco Di Carlo: Investigation, Writing - original draft. Paolo Fraticelli: Conceptualization, Investigation. Colomba Fischetti: Investigation. Andrea Giovagnoni: Conceptualization, Investigation, Writing - review & editing. Armando Gabrielli: Conceptualization, Methodology, Supervision, Writing - review & editing.
[15]
[16]
[17]
Declaration of Competing Interest [18]
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. The authors declare that there is no conflict of interest regarding the publication of this paper.
[19]
[20]
References [21]
[1] Allanore Y, Simms R, Distler O, Trojanowska M, Pope J, Denton CP, et al. Systemic sclerosis. Nat Rev Dis Primers 2015;23(1):15002. [2] Elhai M, Meune C, Avouac J, Kahan A, Allanore Y. Trends in mortality in patients with systemic sclerosis over 40 years: a systematic review and meta-analysis of cohort studies. Rheumatology (Oxford) 2012;51(6):1017–26. [3] Walker UA, Tyndall A, Czirják L, Denton C, Farge-Bancel D, Kowal-Bielecka O, et al. Clinical risk assessment of organ manifestations in systemic sclerosis: a report from the Eular Scleroderma trials and research group database. Ann Rheum Dis 2007;66:754–63. [4] Patiwetwitoon S, Wangkaew S, Euathrongchit J, Kasitanon N, Louthrenoo W. Highresolution computed tomographic findings in systemic sclerosis-associated interstitial lung disease: comparison between diffuse and limited systemic sclerosis. J Clin Rheumatol 2012;18:229–33. [5] Goldin JG, Lynch DA, Strollo DC, Suh RD, Schraufnagel DE, Clements PJ, et al. High resolution CT findings in scleroderma-related lung diseases: findings from scleroderma lung study. Chest 2008;134:358–67. [6] Kim EA, Johkoh T, Lee KS, Ichikado K, Koh EM, Kim TS, et al. Interstitial pneumonia in progressive systemic sclerosis: serial high-resolution CT findings with functional correlation. J Comput Assist Tomogr 2001;25:757–63. [7] Diot E, Boissinot E, Asquier E, Guilmot JL, Lemarié E, Valat C, et al. Relationship between abnormalities on high-resolution CT and pulmonary function in systemic sclerosis. Chest 1998;114:1623–9. [8] Warrick JH, Bhalla M, Schabel SI, Silver RM. High resolution computed tomography in early scleroderma lung disease. J Rheumatol 1991;18:1520–8. [9] Goh N, Desai SR, Veeraraghavan S, Hansell DM, Copley SJ, Maher TM, et al. Interstitial lung disease in systemic sclerosis: a simple staging system. Am J Respir Crit Care Med 2008;177:1248–54. [10] Kim HJ, Li G, Gjertson D, Elashoff R, Shah SK, Ochs R, et al. Classification of parenchymal abnormality in scleroderma lung using a novel approach to denoise images collected via a multicenter study. Acad Radiol 2008;15:1004–16. [11] Yabuuchi H, Matsuo Y, Tsukamoto H, Horiuchi T, Sunami S, Kamitani T, et al. Evaluation of the extent of ground-glass opacityon high-resolution ct in patients with interstitial pneumonia associated with systemic sclerosis: comparison between quantitative and qualitative analysis. Clin Radiol 2014;69:758–64. [12] Kim HG, Tashkin DP, Clements PJ, Li G, Brown MS, Elashoff R, et al. A computeraided diagnosis system for quantitative scoring of extent of lung fibrosis in scleroderma patients. Clin Exp Rheumatol 2010;28(5 Suppl 62):S26–35. [13] Ariani A, Carotti M, Gutierrez M, Bichisecchi E, Grassi W, Giuseppetti GM, et al. Utility of an open-source DICOM viewer software (OsiriX) to assess pulmonary fibrosis in systemic sclerosis: preliminary results. Rheumatol Int 2014;34:511–6. [14] Salaffi F, Carotti M, Bosello S, Ciapetti A, Gutierrez M, Bichisecchi E, et al. Computer-aided quantification of interstitial lung disease from high resolution
[22] [23]
[24] [25] [26]
[27]
[28]
[29]
[30]
[31]
[32]
[33]
5
computed tomography images in systemic sclerosis: correlation with visual readerbased score and physiologic tests. Biomed Res Int 2015;2015:834262. Salaffi F, Carotti M, Di Donato E, Di Carlo M, Ceccarelli M, Giuseppetti G. Computer-Aided tomographic analysis of interstitial lung disease (ILD) in patients with systemic sclerosis (SSc). Correlation with pulmonary physiologic tests and patient-centred measures of perceived dyspnea and functional disability. PLoS One 2016;11:e0149240. Tashkin DP, Elashoff R, Clements PJ, Goldin J, Roth MD, Furst DE, et al. Cyclophosphamide versus placebo in scleroderma lung disease. N Engl J Med 2006;354:2655–66. Roth MD, Tseng CH, Clements PJ, Furst DE, Tashkin DP, Goldin JG, et al. Predicting treatment outcomes and responder subsets in scleroderma-related interstitial lung disease. Arthritis Rheum 2011;63:2797–808. Kim HJ, Tashkin DP, Gjertson DW, Brown MS, Kleerup E, Chong S, et al. Transitions to different patterns of interstitial lung disease in scleroderma with and without treatment. Ann Rheum Dis 2016;75:1367–71. van den Hoogen F, Khanna D, Fransen J, Johnson SR, Baron M, Tyndall A, et al. classification criteria for systemic sclerosis: an American college of Rheumatology/ European League against Rheumatism collaborative initiative. Arthritis Rheum 2013;65:2737–47. 2013. LeRoy EC, Black C, Fleischmajer R, Jablonska S, Krieg T, Medsger TA, et al. Scleroderma (systemic sclerosis): classification, subsets and pathogenesis. J Rheumatol 1988;15:202–5. Shin KE, Chung MJ, Jung MP, Choe BK, Lee KS. Quantitative computed tomographic indexes in diffuse interstitial lung disease: correlation with physiologic tests and computed tomography visual scores. J Comput Assist Tomogr 2011;35:266–71. Husted JA, Cook RJ, Farewell VT, Gladmann DD. Methods for assessing responsiveness: a critical review and recommendations. J Clin Epidemiol 2000;53:459–68. Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol 2008;61:102–9. Efron B, Gong G. A leisurely look at the bootstrap, the jackknife, and cross-validation. Am Statistician 1983;37:36–48. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983;148:839–43. Goldin JG, Kim GHJ, Tseng CH, Volkmann E, Furst D, Clements P, et al. Longitudinal changes in quantitative interstitial lung disease on CT after immunosuppression in the scleroderma lung study II. Annals ATS 2018. Kim GHJ, Tashkin DP, Lo P, Brown M, Volkmann E, Gjertson DW, et al. Using transitional changes on HRCT to monitor the impact of cyclophosphamide or mycophenolate on systemic sclerosis-related interstitial lung disease. Arthritis Rheumatol 2019. Aug 20. Bocchino M, Bruzzese D, D'Alto M, Argiento P, Borgia A, Capaccio A, et al. Performance of a new quantitative computed tomography index for interstitial lung disease assessment in systemic sclerosis. Sci Rep 2019;9(1):9468. Jul 1. Adler S, Huscher D, Siegert E, Allanore Y, Czirják L, DelGaldo F, et al. Systemic sclerosis associated interstitial lung disease – individualized immunosuppressive therapy and course of lung function: results of the Eustar group. Arthritis Res Ther 2018;20(1):17. 30. Khanna D, Brown KK, Clements PJ, Elashoff R, Furst DE, Goldin G, et al. Systemic sclerosis-associated interstitial lung disease-proposed recommendations for future randomized clinical trials. Clin Exp Rheumatol 2010;28(2 Suppl 58):S55–62. Goldin J, Elashoff R, Kim HJ, Yan X, Lynch D, Strollo D, et al. Treatment of scleroderma-interstitial lung disease with cyclophosphamide is associated with less progressive fibrosis on serial thoracic high-resolution CT scan than placebo. Chest 2009;136:1333–40. de Vet HC, Terluin B, Knol D, Roorda LD, Mokkink LB, Ostelo RW, et al. Three ways to quantify uncertainty in individually applied “minimally important change” (MIC) values. J Clin Epidemiol 2010;63:37–45. Salaffi F, Carotti M, Di Carlo M, Tardella M, Giovagnoni A. High-resolution computed tomography of the lung in patients with rheumatoid arthritis: Prevalence of interstitial lung disease involvement and determinants of abnormalities. Medicine (Baltimore) 2019;98(38):e17088. https://doi.org/10.1097/MD. 0000000000017088. Vol:No(e17088).
Contents lists available at ScienceDirect
European Journal of Internal Medicine journal homepage: www.elsevier.com/locate/ejim
Original article
Computed tomography assessment of evolution of interstitial lung disease in systemic sclerosis: Comparison of two scoring systems ⁎
Fausto Salaffia, Marina Carottib, Marika Tardellaa, , Marco Di Carloa, Paolo Fraticellic, Colomba Fischettic, Andrea Giovagnonib, Armando Gabriellid a
Rheumatological Clinic, Ospedale Carlo Urbani Jesi, Università Politecnica delle Marche, Ancona, Italy Department of Radiology, Ospedali Riuniti, Università Politecnica delle Marche, Italy c Department of Internal Medicine, Ospedali Riuniti, Università Politecnica delle Marche, Ancona, Italy d Department of Clinical and Molecular Sciences, Università Politecnica delle Marche, Ancona, Italy b
A R T I C LE I N FO
A B S T R A C T
Keywords: High resolution computed tomography Computed-aided method Conventional visual reader-based score
Background: The aim of this study was to evaluate and compare the internal and external responsiveness of a computed-aided method (CaM) with a conventional visual reader-based score (CoVR) to measure interstitial lung disease (ILD) in patients with systemic sclerosis (SSc) on high resolution computed tomography (HRCT). Methods: Forty-five patients were evaluated in this retrospective cohort. HRCTs were collected at baseline and after 1 year. HRCT abnormalities were evaluated according to a CoVR (Warrick's method) and a quantitative CaM. Internal 1-year responsiveness was tested with a standardized mean response (SRM). Analyses of the receiver operating characteristic curves (ROCs) evaluated the sensitivity and specificity of the two methods to discriminate between clinically relevant progression and no relevant progression, using expert judgment as the gold standard (external responsiveness). Results: In one year, lung involvement was stable/improved in 17 of the 45 patients (37.8%) and worsened in 28 patients (62.2%). HRCT scores changed moderately over the follow-up period. Using SFM, CaM was significantly more responsive in detecting changes due to treatment than the CoVR method. Likewise, in the analysis of the ROC curve, CaM scores showed the highest performance (AUC ROC CaM vs. CoVR, 0.951 vs. 0.807; p = 0.0065). Conclusion: Quantitative analysis of CaM was more responsive than the CoVR method to accurately evaluate and monitor SSc-ILD progression or response to therapy.
1. Introduction Systemic sclerosis (SSc) is a connective tissue disease characterized by extensive skin fibrosis and abnormalities in large and small vessels. Although cutaneous manifestations are the most frequent sign of the disease, there may be involvement of internal organs, particularly the lungs, with a severe impact on prognosis [1,2]. Interstitial lung disease (ILD) is the most frequent manifestation of lung involvement, with a prevalence of 53% of cases with diffuse SSc and 35% of cases with limited SSc [3]. In daily clinical practice, the main tool for the detection and evaluation of SSc-ILDs is high resolution computed tomography (HRCT) [4,5]. In many studies, the high correlation between HRCT results and pulmonary function test (PFT) results has been demonstrated [6,7]. Several methods of semi-quantitative HRCT evaluation
were described and used to characterize and quantify the extent of pulmonary fibrosis [8,9]. Intra- and inter-reader changes and the need for detailed knowledge of pulmonary radiological anatomy limit the implementation and diffusion of these scoring systems. This impairs the possibility of using these methods in ILD practice. In recent years, a computed automatic evaluation of HRCT results has been developed, which has improved the objectivity, sensitivity and repeatability of quantitative changes in lung characteristics compared to traditional visual interpretation [10–12]. Previously, we demonstrated high agreement on semi-quantitative HRCT analysis performed by experienced radiologists and a significant association between descriptive parameters in both quantitative and semi-quantitative HRCT analysis [13]. We also demonstrated good validity, reliability and feasibility in the evaluation of SSc-ILD using a computed-aided method (CaM)
⁎
Corresponding author. E-mail addresses: fausto.salaffi@gmail.com (F. Salaffi), [email protected] (M. Carotti), [email protected] (M. Tardella), [email protected] (M. Di Carlo), [email protected] (P. Fraticelli), colomba.fi[email protected] (C. Fischetti), [email protected] (A. Giovagnoni), [email protected] (A. Gabrielli). https://doi.org/10.1016/j.ejim.2020.02.009 Received 2 November 2019; Received in revised form 29 January 2020; Accepted 11 February 2020 0953-6205/ © 2020 Published by Elsevier B.V. on behalf of European Federation of Internal Medicine.
Please cite this article as: Fausto Salaffi, et al., European Journal of Internal Medicine, https://doi.org/10.1016/j.ejim.2020.02.009
European Journal of Internal Medicine xxx (xxxx) xxx–xxx
F. Salaffi, et al.
intraobserver reliability, each reader examined all HRCTs twice, with an interval of at least four weeks. The Warrick et al. method was used to evaluate lung disease results [8]. The mean values of the two independent readers were used for the final control group. The intraclass correlation coefficient (ICC) of the radiologists' agreement level on total HRCT scores, calculated by the CoVR method, was 0.80 [14,15].
[14,15]. Although HRCT has been used to define the characteristics and extent of SSc-ILD and to predict results, experience with its use as a measure of results in therapeutic trials is limited [16–18]. In the present study, we examined data from serial HRCTs performed in a cohort of SSc patients treated with different protocols. The purpose of the study was to evaluate and compare the internal and external responsiveness of two different HRCT evaluation methods (CaM and conventional visual-reader based score [CoVR]) to detect response to one-year immunosuppressive therapy in SSc-ILD.
2.3. Computer-aided method (CaM) quantification process HRCT images were reconstructed and analyzed by OsiriX MD 7, a DICOM software viewer (OsiriX MD version 7, 64-bit format), on a Mac Mini (Intel Core 2 Duo desktop computer 2.8 GHz, 16 GB random access memory; Apple Computer, Cupertino, CA, USA) with Mac OSX 10.12.2 operating system. The program uses a semiautomatic thresholding technique to isolate the lungs from other tissues. A semiautomatic parenchymal pulmonary segmentation was performed in each section to obtain the analysis of all images. The calculated HRCT lung mapping analysis was performed using a radiodensity of −200 to −1024 HU for the pulmonary parenchyma (isolated from the mediastinal and thoracic wall) and −700 to −500 HU for ILD [21]. Minimal user intervention was required to exclude blood vessels and large hilum bronchi. The CT attenuation of the normal lung parenchyma is reported in a range from −800 to −900 HU, depending on inspiration or expiration, on the level of inspiration achieved for the scan and on the anatomical position, i.e. the ventral or dorsal portion. The attenuation area between −500 and −700 HU has been defined as the radiodensity value for ILD, which includes both the opacity of the ground glass and the reticular opacity. The radiodensity of −500 HU was chosen as the threshold between the consolidation and opacity of ground glass. Therefore, −700 HU was recognized as the default threshold value for normal lung portions. For each HRCT, two CaM measurements were performed with a total agreement between the first and the second calculation of the scores (95% of the agreement limits = 0–0, ICC = 1) [13].
2. Patients and methods 2.1. Study population and protocol This is a retrospective analysis carried out in patients attending the Rheumatological Clinic and the Medical Clinic, Università Politecnica delle Marche, Italy, from January 1, 2014 to June 30, 2017, stored in a dedicated database. We included in the study patients with SSc, defined by the classification criteria of the American College of Rheumatology/ European League against Rheumatism 2013 [19] and classified in limited and diffuse skin involvement (lcSSc and dcSSc, respectively) according to the extension of skin thickness [20]. The criteria for inclusion were: age over 18 years, presence of ILD demonstrated at HRCT for which specific immunosuppressive therapy was started, absence of known pleural and pulmonary pathology (e.g. chronic obstructive bronchopneumopathy, emphysema) other than ILD, absence of other diseases (e.g. chronic heart failure, hypoalbuminemia) or therapy that could influence the results of HRCT (potentially toxic drugs for the lungs). The study protocol focused the analysis on the comparison between the responsiveness of a CaM and a CoVR to measure the SSc-ILD and evaluate the changes in the score. We evaluated the results of HRCT chest investigations performed at baseline, when immunosuppressive therapy was started, and after 12 months. Patients who did not undergo HRCT after 12 months were excluded. The local Ethics Committee (Comitato Unico Regionale – ASUR Marche) approved the protocol, the patient information sheet and the consent form. The study was conducted in accordance with the Helsinki Declaration in its fifth edition (2000). The informed consent was signed by all participants.
2.4. Statistical analysis The data was entered into a Microsoft Excel database and analysed with MedCalc® version 18.6 (MedCalc Software, Ostend, Belgium). The values were expressed both as mean ± standard deviation (SD) and as median (interquartile range [IQR]). The Student's paired-t-test evaluated whether the scores of the CoVR method and the CaM changed over 1-year. Responsiveness was assessed in the 1-year longitudinal HRCT follow-up. Based on the suggestion of Husted et al. [22], internal and external responsiveness were measured. To analyze internal responsiveness over 1-year, the standardized response mean (SRM) was used. The SRM is calculated as the mean change score between baseline and 1-year divided by the SD of this difference [23]. Ninety-five percent confidence intervals (95% CIs) for the SRM were estimated through bootstrap resampling. Bootstrapping consists of resampling with replacement. We selected 1000 samples with replacement and calculated the SRM for each sample [24]. Receiver operating characteristic (ROC) curve analyses assessed the sensitivity and specificity of the two scoring methods to discriminate between clinically relevant and no relevant progression, using judgment of the experts as gold standard. For the ROC curves analyses, the external criterion has to be dichotomized. Patients were classified as “stable/improved” or “worsened” on the HRCT evaluation. The external criterion should be a well-accepted indication of change in the condition of the patient, clinicians should regard change in this standard as clinically meaningful [22]. The areas under the ROC curves (AUC) were calculated to determine the accuracy to discriminate “stable/improved” from “worsened” patients. A non-discriminating test has an AUC of 0.5 and a perfect discriminating test has an AUC of 1.0. The threshold level with the most discriminative utility is the best combination of sensitivity and specificity. ROC curves allow to choose the threshold that is the best compromise between sensitivity and specificity for each CaM
2.2. HRCT assessment and conventional visual reader-based score (CoVR) quantification Thin section volumetric CT examinations using a GE light Speed VCT power scanner were considered eligible for inclusion in the study. These scans were obtained in full inspiration with the patient in a supine position from the apex to the pulmonary base. The scanning parameters were: 120 kV, and 300 mAs, 0.8 s acquisition time, 1 mm slice thickness with 0.6 mm reconstructions and lowest possible field of view (FOV) covering both lungs. They were viewed with a window level of −600 Hounsfield units (HU) and a width of 1600 HU. Contrast media were not added to the HRCT evaluation. The HRCTs in question were evaluated independently by a radiologist (MC – consultant with 15 years of experience in the field of musculoskeletal radiology) and a rheumatologist (FS – trained in CT interpretation). The chronological order of the CT scans, but not the clinical information of the patients, was known to the readers. Readers were asked if they had observed any progression of the SSc-ILD. During the same observation session, if any progression was noticed, readers should state whether it was significant (i.e., to be considered a substantial progression in their relationship). The unanimous opinion of the two readers on the existence of a significant progression was used. A third reader (AGi – expert radiologist) examined the scans reported in a discordant way to obtain a final consensus decision. To estimate 2
European Journal of Internal Medicine xxx (xxxx) xxx–xxx
F. Salaffi, et al.
and CoVR criterion. The comparative accuracy of CoVR and CaM was determined by comparing the AUC using the Wilcoxon signed-rank test [25].
Table. 1 Statistics for the conventional visual reader-based score (CoVR) in the sample of “stable/improved” (n = 17) and “worsened” patients (n = 28). “Stable/improved” patients (n = 17) Lowest value Highest value Arithmetic mean 95% CI for the Arithmetic mean Median 95% CI for the median Variance Standard deviation Relative standard deviation Standard error of the mean Coefficient of Skewness
3. Results Forty-five patients with SSc (29 with lcSSc and 16 with dcSSc), 11 (24.4%) males and 34 (75.6%) females were included, with a mean ± SD age of 54.1 ± 11.0 years and a mean ± SD disease duration of 8.6 ± 4.7 years. Thirteen (28.9%) patients were positive for antitopoisomerase I antibodies, while 32 (71.1%) patients were positive for anticentromere antibodies. Treatment was performed according to different protocols: 17 (37.8%) patients were treated with rituximab (RTX) using two different regimens (1000 mg fortnightly x 2 or 375 mg/m²/week for 4 consecutive weeks) at baseline and after 6 months, associated with mycophenolate mofetil (MMF) 2000 mg/day, 16 (35.6%) patients received cyclophosphamide (CYC), titrated as tolerated at 2 mg/kg/day, 6 (13.3%) patients received a combination of oral imatinib (200 mg/day) and CYC, titrated as tolerated at 2 mg/kg/day and 6 (13.3%) patients received monotheraphy with MMF (2000 mg/day). All patients also received treatment with corticosteroids (≤10 mg/day prednisone equivalent), antiaggregant agents, calcium channel blockers, proton pump inhibitors and, if indicated, bisphosphonates and vitamin D supplement. After 12 months, 17 (37.8%) patients had stable or improved lung involvement (“stable/improved”), while 28 (62.2%) patients worsened (“worsened”) ILD. Longitudinal analysis of baseline and one-year differences in the two groups of patients, using the CoVR method, showed no significant differences (t-testing of paired samples) (Table 1); on the contrary, CaM found a statistically significant difference in either group (Table 2). Table 3 lists the SRM results for the two scoring methods, showing that CaM was more responsive in both the “stable/improved” and “worsened” groups compared to the performance of the CoVR method. A similar result emerges from the analysis of the ROC curves (Fig. 1). The AUC of CaM was 0.951 (standard error = 0.0287) with 95% CI from 0.841 to 0.993, while of CoVR it was 0.807 (standard error = 0.0644) with 95% CI from 0.662 to 0.909. The ability to evaluate changes in HRCT was better when using CaM than CoVR. The difference between the variations in the scores of the two methods was significant (differences between AUCs = 0.144, 95% C.I. 0.0402–0.248, p = 0.0065).
Paired samples t-test Mean difference Standard deviation 95% CI
4. Discussion
Test statistic t Two-tailed probability
Coefficient of Kurtosis Shapiro–Wilk test for Normal distribution Paired samples t-test Mean difference Standard deviation 95% CI Test statistic t Two-tailed probability “Worsened” patients (n = 28) Lowest value Highest value Arithmetic mean 95% CI for the Arithmetic mean Median 95% CI for the median Variance Standard deviation Relative standard deviation Standard error of the mean Coefficient of Skewness Coefficient of Kurtosis Shapiro-Wilk test for Normal distribution
Our study shows that, after 12 months, lung involvement improved or remained stable in 37.8% of patients and worsened in 62.2%. HRCT scores changed moderately in 12 months. CaM showed significantly better internal performance than CoVR. These results are in agreement with previous literature data [12,18]. Goldin et al. evaluated the variations of SSc-ILD extension on HRCT at baseline and after two years in 97 patients, divided into two groups: 47 subjects treated with CYC, 50 subjects with MMF. They used a CaM score able to quantify pulmonary fibrosis, ground glass, honeycombing and normal lung volumes, obtaining a total quantitative ILD score (QILD). They found an average QILD reduction of 2.51% in the whole group and no difference between the two groups. They also found a significant correlation between QILD and forced vital capacity, carbon monoxide diffusion capacity and dyspnea index. They concluded that changes in QILD score provide structural evidence for stabilization or improvement of ILD, suggesting its introduction in the follow-up of SScILD in clinical trials [26]. Recently, the same authors used a CaM to determine its ability to assess the likelihood of change from one ILD pattern to another at baseline and two years after therapy in the same group of SSc-ILD patients (47 and 50 patients in CYC and MMF arms, respectively). They considered four patterns: fibrotic, ground glass,
Baseline 8.00 30.00 16.58 12.97–20.20 15.00 12.00–20.91 49.38 7.02 0.42 (42.36%) 1.70 0.8839 (P = 0.1048) −0.6665 (P = 0.5628) W = 0.8534 reject Normality (P = 0.0121)
1-year changes 8.00 30.00 16.00 12.45–19.54 14.00 11.01–19.91 47.62 6.90 0.43 (43.13%) 1.67 0.9504 (P = 0.0833) −0.4807 (P = 0.7463) W = 0.8529 reject Normality (P = 0.0119)
−0.5882 0.8703 −1.0357 to −0.1408 −2087 P = 0.0632 Baseline 1-year changes 6.00 28.00 16.64 14.28–18.99 15.00 11.71–20.64 36.83 6.06 0.36 (36.47%) 1.14 0.1308 (P = 0.7531) −1.2048 (P = 0.0261) W = 0.9393 accept Normality (P = 0.1064)
7.00 29.00 17.35 15.01–19.70 15.50 14.35–21.00 36.53 6.04 0.34 (34.82%) 1.14 0.09353 (P = 0.8218) −1.1538 (P = 0.0398) W = 0.9450 accept Normality (P = 0.1483)
−0.7136 0.8176 −1.342 to −0.1503 −1987 P = 0.0724
honeycombing and normal pattern. They found an overall improvement from ground glass or fibrotic pattern to normal lung, significant in both groups of treatments (p < 0.001). They stated that the most important issue is the ability to reveal changes from one pattern to another, concluding that the evaluation of CaM must be validated and used in clinical practice [27]. An Italian group recently developed a computerized integrated index (CII) for ILD, which integrates mean lung attenuation, kurtosis and skewness. Eighty-three patients with SSc were studied, of whom 39 (47%) with ILD, comparing the CII with the HRCT visual score of Goh et al. [9] and, in addition, with laboratory data and lung function. They obtained a significant correlation (p < 0.001) between CII and visual score, concluding that CII discriminates between ILD and no ILD with excellent reproducibility (0.99) and strongly correlated with visual score. CII correlates with both lung function and immuno-inflammatory parameters even in early lung disease [28]. In the era of precision medicine, these results confirm the need to develop an easy and feasible computerized score to estimate the extent of ILD in the clinical setting, in order to customize the management of SSc-ILD, a field still debated in the medical literature 17,18,29,30]. 3
European Journal of Internal Medicine xxx (xxxx) xxx–xxx
F. Salaffi, et al.
Table. 2 Statistics for the computer-aided method (CaM) in the sample of “stable/improved” (n = 17) and “worsened” patients (n = 28). “Stable/improved” patients (n = 17) Lowest value Highest value Arithmetic mean 95% CI for the Arithmetic mean Median 95% CI for the median Variance Standard deviation Relative standard deviation Standard error of the mean Coefficient of Skewness Coefficient of Kurtosis Shapiro-Wilk test for Normal distribution Paired samples t-test Mean difference Standard deviation 95% CI Test statistic t Two-tailed probability “Worsened” patients (n = 28) Lowest value Highest value Arithmetic mean 95% CI for the Arithmetic mean Median 95% CI for the median Variance Standard deviation Relative standard deviation Standard error of the mean Coefficient of Skewness Coefficient of Kurtosis Shapiro-Wilk test for Normal distribution Paired samples t-test Mean difference Standard deviation 95% CI Test statistic t Two-tailed probability
Baseline
1-year changes
7.80 35.40 20.42 16.24–24.60 20.60 13.83–25.58 66.17 8.13 0.39 (39.83%) 1.97 0.2474 (P = 0.6345) −1.0030 (P = 0.2618) W = 0.9598 accept Normality (P = 0.6279)
6.60 33.40 17.12 12.99–21.26 13.10 11.91–25.40 64.77 8.04 0.46 (46.99%) 1.95 0.6013 (P = 0.2576) −0.7666 (P = 0.4665) W = 0.9174 accept Normality (P = 0.1335)
3.2941 2.7467 1.8819–4.7063 4.945 P = 0.0001 Baseline
1-year changes
6.90 37.50 16.73 14.26–19.19 15.55 13.58–19.49 40.33 6.35 0.37 (37.96%) 1.20 1.1917 (P = 0.0111) 2.9884 (P = 0.0187) W = 0.9262 reject Normality (P = 0.0493)
5.90 46.50 19.03 15.99–22.07 18.05 16.20–21.14 61.60 7.84 0.41 (41.23%) 1.48 1.4984 (P = 0.0024) 4.6150 (P = 0.0035) W = 0.8955 reject Normality (P = 0.0090)
Fig. 1. ROC curves illustrating the relationship between sensitivity and complement of specificity (100 – specificity) for differences in computer-aided method (CaM) and conventional visual reader based score (CoVR) between baseline and 1 year. The line running diagonally along the Figure from the bottom left to the top right defines an area of 0.5 and represents a tool not able to discriminate the different state of activity of the disease. CaM shows higher accuracy than the CoVR method to distinguish “stable/improved” from “worsened” (difference in AUC = 0.144. 95% C.I. 0.402–0.248, p = 0.0065). The expert's opinion is used as an external criterion.
computed evaluations of pulmonary fibrosis are easier and faster to perform, as it is fully processed by the machine, but above all it reduces errors by eliminating those produced by the personal interpretation of the reader. The CaM we used showed high agreement with visual HRCT analysis [13], good validity, reliability and feasibility in the evaluation of SSc-ILD [14,15]. Further merits are the simplicity and promptness of use even for a non-radiologist. It has a high sensitivity for small changes not estimated with a conventional method. These observations indicate that CaM analysis is useful for objective and reproducible measurements of ILD, improving the objectivity, sensitivity and repeatability of quantitative changes in lung features [11,18]. Our study has limitations. First, the group is small. To achieve more significant results it is necessary to enroll a larger group and compare it with a control group of untreated patients. Secondly, the CaM we use does not evaluate the pattern of fibrosis and the changes from one pattern to another. Moreover, to measure the construct of the ROC approach is necessary an anchor as gold standard, which in this case is the HRCT change of the severity of the fibrosis. Unfortunately, anchors used in the literature, such as expert opinion to estimate HRCT changes, have not been widely validated, so there is a strong criticism of their reliability and validity [32,33]. Finally, our work has evaluated only one aspect of the SSc, the ILD, and therefore our results only concern lung involvement, not considering a global evolution of the disease.
−2.3036 3.5671 −3.6867–−0.9204 −3.417 P = 0.0021
Table. 3 Standardized response mean (SRM) (with 95% CI) for the computer-aided method (CaM) and conventional visual reader-based score (CoVR) in both groups.
CaM “stable/improved” CaM “worsened” CoVR “stable/improved” CoVR “worsened”
Value
95% CIa
1.1993 0.8188 0.6059 0.4589
0.4919–1.8083 0.3658–1.0952 0.1584–1.2318 0.1717–0.8142
5. Conclusions The study confirms that CaM is more sensitive than CoVR, is quick and easy to use and has high reproducibility in estimating fibrosis. Further studies are needed to validate these results on larger patient groups in order to use this tool as a surrogate for the response to therapy.
a
BCa bootstrap confidence interval (1000 iterations; random number seed: 978).
The use of visual scoring systems in quantification of ILD is not widespread due to the complexity of execution and is performed only by dedicated radiologists with high experience, obtaining a low inter and intra-reader agreement [31, 32]. This moderate agreement of visual scoring systems degrades the clinical usefulness of the scores themselves, compromising their application in daily practice and clinical trials. The introduction of imaging techniques brings many advantages, helping to solve some of these difficulties [19,43]. Quantitative
Funding statement This work was not supported by any research grants. CRediT authorship contribution statement Fausto Salaffi: Conceptualization, Data curation, Formal analysis, 4
European Journal of Internal Medicine xxx (xxxx) xxx–xxx
F. Salaffi, et al.
Investigation, Methodology, Supervision, Writing - original draft. Marina Carotti: Investigation, Writing - review & editing. Marika Tardella: Investigation, Writing - original draft. Marco Di Carlo: Investigation, Writing - original draft. Paolo Fraticelli: Conceptualization, Investigation. Colomba Fischetti: Investigation. Andrea Giovagnoni: Conceptualization, Investigation, Writing - review & editing. Armando Gabrielli: Conceptualization, Methodology, Supervision, Writing - review & editing.
[15]
[16]
[17]
Declaration of Competing Interest [18]
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. The authors declare that there is no conflict of interest regarding the publication of this paper.
[19]
[20]
References [21]
[1] Allanore Y, Simms R, Distler O, Trojanowska M, Pope J, Denton CP, et al. Systemic sclerosis. Nat Rev Dis Primers 2015;23(1):15002. [2] Elhai M, Meune C, Avouac J, Kahan A, Allanore Y. Trends in mortality in patients with systemic sclerosis over 40 years: a systematic review and meta-analysis of cohort studies. Rheumatology (Oxford) 2012;51(6):1017–26. [3] Walker UA, Tyndall A, Czirják L, Denton C, Farge-Bancel D, Kowal-Bielecka O, et al. Clinical risk assessment of organ manifestations in systemic sclerosis: a report from the Eular Scleroderma trials and research group database. Ann Rheum Dis 2007;66:754–63. [4] Patiwetwitoon S, Wangkaew S, Euathrongchit J, Kasitanon N, Louthrenoo W. Highresolution computed tomographic findings in systemic sclerosis-associated interstitial lung disease: comparison between diffuse and limited systemic sclerosis. J Clin Rheumatol 2012;18:229–33. [5] Goldin JG, Lynch DA, Strollo DC, Suh RD, Schraufnagel DE, Clements PJ, et al. High resolution CT findings in scleroderma-related lung diseases: findings from scleroderma lung study. Chest 2008;134:358–67. [6] Kim EA, Johkoh T, Lee KS, Ichikado K, Koh EM, Kim TS, et al. Interstitial pneumonia in progressive systemic sclerosis: serial high-resolution CT findings with functional correlation. J Comput Assist Tomogr 2001;25:757–63. [7] Diot E, Boissinot E, Asquier E, Guilmot JL, Lemarié E, Valat C, et al. Relationship between abnormalities on high-resolution CT and pulmonary function in systemic sclerosis. Chest 1998;114:1623–9. [8] Warrick JH, Bhalla M, Schabel SI, Silver RM. High resolution computed tomography in early scleroderma lung disease. J Rheumatol 1991;18:1520–8. [9] Goh N, Desai SR, Veeraraghavan S, Hansell DM, Copley SJ, Maher TM, et al. Interstitial lung disease in systemic sclerosis: a simple staging system. Am J Respir Crit Care Med 2008;177:1248–54. [10] Kim HJ, Li G, Gjertson D, Elashoff R, Shah SK, Ochs R, et al. Classification of parenchymal abnormality in scleroderma lung using a novel approach to denoise images collected via a multicenter study. Acad Radiol 2008;15:1004–16. [11] Yabuuchi H, Matsuo Y, Tsukamoto H, Horiuchi T, Sunami S, Kamitani T, et al. Evaluation of the extent of ground-glass opacityon high-resolution ct in patients with interstitial pneumonia associated with systemic sclerosis: comparison between quantitative and qualitative analysis. Clin Radiol 2014;69:758–64. [12] Kim HG, Tashkin DP, Clements PJ, Li G, Brown MS, Elashoff R, et al. A computeraided diagnosis system for quantitative scoring of extent of lung fibrosis in scleroderma patients. Clin Exp Rheumatol 2010;28(5 Suppl 62):S26–35. [13] Ariani A, Carotti M, Gutierrez M, Bichisecchi E, Grassi W, Giuseppetti GM, et al. Utility of an open-source DICOM viewer software (OsiriX) to assess pulmonary fibrosis in systemic sclerosis: preliminary results. Rheumatol Int 2014;34:511–6. [14] Salaffi F, Carotti M, Bosello S, Ciapetti A, Gutierrez M, Bichisecchi E, et al. Computer-aided quantification of interstitial lung disease from high resolution
[22] [23]
[24] [25] [26]
[27]
[28]
[29]
[30]
[31]
[32]
[33]
5
computed tomography images in systemic sclerosis: correlation with visual readerbased score and physiologic tests. Biomed Res Int 2015;2015:834262. Salaffi F, Carotti M, Di Donato E, Di Carlo M, Ceccarelli M, Giuseppetti G. Computer-Aided tomographic analysis of interstitial lung disease (ILD) in patients with systemic sclerosis (SSc). Correlation with pulmonary physiologic tests and patient-centred measures of perceived dyspnea and functional disability. PLoS One 2016;11:e0149240. Tashkin DP, Elashoff R, Clements PJ, Goldin J, Roth MD, Furst DE, et al. Cyclophosphamide versus placebo in scleroderma lung disease. N Engl J Med 2006;354:2655–66. Roth MD, Tseng CH, Clements PJ, Furst DE, Tashkin DP, Goldin JG, et al. Predicting treatment outcomes and responder subsets in scleroderma-related interstitial lung disease. Arthritis Rheum 2011;63:2797–808. Kim HJ, Tashkin DP, Gjertson DW, Brown MS, Kleerup E, Chong S, et al. Transitions to different patterns of interstitial lung disease in scleroderma with and without treatment. Ann Rheum Dis 2016;75:1367–71. van den Hoogen F, Khanna D, Fransen J, Johnson SR, Baron M, Tyndall A, et al. classification criteria for systemic sclerosis: an American college of Rheumatology/ European League against Rheumatism collaborative initiative. Arthritis Rheum 2013;65:2737–47. 2013. LeRoy EC, Black C, Fleischmajer R, Jablonska S, Krieg T, Medsger TA, et al. Scleroderma (systemic sclerosis): classification, subsets and pathogenesis. J Rheumatol 1988;15:202–5. Shin KE, Chung MJ, Jung MP, Choe BK, Lee KS. Quantitative computed tomographic indexes in diffuse interstitial lung disease: correlation with physiologic tests and computed tomography visual scores. J Comput Assist Tomogr 2011;35:266–71. Husted JA, Cook RJ, Farewell VT, Gladmann DD. Methods for assessing responsiveness: a critical review and recommendations. J Clin Epidemiol 2000;53:459–68. Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol 2008;61:102–9. Efron B, Gong G. A leisurely look at the bootstrap, the jackknife, and cross-validation. Am Statistician 1983;37:36–48. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983;148:839–43. Goldin JG, Kim GHJ, Tseng CH, Volkmann E, Furst D, Clements P, et al. Longitudinal changes in quantitative interstitial lung disease on CT after immunosuppression in the scleroderma lung study II. Annals ATS 2018. Kim GHJ, Tashkin DP, Lo P, Brown M, Volkmann E, Gjertson DW, et al. Using transitional changes on HRCT to monitor the impact of cyclophosphamide or mycophenolate on systemic sclerosis-related interstitial lung disease. Arthritis Rheumatol 2019. Aug 20. Bocchino M, Bruzzese D, D'Alto M, Argiento P, Borgia A, Capaccio A, et al. Performance of a new quantitative computed tomography index for interstitial lung disease assessment in systemic sclerosis. Sci Rep 2019;9(1):9468. Jul 1. Adler S, Huscher D, Siegert E, Allanore Y, Czirják L, DelGaldo F, et al. Systemic sclerosis associated interstitial lung disease – individualized immunosuppressive therapy and course of lung function: results of the Eustar group. Arthritis Res Ther 2018;20(1):17. 30. Khanna D, Brown KK, Clements PJ, Elashoff R, Furst DE, Goldin G, et al. Systemic sclerosis-associated interstitial lung disease-proposed recommendations for future randomized clinical trials. Clin Exp Rheumatol 2010;28(2 Suppl 58):S55–62. Goldin J, Elashoff R, Kim HJ, Yan X, Lynch D, Strollo D, et al. Treatment of scleroderma-interstitial lung disease with cyclophosphamide is associated with less progressive fibrosis on serial thoracic high-resolution CT scan than placebo. Chest 2009;136:1333–40. de Vet HC, Terluin B, Knol D, Roorda LD, Mokkink LB, Ostelo RW, et al. Three ways to quantify uncertainty in individually applied “minimally important change” (MIC) values. J Clin Epidemiol 2010;63:37–45. Salaffi F, Carotti M, Di Carlo M, Tardella M, Giovagnoni A. High-resolution computed tomography of the lung in patients with rheumatoid arthritis: Prevalence of interstitial lung disease involvement and determinants of abnormalities. Medicine (Baltimore) 2019;98(38):e17088. https://doi.org/10.1097/MD. 0000000000017088. Vol:No(e17088).