Longitudinal study of diffusion tensor imaging properties of affected cortical spinal tracts in acute and chronic hemorrhagic stroke

Longitudinal study of diffusion tensor imaging properties of affected cortical spinal tracts in acute and chronic hemorrhagic stroke

Journal of Clinical Neuroscience xxx (2014) xxx–xxx Contents lists available at ScienceDirect Journal of Clinical Neuroscience journal homepage: www...

878KB Sizes 0 Downloads 3 Views

Journal of Clinical Neuroscience xxx (2014) xxx–xxx

Contents lists available at ScienceDirect

Journal of Clinical Neuroscience journal homepage: www.elsevier.com/locate/jocn

Clinical Study

Longitudinal study of diffusion tensor imaging properties of affected cortical spinal tracts in acute and chronic hemorrhagic stroke Chicheng Ma a, Aijun Liu a, Zhenzuo Li a, Xueying Zhou b, Shengnian Zhou b,⇑ a b

The Fourth People’s Hospital of Jinan City, Jinan, Shandong, China Shangdong University of Traditional Chinese Medicine, Jinan 250031, Shandong, China

a r t i c l e

i n f o

Article history: Received 29 January 2013 Accepted 5 November 2013 Available online xxxx Keywords: Basal ganglia hemorrhage Cortical spinal tract Diffusion tensor imaging Fractional anisotropy Motor function score

a b s t r a c t This study investigated the clinical value of diffusion tensor imaging (DTI) in predicting the motor outcome in patients with basal ganglia hemorrhage. This prospective study included 23 patients assessed with DTI to measure the fractional anisotropy (FA) value in affected cortical spinal tract (CST) at three time points: day 0, day 30 and day 90 after onset. The motor function score (MFS) was applied to evaluate motor function and patients were divided into good and poor outcome groups according to the MFS on day 90. The mean FA value on day 0 was significantly lower in the poor outcome group than in the good outcome group (p < 0.01). FA value gradually decreased in the poor outcome group until day 90 after onset, while it continuously increased in the good outcome group. The MFS obtained at day 90 after onset was significantly correlated with the initial FA value in the affected cerebral peduncle (r = 0.926, p < 0.01). Receiver operating characteristic curve analysis showed that the FA value on day 0 could predict motor function outcome with a sensitivity of 88.89% and specificity of 92.86% at the initial FA value of 0.45. The FA value of affected CST in acute cerebral hemorrhage may valuably predict the motor function outcome and its dynamic change may represent the Wallerian degeneration in motor tracts after hemorrhagic stroke. Ó 2014 Elsevier Ltd. All rights reserved.

1. Introduction Stroke is one of the most common causes of death and disability in developed and developing countries [1]. Approximately 13% of strokes are caused by hypertensive intracerebral hemorrhage (ICH). Patients with ICH are sometimes treated surgically in order to save their lives, and surgery can reduce mortality rates to some extent [2]. Recently, it has been reported that the long-term disability rate is associated with impairment of the cortical spinal tract (CST) in patients with basal ganglia hemorrhage [3]. Furthermore, neuropathologic studies have shown that neuronal degeneration of the CST can occur as early as several hours after cerebrovascular disease [4,5]. Thus, an early assessment of this degeneration may be of value in predicting the long-term motor function outcome. Neuroimaging assessment of the CST with methods such as CT scan or conventional MRI has evolved in recent years. However, it is difficult to identify neuronal degeneration in the central nervous system with these traditional methods [6–8]. Diffusion tensor imaging (DTI), a MRI based technique, has proved to be a valuable tool in detecting, visualizing and quantifying the integrity of the cerebral microstructure by measuring the diffusion of water in tis⇑ Corresponding author. Tel./fax: +86 531 8692 7544. E-mail address: [email protected] (S. Zhou).

sue [9]. Fractional anisotropy (FA) is a quantitative index to identify white matter lesions and evaluate the integrity of white matter tracts [10,11]. In addition, the FA value is reported to be very sensitive in detecting Wallerian degeneration [12]. Both FA values and the integrity of the CST in patients with ICH have also been reported to be associated with the motor function outcome in several studies [13,14]. However, these studies focused on the relationship between initial parameters and the final motor function outcome. To our knowledge, the dynamic changes of the FA of the CST following hemorrhagic stroke has not been well addressed [15]. We hypothesized that the dynamic change of the FA value in the CST might predict motor function outcome in patients with hemorrhagic stroke and represent the Wallerian degeneration of the CST following ICH. Therefore, in this prospective study, we sought to observe the dynamic changes of FA in CST and correlate this to the motor function outcome.

2. Material and methods 2.1. Patients This study included 25 patients with basal ganglia hemorrhagic stroke confirmed by CT scan in our department within 24 hours of

http://dx.doi.org/10.1016/j.jocn.2013.11.032 0967-5868/Ó 2014 Elsevier Ltd. All rights reserved.

Please cite this article in press as: Ma C et al. Longitudinal study of diffusion tensor imaging properties of affected cortical spinal tracts in acute and chronic hemorrhagic stroke. J Clin Neurosci (2014), http://dx.doi.org/10.1016/j.jocn.2013.11.032

2

C. Ma et al. / Journal of Clinical Neuroscience xxx (2014) xxx–xxx

onset between January 2010 and December 2010. Of these patients, two were excluded because of missing data. The study population consisted of 23 patients, comprising 15 men and eight women, aged 34–67 years (mean 54 ± standard deviation [SD] 9 years). DTI data were prospectively collected within 24 hours of onset. The initial head CT scan confirmed the hemorrhagic stroke within the basal ganglia. We excluded patients with previous stroke history and medical history of other brain diseases. All of our patients were treated with conservative medical therapies. All patients underwent our current imaging protocol that was approved by the local Ethics Committee after obtaining signed informed consent documents from the patient or responsible relatives. 2.2. DTI All enrolled patients underwent MRI at three time points: on day 0, day 30 and day 90 after the onset of ICH. MRI was performed on a 1.5 Tesla Signa Excite II scanner (GE Healthcare, Milwaukee, WI, USA). All patients underwent T1- and T2-weighted imaging using short inversion time inversion recovery sequences prior to DTI with the following parameters: inversion time 100 ms, matrix 512  384, field of view (FOV) 240 mm, 6.5 mm thickness, and 2 mm gap, as reported previously [14]. For DTI, we applied a single shot spin echo diffusion-weighted echo planar imaging sequence with diffusion gradients applied in six directions (echo time = 64.8 ms, repetition time = 8000 ms, matrix size = 128  128, FOV = 240 mm, slice thickness = 3.5 mm, b values of 0 and 1000 s/ mm2, no intersection gap). 2.3. Image analysis The Volume-One and dTV software (Tokyo University, Tokyo, Japan) were used to perform the imaging post-processing. A seed region of interest was drawn in the CST portion of the affected cerebral peduncle on a T1- or T2-weighted image (Fig. 1, 2). An examiner who was blinded to the patient’s data calculated the FA value with the dTV software. 2.4. Motor function assessments All patients underwent motor function score (MFS) examinations before each MRI at day 0, day 30 and day 90 after onset by an experienced neurosurgeon blinded to the DTI data. All patients underwent the standard physical and post-stroke occupational

A

therapy [14]. The MFS is calculated according to the National Institutes of Health Stroke Scale, which is frequently used to measure the motor function with a score range of 0–8 [16]. Higher scores reflect worse motor deficit [17].

2.5. Statistical analysis One-way analysis of variance and multivariate testing were employed to compare the FA values, MFS and other clinical characteristics between two groups using the Statistical Package for the Social Sciences version 20.0 software (SPSS, Chicago, IL, USA). To determine the relationship between motor function and the status of CST injury, we performed a Pearson analysis of MFS against the FA value of the affected CST. The receiver operating characteristic (ROC) curve was established to determine significant factors for predicting the outcome. A statistical threshold of p < 0.05 was used.

3. Results The clinical and imaging characteristics of the 23 enrolled patients are shown in Table 1. The good and poor outcome groups included 14 and nine patients, respectively. The dynamic changes of FA values are shown in Figure 3. The mean FA value gradually decreased until day 90 in the poor outcome group, while it greatly increased in the good outcome group. The mean FA value on day 90 was statistically different between the two groups (p < 0.01). The initial FA value was significantly lower (p < 0.01) in the poor outcome group (0.491 ± SD 0.040) than in the good outcome group (0.403 ± SD 0.037). Most patients with an initial FA value lower than 0.45 were included in the poor outcome group. In contrast, patients with an initial FA value over 0.45 were mostly included in the good outcome group (Fig. 4). The results of univariate and multivariate logistic analysis are shown in Table 2. There were no statistical differences in the distribution of patient age, sex, hematoma location, hematoma volume and patients with or without ventricular hemorrhage between the two groups (p > 0.05). The FA value on day 0 was the most independent factor for predicting the motor function outcome at day 90 after onset (p = 0.021), while the MFS on day 0 did not predict the motor function outcome (p = 0.101). The MFS obtained at day 90 after onset was significantly different between the two groups (p < 0.05), and it was negatively correlated with the initial FA value (r = 0.926, p < 0.01) (Fig. 5).

B

Fig. 1. (A) T2-weighted axial MRI of a patient with a left basal ganglia hemorrhage in the poor outcome group. (B) Regions of interest in the cerebral peduncle (circle) on the lesion side were drawn manually on the MRI.

Please cite this article in press as: Ma C et al. Longitudinal study of diffusion tensor imaging properties of affected cortical spinal tracts in acute and chronic hemorrhagic stroke. J Clin Neurosci (2014), http://dx.doi.org/10.1016/j.jocn.2013.11.032

3

C. Ma et al. / Journal of Clinical Neuroscience xxx (2014) xxx–xxx

A

B

Fig. 2. (A) Axial fluid attenuated inversion recovery MRI of a patient with a right basal ganglia hemorrhage in the good outcome group. (B) Regions of interest in the cerebral peduncle (circle) on the lesion side were drawn manually on the MRI.

Table 1 Clinical and imaging characteristics of basal ganglia hemorrhage patients Day 0 Patient 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

Sex

Age, years

M M F F M M M M M F F M M M M M F M M F F M F

40 62 57 51 59 56 63 34 45 64 47 60 67 54 38 49 64 64 45 61 56 55 58

Lesion volume, ml 14 15 9 12 7 13 17 20 13 21 22 13 9 12 8 18 22 16 23 11 14 16 13

Lesion side

Ventricular hemorrhage

L L R R L R L R R R L R R L L L R R L L L R L

+

+

+

FA 0.499 0.518 0.497 0.470 0.563 0.415 0.497 0.505 0.463 0.435 0.371 0.424 0.438 0.359 0.397 0.405 0.386 0.361 0.498 0.385 0.375 0.425 0.387

F = female, FA = fractional anisotropy, L = left, M = male, MFS = motor function score, R = right, + = present,

Day 30 MFS 4 3 2 3 2 4 3 2 5 6 6 5 5 8 7 5 6 7 5 8 6 6 6

FA 0.523 0.590 0.521 0.480 0.613 0.495 0.513 0.673 0.512 0.418 0.359 0.399 0.424 0.346 0.382 0.385 0.370 0.352 0.412 0.375 0.368 0.401 0.372

Day 90 MFS 1 0 2 3 0 2 3 0 3 7 8 8 7 8 6 5 6 8 4 7 8 6 8

FA 0.690 0.731 0.679 0.520 0.656 0.623 0.712 0.724 0.627 0.367 0.360 0.400 0.394 0.326 0.377 0.367 0.355 0.341 0.405 0.368 0.358 0.422 0.361

MFS

Outcome

0 0 0 1 0 2 1 0 2 5 6 6 6 8 6 5 7 7 6 8 7 7 8

Good Good Good Good Good Good Good Good Good Poor Poor Poor Poor Poor Poor Poor Poor Poor Poor Poor Poor Poor Poor

= absent.

On one hand, according to the ROC curve, the FA value on day 0 predicted good recovery. The area under the curve was 0.929, with a sensitivity of 88.89% and specificity of 92.86% at an initial FA value of 0.45 (Fig. 6a). On the other hand, the MFS on day 0 also predicted good recovery according to the ROC curve. The area under the curve was 0.984, with a sensitivity of 100% and specificity of 71.43% at an MFS of 3.21 on day 0 (Fig. 6b).

4. Discussion

Fig. 3. Dynamic changes in mean fractional anisotropy (FA) values in patients with hemorrhagic basal ganglia stroke. The mean FA value greatly increased in the good outcome group but decreased gradually in the poor outcome group.

We performed a prospective study to evaluate the dynamic changes of FA value in affected CST following basal ganglia hemorrhagic stroke and found that the initial FA value of the CST on the pathological side was statistically correlated with the motor function outcome at day 90 after onset. Moreover, patients with an initial FA value over 0.45 could be expected to obtain a good motor function outcome after ICH with conservative medical treatment.

Please cite this article in press as: Ma C et al. Longitudinal study of diffusion tensor imaging properties of affected cortical spinal tracts in acute and chronic hemorrhagic stroke. J Clin Neurosci (2014), http://dx.doi.org/10.1016/j.jocn.2013.11.032

4

C. Ma et al. / Journal of Clinical Neuroscience xxx (2014) xxx–xxx

Fig. 5. Relationship between the motor function score on day 90 and fractional anisotropy value on day 0 after onset. FA = fractional anisotropy, MFS = motor function score.

Fig. 4. Mean fractional anisotropy (FA) on day 0 was significantly lower in the poor outcome group than in the good outcome group. Most patients with an initial FA value lower than 0.45 were included in the poor outcome group. In contrast, most patients with an initial FA value over 0.45 were included in the good outcome group.

Our quantitative analysis of 23 patients revealed a highly significant relationship between DTI-derived parameters and motor recovery. These results were consistent with previous studies, which correlated the damage pattern of motor fiber tracts with motor impairment using quantitative DTI parameters [18–20]. In another study, Yoshioka et al. reported that motor functional outcome in patients with ICH could be predicted by measuring FA values using DTI [11]. They considered that the decrease in FA values in patients with poor motor functional outcome was likely due to the damage from hematoma to neural fibers in the acute phase. However there is no direct pathological evidence of damage to the neural fibers. In addition, motor neurons will be irreversibly damaged by 90 days post hemorrhagic stroke even if they are not directly destroyed by hematoma in the acute stage [21,22]. Decreased FA values in the remote CST after stroke has been reported to be associated with Wallerian degeneration [15]. In our study, we illustrated the dynamic changes of FA value in the affected CST in patients with hemorrhagic stroke, and added the direct evidence of a progressive reduction of FA value, which potentially represent the Wallerian degeneration of affected motor fibers. However, the FA value in the good outcome group increased. The mechanism of this diametrical change of FA remains unclear. It appears that Wallerian degeneration is induced by the direct damage of the hematoma and following cytotoxic and vasogenic edema of the motor fibers. Cytotoxic edema along the tracts is not visible via traditional radiological images, however, it might be detected on DTI. Nevertheless, whether this change is related to enlargement of the hematoma or progressive edema requires further study.

Fig. 6. (a) Receiver operating characteristic (ROC) curve predicting good recovery based on the fractional anisotropy (FA) value on day 0 after onset. The area under the curve was 0.929, with a sensitivity of 88.89% and specificity of 92.86% at an FA value of 0.45 on day 0. (b) ROC curve predicting good recovery based on the motor function score (MFS) on day 0 after onset. The area under the curve was 0.984, with a sensitivity of 100% and specificity of 71.43% at an MFS of 3.21 on day 0.

Table 2 Univariate and multivariate logistic analysis in basal ganglia hemorrhage patients Characteristics Good outcome group Poor outcome group Univariate p value Multivariate p value

Age, years

Sex (M/F)

Location (L/R)

Lesion volume, ml

Ventricular hemorrhage (with/without)

52 ± 10a 56 ± 8a 0.323 0.763

7/2 8/6 0.333 0.458

5/4 7/7 0.806 0.999

13.33 ± 1.30a 15.57 ± 1.33a 0.27 0.450

1/8 2/12 0.83 0.27

FA on day 0 0.491 ± 0.040a 0.403 ± 0.037a <0.01 0.021

MFS on day 0 3.11 ± 1.05a 6.14 ± 1.03a <0.01 0.101

F = female, FA = fractional anisotropy, L = left, M = male, MFS = motor function score, R = right. a Data presented as mean ± standard deviation.

Please cite this article in press as: Ma C et al. Longitudinal study of diffusion tensor imaging properties of affected cortical spinal tracts in acute and chronic hemorrhagic stroke. J Clin Neurosci (2014), http://dx.doi.org/10.1016/j.jocn.2013.11.032

C. Ma et al. / Journal of Clinical Neuroscience xxx (2014) xxx–xxx

Conventional MRI can detect Wallerian degeneration a few months after stroke onset [7,23,24]. In contrast, DTI can detect Wallerian degeneration as a decreased FA value in patients with ischemic stroke in the acute stage [25–27]. The reduction of FA mirrors the disintegration of neuronal fibers which occurs in the acute stage of Wallerian degeneration, and DTI can identify subtle changes in water diffusion within the first several days after onset [25]. Traditional MRI fails to detect signal changes until 4 weeks after stroke, whereas DTI reveals these changes after only several days [26]. The scoring tool of MFS was chosen to focus on the clinical perspective of motor function evaluation. Those patients with higher initial MFS developed worse motor function outcome, and vice versa. We also found the motor recovery at day 90 after onset was negatively correlated with the initial FA value (Fig. 5). However, when it came to the logistic analysis, only the FA value on day 0 was an independent factor for predicting the motor function outcome at day 90 after onset (p = 0.021); whilst the MFS on day 0 did not predict the motor function outcome (p = 0.101). Some patients with a poor motor function in the acute phase would be expected to improve in the chronic phase; additionally, patients with a motor function score of 5 or higher may be likely to require surgical therapy for recovery of motor function [28]. Measurement of FA may help to select eligible patients for surgical hematoma evacuation. Therefore, the clinical application of DTI may empower physicians with an earlier prediction for motor recovery and choice of treatment strategy. Inevitably, this study has some limitations. First, patients undergoing surgical intervention were not enrolled in the current study. Therefore, the effect of hematoma evacuation on the change of FA values remains unclear and needs to be clarified in future studies. Also, the estimation of FA in affected CST using our region of interest method in the cerebral peduncle may be not very accurate, because the CST may be displaced by hematoma and/or vasogenic edema [14]. This technique examines limited parts of the white matter in the brain, and may underestimate the full extent of the injury. Last, the surgical indications for patients with ICH is limited to hematoma size and conscious state. Whether surgical treatment is able to affect long-term motor function outcome in ICH patient needs to be explored in future studies. We suggest that further studies include patients with surgery after hemorrhagic stroke, but so far there are no randomized trials focusing on the efficacy of surgical intervention in patients with ICH. 5. Conclusions Our study demonstrates that FA values calculated from DTI can reliably indicate impairment of the CST and effectively predict the motor function outcome in patients with hemorrhagic stroke. The dynamic changes of FA in the affected CST following acute cerebral hemorrhage may represent Wallerian degeneration of white matter tracts after stroke. Conflicts of Interest/Disclosures The authors declare that they have no financial or other conflicts of interest in relation to this research and its publication. References

5

[2] Hosomi N, Naya T, Ohkita H, et al. Predictors of intracerebral hemorrhage severity and its outcome in Japanese stroke patients. Cerebrovasc Dis 2009;27:67–74. [3] Stinear CM, Barber PA, Smale PR, et al. Functional potential in chronic stroke patients depends on corticospinal tract integrity. Brain 2007;130: 170–80. [4] Vargas ME, Barres BA. Why is Wallerian degeneration in the CNS so slow? Annu Rev Neurosci 2007;30:153–79. [5] Concha L, Gross DW, Wheatley BM, et al. Diffusion tensor imaging of timedependent axonal and myelin degradation after corpus callosotomy in epilepsy patients. Neuroimage 2006;32:1090–9. [6] Fumeya H, Hideshima H. Early MR abnormality indicating functional recovery from spontaneous intracerebral hemorrhage. Neurol Med Chir (Tokyo) 1991;31:650–65. [7] Pujol J, Martí-Vilalta JL, Junqué C, et al. Wallerian degeneration of the pyramidal tract in capsular infarction studied by magnetic resonance imaging. Stroke 1990;21:404–9. [8] Sawlani V, Gupta RK, Singh MK, et al. MRI demonstration of Wallerian degeneration in various intracranial lesions and its clinical implications. J Neurol Sci 1997;146:103–8. [9] Pierpaoli C, Jezzard P, Basser PJ, et al. Diffusion tensor MR imaging of the human brain. Radiology 1996;201:637–48. [10] Koyama T, Tsuji M, Nishimura H, et al. Diffusion tensor imaging for intracerebral hemorrhage outcome prediction: comparison using data from the corona radiata/internal capsule and the cerebral peduncle. J Stroke Cerebrovasc Dis 2013;22:72–9. [11] Yoshioka H, Horikoshi T, Aoki S, et al. Diffusion tensor tractography predicts motor functional outcome in patients with spontaneous intracerebral hemorrhage. Neurosurgery 2008;62:97–103. [12] Xie R, Fang M, Zhou L, et al. Diffusion tensor imaging detects Wallerian degeneration of the corticospinal tract early after cerebral infarction. Neural Regen Res 2012;7:900–5. [13] Kusano Y, Seguchi T, Horiuchi T, et al. Prediction of functional outcome in acute cerebral hemorrhage using diffusion tensor imaging at 3T: a prospective study. AJNR Am J Neuroradiol 2009;30:1561–5. [14] Zeng J, Zheng P, Xu J, et al. Prediction of motor function by diffusion tensor tractography in patients with basal ganglion haemorrhage. Arch Med Sci 2011;7:310–4. [15] Kuzu Y, Inoue T, Kanbara Y, et al. Prediction of motor function outcome after intracerebral hemorrhage using fractional anisotropy calculated from diffusion tensor imaging. Cerebrovasc Dis 2012;33:566–73. [16] Goldstein LB, Bertels C, Davis JN. Interrater reliability of the NIH stroke scale. Arch Neurol 1989;46:660–2. [17] Kelly-Hayes M, Robertson JT, Broderick JP, et al. The American heart association stroke outcome classification. Stroke 1998;29:1274–80. [18] Nelles M, Gieseke J, Flacke S, et al. Diffusion tensor pyramidal tractography in patients with anterior choroidal artery infarcts. AJNR Am J Neuroradiol 2008;29:488–93. [19] Konishi J, Yamada K, Kizu O, et al. MR tractography for the evaluation of functional recovery from lenticulostriate infarcts. Neurology 2005;64:108–13. [20] Kwak SY, Yeo SS, Choi BY, et al. Corticospinal tract change in the unaffected hemisphere at the early stage of intracerebral hemorrhage: a diffusion tensor tractography study. Eur Neurol 2010;63:149–53. [21] Yamada K, Kizu O, Ito H, et al. Wallerian degeneration of the inferior cerebellar peduncle depicted by diffusion weighted imaging. J Neurol Neurosurg Psychiatry 2003;74:977–8. [22] Wieshmann UC, Clark CA, Symms MR, et al. Reduced anisotropy of water diffusion in structural cerebral abnormalities demonstrated with diffusion tensor imaging. Magn Reson Imaging 1999;17:1269–74. [23] Inoue Y, Matsumura Y, Fukuda T, et al. MR imaging of Wallerian degeneration in the brainstem: temporal relationships. AJNR Am J Neuroradiol 1990;11:897–902. [24] Kuhn MJ, Mikulis DJ, Ayoub DM, et al. Wallerian degeneration after cerebral infarction: evaluation with sequential MR imaging. Radiology 1989;172:179–82. [25] Thomalla G, Glauche V, Koch MA, et al. Diffusion tensor imaging detects early Wallerian degeneration of the pyramidal tract after ischemic stroke. Neuroimage 2004;22:1767–74. [26] Thomalla G, Glauche V, Weiller C, et al. Time course of Wallerian degeneration after ischaemic stroke revealed by diffusion tensor imaging. J Neurol Neurosurg Psychiatry 2005;76:266–8. [27] Yu C, Zhu C, Zhang Y, et al. A longitudinal diffusion tensor imaging study on Wallerian degeneration of corticospinal tract after motor pathway stroke. Neuroimage 2009;47:451–8. [28] Kuzu Y, Inoue T, Kanbara Y, et al. Prediction of motor function outcome after intracerebral hemorrhage using fractional anisotropy calculated from diffusion tensor imaging. Cerebrovasc Dis 2012;33:566–73.

[1] Pendlebury ST. Worldwide under-funding of stroke research. Int J Stroke 2007;2:80–4.

Please cite this article in press as: Ma C et al. Longitudinal study of diffusion tensor imaging properties of affected cortical spinal tracts in acute and chronic hemorrhagic stroke. J Clin Neurosci (2014), http://dx.doi.org/10.1016/j.jocn.2013.11.032