- Email: [email protected]
MR image super-resolution via manifold regularized sparse learning
Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
Contents lists available at ScienceDirect
Neurocomputing journal homepage: www.elsevier.com/locate/neucom
MR image super-resolution via manifold regularized sparse learning Xiaoqiang Lu n, Zihan Huang, Yuan Yuan Center for OPTical IMagery Analysis and Learning (OPTIMAL), State Key Laboratory of Transient Optics and Photonics, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an 710119, Shaanxi, PR China
art ic l e i nf o
a b s t r a c t
Article history: Received 3 February 2015 Received in revised form 21 March 2015 Accepted 31 March 2015 Communicated by Dacheng Tao
Single image super-resolution (SR) has been shown useful in Magnetic Resonance (MR) image based diagnosis, where the image resolution is still limited. The basic goal of single image SR is to produce a high-resolution (HR) image from corresponding low-resolution (LR) image. However, most existing SR algorithms often fail to: (1) reflect the intrinsic structure between MR images and (2) exploit the intrapatient information of MR images. In fact, MR images are more likely to vary along a low dimensional submanifold, which can be embedded in the high dimensional space. It has also been shown that the structure information of MR images and the priors of the MR images of different modality are important for improving the image resolution. To take full advantage of manifold structure information and intrapatient prior of MR images, a novel single image super-resolution algorithm for MR images is proposed in this paper. Compared with the existing works, the proposed algorithm has the following merits: (1) the proposed sparse coding based algorithm integrates manifold constraints to handle the inverse problem in MR image SR; (2) the manifold structure of the intra-patient MR image is considered for image SR; and (3) the topological structure of the intra-patient MR image can be preserved to improve the reconstructed result. Experiments show that the proposed algorithm is more effective than the stateof-the-art algorithms. & 2015 Elsevier B.V. All rights reserved.
Keywords: Sparse learning Manifold regularization Super-resolution Magnetic Resonance Imaging (MRI)
1. Introduction Recently, image enhancement has received increasing attention in the area of Magnetic Resonance Imaging (MRI) [1–4]. In practical application, high quality MR image is useful for doctor to make diagnosis. For example, when diagnosing neurodegenerative diseases, it is important to observe the anatomical structures in finer details, such as some deep brain structures like hippocampus and caudate nuclei. Moreover, for many other image processing algorithm such as registration and segmentation, a high-resolution MR image is also helpful. Nowadays, 3T MRI has been more often seen in clinical use and even 7T MRI is started to be equipped in labs. The increase of magnetic field strength helps to increase signal-tonoise ratio (SNR), but the image resolution is still limited. A few factors would affect the quality of MRI, including noise, tissue contrast, resolution, and artifacts. Among them, spatial resolution is one of the main factors. Therefore, it is necessary to improve the spatial resolution of MR image. One traditional kind of method to achieve higher spatial resolution is to adjust acquisition parameters, such as decreasing the field of view, or increasing gradient
n
Corresponding author. E-mail address: [email protected] (X. Lu).
intensity. However, these methods will either decrease SNR or appreciably increase the total scan time. The other kind of method is post-processing algorithm, namely image super-resolution (SR) technique. Image SR technique aims to reconstruct one highresolution (HR) image from one or more low-resolution (LR) input images. In general, image SR technique can be categorized into three kinds: interpolation based methods, reconstruction based methods and learning based methods. Previously, much attention has been paid on interpolation based methods, such as nearest-neighbor, and linear interpolation [5–7]. These methods are fast and easy to understand and manipulate, but they are not good at handling edges and complex textures. Blurred edges and textures can often be found on interpolation based methods. The second kind of image SR technique is reconstruction based methods, which have also been proposed to deal with multiple LR images [8–10]. However, there is one prerequisite for this type of methods: sufficient LR images with sub-pixel displacements are needed, which is not always the case. The limitations of the above methods have spurred the emergence of learning based SR methods. Learning based methods use training database to help restore the high frequency details that lost in LR image. Recently sparse learning has arisen great interest in a variety of fields [11–13], and sparse coding based methods are
http://dx.doi.org/10.1016/j.neucom.2015.03.065 0925-2312/& 2015 Elsevier B.V. All rights reserved.
Please cite this article as: X. Lu, et al., MR image super-resolution via manifold regularized sparse learning, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.065i
2
X. Lu et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
Fig. 1. Sketch of the proposed algorithm. LR T2-weighted image is reconstructed using only one HR T1-weighted image as the reference image. In the algorithm, first the intra-patient information is obtained from reference image, then this information is used to guide the SR process.
Fig. 2. T1-weighted image and T2-weighted image of the same patient. left: T1weighted image; right: T2-weighted image.
the main stream of learning based image SR, which show the state-of-the-art performance. For instance, Yang et al. [14] assume that an over-complete dictionary can be learned to represent image patches sparsely, and then an ℓ1-norm sparse regularizer is introduced to ensure that the reconstruction coefficient matrix of the LR image is sparse. Dong et al. [15] build a series of dictionary to enhance the resolution of an image, in which one best dictionary is selected for each image patch to be reconstructed. In their method, the representation of image patches over the selected dictionary is also required to be sparse. However, all of the aforementioned studies have not considered the information within intra-patient MR images, which has recently been proven to be useful for enhancing the resolution of MR image [16]. Generally MR image examination is made up of a series of sequences, and each of them can provide its unique information about the subject tissues. Among them, T1-weighted MRI and T2-weighted MRI are two basic sequences. It is a common occurrence that an HR T1-weighted reference image and a LR T2weighted image of the same patient are available. This often happens when the patient is asked to regularly take MRI scan, or take a series of scans under different acquisition parameters, such as the subsequent examination of brain tumor evolution, neurodegenerative diseases and diffusion tensor imaging. For the convenience of diagnosis, it is necessary to enlarge the LR image so that it achieves the same size as HR image. In other words, it is
necessary to exploit the information of one HR T1-weighted image to improve the resolution of the other modality LR image (T2weighted image). To be more specific, a T1-weighted image can be used as a reference image to help reconstruct one corresponding LR T2-weighted image. The sketch of the proposed algorithm is shown in Fig. 1. To employ the intra-patient information of MR image, Rousseau et al. [16] use the non-local self-similarity of reference image as a prior and solve the SR problem under a patch-based framework. The proposed algorithm has yielded satisfactory achievement. However, it only considered image structure in the Euclidean space. Recent study [17,18] shows that the intrinsic structure of the data may not be accurately reflected by the Euclidean distance in the high dimensional space. Recently, research has shown that high dimensional Euclidean space cannot be uniformly “filled up” by the image data. These image data can be regarded as sampled data from or near a submanifold of an ambient space [19–21]. Therefore, it is necessary to consider the intrinsic structure when reconstructing the HR image. Inspired by recent advances in manifold learning and sparse coding, a new learning based single image SR algorithm is proposed in this paper. The proposed algorithm tries to solve the inverse problem of reconstructing an HR image from the LR T2-weighted image by exploiting an HR T1-weighted reference image of different modality, which has very obvious practical application in medical image diagnosis. T2-weighted image and T1-weighted image of the same patient are shown in Fig. 2. In most case, it is possible to exploit the prior information of HR T1-weighted reference image to enhance the resolution of the LR T2-weighted image. The new learning based algorithm for single MR image SR can be divided into three steps. Firstly, a new sparse coding algorithm is employed as the framework of the SR. Secondly, in order to explicitly utilize the structure information of the reference image, the reference is fused with the estimated HR image, and then a weight estimator is learned from the fused image, which can help reduce the uncertainty of the reconstruction. Finally, by incorporating the manifold learning technique into sparse coding, the proposed algorithm extracts the intrinsic geometric structure from the fused image. The rest of the paper is organized as follows: Section 2 gives a brief description of the problem. Section 3 details the proposed algorithm which enhances the resolution of MRI image using one reference image. Experiment results are shown in Section 4. Finally, conclusion is given in Section 5.
Please cite this article as: X. Lu, et al., MR image super-resolution via manifold regularized sparse learning, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.065i
X. Lu et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
2. Problem description This section presents a generic observation model for image SR. Generally, super-resolution problems can be modeled as y ¼ DHx þ ν;
ð1Þ
where y denotes the LR input image, x denotes the HR image to be estimated, D represents down sampling operator, H represents blurring operator, and ν denotes additive noise. Based on this model, the reconstructed image can be estimated by minimizing the least-square cost function 2 ð2Þ X^ ¼ arg miny DHx2 : x
Eq. (2) presents an ill-posed problem and there is no unique answer to it. Therefore, some prior should be added to regularize the above equation. The regularized model can be referred to as 2 ð3Þ X^ ¼ arg miny DHx2 þ λRðxÞ: x
RðxÞ is a regularization term, which represents the priors of the HR image. λ is a regularization parameter. An appropriate prior is of prime importance to enhance the performance of SR reconstruction. Different methods introduce different image priors into SR, such as edge smoothness [22,23], gradient profile priors [24] and nonlocal similar priors [16]. Recently, it has become a hot research topic for compressed sensing to obtain sparse solutions of under-determined inverse problem [25,26]. Motivated by recent progress in compressed sensing, sparse coding based algorithms have been applied to image SR and achieved considerable SR results [27,28,14]. Sparse coding seeks to develop a simple and efficient way to achieve sparse coding over a proper dictionary. That is, the product of sparse representations a A Rpn and a given dictionary φ can best approximate the original data matrix x A Rmn [29]. In single image SR, therefore, the objective function of sparse coding can be represented as follows: 2 minx DHφa2 þ λkak1 ; ð4Þ φ;α
where λ 4 0 is a regularization parameter. In sparse coding, all the data points are represented by only a few of basis vector, and the representation is often over complete, so a sparse dictionary can represent a wide variety of elements. What is more, study shows that sparse representation accords with biological vision in primary visual cortex (V1) [30]. Recent studies have shown that human generated image data vary smoothly on the geodesics of the manifold [31–33]. Therefore, if the intrinsic manifold structure of both the T2-weighted image and the T1-weighted image can be taken into consideration, the reconstruction result will be improved. In this case, it is assumed that both T1 and T2 images have intrinsically low-dimensional representation, which lie on a very high dimensional space.
3. Proposed algorithm To improve the performance of sparse coding based algorithm, the intra-patient information of MR image that considers the local manifold structure has been incorporated into the sparse coding process in this paper. 3.1. Manifold regularized sparse algorithm for SR In the field of natural image SR, sparse coding based algorithms are very popular. By introducing the concept of sparse coding, each data point can be interpreted more reasonably, which will result in a high performance of SR. However, in the field of MR image, sparse coding may not be an ideal algorithm. Due to the lack of
3
intra-patient information in MR images, the performance of the sparse coding based algorithm is reduced on MR image. To improve the quality of the reconstructed HR MR image, more prior information should be considered. Motivated by recent advances in sparse coding and manifold learning, a novel sparse coding based algorithm is proposed in this paper, which incorporates intra-patient information of MR image that consider the local manifold structure. In this paper, the main work is to reconstruct an HR image from one LR image and one corresponding HR reference image of a different modality. The proposed algorithm can be divided into three steps. In the first step, the principal component analysis (PCA) dictionary based sparse coding algorithm is used for the MR image reconstruction. Let xi ¼ Ri x; i ¼ 1; 2; …; N, denote the ith patch vector of x. Inspired by [15], K orthonormal PCA dictionaries fΦk g; k ¼ 1; 2; …; K, are introduced into sparse coding. For each image patch x^ i , the most suitable sub-dictionary Φki is then chosen, which can best represented the image patch both linearly and sparsely. The concrete step of dictionary learning and sub-dictionary selection can be found in [15]. Then x^ i can be represented as x^i ¼ Φki αi
s:t: kαi k1 r c
ð5Þ
#where αi is the representation of the image patch x^ i . Then by averaging all the image patches, complete image can be reconstructed as !1 N N X X RTi Ri RTi Φki αi x^ ¼ Φ○α ¼ ð6Þ i¼1
i¼1
#where Φ is the concatenation of Φk, α is the concatenation of αi . Combining Eq. (5) with Eq. (6), the reconstruction problem can be described as α^ ¼ arg miny DHΦ○α22 þ λkαk1 ; ð7Þ α
in which λ controls the balance between the approximation of y and the sparsity of solution. In the second step, the weight estimator is introduced based on manifold learning by assuming that LR T2-weighted image and HR T1-weighted image lie on two distinct manifolds with similar local geometry. It can be shown from Fig. 2 that T1-weighted image and T2-weighted image looks very similar. Thus, we believe that they are very likely to have similar local geometrical structure, and the structure information can be used to reduce the uncertainty of the HR T2-weighted image. Moreover, exploiting data in low dimensional space instead of their high dimensional representation can improve the learning performance and the reconstruction accuracy [34]. Given one HR T1-weighted reference image xr, we expect to obtain the weight estimator, which depends on the manifold of each data point and its neighbors. Accordingly, the second step can be divided into three phases: firstly, the fused image xf can be obtained by fusing the HR T1 reference image with the estimated HR image using a wavelet based method [35]. Consequently, the fused image contains the priors of both original image and the reference image. The fused image is shown in Fig. 3. Secondly, according to manifold learning technique, i.e., locally linear embedding (LLE) criterion, the reconstruction weight of the neighbors in xi is computed by minimizing the local reconstruction error: X f f X ω^ fij ¼ arg min J xfi ωij xj J 22 ; ð8Þ ωfij
i
xfj A Ni
where Ni denotes the K nearest neighbor of xfi (here Euclidean distance is used to define neighborhood); ωfij isP the reconstruction ωf ¼ 1, and weight, which is subject to the constraints xf A N i ij j
Please cite this article as: X. Lu, et al., MR image super-resolution via manifold regularized sparse learning, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.065i
X. Lu et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
4
Fig. 3. Example of MRI image fusion. From left to right: T1-weighted image, T2-weighted image, the fused image.
ωfij ¼ 0 for all xfj 2= N i . As is described in [34], the optimization of Eq. (8) can be done with the help of a Gram matrix Gi ¼ ðxfi 1T XÞT ðxfi 1T XÞ;
ð9Þ
where each column of X is one neighbor of xfi, and 1 is a column vector whose elements are all ones. Let ωfi be the concatenation of ωfij. The weight estimator ωfi is then obtained as follows: Gi ωfi ¼ 1:
Table 1 Comparison of algorithms with and without manifold regularizer. Image
No manifold regularizer
Proposed algorithm
Normal brain transversal
26.67 0.8689
26.84 0.8737
Normal brain sagittal
26.21 0.8952
26.50 0.9027
Normal brain coronal
25.84 0.8566
26.03 0.8646
MS brain transversal
27.22 0.8690
27.47 0.8769
MS brain sagittal
27.42 0.8906
27.57 0.8972
MS brain coronal
26.98 0.8625
27.11 0.8709
ð10Þ
P ωf ¼ 1. Thirdly, the reconThen ωfi is normalized so that xfj A N i ij f ^ ij is assigned to the reconstructed image struction weight ω patches x^i , thus the reconstruction error of the reconstructed image patch and its neighbor can be obtained as X X f ε ¼ ‖x^ i ω^ ij x^ j k22 : ð11Þ x^ j A Ni
i
The topology structure of the HR T1-weighted reference image can be preserved in the reconstructed T2-weighted image by the weight estimator. In all, the Pseudo-code can be written as Algorithm 1. Algorithm 1. 1: Fuse x^ with xr , generate xf. ^ fij by minimizing the local 2: Compute reconstruction weight ω reconstruction error:#X X ω^ fij ¼ arg min ‖xfi ωfij xfj k22 ; ω
f ij
Compute
i
xfj A Ni
It is worth noting that the proposed algorithm does not utilize any specific prior knowledge of certain kind of tissue. Thus, it is a general algorithm, and can be applied to solve SR problems other than MR imaging.
ωfi so that Gi ωfi ¼ 1, where
Gi ¼ ðxfi 1T
XÞT ðxfi 1T XÞ. P Normalize ωfi so that xf A N ωfij ¼ 1. j
3.2. Optimization Eq. (11) can be rewritten as follows: ε ¼ I Ω Φ○α22 ;
i
^ fij to calculate the reconstruct error of xi 3: Use ω X X f ε ¼ ‖x^ i ω^ ij x^ j k22 : i
local geometric structure with xf . After adding Eq. (11) into Eq. (7), the manifolds formed by the reconstructed image and the reference image will have similar local geometry, and the structure information of reference image can be preserved in the reconstructed image by learning the reconstruction weight using LLE.
where (
x^ j A N i
Ωði; jÞ ¼ In the third step, by incorporating Eq. (11) into Eq. (7), the object function can be obtained as
α^ ¼ arg min ‖y DH Φ○αk22 þ λkαk1 þ ε α
ωfij if xj A N i 0
i
ð12Þ
xj A N i
where γ is a weighting factor. In Eq. (12) the first term is the fidelity term, which is used to ensure that the reconstructed image x^ ¼ Φ○α agree with super-resolution model y ¼ DHx þ ν; the second term is the sparsity penalty term, guaranteeing that coefficient α is sparse; the last term is the manifold regularizer term, demanding that the reconstructed image shares the same
otherwise:
Thus Eq. (12) is rewritten as α^ ¼ arg miny DHΦ○α22 þ λkαk1 þ γ I Ω Φ○α22 : α
X X f 2 ¼ arg miny DH Φ○α2 þ λkαk1 þ γ ‖xi ωij xj ‖22 ; α
ð13Þ
ð14Þ
ð15Þ
Noticing that the first and third terms of Eq. (15) are very similar, Eq. (15) can be reformulated as α^ ¼ arg miny~ MΦ○α22 þ λkαk1 ð16Þ α
where y y~ ¼ ; 0
Please cite this article as: X. Lu, et al., MR image super-resolution via manifold regularized sparse learning, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.065i
X. Lu et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
Influence of τ on PSNR
27.8
5
Influence of τ on SSIM
0.894 Normal brainweb MS brainweb
27.6
Normal brainweb MS brainweb
0.892 27.4 0.89
27
SSIM
PSNR
27.2
26.8 26.6
0.888
0.886
26.4 0.884 26.2 26
0
0.1
0.2
0.3
0.4
0.5 τ
0.6
0.7
0.8
0.9
0.882
1
0
0.1
0.2
0.3
0.4
0.5 τ
0.6
0.7
0.8
0.9
1
Fig. 4. Influence of parameter τ on (a) PSNR, (b) SSIM.
Influence of the number of neighbour K on PSNR
28
Normal brainweb MS brainweb
27.8
Influence of the number of neighbour K on SSIM
0.887
Normal brainweb MS brainweb
0.886 0.885
27.6
0.884
27.4
SSIM
PSNR
0.883 27.2 27
0.882 0.881
26.8
0.88
26.6
0.879
26.4 26.2
0.878 0
5
10
15
20
25
0.877
0
5
10
K
15
20
25
K
Fig. 5. Influence of parameter K on (a) PSNR, (b) SSIM.
Table 2 PSNR and SSIM results of different super-resolution algorithms. Image
Nearest neighbor
Bicubic interpolation
Non-local means
Proposed algorithm
Normal brain transversal
19.94 0.5893
20.21 0.6104
25.79 0.8540
26.84 0.8737
Normal brain sagittal
19.34 0.6064
19.73 0.6342
25.71 0.8906
26.50 0.9027
Normal brain coronal
19.85 0.6024
20.19 0.6253
25.35 0.7887
26.03 0.8514
MS brain transversal
20.88 0.6295
21.15 0.6467
26.65 0.8670
27.47 0.8769
MS brain sagittal
20.92 0.6411
21.30 0.6646
26.98 0.8246
27.57 0.8841
MS brain coronal
20.90 0.6411
21.26 0.6617
26.55 0.8630
27.11 0.8709
Please cite this article as: X. Lu, et al., MR image super-resolution via manifold regularized sparse learning, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.065i
X. Lu et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
6
Fig. 6. Results on normal brain.
input
nearest-neighbor interpolation
bicubic interpolation
non-local means
proposed algorithm
ground truth
Fig. 7. Results on brain with MS lesions.
Please cite this article as: X. Lu, et al., MR image super-resolution via manifold regularized sparse learning, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.065i
X. Lu et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎ Table 3 PSNR results of 3D reconstruction. Image
Non-local means
Proposed algorithm
Normal brain MS brain
32.16 32.11
32.91 32.50
"
7
and structural similarity (SSIM) were adopted to evaluate the effectiveness of different algorithms. Given two images I and K, the PSNR between them is defined as ! 2552 PSNR ¼ 10 log 10 ; ð17Þ MSE where MSE is the mean squared error, which is defined as
#
DH M ¼ pffiffiffi γ ðI ΩÞ :
MSE ¼
1 n 1 X 1 mX ½Iði; jÞ Kði; jÞ2 : mn i ¼ 0 j ¼ 0
ð18Þ
Eq. (16) is convex, so it is possible to achieve global minima. It is an ℓ1-norm optimization problem, and can be solved by iterative shrinkage algorithm [36]. The whole algorithm is summarized as Algorithm 2.
The SSIM is calculated on various windows of an image (default window is Gaussian). The measure between windows x and y is
Algorithm 2.
where μx is the average of x, μy is the average of y, σx is the variance of x, σy is the variance of y, σxy is the covariance of x and y, c1 ¼ ðk1 LÞ2 ; c2 ¼ ðk2 LÞ2 are two variables to stabilize the division, L is the range of pixel values, and k1 ¼ 0:01, k2 ¼ 0:03 by default.
^ PCA Require: initial estimate of reconstructed image x, dictionary Φ, reference image xr, γ, and K, which is the number of neighbor used to calculate reconstruction weight ω f. 1: Set iteration k ¼0. 2: repeat k 3: Fuse xr with x^ .
4: Generate ω with K neighbor using Eq. (8), then use Eq. (14) f ij
to update Ω. 5: Update x^ k by# ðk þ 1=2Þ ðkÞ ðkÞ ¼ x^ þM T ðy~ M x^ Þ x^ ðkÞ ¼ x^ þ ðDHÞT y ðDHÞT ðDHÞX
γ 2 ðI ΩÞT ðI ΩÞX: h
SSIMðx; yÞ ¼
ð2μx μy þ c1 Þð2σ xy þ c2 Þ ; ðμ2x þ μ2y þ c1 Þðσ 2x þ σ 2y þ c2 Þ
ð19Þ
4.1. Effectiveness of the manifold regularizer To demonstrate the importance of employing the manifold regularizer, a comparison is made between the proposed algorithm with and without manifold regularizer, namely the performance between Eqs. (16) and (7). Results are listed in Table 1. It is easy to find that algorithm with manifold regularizer performs better, which proves the validity of introducing manifold assumption, as well as the advantage of exploiting intra-patient information. 4.2. Influence of parameters
i
6: αðk þ 1=2Þ ¼ φ 1 R1 x^ ðk þ 1=2Þ ; …; φ 1 RN x^ ðk þ 1=2Þ . 7: αðk þ 1Þ ¼ softðαðk þ 1=2Þ ; τÞ, soft() is a soft thresholding function, τ is the threshold. 8: Compute x^ ðk þ 1Þ ¼ Φ○αðk þ 1Þ 9: until convergence
4. Results To verify the proposed algorithm's ability to reconstruct HR images of typical brain MR images, the algorithm was applied on MRI images taken from Brainweb [37]. Brainweb is a 3D MRI Simulated Brain Database, generated using a powerful MRI simulator, and it is often used as a standard to test the capabilities of different medical image algorithms. In this experiment MRI image was generated with slice thickness of 1 mm and in-plane pixel size of 1 mm 1 mm. In Brainweb the database is based on two anatomical models: normal and multiple sclerosis (MS), and they both have three orthogonal views: transversal, sagittal, and coronal. In each image sequence, the slice exactly in the middle was selected as the test image, thus there are 6 images in total. All the images in Brainweb have been registered. In real application, images should be registered at first. Many methods are suitable for this work. In all experiments, the LR images were obtained by first blurring HR images by a 7 7 Gaussian kernel whose standard deviation is 1.6, and then down-sampling the image by a factor of 2. A cubic interpolation algorithm is applied to LR images to ^ Images are cut into 3 3 initialize the estimated HR images x. patches with overlap of 1 pixel, parameter γ is set to 0.01. Empirically this will yield satisfactory results. In the following experiments, the commonly used peak signal-to-noise ratio (PSNR)
In this section, influence of different parameters was tested. Firstly, K was fixed to 5 and the influence of τ was investigated. Fig. 4 shows the influence of sparse regularization parameter τ. From these figures, it appears that when τ changes from 0 to 0.2, the quality of reconstruction images increase remarkably. However, when τ is large enough (more than 0.2), the quality of the reconstructed images does not change a lot, meaning that the proposed algorithm is not sensitive to parameter τ. It can also be observed in these figures that the proposed algorithm has similar performance on normal brain MR images and MS brain MR images, showing that the proposed algorithm is robust with different kinds of MR images. Next, the influence of K was discussed. Fig. 5 shows how the performance of the proposed algorithm is affected by the number of nearest neighbor K used. In this experiment, PSNR and SSIM between reconstructed images and ground-truth images were computed when K varies from 1 to 25. From these figures it can be seen that both PSNR and SSIM decrease sharply when K is larger than 5. And finally experiment results become stable when K is larger than 10. 4.3. Comparison with other algorithms In this section, the proposed algorithm was compared with the methods based on nearest neighbor interpolation, bicubic interpolation and non-local means. To make it fair, in non-local means approach, volume search size was set to 5 and patch size was set to 3, which is the parameter setting that has been claimed to be able to maximize the PSNR of the approach [16]. On the other hand, for the proposed algorithm, τ was set to 0.4 and K was set to 5, following the previous discussion. PSNR and SSIM results of different algorithms are shown in Table 2. In each row, the upper numbers are the PSNR results and the lower numbers are the SSIM
Please cite this article as: X. Lu, et al., MR image super-resolution via manifold regularized sparse learning, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.065i
X. Lu et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
8
Fig. 8. 3D reconstruction results.
results. It can be found in Table 2 that the proposed algorithm distinctly outperforms all the other algorithms listed in the table, which shows that the proposed algorithm, which integrates the sparse coding framework with the manifold structure information, is helpful for reducing the uncertainty of the reconstruction image. Figs. 6 and 7 are the high resolution images reconstructed by different algorithms. The transversal, sagittal, and coronal view of both normal images and multiple sclerosis were compared here. It can be observed that the proposed algorithm provides more details. The curves look more clearly and contrasts between structures are sharper in the images reconstructed using our proposed algorithm. 4.4. 3D reconstruction results To further test the performance of the algorithm, experiments are also done on the original 3D images. The PSNR results are shown in Table 3, and some of the results are shown in Fig. 8. It can be concluded from the results that the proposed algorithm can correctly reconstruct both 2D and 3D images.
5. Conclusion This paper proposes a novel single super-resolution algorithm for MR images. The proposed algorithm exploits the local manifold structure of both a LR image and its corresponding HR reference image of different modalities, and incorporates locally linear embedding (LLE) regularizer into the original sparse coding. As a result, the proposed algorithm can be more effective and robust than the traditional image SR algorithms in dealing with MR images. The experimental results showed that the proposed algorithm performed better than state-of-the-art algorithms in both objective and subjective evaluations.
Acknowledgments This work is supported by the National Basic Research Program of China (973 Program) (Grant no. 2012CB719905), State Key Program of National Natural Science of China (Grant no. 61232010), and the National Natural Science Foundation of China (Grant nos. 61172143 and 61472413). References [1] M. Joshi, S. Chaudhuri, R. Panuganti, A learning-based method for image super-resolution from zoomed observations, IEEE Trans. Syst. Man Cybern. Part B: Cybern. 35 (3) (2005) 527–537. [2] H. Greenspan, Super-resolution in medical imaging, Comput. J. 52 (1) (2009) 43–63.
[3] B. Du, L. Zhang, A discriminative metric learning based anomaly detection method, IEEE Trans. Geosci. Remote Sens. 52 (11) (2014) 6844–6857. [4] B. Du, L. Zhang, Target detection based on a dynamic subspace, IEEE Trans. Geosci. Remote Sens. 47 (1) (2014) 344–358. [5] F. Calamante, J. Tournier, G. Jackson, A. Connelly, Track-density imaging (TDI): super-resolution white matter imaging using whole-brain track-density mapping, NeuroImage 53 (4) (2010) 1233–1243. [6] D. Frakes, L. Dasi, K. Pekkan, H. Kitajima, K. Sundareswaran, A. Yoganathan, M. Smith, A new method forregistration-based medical image interpolation, IEEE Trans. Med.Imaging 27 (3) (2008) 370–377. [7] B. Du, L. Zhang, Random-selection-based anomaly detector for hyperspectral imagery, IEEE Trans. Geosci. Remote Sens. 49 (5) (2014) 1578–1589. [8] E. Carmi, S. Liu, N. Alon, A. Fiat, D. Fiat, Resolution enhancement in MRI, Magn. Reson. Imaging 24 (2) (2006) 133–154. [9] A. Gholipour, J. Estroff, S. Warfield, Robust super-resolution volume reconstruction from slice acquisitions: application to fetal brain MRI, IEEE Trans. Med. Imaging 29 (10) (2010) 1739–1758. [10] D. Tao, X. Lin, L. Jin, X. Li, Principal component 2-dimensional long short-term memory for font recognition on single Chinese characters, IEEE Trans. Cybern. Accepted. http://dx.doi.org/10.1109/TCYB.2015.2414920. [11] S. Chen, X. Hong, C. Harris, P. Sharkey, Sparse modeling using orthogonal forward regression with PRESS statistic and regularization, IEEE Trans. Syst. Man Cybern. Part B: Cybern. 34 (2) (2004) 898–911. [12] R. Zhi, M. Flierl, Q. Ruan, W. Kleijn, Graph-preserving sparse nonnegative matrix factorization with application to facial expression recognition, IEEE Trans. Syst. Man Cybern. Part B: Cybern. 41 (1) (2011) 38–52. [13] D. Tao, X. Li, X. Wu, S.J. Maybank, General tensor discriminant analysis and Gabor features for gait recognition, IEEE Trans. Pattern Anal. Mach. Intell. 29 (10) (2007) 1700–1715. [14] J. Yang, J. Wright, T. Huang, Y. Ma, Image super-resolution as sparse representation of raw image patches, in: IEEE Conference on Computer Vision and Pattern Recognition, 2008, pp. 1–8. [15] W. Dong, L. Zhang, G. Shi, X. Wu, Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization, IEEE Trans. Image Process. 20 (7) (2011) 1838–1857. [16] F. Rousseau, A non-local approach for image super-resolution using intermodality priors, Med. Image Anal. 14 (4) (2010) 594–605. [17] D. Cai, X. He, K. Zhou, J. Han, H. Bao, Locality sensitive discriminant analysis, in: Proceedings of the 20th International Joint Conference on Artificial Intelligence, 2007, pp. 708–713. [18] C. Xu, D. Tao, C. Xu, Multi-view Intact Space Learning, IEEE Transactions on Trans. Pattern. Anal. Mach. Intell. http://dx.doi.org/10.1109/TPAMI.2015. 2417578. [19] J. Tenenbaum, V. Silva, J. Langford, A global geometric framework for nonlinear dimensionality reduction, Science 290 (5500) (2000) 2319. [20] X. Li, S. Lin, S. Yan, D. Xu, Discriminant locally linear embedding with highorder tensor data, IEEE Trans. Syst. Man Cybern. Part B: Cybernetics 38 (2) (2008) 342–352. [21] C. Xu, D. Tao, C. Xu, Large-margin multi-view information bottleneck, IEEE Trans. Pattern Anal. Mach. Intell. 36 (8) (2014) 1559–1572. [22] S. Dai, M. Han, W. Xu, Y. Wu, Y. Gong, A. Katsaggelos, SoftCuts: a soft edge smoothness prior for color image super-resolution, IEEE Trans. Image Process. 18 (5) (2009) 969–981. [23] S. Dai, M. Han, W. Xu, Y. Wu, Y. Gong, Soft edge smoothness prior for alpha channel super resolution, in: IEEE Conference on Computer Vision and Pattern Recognition, 2007, pp. 1–8. [24] J. Sun, Z. Xu, H. Shum, Image super-resolution using gradient profile prior, in: IEEE Conference on Computer Vision and Pattern Recognition, 2008, pp. 1–8. [25] D. Donoho, Compressed sensing, IEEE Trans. Inf. Theory 52 (4) (2006) 1289–1306. [26] M.A.T. Figueiredo, R.D. Nowak, S.J. Wright, Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems, IEEE J. Sel. Top. Signal Process. 1 (4) (2007) 586–597. [27] Y. Li, D. Dai, L. Shen, Multiframe super-resolution reconstruction using sparse directional regularization, IEEE Trans. Circuits Syst. Video Technol. 20 (7) (2010) 945–956.
Please cite this article as: X. Lu, et al., MR image super-resolution via manifold regularized sparse learning, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.065i
X. Lu et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎ [28] S. Mallat, Y. Guoshen, Super-resolution with sparse mixing estimators, IEEE Trans. Image Process. 19 (11) (2010) 2889–2900. [29] M. Zheng, J. Bu, C. Chen, C. Wang, L. Zhang, G. Qiu, D. Cai, Graph regularized sparse coding for image representation, IEEE Trans. Image Process. 20 (5) (2011) 1327–1336. [30] B. Olshausen, et al., Emergence of simple-cell receptive field properties by learning a sparse code for natural images, Nature 381 (6583) (1996) 607–609. [31] M. Belkin, P. Niyogi, Laplacian eigenmaps and spectral techniques for embedding and clustering, in: Advances in Neural Information Processing Systems, vol. 14, 2001, pp. 585–591. [32] K. Weinberger, F. Sha, L. Saul, Learning a kernel matrix for nonlinear dimensionality reduction, in: Proceedings of the Twenty-first International Conference on Machine Learning, 2004, p. 106. [33] S. Roweis, L. Saul, Nonlinear dimensionality reduction by locally linear embedding, Science 290 (5500) (2000) 2323–2326. [34] H. Chang, D. Yeung, Y. Xiong, Super-resolution through neighbor embedding, in: IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, 2004, pp. 275–282. [35] H. Li, B. Manjunath, S. Mitra, Multisensor image fusion using the wavelet transform, Graph. Model Image Process. 57 (3) (1995) 235–245. [36] K. Koh, S. Kim, S. Boyd, An interior-point method for large-scale l1-regularized logistic regression, J. Mach. Learn. Res. 8 (8) (2007) 1519–1555. [37] C. Cocosco, V. Kollokian, K. Remi, G. Pike, A.C. Evans, Brainweb: Online interface to a 3D MRI simulated brain database, NeuroImage 5 (4) (1997).
9 Zihan Huang received the B.S. degree in electrical engineering from Xidian University. He is currently pursuing the M.S. degree in the Center for OPTical IMagery Analysis and Learning (OPTIMAL), State Key Laboratory of Transient Optics and Photonics, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an, China. His research interests include change detection from Remote Sensing imagery.
Yuan Yuan is a Full Professor with the Chinese Academy of Sciences (CAS), China. She has published over 150 papers, including about 100 in reputable journals such as IEEE transactions and Pattern Recognition, as well as conferences papers in CVPR, BMVC, ICIP, and ICASSP. Her current research interests include visual information processing and image/video content analysis.
Xiaoqiang Lu is currently an Associate Professor with the Center for OPTical IMagery Analysis and Learning (OPTIMAL), State Key Laboratory of Transient Optics and Photonics, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an, China. His current research interests include pattern recognition, machine learning, hyperspectral image analysis, cellular automata, and medical imaging.
Please cite this article as: X. Lu, et al., MR image super-resolution via manifold regularized sparse learning, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.065i
Contents lists available at ScienceDirect
Neurocomputing journal homepage: www.elsevier.com/locate/neucom
MR image super-resolution via manifold regularized sparse learning Xiaoqiang Lu n, Zihan Huang, Yuan Yuan Center for OPTical IMagery Analysis and Learning (OPTIMAL), State Key Laboratory of Transient Optics and Photonics, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an 710119, Shaanxi, PR China
art ic l e i nf o
a b s t r a c t
Article history: Received 3 February 2015 Received in revised form 21 March 2015 Accepted 31 March 2015 Communicated by Dacheng Tao
Single image super-resolution (SR) has been shown useful in Magnetic Resonance (MR) image based diagnosis, where the image resolution is still limited. The basic goal of single image SR is to produce a high-resolution (HR) image from corresponding low-resolution (LR) image. However, most existing SR algorithms often fail to: (1) reflect the intrinsic structure between MR images and (2) exploit the intrapatient information of MR images. In fact, MR images are more likely to vary along a low dimensional submanifold, which can be embedded in the high dimensional space. It has also been shown that the structure information of MR images and the priors of the MR images of different modality are important for improving the image resolution. To take full advantage of manifold structure information and intrapatient prior of MR images, a novel single image super-resolution algorithm for MR images is proposed in this paper. Compared with the existing works, the proposed algorithm has the following merits: (1) the proposed sparse coding based algorithm integrates manifold constraints to handle the inverse problem in MR image SR; (2) the manifold structure of the intra-patient MR image is considered for image SR; and (3) the topological structure of the intra-patient MR image can be preserved to improve the reconstructed result. Experiments show that the proposed algorithm is more effective than the stateof-the-art algorithms. & 2015 Elsevier B.V. All rights reserved.
Keywords: Sparse learning Manifold regularization Super-resolution Magnetic Resonance Imaging (MRI)
1. Introduction Recently, image enhancement has received increasing attention in the area of Magnetic Resonance Imaging (MRI) [1–4]. In practical application, high quality MR image is useful for doctor to make diagnosis. For example, when diagnosing neurodegenerative diseases, it is important to observe the anatomical structures in finer details, such as some deep brain structures like hippocampus and caudate nuclei. Moreover, for many other image processing algorithm such as registration and segmentation, a high-resolution MR image is also helpful. Nowadays, 3T MRI has been more often seen in clinical use and even 7T MRI is started to be equipped in labs. The increase of magnetic field strength helps to increase signal-tonoise ratio (SNR), but the image resolution is still limited. A few factors would affect the quality of MRI, including noise, tissue contrast, resolution, and artifacts. Among them, spatial resolution is one of the main factors. Therefore, it is necessary to improve the spatial resolution of MR image. One traditional kind of method to achieve higher spatial resolution is to adjust acquisition parameters, such as decreasing the field of view, or increasing gradient
n
Corresponding author. E-mail address: [email protected] (X. Lu).
intensity. However, these methods will either decrease SNR or appreciably increase the total scan time. The other kind of method is post-processing algorithm, namely image super-resolution (SR) technique. Image SR technique aims to reconstruct one highresolution (HR) image from one or more low-resolution (LR) input images. In general, image SR technique can be categorized into three kinds: interpolation based methods, reconstruction based methods and learning based methods. Previously, much attention has been paid on interpolation based methods, such as nearest-neighbor, and linear interpolation [5–7]. These methods are fast and easy to understand and manipulate, but they are not good at handling edges and complex textures. Blurred edges and textures can often be found on interpolation based methods. The second kind of image SR technique is reconstruction based methods, which have also been proposed to deal with multiple LR images [8–10]. However, there is one prerequisite for this type of methods: sufficient LR images with sub-pixel displacements are needed, which is not always the case. The limitations of the above methods have spurred the emergence of learning based SR methods. Learning based methods use training database to help restore the high frequency details that lost in LR image. Recently sparse learning has arisen great interest in a variety of fields [11–13], and sparse coding based methods are
http://dx.doi.org/10.1016/j.neucom.2015.03.065 0925-2312/& 2015 Elsevier B.V. All rights reserved.
Please cite this article as: X. Lu, et al., MR image super-resolution via manifold regularized sparse learning, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.065i
2
X. Lu et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
Fig. 1. Sketch of the proposed algorithm. LR T2-weighted image is reconstructed using only one HR T1-weighted image as the reference image. In the algorithm, first the intra-patient information is obtained from reference image, then this information is used to guide the SR process.
Fig. 2. T1-weighted image and T2-weighted image of the same patient. left: T1weighted image; right: T2-weighted image.
the main stream of learning based image SR, which show the state-of-the-art performance. For instance, Yang et al. [14] assume that an over-complete dictionary can be learned to represent image patches sparsely, and then an ℓ1-norm sparse regularizer is introduced to ensure that the reconstruction coefficient matrix of the LR image is sparse. Dong et al. [15] build a series of dictionary to enhance the resolution of an image, in which one best dictionary is selected for each image patch to be reconstructed. In their method, the representation of image patches over the selected dictionary is also required to be sparse. However, all of the aforementioned studies have not considered the information within intra-patient MR images, which has recently been proven to be useful for enhancing the resolution of MR image [16]. Generally MR image examination is made up of a series of sequences, and each of them can provide its unique information about the subject tissues. Among them, T1-weighted MRI and T2-weighted MRI are two basic sequences. It is a common occurrence that an HR T1-weighted reference image and a LR T2weighted image of the same patient are available. This often happens when the patient is asked to regularly take MRI scan, or take a series of scans under different acquisition parameters, such as the subsequent examination of brain tumor evolution, neurodegenerative diseases and diffusion tensor imaging. For the convenience of diagnosis, it is necessary to enlarge the LR image so that it achieves the same size as HR image. In other words, it is
necessary to exploit the information of one HR T1-weighted image to improve the resolution of the other modality LR image (T2weighted image). To be more specific, a T1-weighted image can be used as a reference image to help reconstruct one corresponding LR T2-weighted image. The sketch of the proposed algorithm is shown in Fig. 1. To employ the intra-patient information of MR image, Rousseau et al. [16] use the non-local self-similarity of reference image as a prior and solve the SR problem under a patch-based framework. The proposed algorithm has yielded satisfactory achievement. However, it only considered image structure in the Euclidean space. Recent study [17,18] shows that the intrinsic structure of the data may not be accurately reflected by the Euclidean distance in the high dimensional space. Recently, research has shown that high dimensional Euclidean space cannot be uniformly “filled up” by the image data. These image data can be regarded as sampled data from or near a submanifold of an ambient space [19–21]. Therefore, it is necessary to consider the intrinsic structure when reconstructing the HR image. Inspired by recent advances in manifold learning and sparse coding, a new learning based single image SR algorithm is proposed in this paper. The proposed algorithm tries to solve the inverse problem of reconstructing an HR image from the LR T2-weighted image by exploiting an HR T1-weighted reference image of different modality, which has very obvious practical application in medical image diagnosis. T2-weighted image and T1-weighted image of the same patient are shown in Fig. 2. In most case, it is possible to exploit the prior information of HR T1-weighted reference image to enhance the resolution of the LR T2-weighted image. The new learning based algorithm for single MR image SR can be divided into three steps. Firstly, a new sparse coding algorithm is employed as the framework of the SR. Secondly, in order to explicitly utilize the structure information of the reference image, the reference is fused with the estimated HR image, and then a weight estimator is learned from the fused image, which can help reduce the uncertainty of the reconstruction. Finally, by incorporating the manifold learning technique into sparse coding, the proposed algorithm extracts the intrinsic geometric structure from the fused image. The rest of the paper is organized as follows: Section 2 gives a brief description of the problem. Section 3 details the proposed algorithm which enhances the resolution of MRI image using one reference image. Experiment results are shown in Section 4. Finally, conclusion is given in Section 5.
Please cite this article as: X. Lu, et al., MR image super-resolution via manifold regularized sparse learning, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.065i
X. Lu et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
2. Problem description This section presents a generic observation model for image SR. Generally, super-resolution problems can be modeled as y ¼ DHx þ ν;
ð1Þ
where y denotes the LR input image, x denotes the HR image to be estimated, D represents down sampling operator, H represents blurring operator, and ν denotes additive noise. Based on this model, the reconstructed image can be estimated by minimizing the least-square cost function 2 ð2Þ X^ ¼ arg miny DHx2 : x
Eq. (2) presents an ill-posed problem and there is no unique answer to it. Therefore, some prior should be added to regularize the above equation. The regularized model can be referred to as 2 ð3Þ X^ ¼ arg miny DHx2 þ λRðxÞ: x
RðxÞ is a regularization term, which represents the priors of the HR image. λ is a regularization parameter. An appropriate prior is of prime importance to enhance the performance of SR reconstruction. Different methods introduce different image priors into SR, such as edge smoothness [22,23], gradient profile priors [24] and nonlocal similar priors [16]. Recently, it has become a hot research topic for compressed sensing to obtain sparse solutions of under-determined inverse problem [25,26]. Motivated by recent progress in compressed sensing, sparse coding based algorithms have been applied to image SR and achieved considerable SR results [27,28,14]. Sparse coding seeks to develop a simple and efficient way to achieve sparse coding over a proper dictionary. That is, the product of sparse representations a A Rpn and a given dictionary φ can best approximate the original data matrix x A Rmn [29]. In single image SR, therefore, the objective function of sparse coding can be represented as follows: 2 minx DHφa2 þ λkak1 ; ð4Þ φ;α
where λ 4 0 is a regularization parameter. In sparse coding, all the data points are represented by only a few of basis vector, and the representation is often over complete, so a sparse dictionary can represent a wide variety of elements. What is more, study shows that sparse representation accords with biological vision in primary visual cortex (V1) [30]. Recent studies have shown that human generated image data vary smoothly on the geodesics of the manifold [31–33]. Therefore, if the intrinsic manifold structure of both the T2-weighted image and the T1-weighted image can be taken into consideration, the reconstruction result will be improved. In this case, it is assumed that both T1 and T2 images have intrinsically low-dimensional representation, which lie on a very high dimensional space.
3. Proposed algorithm To improve the performance of sparse coding based algorithm, the intra-patient information of MR image that considers the local manifold structure has been incorporated into the sparse coding process in this paper. 3.1. Manifold regularized sparse algorithm for SR In the field of natural image SR, sparse coding based algorithms are very popular. By introducing the concept of sparse coding, each data point can be interpreted more reasonably, which will result in a high performance of SR. However, in the field of MR image, sparse coding may not be an ideal algorithm. Due to the lack of
3
intra-patient information in MR images, the performance of the sparse coding based algorithm is reduced on MR image. To improve the quality of the reconstructed HR MR image, more prior information should be considered. Motivated by recent advances in sparse coding and manifold learning, a novel sparse coding based algorithm is proposed in this paper, which incorporates intra-patient information of MR image that consider the local manifold structure. In this paper, the main work is to reconstruct an HR image from one LR image and one corresponding HR reference image of a different modality. The proposed algorithm can be divided into three steps. In the first step, the principal component analysis (PCA) dictionary based sparse coding algorithm is used for the MR image reconstruction. Let xi ¼ Ri x; i ¼ 1; 2; …; N, denote the ith patch vector of x. Inspired by [15], K orthonormal PCA dictionaries fΦk g; k ¼ 1; 2; …; K, are introduced into sparse coding. For each image patch x^ i , the most suitable sub-dictionary Φki is then chosen, which can best represented the image patch both linearly and sparsely. The concrete step of dictionary learning and sub-dictionary selection can be found in [15]. Then x^ i can be represented as x^i ¼ Φki αi
s:t: kαi k1 r c
ð5Þ
#where αi is the representation of the image patch x^ i . Then by averaging all the image patches, complete image can be reconstructed as !1 N N X X RTi Ri RTi Φki αi x^ ¼ Φ○α ¼ ð6Þ i¼1
i¼1
#where Φ is the concatenation of Φk, α is the concatenation of αi . Combining Eq. (5) with Eq. (6), the reconstruction problem can be described as α^ ¼ arg miny DHΦ○α22 þ λkαk1 ; ð7Þ α
in which λ controls the balance between the approximation of y and the sparsity of solution. In the second step, the weight estimator is introduced based on manifold learning by assuming that LR T2-weighted image and HR T1-weighted image lie on two distinct manifolds with similar local geometry. It can be shown from Fig. 2 that T1-weighted image and T2-weighted image looks very similar. Thus, we believe that they are very likely to have similar local geometrical structure, and the structure information can be used to reduce the uncertainty of the HR T2-weighted image. Moreover, exploiting data in low dimensional space instead of their high dimensional representation can improve the learning performance and the reconstruction accuracy [34]. Given one HR T1-weighted reference image xr, we expect to obtain the weight estimator, which depends on the manifold of each data point and its neighbors. Accordingly, the second step can be divided into three phases: firstly, the fused image xf can be obtained by fusing the HR T1 reference image with the estimated HR image using a wavelet based method [35]. Consequently, the fused image contains the priors of both original image and the reference image. The fused image is shown in Fig. 3. Secondly, according to manifold learning technique, i.e., locally linear embedding (LLE) criterion, the reconstruction weight of the neighbors in xi is computed by minimizing the local reconstruction error: X f f X ω^ fij ¼ arg min J xfi ωij xj J 22 ; ð8Þ ωfij
i
xfj A Ni
where Ni denotes the K nearest neighbor of xfi (here Euclidean distance is used to define neighborhood); ωfij isP the reconstruction ωf ¼ 1, and weight, which is subject to the constraints xf A N i ij j
Please cite this article as: X. Lu, et al., MR image super-resolution via manifold regularized sparse learning, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.065i
X. Lu et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
4
Fig. 3. Example of MRI image fusion. From left to right: T1-weighted image, T2-weighted image, the fused image.
ωfij ¼ 0 for all xfj 2= N i . As is described in [34], the optimization of Eq. (8) can be done with the help of a Gram matrix Gi ¼ ðxfi 1T XÞT ðxfi 1T XÞ;
ð9Þ
where each column of X is one neighbor of xfi, and 1 is a column vector whose elements are all ones. Let ωfi be the concatenation of ωfij. The weight estimator ωfi is then obtained as follows: Gi ωfi ¼ 1:
Table 1 Comparison of algorithms with and without manifold regularizer. Image
No manifold regularizer
Proposed algorithm
Normal brain transversal
26.67 0.8689
26.84 0.8737
Normal brain sagittal
26.21 0.8952
26.50 0.9027
Normal brain coronal
25.84 0.8566
26.03 0.8646
MS brain transversal
27.22 0.8690
27.47 0.8769
MS brain sagittal
27.42 0.8906
27.57 0.8972
MS brain coronal
26.98 0.8625
27.11 0.8709
ð10Þ
P ωf ¼ 1. Thirdly, the reconThen ωfi is normalized so that xfj A N i ij f ^ ij is assigned to the reconstructed image struction weight ω patches x^i , thus the reconstruction error of the reconstructed image patch and its neighbor can be obtained as X X f ε ¼ ‖x^ i ω^ ij x^ j k22 : ð11Þ x^ j A Ni
i
The topology structure of the HR T1-weighted reference image can be preserved in the reconstructed T2-weighted image by the weight estimator. In all, the Pseudo-code can be written as Algorithm 1. Algorithm 1. 1: Fuse x^ with xr , generate xf. ^ fij by minimizing the local 2: Compute reconstruction weight ω reconstruction error:#X X ω^ fij ¼ arg min ‖xfi ωfij xfj k22 ; ω
f ij
Compute
i
xfj A Ni
It is worth noting that the proposed algorithm does not utilize any specific prior knowledge of certain kind of tissue. Thus, it is a general algorithm, and can be applied to solve SR problems other than MR imaging.
ωfi so that Gi ωfi ¼ 1, where
Gi ¼ ðxfi 1T
XÞT ðxfi 1T XÞ. P Normalize ωfi so that xf A N ωfij ¼ 1. j
3.2. Optimization Eq. (11) can be rewritten as follows: ε ¼ I Ω Φ○α22 ;
i
^ fij to calculate the reconstruct error of xi 3: Use ω X X f ε ¼ ‖x^ i ω^ ij x^ j k22 : i
local geometric structure with xf . After adding Eq. (11) into Eq. (7), the manifolds formed by the reconstructed image and the reference image will have similar local geometry, and the structure information of reference image can be preserved in the reconstructed image by learning the reconstruction weight using LLE.
where (
x^ j A N i
Ωði; jÞ ¼ In the third step, by incorporating Eq. (11) into Eq. (7), the object function can be obtained as
α^ ¼ arg min ‖y DH Φ○αk22 þ λkαk1 þ ε α
ωfij if xj A N i 0
i
ð12Þ
xj A N i
where γ is a weighting factor. In Eq. (12) the first term is the fidelity term, which is used to ensure that the reconstructed image x^ ¼ Φ○α agree with super-resolution model y ¼ DHx þ ν; the second term is the sparsity penalty term, guaranteeing that coefficient α is sparse; the last term is the manifold regularizer term, demanding that the reconstructed image shares the same
otherwise:
Thus Eq. (12) is rewritten as α^ ¼ arg miny DHΦ○α22 þ λkαk1 þ γ I Ω Φ○α22 : α
X X f 2 ¼ arg miny DH Φ○α2 þ λkαk1 þ γ ‖xi ωij xj ‖22 ; α
ð13Þ
ð14Þ
ð15Þ
Noticing that the first and third terms of Eq. (15) are very similar, Eq. (15) can be reformulated as α^ ¼ arg miny~ MΦ○α22 þ λkαk1 ð16Þ α
where y y~ ¼ ; 0
Please cite this article as: X. Lu, et al., MR image super-resolution via manifold regularized sparse learning, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.065i
X. Lu et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
Influence of τ on PSNR
27.8
5
Influence of τ on SSIM
0.894 Normal brainweb MS brainweb
27.6
Normal brainweb MS brainweb
0.892 27.4 0.89
27
SSIM
PSNR
27.2
26.8 26.6
0.888
0.886
26.4 0.884 26.2 26
0
0.1
0.2
0.3
0.4
0.5 τ
0.6
0.7
0.8
0.9
0.882
1
0
0.1
0.2
0.3
0.4
0.5 τ
0.6
0.7
0.8
0.9
1
Fig. 4. Influence of parameter τ on (a) PSNR, (b) SSIM.
Influence of the number of neighbour K on PSNR
28
Normal brainweb MS brainweb
27.8
Influence of the number of neighbour K on SSIM
0.887
Normal brainweb MS brainweb
0.886 0.885
27.6
0.884
27.4
SSIM
PSNR
0.883 27.2 27
0.882 0.881
26.8
0.88
26.6
0.879
26.4 26.2
0.878 0
5
10
15
20
25
0.877
0
5
10
K
15
20
25
K
Fig. 5. Influence of parameter K on (a) PSNR, (b) SSIM.
Table 2 PSNR and SSIM results of different super-resolution algorithms. Image
Nearest neighbor
Bicubic interpolation
Non-local means
Proposed algorithm
Normal brain transversal
19.94 0.5893
20.21 0.6104
25.79 0.8540
26.84 0.8737
Normal brain sagittal
19.34 0.6064
19.73 0.6342
25.71 0.8906
26.50 0.9027
Normal brain coronal
19.85 0.6024
20.19 0.6253
25.35 0.7887
26.03 0.8514
MS brain transversal
20.88 0.6295
21.15 0.6467
26.65 0.8670
27.47 0.8769
MS brain sagittal
20.92 0.6411
21.30 0.6646
26.98 0.8246
27.57 0.8841
MS brain coronal
20.90 0.6411
21.26 0.6617
26.55 0.8630
27.11 0.8709
Please cite this article as: X. Lu, et al., MR image super-resolution via manifold regularized sparse learning, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.065i
X. Lu et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
6
Fig. 6. Results on normal brain.
input
nearest-neighbor interpolation
bicubic interpolation
non-local means
proposed algorithm
ground truth
Fig. 7. Results on brain with MS lesions.
Please cite this article as: X. Lu, et al., MR image super-resolution via manifold regularized sparse learning, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.065i
X. Lu et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎ Table 3 PSNR results of 3D reconstruction. Image
Non-local means
Proposed algorithm
Normal brain MS brain
32.16 32.11
32.91 32.50
"
7
and structural similarity (SSIM) were adopted to evaluate the effectiveness of different algorithms. Given two images I and K, the PSNR between them is defined as ! 2552 PSNR ¼ 10 log 10 ; ð17Þ MSE where MSE is the mean squared error, which is defined as
#
DH M ¼ pffiffiffi γ ðI ΩÞ :
MSE ¼
1 n 1 X 1 mX ½Iði; jÞ Kði; jÞ2 : mn i ¼ 0 j ¼ 0
ð18Þ
Eq. (16) is convex, so it is possible to achieve global minima. It is an ℓ1-norm optimization problem, and can be solved by iterative shrinkage algorithm [36]. The whole algorithm is summarized as Algorithm 2.
The SSIM is calculated on various windows of an image (default window is Gaussian). The measure between windows x and y is
Algorithm 2.
where μx is the average of x, μy is the average of y, σx is the variance of x, σy is the variance of y, σxy is the covariance of x and y, c1 ¼ ðk1 LÞ2 ; c2 ¼ ðk2 LÞ2 are two variables to stabilize the division, L is the range of pixel values, and k1 ¼ 0:01, k2 ¼ 0:03 by default.
^ PCA Require: initial estimate of reconstructed image x, dictionary Φ, reference image xr, γ, and K, which is the number of neighbor used to calculate reconstruction weight ω f. 1: Set iteration k ¼0. 2: repeat k 3: Fuse xr with x^ .
4: Generate ω with K neighbor using Eq. (8), then use Eq. (14) f ij
to update Ω. 5: Update x^ k by# ðk þ 1=2Þ ðkÞ ðkÞ ¼ x^ þM T ðy~ M x^ Þ x^ ðkÞ ¼ x^ þ ðDHÞT y ðDHÞT ðDHÞX
γ 2 ðI ΩÞT ðI ΩÞX: h
SSIMðx; yÞ ¼
ð2μx μy þ c1 Þð2σ xy þ c2 Þ ; ðμ2x þ μ2y þ c1 Þðσ 2x þ σ 2y þ c2 Þ
ð19Þ
4.1. Effectiveness of the manifold regularizer To demonstrate the importance of employing the manifold regularizer, a comparison is made between the proposed algorithm with and without manifold regularizer, namely the performance between Eqs. (16) and (7). Results are listed in Table 1. It is easy to find that algorithm with manifold regularizer performs better, which proves the validity of introducing manifold assumption, as well as the advantage of exploiting intra-patient information. 4.2. Influence of parameters
i
6: αðk þ 1=2Þ ¼ φ 1 R1 x^ ðk þ 1=2Þ ; …; φ 1 RN x^ ðk þ 1=2Þ . 7: αðk þ 1Þ ¼ softðαðk þ 1=2Þ ; τÞ, soft() is a soft thresholding function, τ is the threshold. 8: Compute x^ ðk þ 1Þ ¼ Φ○αðk þ 1Þ 9: until convergence
4. Results To verify the proposed algorithm's ability to reconstruct HR images of typical brain MR images, the algorithm was applied on MRI images taken from Brainweb [37]. Brainweb is a 3D MRI Simulated Brain Database, generated using a powerful MRI simulator, and it is often used as a standard to test the capabilities of different medical image algorithms. In this experiment MRI image was generated with slice thickness of 1 mm and in-plane pixel size of 1 mm 1 mm. In Brainweb the database is based on two anatomical models: normal and multiple sclerosis (MS), and they both have three orthogonal views: transversal, sagittal, and coronal. In each image sequence, the slice exactly in the middle was selected as the test image, thus there are 6 images in total. All the images in Brainweb have been registered. In real application, images should be registered at first. Many methods are suitable for this work. In all experiments, the LR images were obtained by first blurring HR images by a 7 7 Gaussian kernel whose standard deviation is 1.6, and then down-sampling the image by a factor of 2. A cubic interpolation algorithm is applied to LR images to ^ Images are cut into 3 3 initialize the estimated HR images x. patches with overlap of 1 pixel, parameter γ is set to 0.01. Empirically this will yield satisfactory results. In the following experiments, the commonly used peak signal-to-noise ratio (PSNR)
In this section, influence of different parameters was tested. Firstly, K was fixed to 5 and the influence of τ was investigated. Fig. 4 shows the influence of sparse regularization parameter τ. From these figures, it appears that when τ changes from 0 to 0.2, the quality of reconstruction images increase remarkably. However, when τ is large enough (more than 0.2), the quality of the reconstructed images does not change a lot, meaning that the proposed algorithm is not sensitive to parameter τ. It can also be observed in these figures that the proposed algorithm has similar performance on normal brain MR images and MS brain MR images, showing that the proposed algorithm is robust with different kinds of MR images. Next, the influence of K was discussed. Fig. 5 shows how the performance of the proposed algorithm is affected by the number of nearest neighbor K used. In this experiment, PSNR and SSIM between reconstructed images and ground-truth images were computed when K varies from 1 to 25. From these figures it can be seen that both PSNR and SSIM decrease sharply when K is larger than 5. And finally experiment results become stable when K is larger than 10. 4.3. Comparison with other algorithms In this section, the proposed algorithm was compared with the methods based on nearest neighbor interpolation, bicubic interpolation and non-local means. To make it fair, in non-local means approach, volume search size was set to 5 and patch size was set to 3, which is the parameter setting that has been claimed to be able to maximize the PSNR of the approach [16]. On the other hand, for the proposed algorithm, τ was set to 0.4 and K was set to 5, following the previous discussion. PSNR and SSIM results of different algorithms are shown in Table 2. In each row, the upper numbers are the PSNR results and the lower numbers are the SSIM
Please cite this article as: X. Lu, et al., MR image super-resolution via manifold regularized sparse learning, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.065i
X. Lu et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
8
Fig. 8. 3D reconstruction results.
results. It can be found in Table 2 that the proposed algorithm distinctly outperforms all the other algorithms listed in the table, which shows that the proposed algorithm, which integrates the sparse coding framework with the manifold structure information, is helpful for reducing the uncertainty of the reconstruction image. Figs. 6 and 7 are the high resolution images reconstructed by different algorithms. The transversal, sagittal, and coronal view of both normal images and multiple sclerosis were compared here. It can be observed that the proposed algorithm provides more details. The curves look more clearly and contrasts between structures are sharper in the images reconstructed using our proposed algorithm. 4.4. 3D reconstruction results To further test the performance of the algorithm, experiments are also done on the original 3D images. The PSNR results are shown in Table 3, and some of the results are shown in Fig. 8. It can be concluded from the results that the proposed algorithm can correctly reconstruct both 2D and 3D images.
5. Conclusion This paper proposes a novel single super-resolution algorithm for MR images. The proposed algorithm exploits the local manifold structure of both a LR image and its corresponding HR reference image of different modalities, and incorporates locally linear embedding (LLE) regularizer into the original sparse coding. As a result, the proposed algorithm can be more effective and robust than the traditional image SR algorithms in dealing with MR images. The experimental results showed that the proposed algorithm performed better than state-of-the-art algorithms in both objective and subjective evaluations.
Acknowledgments This work is supported by the National Basic Research Program of China (973 Program) (Grant no. 2012CB719905), State Key Program of National Natural Science of China (Grant no. 61232010), and the National Natural Science Foundation of China (Grant nos. 61172143 and 61472413). References [1] M. Joshi, S. Chaudhuri, R. Panuganti, A learning-based method for image super-resolution from zoomed observations, IEEE Trans. Syst. Man Cybern. Part B: Cybern. 35 (3) (2005) 527–537. [2] H. Greenspan, Super-resolution in medical imaging, Comput. J. 52 (1) (2009) 43–63.
[3] B. Du, L. Zhang, A discriminative metric learning based anomaly detection method, IEEE Trans. Geosci. Remote Sens. 52 (11) (2014) 6844–6857. [4] B. Du, L. Zhang, Target detection based on a dynamic subspace, IEEE Trans. Geosci. Remote Sens. 47 (1) (2014) 344–358. [5] F. Calamante, J. Tournier, G. Jackson, A. Connelly, Track-density imaging (TDI): super-resolution white matter imaging using whole-brain track-density mapping, NeuroImage 53 (4) (2010) 1233–1243. [6] D. Frakes, L. Dasi, K. Pekkan, H. Kitajima, K. Sundareswaran, A. Yoganathan, M. Smith, A new method forregistration-based medical image interpolation, IEEE Trans. Med.Imaging 27 (3) (2008) 370–377. [7] B. Du, L. Zhang, Random-selection-based anomaly detector for hyperspectral imagery, IEEE Trans. Geosci. Remote Sens. 49 (5) (2014) 1578–1589. [8] E. Carmi, S. Liu, N. Alon, A. Fiat, D. Fiat, Resolution enhancement in MRI, Magn. Reson. Imaging 24 (2) (2006) 133–154. [9] A. Gholipour, J. Estroff, S. Warfield, Robust super-resolution volume reconstruction from slice acquisitions: application to fetal brain MRI, IEEE Trans. Med. Imaging 29 (10) (2010) 1739–1758. [10] D. Tao, X. Lin, L. Jin, X. Li, Principal component 2-dimensional long short-term memory for font recognition on single Chinese characters, IEEE Trans. Cybern. Accepted. http://dx.doi.org/10.1109/TCYB.2015.2414920. [11] S. Chen, X. Hong, C. Harris, P. Sharkey, Sparse modeling using orthogonal forward regression with PRESS statistic and regularization, IEEE Trans. Syst. Man Cybern. Part B: Cybern. 34 (2) (2004) 898–911. [12] R. Zhi, M. Flierl, Q. Ruan, W. Kleijn, Graph-preserving sparse nonnegative matrix factorization with application to facial expression recognition, IEEE Trans. Syst. Man Cybern. Part B: Cybern. 41 (1) (2011) 38–52. [13] D. Tao, X. Li, X. Wu, S.J. Maybank, General tensor discriminant analysis and Gabor features for gait recognition, IEEE Trans. Pattern Anal. Mach. Intell. 29 (10) (2007) 1700–1715. [14] J. Yang, J. Wright, T. Huang, Y. Ma, Image super-resolution as sparse representation of raw image patches, in: IEEE Conference on Computer Vision and Pattern Recognition, 2008, pp. 1–8. [15] W. Dong, L. Zhang, G. Shi, X. Wu, Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization, IEEE Trans. Image Process. 20 (7) (2011) 1838–1857. [16] F. Rousseau, A non-local approach for image super-resolution using intermodality priors, Med. Image Anal. 14 (4) (2010) 594–605. [17] D. Cai, X. He, K. Zhou, J. Han, H. Bao, Locality sensitive discriminant analysis, in: Proceedings of the 20th International Joint Conference on Artificial Intelligence, 2007, pp. 708–713. [18] C. Xu, D. Tao, C. Xu, Multi-view Intact Space Learning, IEEE Transactions on Trans. Pattern. Anal. Mach. Intell. http://dx.doi.org/10.1109/TPAMI.2015. 2417578. [19] J. Tenenbaum, V. Silva, J. Langford, A global geometric framework for nonlinear dimensionality reduction, Science 290 (5500) (2000) 2319. [20] X. Li, S. Lin, S. Yan, D. Xu, Discriminant locally linear embedding with highorder tensor data, IEEE Trans. Syst. Man Cybern. Part B: Cybernetics 38 (2) (2008) 342–352. [21] C. Xu, D. Tao, C. Xu, Large-margin multi-view information bottleneck, IEEE Trans. Pattern Anal. Mach. Intell. 36 (8) (2014) 1559–1572. [22] S. Dai, M. Han, W. Xu, Y. Wu, Y. Gong, A. Katsaggelos, SoftCuts: a soft edge smoothness prior for color image super-resolution, IEEE Trans. Image Process. 18 (5) (2009) 969–981. [23] S. Dai, M. Han, W. Xu, Y. Wu, Y. Gong, Soft edge smoothness prior for alpha channel super resolution, in: IEEE Conference on Computer Vision and Pattern Recognition, 2007, pp. 1–8. [24] J. Sun, Z. Xu, H. Shum, Image super-resolution using gradient profile prior, in: IEEE Conference on Computer Vision and Pattern Recognition, 2008, pp. 1–8. [25] D. Donoho, Compressed sensing, IEEE Trans. Inf. Theory 52 (4) (2006) 1289–1306. [26] M.A.T. Figueiredo, R.D. Nowak, S.J. Wright, Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems, IEEE J. Sel. Top. Signal Process. 1 (4) (2007) 586–597. [27] Y. Li, D. Dai, L. Shen, Multiframe super-resolution reconstruction using sparse directional regularization, IEEE Trans. Circuits Syst. Video Technol. 20 (7) (2010) 945–956.
Please cite this article as: X. Lu, et al., MR image super-resolution via manifold regularized sparse learning, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.065i
X. Lu et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎ [28] S. Mallat, Y. Guoshen, Super-resolution with sparse mixing estimators, IEEE Trans. Image Process. 19 (11) (2010) 2889–2900. [29] M. Zheng, J. Bu, C. Chen, C. Wang, L. Zhang, G. Qiu, D. Cai, Graph regularized sparse coding for image representation, IEEE Trans. Image Process. 20 (5) (2011) 1327–1336. [30] B. Olshausen, et al., Emergence of simple-cell receptive field properties by learning a sparse code for natural images, Nature 381 (6583) (1996) 607–609. [31] M. Belkin, P. Niyogi, Laplacian eigenmaps and spectral techniques for embedding and clustering, in: Advances in Neural Information Processing Systems, vol. 14, 2001, pp. 585–591. [32] K. Weinberger, F. Sha, L. Saul, Learning a kernel matrix for nonlinear dimensionality reduction, in: Proceedings of the Twenty-first International Conference on Machine Learning, 2004, p. 106. [33] S. Roweis, L. Saul, Nonlinear dimensionality reduction by locally linear embedding, Science 290 (5500) (2000) 2323–2326. [34] H. Chang, D. Yeung, Y. Xiong, Super-resolution through neighbor embedding, in: IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, 2004, pp. 275–282. [35] H. Li, B. Manjunath, S. Mitra, Multisensor image fusion using the wavelet transform, Graph. Model Image Process. 57 (3) (1995) 235–245. [36] K. Koh, S. Kim, S. Boyd, An interior-point method for large-scale l1-regularized logistic regression, J. Mach. Learn. Res. 8 (8) (2007) 1519–1555. [37] C. Cocosco, V. Kollokian, K. Remi, G. Pike, A.C. Evans, Brainweb: Online interface to a 3D MRI simulated brain database, NeuroImage 5 (4) (1997).
9 Zihan Huang received the B.S. degree in electrical engineering from Xidian University. He is currently pursuing the M.S. degree in the Center for OPTical IMagery Analysis and Learning (OPTIMAL), State Key Laboratory of Transient Optics and Photonics, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an, China. His research interests include change detection from Remote Sensing imagery.
Yuan Yuan is a Full Professor with the Chinese Academy of Sciences (CAS), China. She has published over 150 papers, including about 100 in reputable journals such as IEEE transactions and Pattern Recognition, as well as conferences papers in CVPR, BMVC, ICIP, and ICASSP. Her current research interests include visual information processing and image/video content analysis.
Xiaoqiang Lu is currently an Associate Professor with the Center for OPTical IMagery Analysis and Learning (OPTIMAL), State Key Laboratory of Transient Optics and Photonics, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an, China. His current research interests include pattern recognition, machine learning, hyperspectral image analysis, cellular automata, and medical imaging.
Please cite this article as: X. Lu, et al., MR image super-resolution via manifold regularized sparse learning, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.065i