- Email: [email protected]
Efficient reconstruction of compressively sensed images and videos using non-iterative method
Accepted Manuscript Efficient reconstruction of compressively sensed images and videos using noniterative method Florence Gnana Poovathy John, Radha Sankararajan PII: DOI: Reference:
S1434-8411(16)31570-9 http://dx.doi.org/10.1016/j.aeue.2016.12.019 AEUE 51759
To appear in:
International Journal of Electronics and Communications
Received Date: Revised Date: Accepted Date:
18 January 2016 28 July 2016 24 December 2016
Please cite this article as: F. Gnana Poovathy John, R. Sankararajan, Efficient reconstruction of compressively sensed images and videos using non-iterative method, International Journal of Electronics and Communications (2016), doi: http://dx.doi.org/10.1016/j.aeue.2016.12.019
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Efficient reconstruction of compressively sensed images and videos using non-iterative method Florence Gnana Poovathy John1, Radha Sankararajan2 1
[Email: [email protected]] 2 [Email: [email protected]] 1,2 Department of ECE, SSN College of Engineering, Kalavakkam, Chennai – 603110, Tamil Nadu, India.
Abstract – Compressed sensing is widely applied for compression and reconstruction of images and videos by projecting the pixel values to smaller dimensional measurements. These measurements are reconstructed at the receiver using various reconstruction procedures. Greedy algorithms are often used for such recovery. These solve the least squares problem to find the best match with minimum error. This is a time consuming and complex process, giving rise to a trade-off between reconstruction performance and algorithmic performance. This work proposes a non-iterative method, viz., non-iterative pseudo inverse based recovery algorithm (NIPIRA), for reconstruction of compressively sensed images and videos with small complexity and time consumption, provided the reconstruction quality is maintained. NIPIRA gives a minimum PSNR of 32 dB for very few measurements (/ = 0.3125) and accuracy of above 97%. There is more than 92% of decrease in elapsed time compared with other iterative algorithms. NIPIRA is tested for its performance with respect to many other objective measures as well. The complexity of NIPIRA is times less than existing recovery algorithms. Keywords: Compressed sensing, image and video reconstruction, non-iterative recovery, objective measures, elapsed time. 1. Introduction Transmission of images and videos over wireless sensor networks (WSNs) require compression for effective usage of bandwidth and storage capacity. Development of an 1
efficient lossy technique, compressed sensing (CS) [1 – 5] has been seen in recent years. This overrules Nyquist criteria by requiring a small number of samples for reconstruction, provided the input data is sparse [6] and the sensing matrix abides by restricted isometry property (RIP) [7, 8]. CS also guarantees improved quality in reconstruction of images and videos [5] and therefore considered as one of the best methods of compression. Compression in CS is carried out by projecting the available sparse samples to a lower dimensional vector space while the reconstruction of the input from these measurements is carried out at the receiver through efficient reconstruction algorithms. Many reconstruction algorithms are available. These may be classified as greedy algorithms like orthogonal matching pursuit (OMP) [9], Stagewise orthogonal matching pursuit (StOMP) [10, 11], compressed sampling matching pursuit (CoSaMP) [12] etc., iterative algorithms like iterative hard thresholding (IHT) [13] and model based algorithms like model based CoSaMP and model based IHT. Other advanced algorithms also exist namely, least absolute shrinkage selection operator (LASSO) [14], sparse Bayesian learning for multiple measurement vector model exploiting temporal correlation (TMSBL) [15] and focal underdetermined system solution using multiple measurement vector (MFOCUSS) [16] have gained attention since they perform better than greedy algorithms. The downside of these algorithms are that they are iterative and hence consume more time and large processing energy. The non-iterative recovery procedure (NIPIRA) is proposed for obviating these disadvantages and avoiding iterations, while providing good perceptual quality for the reconstructed images and videos. The sensing matrix used for projecting the input sparse vector to a lower dimensional space is augmented matrix which is an augmentation of identity matrix and zero matrix. NIPIRA provides a PSNR of above 32 dB for the least number of measurements (31.25% of input information for 8 x 8 block size) with an accuracy of 97% and elapsed time of the order of µs. Thus NIPIRA proves to be a 2
less time consuming non-iterative algorithm for reconstruction of compressively sensed image and video data. Notations used in this work are given in Table 1. Table 1 Notations and denotations
Denotations
Notations
Denotations
Notations
Input vector
x
Threshold
T
Basis function
ψ
Intermediate vector
V
Sensing matrix
ϕ
Position vector
Г
Sensed vector
y
Estimated vector
Sparsity
s
Input vector length
N
Sparsified signal
Measurement vector size
MxN
Block length
N
Number of frames
n
Dell Inspiron laptop with Intel’s i5 core processor and 64 bit operating system has been used for the entire experimentation. This work is distributed into the following sections: Section 2 gives the overall framework of CS; Section 3 provides the proposed NIPIRA and proof of RIP, accuracy and optimality of NIPIRA; Section 4 describes the results and discussions; Section 5 provides the conclusion and future work of the proposed algorithm. 2. Compressed sensing (CS) framework With increase in the flow of data for high end applications, Nyquist rate also increases leading to increase in the sampled data [17]. This further leads to impossibility in physical implementation, high cost variations and complexity and large storage requirements. With the genesis of CS, the number of samples and hence the storage requirements and the complexity are reduced. The main requirements for acceptable recovery performance are that the input must be sparse [6] i.e. the input must have a large number of zeros, and the sensing matrix should satisfy the restricted isometry property (RIP) [8]. The concept of CS can be explained briefly as follows: instead of acquiring the data and then compressing it, as done in traditional compression methods, the compressed form of the data can be directly acquired in 3
the acquisition process itself. Consider the input image as such that , where R is a region with N dimensions. Let be the basis function applied to the input for sparsification and let be the sensing matrix which senses the appropriate information from the sparsified input and projects the same to a smaller dimensional vector called measurement vector y of size . This process can be mathematically figured as in equation (1) [18].
=
(1)
or
= , where =
(2)
This can also be represented as = , where is the sparsified signal. The sparsity constraint [19] that is to be induced in the input vector is expressed as in equation (3). = min‖‖ or : ‖ ‖ ≤
(3)
where, ‖ ‖ is the norm of that indicates the number of non-zero elements in the vector and indicates the sparsity. The sparsity constraint, which represents the real time CS acquisition process, is forcedly induced in simulative environments by the basis function . After sparsification, the sensing matrix collects the important elements from sparsified vector to form the measurement matrix . At the receiver, the estimate of the input data can be obtained by applying a suitable recovery algorithm. The algorithm must be robust enough to reconstruct the image data perfectly with the least number of measurements available. Smaller the number of measurements required for perfect recovery, smaller is the computational complexity and time consumption. Greedy algorithms like StOMP, CoSaMP, ROMP, OMP etc. are iterative in nature and find the perfect match by solving least squares problem (LSP). Hence their performance is stringed with the disadvantages like increase in time consumption and complexity. Moreover, failure to find the best match through erroneous selection of the
4
measuring combination would result in imperfect reconstruction or large error. The results of these algorithms are unstable since they use random matrix as sensing matrix. To overcome the above said shortcomings, stable sensing matrix is designed which paves the way for noniterative recovery algorithm. This work proposes a non-iterative recovery algorithm using pseudo inverse (NIPIRA) for reconstruction of images and videos. NIPIRA satisfies the reconstruction requirements practically and theoretically as explained in detail in section 3. 3. NIPIRA for CS based Image and Video Reconstruction The designer of the recovery algorithm has the sole right to design the sensing matrix that would aid in the proper recovery process. Thus the design of the sensing matrix plays a major role in the development of the recovery algorithm [17]. Random matrices are seemingly robust in sensing but they are tailed with disadvantages like inconsistency, large time consumption etc. Hence a stable measurement matrix called augmented matrix is generated, which does not change its elemental values for every execution. As mentioned in section 2, there are two main properties that the measurement matrix must follow for perfect reconstruction of images and videos. They are the following: the input must be in sparsified [6] form and the sensing matrix must satisfy the restricted isometry property (RIP) [8, 20]. The sparsification of input information is carried out by applying a basis function and a threshold operator to the input. RIP states that if there is a measurement matrix ϕ of size × , there exists a constant ( ∈ 0,1$, such that 1 − $‖ ‖&& ≤ ‖ ‖&& ≤ 1 + $‖ ‖&&, ∀ ∈
(4)
where, is the sparse vector. It is to be noted that the proposed stable augmented matrix completely abides by RIP. Augmented matrix is the combination of identity matrix augmented with zero matrix as given by equation (5). = )*+ = ,-×- ∪ /-× 5
0-$
(5)
where, )*+ is the augmented sensing matrix, ,-×- is the identity matrix of size × , ∪ is the augmentation operator and /-×
0-$
is the zero matrix of size × − $. The
augmentation of zero matrix further reduces the number of sparse values to be selected, thereby providing greater compression. The proposed matrix is shown as in Fig. 1.
)*+
1 30 20 2 0 =2 20 20 20 10
0 1 ⋱ ⋱ ⋱ ⋱ ⋱ 0
0 ⋱ 1 ⋱ ⋱ ⋱ ⋱ 0
0 ⋱ ⋱ 1 ⋱ ⋱ ⋱ 0
0 ⋱ ⋱ ⋱ 1 ⋱ ⋱ 0
0 ⋱ ⋱ ⋱ ⋱ 1 ⋱ 0
0 ⋱ ⋱ ⋱ ⋱ ⋱ 1 0
Identity matrix ×
0 0 0 0 0 0 0 1
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
… … ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ … …
… ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ …
0 08 07 7 07 07 07 07 06
Zero matrix × − $
Fig. 1 Pictorial representation of the augmented matrix
The augmented zero matrix allows limited selection of the elements from the sparse input vector, avoiding the necessity of solving the least squares problem (LSP). The augmented matrix that abides by the RIP, selects only the appropriate sparse values from the sparsified input signal as measurements with the help of its leading diagonal, avoiding iterations. It is to be noted that the popular iterative procedures use random matrices that generate a number of equations which are less than number of unknowns, requiring iterative solving procedures. The proposed non-iterative pseudo inverse based recovery algorithm (NIPIRA) is given in Table 2. Table 2 Non-Iterative Pseudo Inverse based Recovery Algorithm (NIPIRA)
NIPIRA Algorithm Input:
, , Initialisations: Г=∅ ;ℎ=> = ∅ = ?@A B, 1C
6
Algorithm:
D = 10 AE ∑| |$JHJI& K L = MNAE |O $|$PQ R = | S | + D ∀ = ∈ 1, $, where = I& is the block size if RT > L Г = Г ∪ = ;ℎ=> = ;ℎ=> ∪ T V= = ;ℎ=> W , where † denotes the pseudo inverse V=Г = V= @> = V=Г ∪ ?@A I& − $, where is the number of rows in @> = =DY> @>$
CS procedure can be comfortably applied for compression and recovery only of the input signal is sparse. Many signals are naturally sparse while some are not. Hence, at the transmitter, sparsification of the non-sparse signals is attained by applying a basis function . Generally, transform functions such as discrete cosine transform (DCT) [21], discrete wavelet transform (DWT), etc. are some of the prominent basis functions available for sparsifying the input signal. The measurement matrix ( × ) is then applied to the sparsified input vector ( × 1), which extracts high priority information with the help of its leading diagonal to form the measurement vector ( × 1). Once is formed, it is transmitted through the wireless channel to the receiver along with the and , where is the sparsity of the input signal. Since the input image or video information of size × 1 is dimensionally projected to a smaller sized measurement vector of size × 1, the storage requirements, traffic factors and other transmission based parameters are lightly loaded. The receiver receives the transmitted , and and with the help of NIPIRA, estimates the input image information perfectly with the least number of measurements, small elapsed time and with failure proof recovery. The NIPIRA algorithm first finds the add-constant D and the threshold L which are given by equations 6 and 7.
7
D = 10 AE ∑| |$JHJI& K
(6)
L = MNAE |O $|$PQ
(7)
The add-constant D extracts the log value of summed up measurements divided by sparsity and block size I. The add-constant boosts up the pixel quality after the first intermediate reconstruction since D depends upon that has the information about the compressed pixel values. The calculated add-constant is then added to the intermediate reconstruction quantity R = | S |. The threshold L is the log of maximum value present in the measurement vector. The threshold value can be set as per the user’s need. All the elements of the intermediate result R that are greater than L, are selected for further processing and the other elements which do not hold for L are eliminated. This thresholding process at the receiver end will further reduce the load to the upcoming recovery process of finding the second intermediate since some more of the unimportant elements are eliminated. The support of those elements that are greater than L are found and they are augmented to the null vector Г. In the same way, the corresponding columns of the measurement matrix , that are included in the support vector Г, are augmented to form the ;ℎ=> matrix. Formation of ;ℎ=>, where the position of the matrix column corresponds to the position of the best intermediate term ensures easy recovery. The newly formed ;ℎ=> has the similar properties of )*+ , since their columns are the same and only the positions of the columns are dependent upon the position of R T > L. Once ;ℎ=> is ready, the pseudo inverse of ;ℎ=> and are multiplied, which forms the second intermediate vector. The values of this intermediator are arranged in accordance with the indexes provided by the support vector Г. The values in the vector V=Г are then applied with the inverse basis function to give the estimated input vector . Thus without the usage of iterations, NIPIRA chooses indices of the best elements with the help of L. Simple arithmetic 8
operations are used for calculation of the estimated input vector with small time consumption. Since the leading diagonal of augmented matrix chooses the appropriate elements in the sparse vector, the process of finding the best indices does not make a wrong turn thus avoiding failures in reconstruction. NIPIRA is thus suitable for image and video reconstruction since it has no iterations and hence less complexity and time consumption. Besides exhibiting better performance in script based experimentations, NIPIRA showcases its performance theoretically as well. The mathematical proof of NIPIRA algorithm can be done by proving the RIP for augmented matrix and deriving the accuracy. 3.1 Mathematical proof for NIPIRA Any algorithm must be proved both practically and theoretically. Theoretical performance may be calculated approximately with mathematical proof of various parameters. For proper reconstruction, RIP must be satisfied by the sensing matrix, whose theoretical proof is derived. Accuracy, can be considered as the ability of the algorithm to perfectly reconstruct the image or video data with minimum error. Certain lemmas are considered to establish isometry property and accuracy as follows: Lemma 1 Let be a matrix of size × which follows the concentration inequality as follows [8]: ; |‖ ‖&& − ‖ ‖&& | ≥ [‖ ‖&& $ ≤ 2@ 0]^_
,0<[<1
`$
where [ is a constant. Then, for any set b, with # b$ = d < and any 0 < < 1, we have, 1 − $‖ ‖& ≤ ‖ ‖& ≤ 1 + $‖ ‖& ∀ ∈ . Lemma 2 If satisfies RIP [22], ‖ ‖& ≤ e1 + ‖ ‖&, ∀ : ‖ ‖ ≤ , then ‖ ‖& ≤ e1 + ‖ ‖& + e1 + 9
‖ ‖E √
3.1.1 Restricted Isometry Property (RIP) Perfect reconstruction of the original data, requires satisfaction of RIP. RIP can be mathematically expressed as in equation (4): 1 − $‖ ‖&& ≤ ‖ ‖&& ≤ 1 + $‖ ‖&&, ∀ ∈ . Proof In this case, it is sufficient to prove ‖ ‖& = 1, since ϕ is linear. Let a set of points b be considered such that b ∈ , ‖‖& = 1 for all ∈ b
. For all ∈ , with ‖‖& = 1,
we have min∈g ‖ − b ‖& ≤ J4 (8) The probability that the term ‖ ‖& is concentrated about its expected value and can be expressed as in Lemma 1. Considering Lemma 1 to a set b with [ = J2, we have, H1 − J2K‖‖&& ≤ ‖ ‖&& ≤ H1 + J2K‖‖&&, ∀ ∈ b
(9)
H1 − J2K‖‖& ≤ ‖ ‖& ≤ H1 + J2K‖‖& ∀ ∈ b
(10)
This obviously states,
Now, let us consider a small constant i such that ‖ ‖& ≤ 1 + i$‖ ‖&, ∀ ∈
(11)
If it is proved that i < , then it obviously states that ‖ ‖& ≤ 1 + $‖ ‖& . For any ∈ , with ‖ ‖& = 1, a is selected such that ‖ − ‖& ≤ J4 with reference to equation (8). Thus, ‖ ‖& can be written as
10
(12)
‖ ‖& ≤ ‖ ‖& + ‖ − $‖&
(13)
Substituting equations (10) and (12) in (13), and equating the corresponding terms, we get, 1+ J2 + j 1 + i$ J4k i is known to be the smallest number for equation 11 to be true. Therefore, i ≤ 1 + J2 + j 1 + i$ J4k ≤ 1 + J2 + J4 + iH J4K iH1 − J4K ≤ 1 + 3 J4 Equating the i and terms on both sides, we get, i≤
3 J 4 ≤ 1 − J4
Thus, i ≤ , which proves the upper inequality of Lemma 1. The lower inequality can be proved in a similar fashion. Hence, RIP of NIPIRA is proved. 3.1.2 Accuracy Theorem Given a noisy observation, = + @, where is the input vector. Let be sparse representation to . If ϕ follows RIP, then the algorithm will recover an approximation satisfying, E
‖ − ‖& ≤ m − m + &
&
E
√
‖ − ‖E + ‖@‖&
(14)
Proof If ϕ follows RIP [13], with d iterations, the algorithm will recover the input, which satisfies ‖ − ‖& ≤ 20n ‖ ‖& + [̃ where,
[̃ = ‖ − ‖& + 11
E
√
‖ − ‖E + ‖@‖&
(15) (16)
NIPIRA is non iterative, thus d = 1. Therefore, ‖ − ‖& ≤ 20E ‖ ‖& + [̃ Substituting for [̃ , we get, ‖ − ‖& ≤ 20E ‖ ‖& + ‖ − ‖& +
E
√
(17)
‖ − ‖E + ‖@‖& (18)
Separating the norms and rearranging the like terms together, we obtain, E
‖ − ‖& ≤ ‖ ‖& + ‖ ‖& − ‖ ‖& + & E
‖ − ‖& ≤ m − m + & &
E
√
E
√
‖ ‖E −
E
√
‖ ‖E + ‖@‖&
(19)
‖ − ‖E + ‖@‖&
(20)
Hence the difference between the input vector and the estimated vector is less than the sum of first and the second norms of the differences between the input and the sparsified vectors. 4. Results and Discussions NIPIRA has been designed mainly to reconstruct the images and video frames with the least time consumption and utmost perfection. NIPIRA proves itself worthy in the above said regards when applied for the stated purpose. The images considered for the performance evaluation are of uncompressed TIFF format while the videos are of CIF format. The test images taken for experimentation are Lena, Airplane F-16 [23] and the test videos are Akiyo, and Mother – Daughter [24] in uncompressed TIFF and CIF formats respectively. Applying the DCT for the whole image is time and energy consuming since DCT depends upon the input matrix size. Hence the entire image is divided into smaller blocks and DCT is applied for each block such that total time consumption is reduced due to smaller matrix size. The size of the sensing matrix also depends upon the size of the block and hence the complexity induced due to multiplication of sensing matrix and sparse vector is also reduced. The images and video frames are divided into blocks of sizes 4x4, 8x8 and 16x16 and each block gets the application of the algorithms. The blocks are non-overlapping in order to avoid introduction of time complexity in this regard. The ⁄ ratio, which gives the number of measurements
12
selected over the available input , is maintained standard for all block sizes; the selected constant ⁄ ratios are 0.1875, 0.3125, 0.4375 and 0.5625 respectively. All algorithms, including NIPIRA and other comparing algorithms, have been designed as MATLAB scripts in MATLAB 2014a. For benchmarking the results of NIPIRA, the same is compared with those of other iterative algorithms like OMP, StOMP, LASSO and TMSBL. OMP and StOMP are chosen for comparison with NIPIRA as they are well known as complex iterative greedy algorithms of small magnitude and comparing NIPIRA with them would bring the performance of the proposed algorithm into limelight. TMSBL and LASSO that depend upon the temporal information available in the data and are more productive than OMP and StOMP. Hence they are also taken as algorithmic comparison metric. The parameters like number of measurements , sparsity etc., for all the algorithms are the same so that comparison among them would provide better revelation. The measurement matrix applied for OMP and StOMP is the traditional ransom Gaussian matrix and the same for NIPIRA is the proposed augmented matrix. The performance of an algorithm can be tested in two ways: 1) by evaluating the performance of the results generated by the algorithm. NIPIRA is tested on this basis in terms of peak signal to noise ratio (PSNR), mean square error (MSE), structural similarity (SSIM), maximum difference (MD), normalised absolute error (NAE) as the objective measures [25 – 27]; 2) by testing the performance of the algorithm itself, which is done by calculating the elapsed time, runtime [28] and the algorithmic complexity [29, 30]. 4.1 Evaluation of NIPIRA by Objective measures Objective measures are those that evaluate the quality of the reconstructed images and videos with precision since they perform pixel level quantification of the images and videos under consideration. PSNR is considered to be the most relevant quality metric when it
13
comes to evaluating the quality of images and videos. Hence the PSNR for each example image and video frame, with varying block sizes and number of measurements is calculated for OMP, StOMP, LASSO, TMSBL and NIPIRA and compared. Fig. 2 shows comparison of PSNR of NIPIRA for images Lena and Airplane F-16, with all the above mentioned algorithms, considering block size to be 8x8 and = 20.
a) Lena
b) OMP, PSNR = 23.5500 dB
c) StOMP, PSNR d) LASSO, PSNR = 20.6382 dB = 24.4353 dB
e) TMSBL, PSNR = 24.0547 dB
f) NIPIRA, PSNR = 31.6173 dB
g) Airplane F-16
h) OMP, PSNR = 20.7029 dB
i) StOMP, PSNR = 16.1586 dB
j) LASSO, PSNR k) TMSBL, PSNR = 22.2419 dB = 22.5375 dB
l) NIPIRA, PSNR = 28.1711 dB
Fig. 2 Comparison of perceptual quality of the images reconstructed by NIPIRA with OMP, StOMP, LASSO and TMSBL: a) and g) are the original input images; b) and h) are the images reconstructed by OMP; c) and i) are the images reconstructed by StOMP; d) and j) are the images reconstructed by LASSO; e) and k) are the images reconstructed by TMSBL; f) and l) are the images reconstructed by NIPIRA. It is evident that NIPIRA outperforms the other two algorithms in perceptual quality and PSNR (~30 dB) value.
Considering the test images in Fig. 2, the approximate average PSNRs of OMP, StOMP, LASSO, TMSBL and NIPIRA are 24 dB, 19 dB, 23 dB, 23 dB and 30 dB respectively. NIPIRA gives minimum increase in PSNR of 7.2 dB and maximum increase of 10.99 dB against LASSO and StOMP (for Lena) and their corresponding percentage increases are 29.4655% and 53.2507% respectively. These percentages in increase provide evidence that NIPIRA is ideally suited for reconstruction of images compared to all the algorithms considered for comparison. The best among the iterative algorithms is either LASSO or TMSBL, but NIPIRA outperforms them with respect to PSNR.
14
A similar comparison is suitable for video frames. The average PSNR values of the video sequences (averages over both the videos) considered in Fig. 3 that are reconstructed by OMP, StOMP, LASSO, TMSBL and NIPIRA are 25.6106 dB, 19.4065 dB, 28.381 dB, 29.1163 dB and 34.7952 dB respectively. The corresponding percentage increase in PSNR exhibited by NIPIRA compared to OMP, StOMP, LASSO and TMSBL are is 39.53%, 72.3097%, 38.9383% and 37.8403% (considering Akiyo sequence). This proves better reconstruction of NIPIRA when compared to above mentioned algorithms with the highest PSNR for = 20. It is to be noted that the performance of iterative algorithms in terms of PSNR is dependent upon the random sensing matrix and the number of iterations. Hence the PSNR values obtained for the same test images and videos of Fig. 2 and Fig. 3 can vary with number of executions of the script. This instability is avoided in NIPIRA since it uses a fixed augmented sensing matrix. Irrespective of the number of executions, the PSNR remains constant thus leading to stability and consistency in reconstruction.
a) Akiyo
b) OMP, PSNR = 27.2150 dB
c) StOMP, PSNR = 22.0377 dB
e) TMSBL, PSNR = 27.5486 dB
f) NIPIRA, PSNR = 37.9731 dB
15
d) LASSO, PSNR = 27.3309 dB
g) Mother-daughter
h) OMP, PSNR = 24.0062 dB
i) StOMP, PSNR 16.7753 = dB
k) TMSBL, PSNR = 30.6841 dB
l) NIPIRA, PSNR = 31.6173 dB
j) LASSO, PSNR = 29.4311 dB
Fig. 3 Comparison of perceptual quality of video frames reconstructed by NIPIRA with OMP, StOMP, LASSO and TMSBL: a) and g) are the original input video frames (frame No. 4); b) and h) are the video frames reconstructed by OMP; c) and i) are the video frames reconstructed by StOMP; d) and j) are the video frames reconstructed by LASSO; e) and k) are the video frames reconstructed by TMSBL; f) and l) are the video frames reconstructed by NIPIRA. It is evident that NIPIRA outperforms the other two algorithms in perceptual quality and PSNR value.
Besides PSNR there are other objective quality measuring metrics like MSE, SSIM, MD and NAE. These parameters add to the pixel level details whether the images and videos are reconstructed with preservation of pixel level properties. MSE and NAE represent the error in reconstruction, MD gives the maximum deviation from original pixel values to reconstructed pixel values and SSIM gives the structural similarity in the reconstructed image. Tables 3 and 4 give the comparison of all the above said parameters for NIPIRA and iterative algorithms for image (Lena) and video (Akiyo). Table 3 Comparison of various objective parameters for images reconstructed by iterative algorithms and NIPIRA
Algorithm
PSNR
MSE
SSIM
MD
NAE
OMP StOMP LASSO TMSBL NIPIRA
23.5500 20.6382 24.4353 24.0547 31.6173
287.1317 561.3801 234.1799 255.6291 44.8871
0.7117 0.6097 0.7296 0.7320 0.9301
221 222 184 209 74
0.6877 x 10^-6 4.6820 x 10^-6 2.0448 x 10^-4 7.9881 x 10^-5 3.2009 x 10^-8
Table 4 Comparison of various objective parameters for video frames reconstructed by iterative algorithms and NIPIRA
16
Algorithm
PSNR
MSE
SSIM
MD
NAE
OMP StOMP LASSO TMSBL NIPIRA
27.2150 22.0377 27.3309 27.5486 34.9119
123.5321 406.7351 120.2227 114.3631 20.9844
0.8307 0.6427 0.8437 0.8610 0.9558
216.75 247 160 247 78
4.0839 x 10^-5 5.9950 x 10^-6 2.6201 x 10^-4 1.5335 x 10^-5 2.5979 x 10^-8
The number of frames considered for experimentation is n = 4 and thus all the values in the Table 4 has been obtained by averaging them out by the number of frames. From Table 4 it can be observed that, the MSE is substantially reduced to 44.8871 which is a nearly 84% decrease from the same of OMP and 92% decrease from StOMP. The structural similarity retained by NIPIRA is above 93% which has not been achieved by any of the algorithms considered for experimentation. The maximum deviation in pixel value created by NIPIRA is 74 while the values obtained by other algorithms are above 100 which shows the lenient reconstruction performance of other algorithms. The MD reaches a maximum of around 560 in case of StOMP. The error in the reconstruction of pixel values using NIPIRA is at least 100 times smaller than in case of OMP, StOMP, LASSO and TMSBL. 4.1.1 Performance comparison for various measurement values: So far the number of measurements considered for experimentation and evaluation is = 20 which accounts for about 31.25% of input information. All the recovery algorithms vary their performance vis-a-vis increase or decrease in the number of measurements. Hence a comparison between the same parameters obtained by the same algorithms but for varying number of measurements is carried out. The block size is also an important factor since it influences the objective measures and number of measurements which in turn results in alterations of reconstruction quality. Hence the block sizes and the number of measurements were varied simultaneously for every algorithm. Table 5 shows the performance metrics for varying block sizes and number of measurements for NIPIRA.
17
Table 5 Performance of NIPIRA for different block sizes and measurements. Test image is Lena in TIFF format.
Block size
4x4
8x8
16 x 16
M
PSNR (dB)
MSE
SSIM
MD
NAE
3 5 7 9 12 20 28 36 48 80 112 144
27.2603 29.2003 33.0816 35.6003 27.9900 31.6173 34.1508 36.8908 27.6597 31.0026 33.6776 35.5745
122.1935 78.1721 31.9833 17.9081 103.2943 44.8871 25.0032 13.3045 111.4568 51.6198 27.8819 18.0148
0.8458 0.8923 0.9539 0.9728 0.8519 0.9301 0.9590 0.9766 0.8383 0.9151 0.9514 0.9601
115 108 78 52 107 74 67 43 102 96 76 42
5.1907 x 10^-8 1.0034 x 10^-8 1.8167 x 10^-8 8.2186 x 10^-8 1.0468 x 10^-7 3.2009 x 10^-8 5.1907 x 10^-9 2.5088 x 10^-8 2.5088 x 10^-8 5.5367 x 10^-8 8.672 x 10^-9 3.3739 x 10^-8
The ratio of input information measured to the number of information in the selected block is taken in four levels, i.e.
= 0.1875, 0.3125, 0.4375 and 0.5625, corresponding to
= 3, 5, 7, 9 when block size is 4x4 and the same ratio prevails for all the block sizes. Table 5 shows that, with increase in the number of measurements, the performance with respect to PSNR also increases and so does the other objective parameters. With increase in number of measurements, the maximum deviation in the pixel decreases to a greater extend for each block size. The MSE also shows dramatic decrease. The same parameters can be calculated for the video frames too for the same block sizes and number of measurements. Fig. 4 gives the comparative performance in PSNR of NIPIRA and other algorithms under consideration for various number of measurements for images.
18
OMP
StOMP
TMSBL
NIPIRA
LASSO
35
PSNR in dB
PSNR in dB
40
30 25 20 15 10 3
5
7
9
Number of measurements ()
Number of measurements () b) PSNR vs for 8 x 8 block size (Lena)
for 4 x LASSO a) PSNR vsStOMP 4 block size OMP (Lena) TMSBL NIPIRA 40 35 30 25 20 15 10 12
20
28
36
OMP
StOMP
TMSBL
NIPIRA
LASSO
40
PSNR in dB
35 30 25 20 15 10
48
80
112
144
Number of measurements ()
c) PSNR vs for 16 x 16 block size (Lena) Fig. 4 Performance in terms of PSNR using OMP, StOMP, LASSO, TMSBL and NIPIRA for various block sizes and number of measurements for image Lena: a) Block size is 4 x 4, M = 3, 5, 7 and 9, b) Block size is 8 x 8, M = 12, 20, 28 and 36, c) Block size is 16 x 16, M = 48, 80, 112 and 144.
19
Close observation of Fig. 4 reveals the highest PSNR obtained by NIPIRA i.e. 41 dB as not attained by the other algorithms even with highest number of measurements. The least PSNR obtained by NIPIRA (27.2603 dB, block size = 4x4) using the lowest number of measurements ( = 3) is achieved by other algorithms, only with larger number of measurements. It can be noted that irrespective of block sizes and number of measurements, NIPIRA always outperforms OMP, StOMP, LASSO and TMSBL in terms of various objective measures with the highest PSNR and minimum error. The same performance is observed for videos while using all the algorithms with varying number of measurements. NIPIRA gives comparatively higher PSNR than other algorithms for any number of measurements and block size. 4.2 Algorithmic evaluation The second method of evaluation of algorithms is by checking the performance of the algorithm itself. Three parameters are considered for the said purpose, they are: elapsed time [28], runtime and complexity. In this work, elapsed time is considered as the time taken for the algorithm to reconstruct a single block of the image or video from the input measurements. The runtime denotes the total time consumption of the algorithm including compression as well as reconstruction. The algorithmic complexity reports the complex operations carried out by the algorithm in the process of reconstructing the pixel values when the size of the input tends to infinity. 4.2.1 Elapsed time Elapsed time is the time taken by the algorithm to reconstruct a single block in the image or video frame. The smaller the elapsed time, the smaller the complexity and total runtime, which naturally leads to minimised hardware burden, which ultimately increases the longevity of the processing nodes in the network. Elapsed time comparison of NIPIRA, OMP, StOMP, LASSO and TMSBL is carried out for each block size and various number of 20
measurements. The comparative elapsed times for images and videos with 8 x 8 block and = 20, n = 4 is shown by Table 6. Table 6 Elapsed time (ms) comparison between NIPIRA and iterative algorithms, 8 x 8 block size, = 20, n = 4.
Elapsed Time (ms) Algorithms OMP StOMP LASSO TMSBL NIPIRA
Image (Lena) 10.5 19.9 541.3 65.1 0.037323
Video (Akiyo) 18.4 14.9 244.5 55.6 1.1
The elapsed times of the iterative algorithms are at least 100 times greater than NIPIRA for images, since they have random sensing matrices that demand iterative process of finding the best match. Since NIPIRA uses augmented sensing matrix which selects the appropriate measurements with the help of its leading diagonals, there is no need for iterative procedure of finding the best match and hence the elapsed time is reduced to a range of µs. NIPIRA shows the lowest elapsed time of 0.037323 ms and LASSO exhibits highest elapsed time of 541 ms. For videos too, there is a notable 100 fold decrease in elapsed time exhibited by NIPIRA compared to other algorithms. Fig. 5 gives the elapsed time comparison obtained for the reconstruction of images and videos in a visual perspective.
21
StOMP
LASSO
TMSBL
NIPIRA
Elapsed Time (log scale)
Elapsed Time (log scale)
OMP
1000 100 10 1 12
20
28
36
0.1 0.01 Number of measurements ()
Number of measurements ()
b) Elapsed Time vs for 8 x 8 block size (Akiyo)
a) Elapsed Time vs for LASSO 8 x 8 block size OMP StOMP (Lena) TMSBL NIPIRA 1000 100 10 1 12
20
28
36
0.1 0.01
Fig. 5 Elapsed time comparison of OMP, StOMP, LASSO, TMSBL and NIPIRA for various measurements: a) Lena, b) Akiyo
The elapsed time is represented as logarithmic values of base 10 in order to get a good perceptual comparison. Similar comparisons are made for block sizes of 4 x 4 and 16 x 16 with varying number of measurements. In all the cases, NIPIRA showed at least 10 times decrease in elapsed time. 4.2.2 Total runtime Decrease in elapsed time accounts for reduction of total runtime. As mentioned earlier, total runtime is the time taken for the entire compression-reconstruction process. Fig. 6 indicates the comparison of runtimes of various algorithms with NIPIRA. NIPIRA exhibits a small runtime of around 3.9541 s for image and 29.6993 s for video (n = 4). The figures are plot in log scale for easy understanding of the vast difference in time consumption.
22
StOMP
LASSO
TMSBL
NIPIRA
Total Runtime (log scale)
Total Runtime (log scale)
OMP
1000 100 10 1 12
20
28
36
Number of measurements ()
Number of measurements ()
a) Elapsed Time vs for LASSO 8 x 8 block size OMP StOMP
a) Elapsed Time vs for 8 x 8 block size
TMSBL
NIPIRA
10000 1000 100 10 1 12
20
28
36
Fig. 6 Total runtime comparison of OMP, StOMP, LASSO, TMSBL and NIPIRA for various measurements: a) Lena, b) Akiyo
4.2.3 Complexity Operational complexity is the contribution of each mathematical operation in an algorithm. It can also be described as the complex behaviour exhibited by the algorithm when the input size tends to infinity, accounts for the complexity of the algorithm. The mathematical operations and the number of occurrences of these operations in the algorithm are given in Table 7. Table 7 Mathematical operations in the algorithm and their contribution towards complexity
Mathematical operations
Number of occurrences
Addition Subtraction Summation Squaring
N(M-1)+1 1 M 1
23
Matrix Multiplication Augmentation Maximum operation Determinant Pseudo Inverse Transpose
NM N 1 3 M NM
Adding the contributions of all the various operations in the algorithm, we get 3 + 2 + 7. But when → ∞, the term making a substantial contribution will be and other terms can be eliminated since ≪ . Also, > which indicates that the first term is times larger than the second term. Thus, when the input size gets close to infinity, complexity of NIPIRA can be given by u $, where u represents the order. The complexity for OMP and StOMP is generally represented as u $ [31, 10, 11] where is the sparsity or a fixed number of iterations. This shows that since there are no iterations in NIPIRA, the complexity of the algorithm is reduced by times with respect to greedy algorithms. LASSO and TMSBL have a fixed number of iterations that the user can select according to his/her utility. If ℎ is a common representation of the number of iterations of both greedy algorithms (OMP, StOMP) and the temporal information based algorithms (LASSO, TMSBL), then complexity of NIPIRA is ℎ times less than any of the algorithms u $ ≪ u ℎ$$. All the above experiments prove that NIPIRA is best suited for applications requiring faster recovery process and higher clarity. 5 Conclusions Compressed sensing overrules the traditional compression techniques that follow Nyquist criterion for perfect reconstruction. It projects the input image or video information to ≪ dimensional measurements which can be recovered through perfect reconstruction algorithms at the receiver. Most of the present greedy algorithms are iterative and highly time consuming and inconsistent. In order to address these issues, Non-Iterative Pseudo Inverse
24
based Recovery Algorithm (NIPIRA) is proposed which uses augmented matrix as the sensing matrix to project the input data into measurements. Augmented matrix maintains the stability of the algorithm by providing consistent results for any number of executions. The PSNR obtained by NIPIRA is around 32 dB for just 31.25% of the input data. NIPIRA exhibits SSIM of above 92% and has minimum pixel level deviations compared to popular iterative algorithms. Since iterations are eliminated in NIPIRA, the elapsed time and runtime obtained are 0.037 ms and 3.9541 s respectively for images, which are at least 100 times smaller compared to LASSO and TMSBL. The complexity of NIPIRA is u $ which is ℎ times less than iterative algorithms whose complexities are u ℎ$. These results project NIPIRA to be best suited for reconstruction of compressively sensed images and videos that are dimensionally diminished into measurements by the use of process of compressed sensing. In future, NIPIRA can be applied for reconstruction of surveillance applications and streaming of real time videos. References [1]
Dana M. Compressed Sensing Makes Every Pixel Count. In: Proc. of J What Happens in Math Sci, 7 (2009), pp. 114-127.
[2]
R. Masiero, G. Quer, M. Rossi and M. Zorzi. A Bayesian analysis of Compressive Sensing data recovery in Wireless Sensor Networks. In: Proc. of Int Conf on Ultra-Modern Telecomm & Workshops, Oct 12-14, 2009, pp.1-6.
[3]
E. Candes, J. Romberg, and T. Tao. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans on Info Theory, 52 (2) (2006), pp. 489–509.
[4]
David L Donoho. Compressed sensing. IEEE Trans on Info Theory, 52 (4) (2006), pp. 1289–1306.
[5]
I. Drori. Compressed Video Sensing. In: BMVA Symposium on 3D Video - Analysis, Display, and Applications, 2008.
[6]
Vishal M. Patel, Ramalingam Chellappa. Sparse Representation and Compressive Sensing for Imaging and Vision. Springer, 2013.
[7]
M. Fazel, E. Candes, B. Recht, P. Parrilo. Compressed sensing and robust recovery of low rank matrices. In: 42nd Asilomar Conf on Sig, Sys and Computers. pp. 1043-1047, 26-29 Oct. 2008.
[8]
E. Candes. The Restricted Isometry Property and its implications. J Comptes Rendus Mathematique, 346 (9-10) (2008), pp. 589 – 592.
25
[9]
T. Tony Cai and Lie Wang. Orthogonal Matching Pursuit for Sparse Signal Recovery with Noise. IEEE Trans on Info Theory, 57 (7) (2011), pp. 4680-4688.
[10]
D. L. Donoho, Y. Tsaig, I. Drori and J.-L. Starck. Sparse Solution of Underdetermined Linear Equations by Stagewise Orthogonal Matching Pursuit (StOMP). IEEE Trans on Info Theory. 58(2), pp. 1094-1121, 2012.
[11]
Liu Zhi-xue, Li Gang, Zhang Hao, and Wang Xi-qin. Sparse-Driven SAR Imaging Using MMVStOMP. In: Proc. of 1st Int Workshop on Compressed Sensing applied to Radar, IEEE press, May 14 – 16, 2012.
[12]
D. Needell and J.A. Tropp. CoSaMP: Iterative signal recovery from incomplete and inaccurate sample. J of App and Comput Analysis, 26 (3) (2009), pp. 301-321.
[13]
T. Blumensath, Mike E. Davies. Iterative hard thresholding for compressed sensing. J of Applied and Computational Harmonic Analysis, Elsevier, 27 (3) (2009), pp. 265–274.
[14]
Nowak, Robert D., and Stephen J. Wright. "Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems." IEEE Journal of selected topics in signal processing 1.4 (2007): 586-597.
[15]
Zhang, Zhilin. "Comparison of Sparse Signal Recovery Algorithms with Highly Coherent Dictionary Matrices: The Advantage of T-MSBL." Research note (2012).
[16]
Zdunek, Rafal, and Andrzej Cichocki. "Improved M-FOCUSS algorithm with overlapping blocks for locally smooth sparse signals." IEEE Transactions on Signal Processing 56.10 (2008): 4752-4761.
[17]
Yonina C. Eldar, Gitta Kutyniok. Compressed Sensing: Theory and Applications. Cambridge University Press, 2012.
[18]
G. Kutyniok. Theory and applications of compressed sensing. arXiv preprint arXiv:1203.3815, 2012. http://www.math.tuberlin.de/fileadmin/i26_fgkutyniok/Kutyniok/Papers/SurveyCompressedSensing_R evision.pdf
[19]
Beck, Amir, and Yonina C. Eldar. "Sparsity constrained nonlinear optimization: Optimality conditions and algorithms." SIAM Journal on Optimization 23.3 (2013): 1480-1509.
[20]
R. Baraniuk, M. Davenport, R. DeVore, and M. Wakin. A simple proof of the restricted isometry property for random matrices. Const App. 28 (3), pp. 253 – 263.
[21]
Mun, Sungkwang, and James E. Fowler. "Block compressed sensing of images using directional transforms." 2009 16th IEEE international conference on image processing (ICIP). IEEE, 2009.
[22]
J. Florence Gnana Poovathy, S. Radha. Non-Iterative Threshold based Recovery Algorithm (NITRA) for Compressively Sensed Images and Videos. KSII Trans on Int and Info Sys, 9 (10) (2015), pp. 4160 – 4176.
[23]
http://sipi.usc.edu/database/database.php?volume=misc accessed on December 2015.
[24]
http://see.xidian.edu.cn/vipsl/database_Video.html accessed on November 2014.
[25]
M. Mrak, S. Grgic, M. Grgic. Picture quality measures in image systems. EUROCON 2003. Computer as a Tool, The IEEE, 1 (2003), pp.233-236.
[26]
T. Kratochvil & P. Simicek. Utilization of MATLAB for Picture Quality Evaluation. Institute of Radio Electronics, Brno University of Technology, Brno.
[27]
Lauterjung, J. Picture quality measurement. In: International Broadcasting Convention, Amsterdam, 11-15 September 1998. IEE, London, 1998. pp. 413-417.
26
compression
[28]
Martin Knapp-Cordes and Bill McKeeman. Improvements to tic and toc Functions for Measuring Absolute Elapsed Time Performance in MATLAB. Matlab Digest.
[29]
Frank Hutter, Lin Xu, Holger H. Hoos and Kevin Leyton-Brown. Algorithm Runtime Prediction: Methods and Evaluation. Artificial Intelligence. 206 (2014), pp. 79 – 111.
[30]
Sandeep Sen. Lecture Notes for Algorithm Analysis and Design. November 6, 2013. http://www.cse.iitd.ernet.in/~ssen/csl356/root.pdf
[31]
Bob L. Sturm and Mads Græsbøll Christensen. Comparison of Orthogonal Matching Pursuit Implementations. 20th European Signal Processing Conference (EUSIPCO 2012). Romania, August 27 - 31, 2012.
27
S1434-8411(16)31570-9 http://dx.doi.org/10.1016/j.aeue.2016.12.019 AEUE 51759
To appear in:
International Journal of Electronics and Communications
Received Date: Revised Date: Accepted Date:
18 January 2016 28 July 2016 24 December 2016
Please cite this article as: F. Gnana Poovathy John, R. Sankararajan, Efficient reconstruction of compressively sensed images and videos using non-iterative method, International Journal of Electronics and Communications (2016), doi: http://dx.doi.org/10.1016/j.aeue.2016.12.019
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Efficient reconstruction of compressively sensed images and videos using non-iterative method Florence Gnana Poovathy John1, Radha Sankararajan2 1
[Email: [email protected]] 2 [Email: [email protected]] 1,2 Department of ECE, SSN College of Engineering, Kalavakkam, Chennai – 603110, Tamil Nadu, India.
Abstract – Compressed sensing is widely applied for compression and reconstruction of images and videos by projecting the pixel values to smaller dimensional measurements. These measurements are reconstructed at the receiver using various reconstruction procedures. Greedy algorithms are often used for such recovery. These solve the least squares problem to find the best match with minimum error. This is a time consuming and complex process, giving rise to a trade-off between reconstruction performance and algorithmic performance. This work proposes a non-iterative method, viz., non-iterative pseudo inverse based recovery algorithm (NIPIRA), for reconstruction of compressively sensed images and videos with small complexity and time consumption, provided the reconstruction quality is maintained. NIPIRA gives a minimum PSNR of 32 dB for very few measurements (/ = 0.3125) and accuracy of above 97%. There is more than 92% of decrease in elapsed time compared with other iterative algorithms. NIPIRA is tested for its performance with respect to many other objective measures as well. The complexity of NIPIRA is times less than existing recovery algorithms. Keywords: Compressed sensing, image and video reconstruction, non-iterative recovery, objective measures, elapsed time. 1. Introduction Transmission of images and videos over wireless sensor networks (WSNs) require compression for effective usage of bandwidth and storage capacity. Development of an 1
efficient lossy technique, compressed sensing (CS) [1 – 5] has been seen in recent years. This overrules Nyquist criteria by requiring a small number of samples for reconstruction, provided the input data is sparse [6] and the sensing matrix abides by restricted isometry property (RIP) [7, 8]. CS also guarantees improved quality in reconstruction of images and videos [5] and therefore considered as one of the best methods of compression. Compression in CS is carried out by projecting the available sparse samples to a lower dimensional vector space while the reconstruction of the input from these measurements is carried out at the receiver through efficient reconstruction algorithms. Many reconstruction algorithms are available. These may be classified as greedy algorithms like orthogonal matching pursuit (OMP) [9], Stagewise orthogonal matching pursuit (StOMP) [10, 11], compressed sampling matching pursuit (CoSaMP) [12] etc., iterative algorithms like iterative hard thresholding (IHT) [13] and model based algorithms like model based CoSaMP and model based IHT. Other advanced algorithms also exist namely, least absolute shrinkage selection operator (LASSO) [14], sparse Bayesian learning for multiple measurement vector model exploiting temporal correlation (TMSBL) [15] and focal underdetermined system solution using multiple measurement vector (MFOCUSS) [16] have gained attention since they perform better than greedy algorithms. The downside of these algorithms are that they are iterative and hence consume more time and large processing energy. The non-iterative recovery procedure (NIPIRA) is proposed for obviating these disadvantages and avoiding iterations, while providing good perceptual quality for the reconstructed images and videos. The sensing matrix used for projecting the input sparse vector to a lower dimensional space is augmented matrix which is an augmentation of identity matrix and zero matrix. NIPIRA provides a PSNR of above 32 dB for the least number of measurements (31.25% of input information for 8 x 8 block size) with an accuracy of 97% and elapsed time of the order of µs. Thus NIPIRA proves to be a 2
less time consuming non-iterative algorithm for reconstruction of compressively sensed image and video data. Notations used in this work are given in Table 1. Table 1 Notations and denotations
Denotations
Notations
Denotations
Notations
Input vector
x
Threshold
T
Basis function
ψ
Intermediate vector
V
Sensing matrix
ϕ
Position vector
Г
Sensed vector
y
Estimated vector
Sparsity
s
Input vector length
N
Sparsified signal
Measurement vector size
MxN
Block length
N
Number of frames
n
Dell Inspiron laptop with Intel’s i5 core processor and 64 bit operating system has been used for the entire experimentation. This work is distributed into the following sections: Section 2 gives the overall framework of CS; Section 3 provides the proposed NIPIRA and proof of RIP, accuracy and optimality of NIPIRA; Section 4 describes the results and discussions; Section 5 provides the conclusion and future work of the proposed algorithm. 2. Compressed sensing (CS) framework With increase in the flow of data for high end applications, Nyquist rate also increases leading to increase in the sampled data [17]. This further leads to impossibility in physical implementation, high cost variations and complexity and large storage requirements. With the genesis of CS, the number of samples and hence the storage requirements and the complexity are reduced. The main requirements for acceptable recovery performance are that the input must be sparse [6] i.e. the input must have a large number of zeros, and the sensing matrix should satisfy the restricted isometry property (RIP) [8]. The concept of CS can be explained briefly as follows: instead of acquiring the data and then compressing it, as done in traditional compression methods, the compressed form of the data can be directly acquired in 3
the acquisition process itself. Consider the input image as such that , where R is a region with N dimensions. Let be the basis function applied to the input for sparsification and let be the sensing matrix which senses the appropriate information from the sparsified input and projects the same to a smaller dimensional vector called measurement vector y of size . This process can be mathematically figured as in equation (1) [18].
=
(1)
or
= , where =
(2)
This can also be represented as = , where is the sparsified signal. The sparsity constraint [19] that is to be induced in the input vector is expressed as in equation (3). = min‖‖ or : ‖ ‖ ≤
(3)
where, ‖ ‖ is the norm of that indicates the number of non-zero elements in the vector and indicates the sparsity. The sparsity constraint, which represents the real time CS acquisition process, is forcedly induced in simulative environments by the basis function . After sparsification, the sensing matrix collects the important elements from sparsified vector to form the measurement matrix . At the receiver, the estimate of the input data can be obtained by applying a suitable recovery algorithm. The algorithm must be robust enough to reconstruct the image data perfectly with the least number of measurements available. Smaller the number of measurements required for perfect recovery, smaller is the computational complexity and time consumption. Greedy algorithms like StOMP, CoSaMP, ROMP, OMP etc. are iterative in nature and find the perfect match by solving least squares problem (LSP). Hence their performance is stringed with the disadvantages like increase in time consumption and complexity. Moreover, failure to find the best match through erroneous selection of the
4
measuring combination would result in imperfect reconstruction or large error. The results of these algorithms are unstable since they use random matrix as sensing matrix. To overcome the above said shortcomings, stable sensing matrix is designed which paves the way for noniterative recovery algorithm. This work proposes a non-iterative recovery algorithm using pseudo inverse (NIPIRA) for reconstruction of images and videos. NIPIRA satisfies the reconstruction requirements practically and theoretically as explained in detail in section 3. 3. NIPIRA for CS based Image and Video Reconstruction The designer of the recovery algorithm has the sole right to design the sensing matrix that would aid in the proper recovery process. Thus the design of the sensing matrix plays a major role in the development of the recovery algorithm [17]. Random matrices are seemingly robust in sensing but they are tailed with disadvantages like inconsistency, large time consumption etc. Hence a stable measurement matrix called augmented matrix is generated, which does not change its elemental values for every execution. As mentioned in section 2, there are two main properties that the measurement matrix must follow for perfect reconstruction of images and videos. They are the following: the input must be in sparsified [6] form and the sensing matrix must satisfy the restricted isometry property (RIP) [8, 20]. The sparsification of input information is carried out by applying a basis function and a threshold operator to the input. RIP states that if there is a measurement matrix ϕ of size × , there exists a constant ( ∈ 0,1$, such that 1 − $‖ ‖&& ≤ ‖ ‖&& ≤ 1 + $‖ ‖&&, ∀ ∈
(4)
where, is the sparse vector. It is to be noted that the proposed stable augmented matrix completely abides by RIP. Augmented matrix is the combination of identity matrix augmented with zero matrix as given by equation (5). = )*+ = ,-×- ∪ /-× 5
0-$
(5)
where, )*+ is the augmented sensing matrix, ,-×- is the identity matrix of size × , ∪ is the augmentation operator and /-×
0-$
is the zero matrix of size × − $. The
augmentation of zero matrix further reduces the number of sparse values to be selected, thereby providing greater compression. The proposed matrix is shown as in Fig. 1.
)*+
1 30 20 2 0 =2 20 20 20 10
0 1 ⋱ ⋱ ⋱ ⋱ ⋱ 0
0 ⋱ 1 ⋱ ⋱ ⋱ ⋱ 0
0 ⋱ ⋱ 1 ⋱ ⋱ ⋱ 0
0 ⋱ ⋱ ⋱ 1 ⋱ ⋱ 0
0 ⋱ ⋱ ⋱ ⋱ 1 ⋱ 0
0 ⋱ ⋱ ⋱ ⋱ ⋱ 1 0
Identity matrix ×
0 0 0 0 0 0 0 1
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
… … ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ … …
… ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ …
0 08 07 7 07 07 07 07 06
Zero matrix × − $
Fig. 1 Pictorial representation of the augmented matrix
The augmented zero matrix allows limited selection of the elements from the sparse input vector, avoiding the necessity of solving the least squares problem (LSP). The augmented matrix that abides by the RIP, selects only the appropriate sparse values from the sparsified input signal as measurements with the help of its leading diagonal, avoiding iterations. It is to be noted that the popular iterative procedures use random matrices that generate a number of equations which are less than number of unknowns, requiring iterative solving procedures. The proposed non-iterative pseudo inverse based recovery algorithm (NIPIRA) is given in Table 2. Table 2 Non-Iterative Pseudo Inverse based Recovery Algorithm (NIPIRA)
NIPIRA Algorithm Input:
, , Initialisations: Г=∅ ;ℎ=> = ∅ = ?@A B, 1C
6
Algorithm:
D = 10 AE ∑| |$JHJI& K L = MNAE |O $|$PQ R = | S | + D ∀ = ∈ 1, $, where = I& is the block size if RT > L Г = Г ∪ = ;ℎ=> = ;ℎ=> ∪ T V= = ;ℎ=> W , where † denotes the pseudo inverse V=Г = V= @> = V=Г ∪ ?@A I& − $, where is the number of rows in @> = =DY> @>$
CS procedure can be comfortably applied for compression and recovery only of the input signal is sparse. Many signals are naturally sparse while some are not. Hence, at the transmitter, sparsification of the non-sparse signals is attained by applying a basis function . Generally, transform functions such as discrete cosine transform (DCT) [21], discrete wavelet transform (DWT), etc. are some of the prominent basis functions available for sparsifying the input signal. The measurement matrix ( × ) is then applied to the sparsified input vector ( × 1), which extracts high priority information with the help of its leading diagonal to form the measurement vector ( × 1). Once is formed, it is transmitted through the wireless channel to the receiver along with the and , where is the sparsity of the input signal. Since the input image or video information of size × 1 is dimensionally projected to a smaller sized measurement vector of size × 1, the storage requirements, traffic factors and other transmission based parameters are lightly loaded. The receiver receives the transmitted , and and with the help of NIPIRA, estimates the input image information perfectly with the least number of measurements, small elapsed time and with failure proof recovery. The NIPIRA algorithm first finds the add-constant D and the threshold L which are given by equations 6 and 7.
7
D = 10 AE ∑| |$JHJI& K
(6)
L = MNAE |O $|$PQ
(7)
The add-constant D extracts the log value of summed up measurements divided by sparsity and block size I. The add-constant boosts up the pixel quality after the first intermediate reconstruction since D depends upon that has the information about the compressed pixel values. The calculated add-constant is then added to the intermediate reconstruction quantity R = | S |. The threshold L is the log of maximum value present in the measurement vector. The threshold value can be set as per the user’s need. All the elements of the intermediate result R that are greater than L, are selected for further processing and the other elements which do not hold for L are eliminated. This thresholding process at the receiver end will further reduce the load to the upcoming recovery process of finding the second intermediate since some more of the unimportant elements are eliminated. The support of those elements that are greater than L are found and they are augmented to the null vector Г. In the same way, the corresponding columns of the measurement matrix , that are included in the support vector Г, are augmented to form the ;ℎ=> matrix. Formation of ;ℎ=>, where the position of the matrix column corresponds to the position of the best intermediate term ensures easy recovery. The newly formed ;ℎ=> has the similar properties of )*+ , since their columns are the same and only the positions of the columns are dependent upon the position of R T > L. Once ;ℎ=> is ready, the pseudo inverse of ;ℎ=> and are multiplied, which forms the second intermediate vector. The values of this intermediator are arranged in accordance with the indexes provided by the support vector Г. The values in the vector V=Г are then applied with the inverse basis function to give the estimated input vector . Thus without the usage of iterations, NIPIRA chooses indices of the best elements with the help of L. Simple arithmetic 8
operations are used for calculation of the estimated input vector with small time consumption. Since the leading diagonal of augmented matrix chooses the appropriate elements in the sparse vector, the process of finding the best indices does not make a wrong turn thus avoiding failures in reconstruction. NIPIRA is thus suitable for image and video reconstruction since it has no iterations and hence less complexity and time consumption. Besides exhibiting better performance in script based experimentations, NIPIRA showcases its performance theoretically as well. The mathematical proof of NIPIRA algorithm can be done by proving the RIP for augmented matrix and deriving the accuracy. 3.1 Mathematical proof for NIPIRA Any algorithm must be proved both practically and theoretically. Theoretical performance may be calculated approximately with mathematical proof of various parameters. For proper reconstruction, RIP must be satisfied by the sensing matrix, whose theoretical proof is derived. Accuracy, can be considered as the ability of the algorithm to perfectly reconstruct the image or video data with minimum error. Certain lemmas are considered to establish isometry property and accuracy as follows: Lemma 1 Let be a matrix of size × which follows the concentration inequality as follows [8]: ; |‖ ‖&& − ‖ ‖&& | ≥ [‖ ‖&& $ ≤ 2@ 0]^_
,0<[<1
`$
where [ is a constant. Then, for any set b, with # b$ = d < and any 0 < < 1, we have, 1 − $‖ ‖& ≤ ‖ ‖& ≤ 1 + $‖ ‖& ∀ ∈ . Lemma 2 If satisfies RIP [22], ‖ ‖& ≤ e1 + ‖ ‖&, ∀ : ‖ ‖ ≤ , then ‖ ‖& ≤ e1 + ‖ ‖& + e1 + 9
‖ ‖E √
3.1.1 Restricted Isometry Property (RIP) Perfect reconstruction of the original data, requires satisfaction of RIP. RIP can be mathematically expressed as in equation (4): 1 − $‖ ‖&& ≤ ‖ ‖&& ≤ 1 + $‖ ‖&&, ∀ ∈ . Proof In this case, it is sufficient to prove ‖ ‖& = 1, since ϕ is linear. Let a set of points b be considered such that b ∈ , ‖‖& = 1 for all ∈ b
. For all ∈ , with ‖‖& = 1,
we have min∈g ‖ − b ‖& ≤ J4 (8) The probability that the term ‖ ‖& is concentrated about its expected value and can be expressed as in Lemma 1. Considering Lemma 1 to a set b with [ = J2, we have, H1 − J2K‖‖&& ≤ ‖ ‖&& ≤ H1 + J2K‖‖&&, ∀ ∈ b
(9)
H1 − J2K‖‖& ≤ ‖ ‖& ≤ H1 + J2K‖‖& ∀ ∈ b
(10)
This obviously states,
Now, let us consider a small constant i such that ‖ ‖& ≤ 1 + i$‖ ‖&, ∀ ∈
(11)
If it is proved that i < , then it obviously states that ‖ ‖& ≤ 1 + $‖ ‖& . For any ∈ , with ‖ ‖& = 1, a is selected such that ‖ − ‖& ≤ J4 with reference to equation (8). Thus, ‖ ‖& can be written as
10
(12)
‖ ‖& ≤ ‖ ‖& + ‖ − $‖&
(13)
Substituting equations (10) and (12) in (13), and equating the corresponding terms, we get, 1+ J2 + j 1 + i$ J4k i is known to be the smallest number for equation 11 to be true. Therefore, i ≤ 1 + J2 + j 1 + i$ J4k ≤ 1 + J2 + J4 + iH J4K iH1 − J4K ≤ 1 + 3 J4 Equating the i and terms on both sides, we get, i≤
3 J 4 ≤ 1 − J4
Thus, i ≤ , which proves the upper inequality of Lemma 1. The lower inequality can be proved in a similar fashion. Hence, RIP of NIPIRA is proved. 3.1.2 Accuracy Theorem Given a noisy observation, = + @, where is the input vector. Let be sparse representation to . If ϕ follows RIP, then the algorithm will recover an approximation satisfying, E
‖ − ‖& ≤ m − m + &
&
E
√
‖ − ‖E + ‖@‖&
(14)
Proof If ϕ follows RIP [13], with d iterations, the algorithm will recover the input, which satisfies ‖ − ‖& ≤ 20n ‖ ‖& + [̃ where,
[̃ = ‖ − ‖& + 11
E
√
‖ − ‖E + ‖@‖&
(15) (16)
NIPIRA is non iterative, thus d = 1. Therefore, ‖ − ‖& ≤ 20E ‖ ‖& + [̃ Substituting for [̃ , we get, ‖ − ‖& ≤ 20E ‖ ‖& + ‖ − ‖& +
E
√
(17)
‖ − ‖E + ‖@‖& (18)
Separating the norms and rearranging the like terms together, we obtain, E
‖ − ‖& ≤ ‖ ‖& + ‖ ‖& − ‖ ‖& + & E
‖ − ‖& ≤ m − m + & &
E
√
E
√
‖ ‖E −
E
√
‖ ‖E + ‖@‖&
(19)
‖ − ‖E + ‖@‖&
(20)
Hence the difference between the input vector and the estimated vector is less than the sum of first and the second norms of the differences between the input and the sparsified vectors. 4. Results and Discussions NIPIRA has been designed mainly to reconstruct the images and video frames with the least time consumption and utmost perfection. NIPIRA proves itself worthy in the above said regards when applied for the stated purpose. The images considered for the performance evaluation are of uncompressed TIFF format while the videos are of CIF format. The test images taken for experimentation are Lena, Airplane F-16 [23] and the test videos are Akiyo, and Mother – Daughter [24] in uncompressed TIFF and CIF formats respectively. Applying the DCT for the whole image is time and energy consuming since DCT depends upon the input matrix size. Hence the entire image is divided into smaller blocks and DCT is applied for each block such that total time consumption is reduced due to smaller matrix size. The size of the sensing matrix also depends upon the size of the block and hence the complexity induced due to multiplication of sensing matrix and sparse vector is also reduced. The images and video frames are divided into blocks of sizes 4x4, 8x8 and 16x16 and each block gets the application of the algorithms. The blocks are non-overlapping in order to avoid introduction of time complexity in this regard. The ⁄ ratio, which gives the number of measurements
12
selected over the available input , is maintained standard for all block sizes; the selected constant ⁄ ratios are 0.1875, 0.3125, 0.4375 and 0.5625 respectively. All algorithms, including NIPIRA and other comparing algorithms, have been designed as MATLAB scripts in MATLAB 2014a. For benchmarking the results of NIPIRA, the same is compared with those of other iterative algorithms like OMP, StOMP, LASSO and TMSBL. OMP and StOMP are chosen for comparison with NIPIRA as they are well known as complex iterative greedy algorithms of small magnitude and comparing NIPIRA with them would bring the performance of the proposed algorithm into limelight. TMSBL and LASSO that depend upon the temporal information available in the data and are more productive than OMP and StOMP. Hence they are also taken as algorithmic comparison metric. The parameters like number of measurements , sparsity etc., for all the algorithms are the same so that comparison among them would provide better revelation. The measurement matrix applied for OMP and StOMP is the traditional ransom Gaussian matrix and the same for NIPIRA is the proposed augmented matrix. The performance of an algorithm can be tested in two ways: 1) by evaluating the performance of the results generated by the algorithm. NIPIRA is tested on this basis in terms of peak signal to noise ratio (PSNR), mean square error (MSE), structural similarity (SSIM), maximum difference (MD), normalised absolute error (NAE) as the objective measures [25 – 27]; 2) by testing the performance of the algorithm itself, which is done by calculating the elapsed time, runtime [28] and the algorithmic complexity [29, 30]. 4.1 Evaluation of NIPIRA by Objective measures Objective measures are those that evaluate the quality of the reconstructed images and videos with precision since they perform pixel level quantification of the images and videos under consideration. PSNR is considered to be the most relevant quality metric when it
13
comes to evaluating the quality of images and videos. Hence the PSNR for each example image and video frame, with varying block sizes and number of measurements is calculated for OMP, StOMP, LASSO, TMSBL and NIPIRA and compared. Fig. 2 shows comparison of PSNR of NIPIRA for images Lena and Airplane F-16, with all the above mentioned algorithms, considering block size to be 8x8 and = 20.
a) Lena
b) OMP, PSNR = 23.5500 dB
c) StOMP, PSNR d) LASSO, PSNR = 20.6382 dB = 24.4353 dB
e) TMSBL, PSNR = 24.0547 dB
f) NIPIRA, PSNR = 31.6173 dB
g) Airplane F-16
h) OMP, PSNR = 20.7029 dB
i) StOMP, PSNR = 16.1586 dB
j) LASSO, PSNR k) TMSBL, PSNR = 22.2419 dB = 22.5375 dB
l) NIPIRA, PSNR = 28.1711 dB
Fig. 2 Comparison of perceptual quality of the images reconstructed by NIPIRA with OMP, StOMP, LASSO and TMSBL: a) and g) are the original input images; b) and h) are the images reconstructed by OMP; c) and i) are the images reconstructed by StOMP; d) and j) are the images reconstructed by LASSO; e) and k) are the images reconstructed by TMSBL; f) and l) are the images reconstructed by NIPIRA. It is evident that NIPIRA outperforms the other two algorithms in perceptual quality and PSNR (~30 dB) value.
Considering the test images in Fig. 2, the approximate average PSNRs of OMP, StOMP, LASSO, TMSBL and NIPIRA are 24 dB, 19 dB, 23 dB, 23 dB and 30 dB respectively. NIPIRA gives minimum increase in PSNR of 7.2 dB and maximum increase of 10.99 dB against LASSO and StOMP (for Lena) and their corresponding percentage increases are 29.4655% and 53.2507% respectively. These percentages in increase provide evidence that NIPIRA is ideally suited for reconstruction of images compared to all the algorithms considered for comparison. The best among the iterative algorithms is either LASSO or TMSBL, but NIPIRA outperforms them with respect to PSNR.
14
A similar comparison is suitable for video frames. The average PSNR values of the video sequences (averages over both the videos) considered in Fig. 3 that are reconstructed by OMP, StOMP, LASSO, TMSBL and NIPIRA are 25.6106 dB, 19.4065 dB, 28.381 dB, 29.1163 dB and 34.7952 dB respectively. The corresponding percentage increase in PSNR exhibited by NIPIRA compared to OMP, StOMP, LASSO and TMSBL are is 39.53%, 72.3097%, 38.9383% and 37.8403% (considering Akiyo sequence). This proves better reconstruction of NIPIRA when compared to above mentioned algorithms with the highest PSNR for = 20. It is to be noted that the performance of iterative algorithms in terms of PSNR is dependent upon the random sensing matrix and the number of iterations. Hence the PSNR values obtained for the same test images and videos of Fig. 2 and Fig. 3 can vary with number of executions of the script. This instability is avoided in NIPIRA since it uses a fixed augmented sensing matrix. Irrespective of the number of executions, the PSNR remains constant thus leading to stability and consistency in reconstruction.
a) Akiyo
b) OMP, PSNR = 27.2150 dB
c) StOMP, PSNR = 22.0377 dB
e) TMSBL, PSNR = 27.5486 dB
f) NIPIRA, PSNR = 37.9731 dB
15
d) LASSO, PSNR = 27.3309 dB
g) Mother-daughter
h) OMP, PSNR = 24.0062 dB
i) StOMP, PSNR 16.7753 = dB
k) TMSBL, PSNR = 30.6841 dB
l) NIPIRA, PSNR = 31.6173 dB
j) LASSO, PSNR = 29.4311 dB
Fig. 3 Comparison of perceptual quality of video frames reconstructed by NIPIRA with OMP, StOMP, LASSO and TMSBL: a) and g) are the original input video frames (frame No. 4); b) and h) are the video frames reconstructed by OMP; c) and i) are the video frames reconstructed by StOMP; d) and j) are the video frames reconstructed by LASSO; e) and k) are the video frames reconstructed by TMSBL; f) and l) are the video frames reconstructed by NIPIRA. It is evident that NIPIRA outperforms the other two algorithms in perceptual quality and PSNR value.
Besides PSNR there are other objective quality measuring metrics like MSE, SSIM, MD and NAE. These parameters add to the pixel level details whether the images and videos are reconstructed with preservation of pixel level properties. MSE and NAE represent the error in reconstruction, MD gives the maximum deviation from original pixel values to reconstructed pixel values and SSIM gives the structural similarity in the reconstructed image. Tables 3 and 4 give the comparison of all the above said parameters for NIPIRA and iterative algorithms for image (Lena) and video (Akiyo). Table 3 Comparison of various objective parameters for images reconstructed by iterative algorithms and NIPIRA
Algorithm
PSNR
MSE
SSIM
MD
NAE
OMP StOMP LASSO TMSBL NIPIRA
23.5500 20.6382 24.4353 24.0547 31.6173
287.1317 561.3801 234.1799 255.6291 44.8871
0.7117 0.6097 0.7296 0.7320 0.9301
221 222 184 209 74
0.6877 x 10^-6 4.6820 x 10^-6 2.0448 x 10^-4 7.9881 x 10^-5 3.2009 x 10^-8
Table 4 Comparison of various objective parameters for video frames reconstructed by iterative algorithms and NIPIRA
16
Algorithm
PSNR
MSE
SSIM
MD
NAE
OMP StOMP LASSO TMSBL NIPIRA
27.2150 22.0377 27.3309 27.5486 34.9119
123.5321 406.7351 120.2227 114.3631 20.9844
0.8307 0.6427 0.8437 0.8610 0.9558
216.75 247 160 247 78
4.0839 x 10^-5 5.9950 x 10^-6 2.6201 x 10^-4 1.5335 x 10^-5 2.5979 x 10^-8
The number of frames considered for experimentation is n = 4 and thus all the values in the Table 4 has been obtained by averaging them out by the number of frames. From Table 4 it can be observed that, the MSE is substantially reduced to 44.8871 which is a nearly 84% decrease from the same of OMP and 92% decrease from StOMP. The structural similarity retained by NIPIRA is above 93% which has not been achieved by any of the algorithms considered for experimentation. The maximum deviation in pixel value created by NIPIRA is 74 while the values obtained by other algorithms are above 100 which shows the lenient reconstruction performance of other algorithms. The MD reaches a maximum of around 560 in case of StOMP. The error in the reconstruction of pixel values using NIPIRA is at least 100 times smaller than in case of OMP, StOMP, LASSO and TMSBL. 4.1.1 Performance comparison for various measurement values: So far the number of measurements considered for experimentation and evaluation is = 20 which accounts for about 31.25% of input information. All the recovery algorithms vary their performance vis-a-vis increase or decrease in the number of measurements. Hence a comparison between the same parameters obtained by the same algorithms but for varying number of measurements is carried out. The block size is also an important factor since it influences the objective measures and number of measurements which in turn results in alterations of reconstruction quality. Hence the block sizes and the number of measurements were varied simultaneously for every algorithm. Table 5 shows the performance metrics for varying block sizes and number of measurements for NIPIRA.
17
Table 5 Performance of NIPIRA for different block sizes and measurements. Test image is Lena in TIFF format.
Block size
4x4
8x8
16 x 16
M
PSNR (dB)
MSE
SSIM
MD
NAE
3 5 7 9 12 20 28 36 48 80 112 144
27.2603 29.2003 33.0816 35.6003 27.9900 31.6173 34.1508 36.8908 27.6597 31.0026 33.6776 35.5745
122.1935 78.1721 31.9833 17.9081 103.2943 44.8871 25.0032 13.3045 111.4568 51.6198 27.8819 18.0148
0.8458 0.8923 0.9539 0.9728 0.8519 0.9301 0.9590 0.9766 0.8383 0.9151 0.9514 0.9601
115 108 78 52 107 74 67 43 102 96 76 42
5.1907 x 10^-8 1.0034 x 10^-8 1.8167 x 10^-8 8.2186 x 10^-8 1.0468 x 10^-7 3.2009 x 10^-8 5.1907 x 10^-9 2.5088 x 10^-8 2.5088 x 10^-8 5.5367 x 10^-8 8.672 x 10^-9 3.3739 x 10^-8
The ratio of input information measured to the number of information in the selected block is taken in four levels, i.e.
= 0.1875, 0.3125, 0.4375 and 0.5625, corresponding to
= 3, 5, 7, 9 when block size is 4x4 and the same ratio prevails for all the block sizes. Table 5 shows that, with increase in the number of measurements, the performance with respect to PSNR also increases and so does the other objective parameters. With increase in number of measurements, the maximum deviation in the pixel decreases to a greater extend for each block size. The MSE also shows dramatic decrease. The same parameters can be calculated for the video frames too for the same block sizes and number of measurements. Fig. 4 gives the comparative performance in PSNR of NIPIRA and other algorithms under consideration for various number of measurements for images.
18
OMP
StOMP
TMSBL
NIPIRA
LASSO
35
PSNR in dB
PSNR in dB
40
30 25 20 15 10 3
5
7
9
Number of measurements ()
Number of measurements () b) PSNR vs for 8 x 8 block size (Lena)
for 4 x LASSO a) PSNR vsStOMP 4 block size OMP (Lena) TMSBL NIPIRA 40 35 30 25 20 15 10 12
20
28
36
OMP
StOMP
TMSBL
NIPIRA
LASSO
40
PSNR in dB
35 30 25 20 15 10
48
80
112
144
Number of measurements ()
c) PSNR vs for 16 x 16 block size (Lena) Fig. 4 Performance in terms of PSNR using OMP, StOMP, LASSO, TMSBL and NIPIRA for various block sizes and number of measurements for image Lena: a) Block size is 4 x 4, M = 3, 5, 7 and 9, b) Block size is 8 x 8, M = 12, 20, 28 and 36, c) Block size is 16 x 16, M = 48, 80, 112 and 144.
19
Close observation of Fig. 4 reveals the highest PSNR obtained by NIPIRA i.e. 41 dB as not attained by the other algorithms even with highest number of measurements. The least PSNR obtained by NIPIRA (27.2603 dB, block size = 4x4) using the lowest number of measurements ( = 3) is achieved by other algorithms, only with larger number of measurements. It can be noted that irrespective of block sizes and number of measurements, NIPIRA always outperforms OMP, StOMP, LASSO and TMSBL in terms of various objective measures with the highest PSNR and minimum error. The same performance is observed for videos while using all the algorithms with varying number of measurements. NIPIRA gives comparatively higher PSNR than other algorithms for any number of measurements and block size. 4.2 Algorithmic evaluation The second method of evaluation of algorithms is by checking the performance of the algorithm itself. Three parameters are considered for the said purpose, they are: elapsed time [28], runtime and complexity. In this work, elapsed time is considered as the time taken for the algorithm to reconstruct a single block of the image or video from the input measurements. The runtime denotes the total time consumption of the algorithm including compression as well as reconstruction. The algorithmic complexity reports the complex operations carried out by the algorithm in the process of reconstructing the pixel values when the size of the input tends to infinity. 4.2.1 Elapsed time Elapsed time is the time taken by the algorithm to reconstruct a single block in the image or video frame. The smaller the elapsed time, the smaller the complexity and total runtime, which naturally leads to minimised hardware burden, which ultimately increases the longevity of the processing nodes in the network. Elapsed time comparison of NIPIRA, OMP, StOMP, LASSO and TMSBL is carried out for each block size and various number of 20
measurements. The comparative elapsed times for images and videos with 8 x 8 block and = 20, n = 4 is shown by Table 6. Table 6 Elapsed time (ms) comparison between NIPIRA and iterative algorithms, 8 x 8 block size, = 20, n = 4.
Elapsed Time (ms) Algorithms OMP StOMP LASSO TMSBL NIPIRA
Image (Lena) 10.5 19.9 541.3 65.1 0.037323
Video (Akiyo) 18.4 14.9 244.5 55.6 1.1
The elapsed times of the iterative algorithms are at least 100 times greater than NIPIRA for images, since they have random sensing matrices that demand iterative process of finding the best match. Since NIPIRA uses augmented sensing matrix which selects the appropriate measurements with the help of its leading diagonals, there is no need for iterative procedure of finding the best match and hence the elapsed time is reduced to a range of µs. NIPIRA shows the lowest elapsed time of 0.037323 ms and LASSO exhibits highest elapsed time of 541 ms. For videos too, there is a notable 100 fold decrease in elapsed time exhibited by NIPIRA compared to other algorithms. Fig. 5 gives the elapsed time comparison obtained for the reconstruction of images and videos in a visual perspective.
21
StOMP
LASSO
TMSBL
NIPIRA
Elapsed Time (log scale)
Elapsed Time (log scale)
OMP
1000 100 10 1 12
20
28
36
0.1 0.01 Number of measurements ()
Number of measurements ()
b) Elapsed Time vs for 8 x 8 block size (Akiyo)
a) Elapsed Time vs for LASSO 8 x 8 block size OMP StOMP (Lena) TMSBL NIPIRA 1000 100 10 1 12
20
28
36
0.1 0.01
Fig. 5 Elapsed time comparison of OMP, StOMP, LASSO, TMSBL and NIPIRA for various measurements: a) Lena, b) Akiyo
The elapsed time is represented as logarithmic values of base 10 in order to get a good perceptual comparison. Similar comparisons are made for block sizes of 4 x 4 and 16 x 16 with varying number of measurements. In all the cases, NIPIRA showed at least 10 times decrease in elapsed time. 4.2.2 Total runtime Decrease in elapsed time accounts for reduction of total runtime. As mentioned earlier, total runtime is the time taken for the entire compression-reconstruction process. Fig. 6 indicates the comparison of runtimes of various algorithms with NIPIRA. NIPIRA exhibits a small runtime of around 3.9541 s for image and 29.6993 s for video (n = 4). The figures are plot in log scale for easy understanding of the vast difference in time consumption.
22
StOMP
LASSO
TMSBL
NIPIRA
Total Runtime (log scale)
Total Runtime (log scale)
OMP
1000 100 10 1 12
20
28
36
Number of measurements ()
Number of measurements ()
a) Elapsed Time vs for LASSO 8 x 8 block size OMP StOMP
a) Elapsed Time vs for 8 x 8 block size
TMSBL
NIPIRA
10000 1000 100 10 1 12
20
28
36
Fig. 6 Total runtime comparison of OMP, StOMP, LASSO, TMSBL and NIPIRA for various measurements: a) Lena, b) Akiyo
4.2.3 Complexity Operational complexity is the contribution of each mathematical operation in an algorithm. It can also be described as the complex behaviour exhibited by the algorithm when the input size tends to infinity, accounts for the complexity of the algorithm. The mathematical operations and the number of occurrences of these operations in the algorithm are given in Table 7. Table 7 Mathematical operations in the algorithm and their contribution towards complexity
Mathematical operations
Number of occurrences
Addition Subtraction Summation Squaring
N(M-1)+1 1 M 1
23
Matrix Multiplication Augmentation Maximum operation Determinant Pseudo Inverse Transpose
NM N 1 3 M NM
Adding the contributions of all the various operations in the algorithm, we get 3 + 2 + 7. But when → ∞, the term making a substantial contribution will be and other terms can be eliminated since ≪ . Also, > which indicates that the first term is times larger than the second term. Thus, when the input size gets close to infinity, complexity of NIPIRA can be given by u $, where u represents the order. The complexity for OMP and StOMP is generally represented as u $ [31, 10, 11] where is the sparsity or a fixed number of iterations. This shows that since there are no iterations in NIPIRA, the complexity of the algorithm is reduced by times with respect to greedy algorithms. LASSO and TMSBL have a fixed number of iterations that the user can select according to his/her utility. If ℎ is a common representation of the number of iterations of both greedy algorithms (OMP, StOMP) and the temporal information based algorithms (LASSO, TMSBL), then complexity of NIPIRA is ℎ times less than any of the algorithms u $ ≪ u ℎ$$. All the above experiments prove that NIPIRA is best suited for applications requiring faster recovery process and higher clarity. 5 Conclusions Compressed sensing overrules the traditional compression techniques that follow Nyquist criterion for perfect reconstruction. It projects the input image or video information to ≪ dimensional measurements which can be recovered through perfect reconstruction algorithms at the receiver. Most of the present greedy algorithms are iterative and highly time consuming and inconsistent. In order to address these issues, Non-Iterative Pseudo Inverse
24
based Recovery Algorithm (NIPIRA) is proposed which uses augmented matrix as the sensing matrix to project the input data into measurements. Augmented matrix maintains the stability of the algorithm by providing consistent results for any number of executions. The PSNR obtained by NIPIRA is around 32 dB for just 31.25% of the input data. NIPIRA exhibits SSIM of above 92% and has minimum pixel level deviations compared to popular iterative algorithms. Since iterations are eliminated in NIPIRA, the elapsed time and runtime obtained are 0.037 ms and 3.9541 s respectively for images, which are at least 100 times smaller compared to LASSO and TMSBL. The complexity of NIPIRA is u $ which is ℎ times less than iterative algorithms whose complexities are u ℎ$. These results project NIPIRA to be best suited for reconstruction of compressively sensed images and videos that are dimensionally diminished into measurements by the use of process of compressed sensing. In future, NIPIRA can be applied for reconstruction of surveillance applications and streaming of real time videos. References [1]
Dana M. Compressed Sensing Makes Every Pixel Count. In: Proc. of J What Happens in Math Sci, 7 (2009), pp. 114-127.
[2]
R. Masiero, G. Quer, M. Rossi and M. Zorzi. A Bayesian analysis of Compressive Sensing data recovery in Wireless Sensor Networks. In: Proc. of Int Conf on Ultra-Modern Telecomm & Workshops, Oct 12-14, 2009, pp.1-6.
[3]
E. Candes, J. Romberg, and T. Tao. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans on Info Theory, 52 (2) (2006), pp. 489–509.
[4]
David L Donoho. Compressed sensing. IEEE Trans on Info Theory, 52 (4) (2006), pp. 1289–1306.
[5]
I. Drori. Compressed Video Sensing. In: BMVA Symposium on 3D Video - Analysis, Display, and Applications, 2008.
[6]
Vishal M. Patel, Ramalingam Chellappa. Sparse Representation and Compressive Sensing for Imaging and Vision. Springer, 2013.
[7]
M. Fazel, E. Candes, B. Recht, P. Parrilo. Compressed sensing and robust recovery of low rank matrices. In: 42nd Asilomar Conf on Sig, Sys and Computers. pp. 1043-1047, 26-29 Oct. 2008.
[8]
E. Candes. The Restricted Isometry Property and its implications. J Comptes Rendus Mathematique, 346 (9-10) (2008), pp. 589 – 592.
25
[9]
T. Tony Cai and Lie Wang. Orthogonal Matching Pursuit for Sparse Signal Recovery with Noise. IEEE Trans on Info Theory, 57 (7) (2011), pp. 4680-4688.
[10]
D. L. Donoho, Y. Tsaig, I. Drori and J.-L. Starck. Sparse Solution of Underdetermined Linear Equations by Stagewise Orthogonal Matching Pursuit (StOMP). IEEE Trans on Info Theory. 58(2), pp. 1094-1121, 2012.
[11]
Liu Zhi-xue, Li Gang, Zhang Hao, and Wang Xi-qin. Sparse-Driven SAR Imaging Using MMVStOMP. In: Proc. of 1st Int Workshop on Compressed Sensing applied to Radar, IEEE press, May 14 – 16, 2012.
[12]
D. Needell and J.A. Tropp. CoSaMP: Iterative signal recovery from incomplete and inaccurate sample. J of App and Comput Analysis, 26 (3) (2009), pp. 301-321.
[13]
T. Blumensath, Mike E. Davies. Iterative hard thresholding for compressed sensing. J of Applied and Computational Harmonic Analysis, Elsevier, 27 (3) (2009), pp. 265–274.
[14]
Nowak, Robert D., and Stephen J. Wright. "Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems." IEEE Journal of selected topics in signal processing 1.4 (2007): 586-597.
[15]
Zhang, Zhilin. "Comparison of Sparse Signal Recovery Algorithms with Highly Coherent Dictionary Matrices: The Advantage of T-MSBL." Research note (2012).
[16]
Zdunek, Rafal, and Andrzej Cichocki. "Improved M-FOCUSS algorithm with overlapping blocks for locally smooth sparse signals." IEEE Transactions on Signal Processing 56.10 (2008): 4752-4761.
[17]
Yonina C. Eldar, Gitta Kutyniok. Compressed Sensing: Theory and Applications. Cambridge University Press, 2012.
[18]
G. Kutyniok. Theory and applications of compressed sensing. arXiv preprint arXiv:1203.3815, 2012. http://www.math.tuberlin.de/fileadmin/i26_fgkutyniok/Kutyniok/Papers/SurveyCompressedSensing_R evision.pdf
[19]
Beck, Amir, and Yonina C. Eldar. "Sparsity constrained nonlinear optimization: Optimality conditions and algorithms." SIAM Journal on Optimization 23.3 (2013): 1480-1509.
[20]
R. Baraniuk, M. Davenport, R. DeVore, and M. Wakin. A simple proof of the restricted isometry property for random matrices. Const App. 28 (3), pp. 253 – 263.
[21]
Mun, Sungkwang, and James E. Fowler. "Block compressed sensing of images using directional transforms." 2009 16th IEEE international conference on image processing (ICIP). IEEE, 2009.
[22]
J. Florence Gnana Poovathy, S. Radha. Non-Iterative Threshold based Recovery Algorithm (NITRA) for Compressively Sensed Images and Videos. KSII Trans on Int and Info Sys, 9 (10) (2015), pp. 4160 – 4176.
[23]
http://sipi.usc.edu/database/database.php?volume=misc accessed on December 2015.
[24]
http://see.xidian.edu.cn/vipsl/database_Video.html accessed on November 2014.
[25]
M. Mrak, S. Grgic, M. Grgic. Picture quality measures in image systems. EUROCON 2003. Computer as a Tool, The IEEE, 1 (2003), pp.233-236.
[26]
T. Kratochvil & P. Simicek. Utilization of MATLAB for Picture Quality Evaluation. Institute of Radio Electronics, Brno University of Technology, Brno.
[27]
Lauterjung, J. Picture quality measurement. In: International Broadcasting Convention, Amsterdam, 11-15 September 1998. IEE, London, 1998. pp. 413-417.
26
compression
[28]
Martin Knapp-Cordes and Bill McKeeman. Improvements to tic and toc Functions for Measuring Absolute Elapsed Time Performance in MATLAB. Matlab Digest.
[29]
Frank Hutter, Lin Xu, Holger H. Hoos and Kevin Leyton-Brown. Algorithm Runtime Prediction: Methods and Evaluation. Artificial Intelligence. 206 (2014), pp. 79 – 111.
[30]
Sandeep Sen. Lecture Notes for Algorithm Analysis and Design. November 6, 2013. http://www.cse.iitd.ernet.in/~ssen/csl356/root.pdf
[31]
Bob L. Sturm and Mads Græsbøll Christensen. Comparison of Orthogonal Matching Pursuit Implementations. 20th European Signal Processing Conference (EUSIPCO 2012). Romania, August 27 - 31, 2012.
27