- Email: [email protected]
“Synergy effect” and its application in LED-multispectral imaging for improving image quality
Accepted Manuscript “Synergy Effect” and its application in LED-multispectral imaging for improving image quality He Li, Gang Li, Wenjuan Yan, Guoquan He, Ling Lin
PII: DOI: Reference:
S0030-4018(18)31149-0 https://doi.org/10.1016/j.optcom.2018.12.091 OPTICS 23759
To appear in:
Optics Communications
Received date : 1 November 2018 Revised date : 24 December 2018 Accepted date : 29 December 2018 Please cite this article as: H. Li, G. Li, W. Yan et al., “Synergy Effect” and its application in LED-multispectral imaging for improving image quality, Optics Communications (2019), https://doi.org/10.1016/j.optcom.2018.12.091 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Highlights
From: “Synergy Effect” and Its Application in LED-multispectral Imaging for Improving Image Quality Highlights:
The quality of single-waveband image has great influence on the analysis of multispectral images.
Multi-wavelength “Synergy Effect” is proposed to improve the image quality.
The image quality of each waveband is improved according to the “Synergy Effect”.
Providing a reference for the acquisition of high-quality multispectral images.
*Manuscript Click here to view linked References
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
“Synergy Effect” and Its Application in LED-multispectral Imaging for Improving Image Quality He Li a,b, Gang Li a,b, Wenjuan Yan c, Guoquan He c, Ling Lin a,b,* a
State Key Laboratory of Precision Measuring Technology and Instruments, Tianjin University, Tianjin 300072, China b Tianjin Key Laboratory of Biomedical Detecting Techniques and Instruments, Tianjin University, Tianjin 300072, China c School of Electronic Information Engineering, Yangtze Normal University, Chongqing 408100, China ∗ Corresponding author. E-mail addresses: (Ling Lin*)[email protected]; (He Li)[email protected]; (Gang Li) [email protected]; (Wenjuan Yan) [email protected]; (Guoquan He)[email protected]
Abstract: This paper proposes and demonstrates the multi-wavelength “synergy effect” in light emitting diode (LED)-multispectral images obtained by frequency-division modulation, which can be used to improve the image quality of each waveband. Taking three wavelengths in LED-multispectral imaging as an example, the following experiment was designed: The three different center wavelength ( 1 、 2 and 3 ) LEDs modulated by sinusoidal signals with different frequencies are used as the light source to acquire image sequences. The time series composed of corresponding pixels of each frame are demodulated by Fourier transform(FT) to obtain the three single-waveband images I S 、I S and I S3 respectively. Similarly, at the same driving intensity, the above three LEDs are separately used as the light source to acquire image sequences, and three groups of images are obtained respectively. Then FT is used to perform demodulation in the same way and three single-waveband images I A 、 I A and I A3 are obtained. By comparison, it can be concluded that the image quality of each waveband obtained by demodulating the three-wavelength images has been improved, which proves the “synergy effect” and provides a reference for improving the quality of multispectral images. 1
2
1
2
Keywords: Image quality; Multispectral image; Modulation and demodulation; Fourier transform; Single-waveband image 1.
Introduction
Multispectral image is composed of a set of images in a limited number of narrow or wider spectral bands. Multispectral imaging is intended to capture physical properties of objects beyond color, in the form of spectral transmittance or reflectance [1]. Now multispectral images have begun to be widely used in many fields, such as biometric security (face recognition [2,3] and palmprint recognition [4,5]), optical mammography [6-9] food inspection [10] and cultural heritage [11]. Multispectral images are widely used, but the application value is limited by the image quality. Researchers seek various methods to obtain high-quality images under different application backgrounds. Among them, LED illumination-based multispectral imaging (LEDMSI) is one kind of promising technique of fast and effective spectral image acquisition, which adopts the method of active light source illumination. Currently, researchers have proposed a large number of LEDMSI systems, which can be divided into two major categories according to the type of camera: monochrome camera based on the LEDMSI system (Mono-LEDMSI) and red-green-blue (RGB) camera based on the LEDMSI system (RGB-LEDMSI). The mono-LEDMSI system is considered in this paper, in which only one type of color LED light is lit at a time in principle. In digital image acquisition systems, quantization noise and resistance thermal noise in the system are two main types of random noise. The quantization noise is the difference between the actual analog value and the quantized digital value in analog-to-digital conversion (ADC), and the limited ADC resolution results in this kind of noise. Noise is a major factor affecting the performance of LEDMSI systems and the quality of spectral images, suppressing noise and
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
improving the quality of each single-waveband image is beneficial for further feature extraction, classification, target detection and recognition of images [12]. And the analysis of multispectral image is based on the quality of each single-spectral image, hence improving the quality of each single-waveband image is of great significance for the analysis of multispectral images. Superposition averaging method is an effective way to improve data accuracy. It has different representations in different applications and can be divided into spatial domain superposition averaging and time domain superposition averaging. In 2010, by calculating the spatial average value of the pixels in the interest region of RGB three-channel images separately, Ming-Zher Poh et al. [13] finally realized the heart rate detection through face video images. In 2011, the Single-Trial method proposed by Gang Li et al. [14] improved the efficiency and accuracy of dynamic spectral data processing by averaging the wavelength. In essence, FT (or phase-locked demodulation) is also one kind of superposition averaging. In this paper, through the demodulation of multi-wavelength images, the equal-weighted superposition averaging of multi-sampled points under three different frequencies is achieved, which greatly suppresses random noise and improves image quality. In previous studies, many researchers have focused on improving the quantization level of ADC to improve image quality. In 2018, J. Yu et al. [15] employed the appropriate range of sawtooth-shaped-function illumination intensity to raise the digital level and then improved the image quality. But this method is for ordinary color images and does not consider the characteristics of multispectral images. In 2018, X. Yang et al. [16] introduced auxiliary light at a higher sensitivity-camera area to increase the ADC quantization levels that are within the linear response zone of ADC. This method uses the shaped signal to improve the grayscale resolution from the perspective of multispectral acquisition, but the selection of LED is limited by the quantum efficiency of image sensor and this method is aimed at improving the quality of only one single-waveband image. And this paper combines the characteristics of Mono-LEDMSI system and breaks the principle that Mono-LEDMSI system only lights one color LED at a time, and simultaneously illuminates three LEDs by frequency-division modulation. Frequency-division modulation can not only suppress interference, such as dark current, ambient light, etc., but also achieve simultaneous acquisition of multi-wavelength images, saving time of image acquisition. The equal-weighted superposition averaging of multi-sampled data points under a certain frequency can be achieved by demodulation. At the same time, there exists “synergy effect” in the process of acquiring multi-spectral images by multi-wavelength frequency-division method, which can improve the image quality of each waveband obtained by demodulation and provide a reference for improving the quality of tissue hyperspectral transmission image in the future. 2.
Theories and Methods
2.1 Fourier Transform and digital demodulation of sinusoid-modulation signals In the data acquisition system, the signal in amplitude and time are both discrete, and the corresponding digital signal FT pair is shown in formula (1)-(2): ak n N x[n]e jkw0 n n N x[n]e jk (2 / N) n (1)
x[n]
1 N
k N
ak e jkw0 n
1 N
k N
ak e jk (2 / N) n (2)
In the multi-channel signal measurement of the frequency-division method, each information signal is modulated in the sending end, and its spectrum is moved into frequency bands that do not overlap each other. Then, they are transmitted together by one channel, and demodulated by the center frequency at the receiving end to separate the signals, thereby recovering the information signals that were originally transmitted. Demodulation according to formula (1) can restore each frequency signal and effectively suppress noise or interference of other frequencies. After all, in an actual data acquisition system, not only sine wave signals x[n] of different frequencies that need to be measured, but also noise and interference signals of other frequencies mixed in are acquired.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
1 N
is equivalent to the “average” in the measurement domain term, which means that
the more sampling values are used, the higher the accuracy of the result. But the premise that “average” has its effect is that white noise is inevitable in the system. 2.2 “Synergy effect” 2.2.1 Reason of poor image quality In the digital image acquisition system [17] of multispectral reflected images under active illumination, the image quality of some wavebands in a multispectral image may be poor due to one or more of the following reasons. (1) The weak intensity of the light source [18] at some wavebands. For active illumination multispectral acquisition systems, there may be not enough intensity of illumination at some waveband. (2) The different reflection characteristics [19] of the illuminated object to different wavelengths. Different objects show different optical characteristics at different wavelengths. Therefore, the image sensor may not receive enough intensity of light signal at some wavebands. (3) The different quantum efficiency [18] of the image sensor at different wavelengths. The sensitivity of the image sensor at different wavebands are different, so the image sensor may obtain electrical signals with low signal to noise ratio (SNR) at some low sensitivity wavebands. The essential reason why the above reasons lead to poor image quality at some wavebands comes down to the low amplitude of analog signal received by the ADC, resulting in a low SNR of the image. 2.2.2 Traditional methods In the current research, the following traditional methods are used to achieve the purpose of improving the SNR, but they all have great limitations. (1) Increase the exposure time of each frame to improve the SNR of images. But the extension of exposure time is limited by the dark current accumulation [21] of image sensors. (2) Acquire images with higher quality through the use of ultra-low noise detector. However, the performance of high-sensitivity imaging sensors, such as photomultiplier tube, intensified charge coupled device (CCD), back-illuminated CCD, or electron-multiplying CCD [20] are still limited under extremely low-light-level (LLL) conditions. (3) By using high-intensity illumination corresponding to the single-waveband images with poor quality, the amplitude of the analog signal received by the ADC can be improved. But if the light source device such as LED has reached its maximum brightness, or the light intensity irradiated on the object cannot be too strong, especially in biological tissue imaging, then this method will not work. 2.2.3 Principle of “synergy effect” In this paper, under the premise of fixed exposure time, the probability of each pixel point being at a higher level is greatly increased by multi-wavelength LED frequency-division modulation illumination, and the limitations of the traditional solution are reduced. In a multispectral imaging system, the combination of different wavelength sources (multi-wavelength source) can raise the level of the analog signal received by the ADC, thereby improving the quality of images in each waveband, that is the multi-wavelength “synergy effect”. There is always a variety of noise in circuits. And in digital image acquisition systems, there are two main types of random noise, one is the quantization noise in the ADC, as shown in Fig. 1, and the other is the resistance thermal noise in the circuit (such as dark current shot noise). Generally, ADC adopts fixed point system and the mantissa rounding method. Assume that the resolution of ADC is 8 bits, so the ADC can encode an analog input to one in q=2b different levels. Suppose that an image signal at point is x(n) ( x(n) x(n) e(n) ). x(n) represents the digital signal without quantization error, and e(n) represents quantization error. For the sake of simple analysis, it is assumed that e(n) is a stationary random noise independent of signal, which has the property of
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
white noise and obeys uniform distribution, and the probability density is expressed by ps(e), as shown in Fig. 2. The statistical mean and variance of e(n) are shown in formula (3)-(4): me
q/2
q / 2
e2
q/2
q / 2
eps (e)de 0 (3)
(e me )2 ps (e)de
1 2 q (4) 12
Fig. 1. Quantization. (a) An 8-level ADC coding scheme. (b) Quantization of sinusoidal signal.
ps ( e ) q 1
q / 2
0
q/2
Fig. 2. Probability density of quantization error.
According to equation (3) and (4), the statistical average of e(n) is me=0, and the average power (ie, mean square error) e2 = q2/12. So the quantization SNR of the ADC is: SNR( AD ) 10lg
x2 6.02b 1.76 10 lg x2 (5) e2
Therefore, the SNR of ADC is related to the resolution of ADC and the average power of the input signal. For an ADC with a certain resolution and within a certain dynamic range, the higher the input signal amplitude, the higher the quantization SNR. The multi-wavelength LEDs are introduced to strengthen the light intensity. On the one hand, as shown in equation (5)the increase in light intensity improve the SNR of data obtained by ADC, especially when the signal is close to full scale. On the other hand, since the thermal noise of the circuit is random and independent of the input signal, as the input signal increases, the proportion of the thermal noise in the total signal is greatly reduced. Therefore, the SNR of data is improved by introducing multi-wavelength LEDs, then the image quality can be improved. From another perspective, ADC is nonlinear, so when the amplitude of the analog signal is too close to the low or high rails of signal channels (photoelectric receiving, signal conditioning and ADC, et al.), the SNR decreases due to the large nonlinear error. The multi-wavelength LEDs are introduced to raise the quantization level, so that the signal amplitude of multi-wavelength superposition is close to the A/D full-scale range. But avoid too close to the upper rail of signal channels, then the A/D values will be in the linear working zone of the ADC. Hence the measurement accuracy can be improved, and the image quality can be improved further. In summary, according to the multi-wavelength “synergy effect”, the mutual improvement of image quality in each waveband can be achieved, especially the waveband that obtains stronger electrical signal in ADC has a better promotion effect on other wavebands. The problem of poor
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
quality of single-waveband images caused by light intensity, object characteristics, or the spectral response curve is improved. At the same time, under active illumination source, the superposition averaging of multi-sampled data points under a certain frequency is achieved by using demodulation of multi-wavelength images, which further suppresses random noise, thereby improves image quality. The multi-wavelength “synergy effect” and the method of superposition averaging also provide a reference for improving the quality of hyperspectral transmission images of breast in the future. 3.
Experiments
3.1 Experimental device and parameters Fig. 3 depicts a schematic of the experimental system configuration. The system consists of the following: three high-power LEDs (LED1、LED2 and LED3, with central wavelengths of 430nm、 590nm and 860nm ), an illuminated object, a CMOS industrial camera(JHSM500Bf), a program-control direct current voltage regulator power supply(0-30 V and 0-10 A adjustable), three multifunctional signal generators(MFSG), three constant current sources (CCS), and a computer used for image acquisition and image processing. The frame rate is 30 frames/s (fps), the captured image resolution is 400×450. In the experiment, the three LEDs are arranged at equal intervals according to the triangular shape, LED1、LED2、 LED3 are driven by sinusoidal signals with different frequencies of 0.5Hz、1Hz、2Hz. The sinusoidal signal is generated by MFSG and constant current source driving circuit. According to the rated current of the three LEDs, the working current of the LED varies from 0 to 600 mA. The CCS driving circuit is a voltage-to-current conversion circuit, and PT4115 constant current drive is chosen as the CCS driving circuit and supplied by program-control direct current voltage regulator power supply in this experiment. According to the parameters of PT4115 and LED working voltage, we choose the input voltage range of 0-10V to drive the LEDs, the voltage amplitude of the three sinusoidal signals with different frequencies is all set to 0.38V and the offset voltage is all set to 1V. The object is directly illuminated by LEDs in this experiment.
Fig. 3. System configuration.
3.2 Experimental process A. Image acquisition (1) The object is illuminated by LED1、LED2 and LED3 simultaneously. The three LEDs are
modulated by sinusoidal signals with different frequencies of 0.5 Hz, 1 Hz, and 2 Hz respectively. The effect of the third harmonic which has the greatest influence on the adjacent channel can be effectively avoided according to this kind of frequency selection. Then a group of 1200-frame image sequence is collected by the camera, which are represented as Xdei (i=1,2…….1200). One of the frames is shown in Fig. 4(a).
(2) The object is illuminated by LED1、LED2 and LED3 separately. The three LEDs are
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
modulated in the way of step (1). The captured image sequences are expressed as Pdei 、Ydei 、 Idei (i=1,2…….1200), as shown in Fig. 4(b-d).
Fig. 4. Original image collected by the camera. (a) Three-waveband original image. (b) Single-waveband original image with a central wavelength of 860nm. (c) Single-waveband original image with a central wavelength of 590nm. (d) Single-waveband original image with a central wavelength of 430nm.
B. Determination of the maximum number of image frames used for Fourier transform (1)
Xdei 、 Pdei 、 Ydei 、 Idei (i=1,2…….1200) are read into MATLAB program separately.
(2) Add the gray value of each frame, xi sum(sum( Xdei )) ( Pdei 、 Ydei 、 Idei in the same way), draw xi with i as the x-coordinate and the drawing result is shown in Fig. 5.
Fig. 5. Curve of the total gray value of each frame. The x-coordinate represents the frame of images and the y-coordinate represents the total gray value of each frame.
(3) Determine the number of image frames included in integer periods according to the curve shown in Fig. 5. By using the integer periodic FT, the spectral leakage caused by demodulation can be avoided. The x-coordinate of the point marked by the ellipse represent the location of start-stop frames of the image used for FT. C. Fast Fourier Transform(FFT) (1) Do FFT on xi (i=6,7……1188), the result after transform is shown in Fig. 6, then determine the x-coordinate values corresponding to the three frequency components with the highest amplitude and expressed as x pu 、 x ye 、 xin .
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
Fig. 6. Result after FFT. (a) The x-coordinate represents the frame of images and the y-coordinate represents the amplitude characteristics corresponding to each frequency point. (b) The x-coordinate represents frequency and the y-coordinate represents the amplitude of the original signal.
(2) According to the determined x-coordinate value x pu 、x ye 、xin , the time series composed of corresponding pixels of each frame is extracted to realize demodulation. Traversing the entire image, all pixels of the three-wavelength and single-wavelength images are demodulated respectively. Finally, single-waveband images( I S , I S , I S3 , I A , I A and I A3 ) of all center wavelengths in both cases are obtained by using the above method (3) In order to adapt to human eyes and improve the display effect of images on the 8-bit displayer, improve the contrast of the image using the 256-level grayscale stretch method: ’ I S1 255 X / (max(max( I S1 ))) ( I S 2 , I S3 , I A1 , I A2 and I A3 in the same way). The images after stretch are shown in Fig. 7. (4) Evaluate the image quality by using a non-reference image quality assessment(NR-IQA) method, and the evaluation results are shown in Table. 1. 1
4.
2
1
2
Results and Discussion
Whether it is the three-wavelength LED or the single-wavelength LED as the light source, single-waveband images are both obtained by demodulation. So in both situations, the equal-weighted superposition averaging of multi-sampled data points under a certain frequency is achieved, and the random noise is suppressed in a large degree. So each situation obtains a relatively better image quality. In general, there are two branches for evaluating image quality: subjective evaluation and objective evaluation. Subjective evaluation is the simplest and most reliable method, but it is limited by the resolution of the human eye. Therefore, objective evaluation methods are usually chosen to evaluate the image quality. In this paper, the reference image is unavailable, and thus full-reference methods are not applicable. Therefore, NR-IQA methods [22] that does not require any access to reference images are adopted to evaluate the images, such as Gray level (GL), Standard Deviation (SD), SNR based on human visual system (HSNR), Entropy Function Method (EFM), Spatial Frequency(SF), Energy of image gradient (EOG), Gray Mean Grads(GMG), and Brenner gradient (Brenner). The GL reflects the gray-scale resolution which determines the amount of information in the image. In LLL imaging, the gray-scale resolution of transmitted images has a crucial role in tissue classification and spatial information extraction. SD is the measurement of the dispersion degree of an image gray value relative to its mean value, as shown in equation (6). HSNR can evaluate the SNR of images and has a high correlation with human visual characteristics. EFM reflects the average amount of information carried by the image, which measures the amount of information in the image from the perspective of information theory, as shown in equation (7). SF indicates the overall activity of an image space, as shown in equation (8). Other gradient-based methods reflect the ability of image to express the contrast of small details. What's more, the higher the values of these image quality assessment criteria, the better the image quality. SD
1 m n 1 m n ( f (i, j ) ) 2 , where f (i, j ) (6) m n i 1 j 1 m n i 1 j 1
g
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
EFM P( x) log 2 P( x) (7) x 0
SF
1 m n 1 m n ( fi , j fi , j 1 ) 2 ( fi, j fi 1, j )2 (8) m n i 1 j 2 m n i 2 j 1
Where, m represents the row number of the image, n represents the column number of the image, and f(i,j) represents the gray value of the image in row i and column j. P(x) is the probability that some pixel value x appears in the image, and g is the range of gray value. In this section, we describe the experimental results in detail. Fig. 7(a-c) show three single-waveband images obtained by demodulating three-wavelength images with central wavelengths of 430nm、590nm、860nm respectively. Fig. 7(d-f) show three single-waveband images obtained by demodulating single-wavelength images with central wavelengths of 430nm、 590nm、860nm respectively. Since the conventional displayer can only display 8-bit grayscale image, part of the image information is lost. On the other hand, due to the limitation of the human eye resolution, there is no significant difference between the single-waveband images obtained in the two situations. Therefore, we need to compare the images with the aid of objective assessment, and the direct comparison of the two demodulation results under different objective assessment criteria is recorded in Table. 1. The equal-weighted superposition of images under different frequencies achieved by demodulation will greatly improve the SNR. Since the number of frames used in the demodulation of three-waveband images is exactly same as that used in the demodulation of single-waveband images. Then the difference between the two is caused by the multi-wavelength “synergy effect”. The assessment results of these criteria prove the existence of the “synergy effect” and its effectiveness in improving the quality of single-waveband images.
Fig. 7. Single-waveband images with central wavelengths of 430nm、590nm、860nm respectively. (a-c) The lighting source is a combination of three LEDs with wavelengths of 430nm、590nm and 860nm. Among them, (a) is the single-waveband image with a central wavelength of 430nm; (b) is the single-waveband image with a central wavelength of 590nm. (c) is the single-waveband image with a central wavelength of 860nm. (d-f) The light source is a single LED with three wavelengths of 430nm, 590nm and 860nm respectively. Among them, (d) is the single-waveband image with a central wavelength of 430nm; (e) is the single-waveband image with a central wavelength of 590nm. (f) is the single-waveband image with a central wavelength of 860nm. Table. 1. Comparison of the single-waveband images obtained by three-wavelength image demodulation and single-wavelength image demodulation by using the objective evaluation criteria. criteria
λ=430nm
single-waveband image λ=590nm
λ=860nm
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
GL SD HSNR EFM SF EOG GMG Brenner
three-wavele ngth LED 12644 37.7107 40.5724 6.0299 8.3902 1.2364e07 0.0142 1.7033e07
single-wavel ength LED 11003 32.6910 38.4460 5.7683 6.9267 8.4261e06 0.0117 1.2245e07
three-wavele ngth LED 14826 40.5302 39.6633 6.5183 6.5350 7.5028e06 0.0111 1.4450e07
single-wavel ength LED 14117 38.1353 36.4733 6.4421 5.9393 6.1978e06 0.0101 1.2649e07
three-wavele ngth LED 18968 53.4715 34.1814 7.0839 5.8538 6.0199e06 0.0099 1.1165e07
single-wavel ength LED 18636 52.3383 34.1572 7.0697 5.7375 5.7833e06 0.0097 1.0897e07
The acquisition of high-quality multispectral images depends on each aspect of image detection and processing. Only through strict control and improvement of enough aspects, can the quality of multispectral images be improved significantly. It is impossible to greatly improve the image quality only through the improvement of one aspect. In the future work, on the one hand, we hope to consider more aspects of image detection and processing and prepare for the improvement of multispectral image quality, then further provide a reference for improving the quality of hyperspectral transmission images. On the other hand, we will consider the combination with practical applications. After all, in a practical application, it takes too long for an imaging system to spend dozens of seconds collecting images. 5.
Conclusion
In this paper, we propose the multi-wavelength “synergy effect” and demonstrate its effectiveness in improving the image quality of each waveband. Through demodulation, the equal-weighted superposition averaging of multi-sampled data points under three different frequencies is achieved. Further, according to the multi-wavelength “synergy effect”, the image quality of each waveband is improved. The acquisition of high-quality multispectral images depends on each aspect of image detection and processing, and the analysis of multispectral images is based on the quality of each single-spectral image. In conclusion, this paper considers from one aspect of image detection and processing and improves the image quality of each waveband according to the multi-wavelength “synergy effect”, which provides a reference for the acquisition of high-quality multispectral images. Funding This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. Acknowledgements We thank the State Key Laboratory of Precision Measuring Technology and Instruments for the use of their equipment. Conflict of Interest The authors declare that they have no conflict of interest.
Reference [1]
[2] [3] [4]
[5]
[6]
R. Shrestha, J.Y. Hardeberg, How are LED illumination based multispectral imaging systems influenced by different factors?, A. Elmoataz, O. Lezoray, F. Nouboud, D. Mammass (eds) Image and Signal Processing(ICISP) 8539(2014) 61–71, https://doi.org/10.1007/978-3-319-07998-1_8. X. Jing, F. Wu, X. Zhu, X. Dong, F. Ma, Z. Li, Multi-spectral lowrank structured dictionary learning for face recognition, Pattern Recognit. 59(2016) 14-25, https://doi.org/10.1016/j.patcog.2016.01.023. S. Sun, H. Zhao, B. Jin, Robust tensor preserving projection for multispectral face recognition, Math. Probl. Eng. 2014(2014) 597245, https://doi.org/10.1155/2014/597245. M.D. Bounneche, L. Boubchir, A. Bouridane, B. Nekhoul, A. Ali-Cherif, Multi-spectral palmprint recognition based on oriented multiscale log-Gabor filters, Neurocomputing 205(2016) 274-286, https://doi.org/10.1016/j.neucom.2016.05.005. X. Xu, L. Lu, X. Zhang, H. Lu, W. Deng, Multispectral palmprint recognition using multiclass projection extreme learning machine and digital shearlet transform, Neural Comput. Appl. 27(2016) 143-153, https://doi.org/10.1007/s00521-014-1570-8. T. Durduran, R. Choe, W.B. Baker, A.G. Yodh, Diffuse optics for tissue monitoring and tomography, Rep. Prog. Phys. 73(2010) 076701, https://doi.org/10.1088/0034-4885/73/7/076701.
[7]
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
[8]
[9]
[10]
[11]
[12]
[13]
[14] [15]
[16]
[17] [18] [19]
[20] [21]
[22]
P. Taroni, A. Pifferi, E. Salvagnini, L. Spinelli, A. Torricelli, R. Cubeddu, Seven-wavelength time-resolved optical mammography extending beyond 1000 nm for breast collagen quantification, Opt. Express 17(2009) 15932-15946, https://doi.org/10.1364/OE.17.015932. P. Taroni, G. Danesini, A. Torricelli, A. Pifferi, L. Spinelli, R. Cubeddu, Clinical trial of time-resolved scanning optical mammography at 4 wavelengths between 683 and 975 nm, J. Biomed. Opt. 9(2004) 464–473, https://doi.org/10.1117/1.1695561. E.J. Walter, J.A. Knight, L. Lilge, A multi-wavelength, laser-based optical spectroscopy device for breast density and breast cancer risk pre-screening, J. Biophotonics 10(2017) 565-576, https://doi.org/10.1002/jbio.201600033. X. Li, Y. Wei, J. Xu, X. Feng, F. Wu, R. Zhou, J. Jin, K. Xu, X. Yu, Y. He, SSC and pH for sweet assessment and maturity classification of harvested cherry fruit based on NIR hyperspectral imaging technology, Postharvest Biol. Technol. 143(2018) 112-118, https://doi.org/10.1016/j.postharvbio.2018.05.003. P. Tsakanikas, D. Pavlidis, E. Panagou, G. Nychas, Exploiting multispectral imaging for non-invasive contamination assessment and mapping of meat samples, Talanta 161(2016) 606-614, https://doi.org/10.1016/j.talanta.2016.09.019. H.Y. Cen, R.F. Lu, D.P. Ariana, F. Mendoza, Hyperspectral Imaging-Based Classification and Wavebands Selection for Internal Defect Detection of Pickling Cucumbers, Food Bioprocess Technol. 7(2014) 1689-1700, https://doi.org/10.1007/s11947-013-1177-6. M.Z. Poh, D.J. McDuff, R.W. Picard, Non-contact, automated cardiac pulse measurements using video imaging and blind source separation, Opt. Express. 18(2010) 10762-10774, https://doi.org/10.1364/OE.18.010762. G. Li, C. Xiong, H.Q. Wang, L. Lin, B.J. Zhang, Y. Tong, Single-Trial Estimation of Dynamic Spectrum, Spectrosc. Spect. Anal. 31(2011) 1857-1861, https://doi.org/10.3964/j.issn.1000-0593(2011)07-1857-05. J. Yu, G. Li, S. Wang, L. Lin, Employment of the appropriate range of sawtooth-shaped-function illumination intensity to improve the image quality, OPTIK. 175(2018) 189-196, https://doi.org/10.1016/j.ijleo.2018.08.136. X. Yang, Y. Hu, G. Li, L. Lin, Optimized lighting method of applying shaped-function signal for increasing the dynamic range of LED-multispectral imaging system, Rev. Sci. Instrum. 89(2018) 025104, https://doi.org/10.1063/1.5022700. E.R. Fossum, D.B. Hondongwa, A Review of the Pinned Photodiode for CCD and CMOS Image Sensors, IEEE J. Electron Devices Soc. 2(2014) 33-43, https://doi.org/10.1109/JEDS.2014.2306412. J. Nakamura, Image Sensors and Signal Processing for Digital Still Cameras, first ed., CRC Press, 2005. S. Srinivasan, B.W. Pogue, S.D. Jiang, H. Dehghani, C. Kogel, S. Soho, J.J. Gibson, T.D. Tosteson, S.P. Poplack, K.D. Paulsen, Interpreting hemoglobin and. water concentration, oxygen saturation, and scattering measured in vivo by near-infrared breast tomography, PNAS. 100(2003) 12349-12354, https://doi.org/10.1073/pnas.2032822100. J.M. Roth, T.E. Murphy, C. Xu, Ultrasensitive and high-dynamic-range two-photon absorption in a GaAs photomultiplier tube, Opt. Lett. 27(2002) 2076-2078, https://dx.doi.org/10.1364/OL.27.002076. J.P. Carrere, S. Place, J.P. Oddou, D. Benoit, F. Roy, CMOS Image Sensor: Process impact on Dark current, IEEE International Reliability Physics Symposium (IRPS) 2014, https://doi.org/10.1109/IRPS.2014.6860620. V. Kamble, K.M. Bhurchandi, No-reference image quality assessment algorithms: A survey, OPTIK. 126(2015) 1080-1097, https://doi.org/10.1016/j.ijleo.2015.02.093.
PII: DOI: Reference:
S0030-4018(18)31149-0 https://doi.org/10.1016/j.optcom.2018.12.091 OPTICS 23759
To appear in:
Optics Communications
Received date : 1 November 2018 Revised date : 24 December 2018 Accepted date : 29 December 2018 Please cite this article as: H. Li, G. Li, W. Yan et al., “Synergy Effect” and its application in LED-multispectral imaging for improving image quality, Optics Communications (2019), https://doi.org/10.1016/j.optcom.2018.12.091 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Highlights
From: “Synergy Effect” and Its Application in LED-multispectral Imaging for Improving Image Quality Highlights:
The quality of single-waveband image has great influence on the analysis of multispectral images.
Multi-wavelength “Synergy Effect” is proposed to improve the image quality.
The image quality of each waveband is improved according to the “Synergy Effect”.
Providing a reference for the acquisition of high-quality multispectral images.
*Manuscript Click here to view linked References
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
“Synergy Effect” and Its Application in LED-multispectral Imaging for Improving Image Quality He Li a,b, Gang Li a,b, Wenjuan Yan c, Guoquan He c, Ling Lin a,b,* a
State Key Laboratory of Precision Measuring Technology and Instruments, Tianjin University, Tianjin 300072, China b Tianjin Key Laboratory of Biomedical Detecting Techniques and Instruments, Tianjin University, Tianjin 300072, China c School of Electronic Information Engineering, Yangtze Normal University, Chongqing 408100, China ∗ Corresponding author. E-mail addresses: (Ling Lin*)[email protected]; (He Li)[email protected]; (Gang Li) [email protected]; (Wenjuan Yan) [email protected]; (Guoquan He)[email protected]
Abstract: This paper proposes and demonstrates the multi-wavelength “synergy effect” in light emitting diode (LED)-multispectral images obtained by frequency-division modulation, which can be used to improve the image quality of each waveband. Taking three wavelengths in LED-multispectral imaging as an example, the following experiment was designed: The three different center wavelength ( 1 、 2 and 3 ) LEDs modulated by sinusoidal signals with different frequencies are used as the light source to acquire image sequences. The time series composed of corresponding pixels of each frame are demodulated by Fourier transform(FT) to obtain the three single-waveband images I S 、I S and I S3 respectively. Similarly, at the same driving intensity, the above three LEDs are separately used as the light source to acquire image sequences, and three groups of images are obtained respectively. Then FT is used to perform demodulation in the same way and three single-waveband images I A 、 I A and I A3 are obtained. By comparison, it can be concluded that the image quality of each waveband obtained by demodulating the three-wavelength images has been improved, which proves the “synergy effect” and provides a reference for improving the quality of multispectral images. 1
2
1
2
Keywords: Image quality; Multispectral image; Modulation and demodulation; Fourier transform; Single-waveband image 1.
Introduction
Multispectral image is composed of a set of images in a limited number of narrow or wider spectral bands. Multispectral imaging is intended to capture physical properties of objects beyond color, in the form of spectral transmittance or reflectance [1]. Now multispectral images have begun to be widely used in many fields, such as biometric security (face recognition [2,3] and palmprint recognition [4,5]), optical mammography [6-9] food inspection [10] and cultural heritage [11]. Multispectral images are widely used, but the application value is limited by the image quality. Researchers seek various methods to obtain high-quality images under different application backgrounds. Among them, LED illumination-based multispectral imaging (LEDMSI) is one kind of promising technique of fast and effective spectral image acquisition, which adopts the method of active light source illumination. Currently, researchers have proposed a large number of LEDMSI systems, which can be divided into two major categories according to the type of camera: monochrome camera based on the LEDMSI system (Mono-LEDMSI) and red-green-blue (RGB) camera based on the LEDMSI system (RGB-LEDMSI). The mono-LEDMSI system is considered in this paper, in which only one type of color LED light is lit at a time in principle. In digital image acquisition systems, quantization noise and resistance thermal noise in the system are two main types of random noise. The quantization noise is the difference between the actual analog value and the quantized digital value in analog-to-digital conversion (ADC), and the limited ADC resolution results in this kind of noise. Noise is a major factor affecting the performance of LEDMSI systems and the quality of spectral images, suppressing noise and
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
improving the quality of each single-waveband image is beneficial for further feature extraction, classification, target detection and recognition of images [12]. And the analysis of multispectral image is based on the quality of each single-spectral image, hence improving the quality of each single-waveband image is of great significance for the analysis of multispectral images. Superposition averaging method is an effective way to improve data accuracy. It has different representations in different applications and can be divided into spatial domain superposition averaging and time domain superposition averaging. In 2010, by calculating the spatial average value of the pixels in the interest region of RGB three-channel images separately, Ming-Zher Poh et al. [13] finally realized the heart rate detection through face video images. In 2011, the Single-Trial method proposed by Gang Li et al. [14] improved the efficiency and accuracy of dynamic spectral data processing by averaging the wavelength. In essence, FT (or phase-locked demodulation) is also one kind of superposition averaging. In this paper, through the demodulation of multi-wavelength images, the equal-weighted superposition averaging of multi-sampled points under three different frequencies is achieved, which greatly suppresses random noise and improves image quality. In previous studies, many researchers have focused on improving the quantization level of ADC to improve image quality. In 2018, J. Yu et al. [15] employed the appropriate range of sawtooth-shaped-function illumination intensity to raise the digital level and then improved the image quality. But this method is for ordinary color images and does not consider the characteristics of multispectral images. In 2018, X. Yang et al. [16] introduced auxiliary light at a higher sensitivity-camera area to increase the ADC quantization levels that are within the linear response zone of ADC. This method uses the shaped signal to improve the grayscale resolution from the perspective of multispectral acquisition, but the selection of LED is limited by the quantum efficiency of image sensor and this method is aimed at improving the quality of only one single-waveband image. And this paper combines the characteristics of Mono-LEDMSI system and breaks the principle that Mono-LEDMSI system only lights one color LED at a time, and simultaneously illuminates three LEDs by frequency-division modulation. Frequency-division modulation can not only suppress interference, such as dark current, ambient light, etc., but also achieve simultaneous acquisition of multi-wavelength images, saving time of image acquisition. The equal-weighted superposition averaging of multi-sampled data points under a certain frequency can be achieved by demodulation. At the same time, there exists “synergy effect” in the process of acquiring multi-spectral images by multi-wavelength frequency-division method, which can improve the image quality of each waveband obtained by demodulation and provide a reference for improving the quality of tissue hyperspectral transmission image in the future. 2.
Theories and Methods
2.1 Fourier Transform and digital demodulation of sinusoid-modulation signals In the data acquisition system, the signal in amplitude and time are both discrete, and the corresponding digital signal FT pair is shown in formula (1)-(2): ak n N x[n]e jkw0 n n N x[n]e jk (2 / N) n (1)
x[n]
1 N
k N
ak e jkw0 n
1 N
k N
ak e jk (2 / N) n (2)
In the multi-channel signal measurement of the frequency-division method, each information signal is modulated in the sending end, and its spectrum is moved into frequency bands that do not overlap each other. Then, they are transmitted together by one channel, and demodulated by the center frequency at the receiving end to separate the signals, thereby recovering the information signals that were originally transmitted. Demodulation according to formula (1) can restore each frequency signal and effectively suppress noise or interference of other frequencies. After all, in an actual data acquisition system, not only sine wave signals x[n] of different frequencies that need to be measured, but also noise and interference signals of other frequencies mixed in are acquired.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
1 N
is equivalent to the “average” in the measurement domain term, which means that
the more sampling values are used, the higher the accuracy of the result. But the premise that “average” has its effect is that white noise is inevitable in the system. 2.2 “Synergy effect” 2.2.1 Reason of poor image quality In the digital image acquisition system [17] of multispectral reflected images under active illumination, the image quality of some wavebands in a multispectral image may be poor due to one or more of the following reasons. (1) The weak intensity of the light source [18] at some wavebands. For active illumination multispectral acquisition systems, there may be not enough intensity of illumination at some waveband. (2) The different reflection characteristics [19] of the illuminated object to different wavelengths. Different objects show different optical characteristics at different wavelengths. Therefore, the image sensor may not receive enough intensity of light signal at some wavebands. (3) The different quantum efficiency [18] of the image sensor at different wavelengths. The sensitivity of the image sensor at different wavebands are different, so the image sensor may obtain electrical signals with low signal to noise ratio (SNR) at some low sensitivity wavebands. The essential reason why the above reasons lead to poor image quality at some wavebands comes down to the low amplitude of analog signal received by the ADC, resulting in a low SNR of the image. 2.2.2 Traditional methods In the current research, the following traditional methods are used to achieve the purpose of improving the SNR, but they all have great limitations. (1) Increase the exposure time of each frame to improve the SNR of images. But the extension of exposure time is limited by the dark current accumulation [21] of image sensors. (2) Acquire images with higher quality through the use of ultra-low noise detector. However, the performance of high-sensitivity imaging sensors, such as photomultiplier tube, intensified charge coupled device (CCD), back-illuminated CCD, or electron-multiplying CCD [20] are still limited under extremely low-light-level (LLL) conditions. (3) By using high-intensity illumination corresponding to the single-waveband images with poor quality, the amplitude of the analog signal received by the ADC can be improved. But if the light source device such as LED has reached its maximum brightness, or the light intensity irradiated on the object cannot be too strong, especially in biological tissue imaging, then this method will not work. 2.2.3 Principle of “synergy effect” In this paper, under the premise of fixed exposure time, the probability of each pixel point being at a higher level is greatly increased by multi-wavelength LED frequency-division modulation illumination, and the limitations of the traditional solution are reduced. In a multispectral imaging system, the combination of different wavelength sources (multi-wavelength source) can raise the level of the analog signal received by the ADC, thereby improving the quality of images in each waveband, that is the multi-wavelength “synergy effect”. There is always a variety of noise in circuits. And in digital image acquisition systems, there are two main types of random noise, one is the quantization noise in the ADC, as shown in Fig. 1, and the other is the resistance thermal noise in the circuit (such as dark current shot noise). Generally, ADC adopts fixed point system and the mantissa rounding method. Assume that the resolution of ADC is 8 bits, so the ADC can encode an analog input to one in q=2b different levels. Suppose that an image signal at point is x(n) ( x(n) x(n) e(n) ). x(n) represents the digital signal without quantization error, and e(n) represents quantization error. For the sake of simple analysis, it is assumed that e(n) is a stationary random noise independent of signal, which has the property of
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
white noise and obeys uniform distribution, and the probability density is expressed by ps(e), as shown in Fig. 2. The statistical mean and variance of e(n) are shown in formula (3)-(4): me
q/2
q / 2
e2
q/2
q / 2
eps (e)de 0 (3)
(e me )2 ps (e)de
1 2 q (4) 12
Fig. 1. Quantization. (a) An 8-level ADC coding scheme. (b) Quantization of sinusoidal signal.
ps ( e ) q 1
q / 2
0
q/2
Fig. 2. Probability density of quantization error.
According to equation (3) and (4), the statistical average of e(n) is me=0, and the average power (ie, mean square error) e2 = q2/12. So the quantization SNR of the ADC is: SNR( AD ) 10lg
x2 6.02b 1.76 10 lg x2 (5) e2
Therefore, the SNR of ADC is related to the resolution of ADC and the average power of the input signal. For an ADC with a certain resolution and within a certain dynamic range, the higher the input signal amplitude, the higher the quantization SNR. The multi-wavelength LEDs are introduced to strengthen the light intensity. On the one hand, as shown in equation (5)the increase in light intensity improve the SNR of data obtained by ADC, especially when the signal is close to full scale. On the other hand, since the thermal noise of the circuit is random and independent of the input signal, as the input signal increases, the proportion of the thermal noise in the total signal is greatly reduced. Therefore, the SNR of data is improved by introducing multi-wavelength LEDs, then the image quality can be improved. From another perspective, ADC is nonlinear, so when the amplitude of the analog signal is too close to the low or high rails of signal channels (photoelectric receiving, signal conditioning and ADC, et al.), the SNR decreases due to the large nonlinear error. The multi-wavelength LEDs are introduced to raise the quantization level, so that the signal amplitude of multi-wavelength superposition is close to the A/D full-scale range. But avoid too close to the upper rail of signal channels, then the A/D values will be in the linear working zone of the ADC. Hence the measurement accuracy can be improved, and the image quality can be improved further. In summary, according to the multi-wavelength “synergy effect”, the mutual improvement of image quality in each waveband can be achieved, especially the waveband that obtains stronger electrical signal in ADC has a better promotion effect on other wavebands. The problem of poor
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
quality of single-waveband images caused by light intensity, object characteristics, or the spectral response curve is improved. At the same time, under active illumination source, the superposition averaging of multi-sampled data points under a certain frequency is achieved by using demodulation of multi-wavelength images, which further suppresses random noise, thereby improves image quality. The multi-wavelength “synergy effect” and the method of superposition averaging also provide a reference for improving the quality of hyperspectral transmission images of breast in the future. 3.
Experiments
3.1 Experimental device and parameters Fig. 3 depicts a schematic of the experimental system configuration. The system consists of the following: three high-power LEDs (LED1、LED2 and LED3, with central wavelengths of 430nm、 590nm and 860nm ), an illuminated object, a CMOS industrial camera(JHSM500Bf), a program-control direct current voltage regulator power supply(0-30 V and 0-10 A adjustable), three multifunctional signal generators(MFSG), three constant current sources (CCS), and a computer used for image acquisition and image processing. The frame rate is 30 frames/s (fps), the captured image resolution is 400×450. In the experiment, the three LEDs are arranged at equal intervals according to the triangular shape, LED1、LED2、 LED3 are driven by sinusoidal signals with different frequencies of 0.5Hz、1Hz、2Hz. The sinusoidal signal is generated by MFSG and constant current source driving circuit. According to the rated current of the three LEDs, the working current of the LED varies from 0 to 600 mA. The CCS driving circuit is a voltage-to-current conversion circuit, and PT4115 constant current drive is chosen as the CCS driving circuit and supplied by program-control direct current voltage regulator power supply in this experiment. According to the parameters of PT4115 and LED working voltage, we choose the input voltage range of 0-10V to drive the LEDs, the voltage amplitude of the three sinusoidal signals with different frequencies is all set to 0.38V and the offset voltage is all set to 1V. The object is directly illuminated by LEDs in this experiment.
Fig. 3. System configuration.
3.2 Experimental process A. Image acquisition (1) The object is illuminated by LED1、LED2 and LED3 simultaneously. The three LEDs are
modulated by sinusoidal signals with different frequencies of 0.5 Hz, 1 Hz, and 2 Hz respectively. The effect of the third harmonic which has the greatest influence on the adjacent channel can be effectively avoided according to this kind of frequency selection. Then a group of 1200-frame image sequence is collected by the camera, which are represented as Xdei (i=1,2…….1200). One of the frames is shown in Fig. 4(a).
(2) The object is illuminated by LED1、LED2 and LED3 separately. The three LEDs are
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
modulated in the way of step (1). The captured image sequences are expressed as Pdei 、Ydei 、 Idei (i=1,2…….1200), as shown in Fig. 4(b-d).
Fig. 4. Original image collected by the camera. (a) Three-waveband original image. (b) Single-waveband original image with a central wavelength of 860nm. (c) Single-waveband original image with a central wavelength of 590nm. (d) Single-waveband original image with a central wavelength of 430nm.
B. Determination of the maximum number of image frames used for Fourier transform (1)
Xdei 、 Pdei 、 Ydei 、 Idei (i=1,2…….1200) are read into MATLAB program separately.
(2) Add the gray value of each frame, xi sum(sum( Xdei )) ( Pdei 、 Ydei 、 Idei in the same way), draw xi with i as the x-coordinate and the drawing result is shown in Fig. 5.
Fig. 5. Curve of the total gray value of each frame. The x-coordinate represents the frame of images and the y-coordinate represents the total gray value of each frame.
(3) Determine the number of image frames included in integer periods according to the curve shown in Fig. 5. By using the integer periodic FT, the spectral leakage caused by demodulation can be avoided. The x-coordinate of the point marked by the ellipse represent the location of start-stop frames of the image used for FT. C. Fast Fourier Transform(FFT) (1) Do FFT on xi (i=6,7……1188), the result after transform is shown in Fig. 6, then determine the x-coordinate values corresponding to the three frequency components with the highest amplitude and expressed as x pu 、 x ye 、 xin .
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
Fig. 6. Result after FFT. (a) The x-coordinate represents the frame of images and the y-coordinate represents the amplitude characteristics corresponding to each frequency point. (b) The x-coordinate represents frequency and the y-coordinate represents the amplitude of the original signal.
(2) According to the determined x-coordinate value x pu 、x ye 、xin , the time series composed of corresponding pixels of each frame is extracted to realize demodulation. Traversing the entire image, all pixels of the three-wavelength and single-wavelength images are demodulated respectively. Finally, single-waveband images( I S , I S , I S3 , I A , I A and I A3 ) of all center wavelengths in both cases are obtained by using the above method (3) In order to adapt to human eyes and improve the display effect of images on the 8-bit displayer, improve the contrast of the image using the 256-level grayscale stretch method: ’ I S1 255 X / (max(max( I S1 ))) ( I S 2 , I S3 , I A1 , I A2 and I A3 in the same way). The images after stretch are shown in Fig. 7. (4) Evaluate the image quality by using a non-reference image quality assessment(NR-IQA) method, and the evaluation results are shown in Table. 1. 1
4.
2
1
2
Results and Discussion
Whether it is the three-wavelength LED or the single-wavelength LED as the light source, single-waveband images are both obtained by demodulation. So in both situations, the equal-weighted superposition averaging of multi-sampled data points under a certain frequency is achieved, and the random noise is suppressed in a large degree. So each situation obtains a relatively better image quality. In general, there are two branches for evaluating image quality: subjective evaluation and objective evaluation. Subjective evaluation is the simplest and most reliable method, but it is limited by the resolution of the human eye. Therefore, objective evaluation methods are usually chosen to evaluate the image quality. In this paper, the reference image is unavailable, and thus full-reference methods are not applicable. Therefore, NR-IQA methods [22] that does not require any access to reference images are adopted to evaluate the images, such as Gray level (GL), Standard Deviation (SD), SNR based on human visual system (HSNR), Entropy Function Method (EFM), Spatial Frequency(SF), Energy of image gradient (EOG), Gray Mean Grads(GMG), and Brenner gradient (Brenner). The GL reflects the gray-scale resolution which determines the amount of information in the image. In LLL imaging, the gray-scale resolution of transmitted images has a crucial role in tissue classification and spatial information extraction. SD is the measurement of the dispersion degree of an image gray value relative to its mean value, as shown in equation (6). HSNR can evaluate the SNR of images and has a high correlation with human visual characteristics. EFM reflects the average amount of information carried by the image, which measures the amount of information in the image from the perspective of information theory, as shown in equation (7). SF indicates the overall activity of an image space, as shown in equation (8). Other gradient-based methods reflect the ability of image to express the contrast of small details. What's more, the higher the values of these image quality assessment criteria, the better the image quality. SD
1 m n 1 m n ( f (i, j ) ) 2 , where f (i, j ) (6) m n i 1 j 1 m n i 1 j 1
g
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
EFM P( x) log 2 P( x) (7) x 0
SF
1 m n 1 m n ( fi , j fi , j 1 ) 2 ( fi, j fi 1, j )2 (8) m n i 1 j 2 m n i 2 j 1
Where, m represents the row number of the image, n represents the column number of the image, and f(i,j) represents the gray value of the image in row i and column j. P(x) is the probability that some pixel value x appears in the image, and g is the range of gray value. In this section, we describe the experimental results in detail. Fig. 7(a-c) show three single-waveband images obtained by demodulating three-wavelength images with central wavelengths of 430nm、590nm、860nm respectively. Fig. 7(d-f) show three single-waveband images obtained by demodulating single-wavelength images with central wavelengths of 430nm、 590nm、860nm respectively. Since the conventional displayer can only display 8-bit grayscale image, part of the image information is lost. On the other hand, due to the limitation of the human eye resolution, there is no significant difference between the single-waveband images obtained in the two situations. Therefore, we need to compare the images with the aid of objective assessment, and the direct comparison of the two demodulation results under different objective assessment criteria is recorded in Table. 1. The equal-weighted superposition of images under different frequencies achieved by demodulation will greatly improve the SNR. Since the number of frames used in the demodulation of three-waveband images is exactly same as that used in the demodulation of single-waveband images. Then the difference between the two is caused by the multi-wavelength “synergy effect”. The assessment results of these criteria prove the existence of the “synergy effect” and its effectiveness in improving the quality of single-waveband images.
Fig. 7. Single-waveband images with central wavelengths of 430nm、590nm、860nm respectively. (a-c) The lighting source is a combination of three LEDs with wavelengths of 430nm、590nm and 860nm. Among them, (a) is the single-waveband image with a central wavelength of 430nm; (b) is the single-waveband image with a central wavelength of 590nm. (c) is the single-waveband image with a central wavelength of 860nm. (d-f) The light source is a single LED with three wavelengths of 430nm, 590nm and 860nm respectively. Among them, (d) is the single-waveband image with a central wavelength of 430nm; (e) is the single-waveband image with a central wavelength of 590nm. (f) is the single-waveband image with a central wavelength of 860nm. Table. 1. Comparison of the single-waveband images obtained by three-wavelength image demodulation and single-wavelength image demodulation by using the objective evaluation criteria. criteria
λ=430nm
single-waveband image λ=590nm
λ=860nm
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
GL SD HSNR EFM SF EOG GMG Brenner
three-wavele ngth LED 12644 37.7107 40.5724 6.0299 8.3902 1.2364e07 0.0142 1.7033e07
single-wavel ength LED 11003 32.6910 38.4460 5.7683 6.9267 8.4261e06 0.0117 1.2245e07
three-wavele ngth LED 14826 40.5302 39.6633 6.5183 6.5350 7.5028e06 0.0111 1.4450e07
single-wavel ength LED 14117 38.1353 36.4733 6.4421 5.9393 6.1978e06 0.0101 1.2649e07
three-wavele ngth LED 18968 53.4715 34.1814 7.0839 5.8538 6.0199e06 0.0099 1.1165e07
single-wavel ength LED 18636 52.3383 34.1572 7.0697 5.7375 5.7833e06 0.0097 1.0897e07
The acquisition of high-quality multispectral images depends on each aspect of image detection and processing. Only through strict control and improvement of enough aspects, can the quality of multispectral images be improved significantly. It is impossible to greatly improve the image quality only through the improvement of one aspect. In the future work, on the one hand, we hope to consider more aspects of image detection and processing and prepare for the improvement of multispectral image quality, then further provide a reference for improving the quality of hyperspectral transmission images. On the other hand, we will consider the combination with practical applications. After all, in a practical application, it takes too long for an imaging system to spend dozens of seconds collecting images. 5.
Conclusion
In this paper, we propose the multi-wavelength “synergy effect” and demonstrate its effectiveness in improving the image quality of each waveband. Through demodulation, the equal-weighted superposition averaging of multi-sampled data points under three different frequencies is achieved. Further, according to the multi-wavelength “synergy effect”, the image quality of each waveband is improved. The acquisition of high-quality multispectral images depends on each aspect of image detection and processing, and the analysis of multispectral images is based on the quality of each single-spectral image. In conclusion, this paper considers from one aspect of image detection and processing and improves the image quality of each waveband according to the multi-wavelength “synergy effect”, which provides a reference for the acquisition of high-quality multispectral images. Funding This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. Acknowledgements We thank the State Key Laboratory of Precision Measuring Technology and Instruments for the use of their equipment. Conflict of Interest The authors declare that they have no conflict of interest.
Reference [1]
[2] [3] [4]
[5]
[6]
R. Shrestha, J.Y. Hardeberg, How are LED illumination based multispectral imaging systems influenced by different factors?, A. Elmoataz, O. Lezoray, F. Nouboud, D. Mammass (eds) Image and Signal Processing(ICISP) 8539(2014) 61–71, https://doi.org/10.1007/978-3-319-07998-1_8. X. Jing, F. Wu, X. Zhu, X. Dong, F. Ma, Z. Li, Multi-spectral lowrank structured dictionary learning for face recognition, Pattern Recognit. 59(2016) 14-25, https://doi.org/10.1016/j.patcog.2016.01.023. S. Sun, H. Zhao, B. Jin, Robust tensor preserving projection for multispectral face recognition, Math. Probl. Eng. 2014(2014) 597245, https://doi.org/10.1155/2014/597245. M.D. Bounneche, L. Boubchir, A. Bouridane, B. Nekhoul, A. Ali-Cherif, Multi-spectral palmprint recognition based on oriented multiscale log-Gabor filters, Neurocomputing 205(2016) 274-286, https://doi.org/10.1016/j.neucom.2016.05.005. X. Xu, L. Lu, X. Zhang, H. Lu, W. Deng, Multispectral palmprint recognition using multiclass projection extreme learning machine and digital shearlet transform, Neural Comput. Appl. 27(2016) 143-153, https://doi.org/10.1007/s00521-014-1570-8. T. Durduran, R. Choe, W.B. Baker, A.G. Yodh, Diffuse optics for tissue monitoring and tomography, Rep. Prog. Phys. 73(2010) 076701, https://doi.org/10.1088/0034-4885/73/7/076701.
[7]
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
[8]
[9]
[10]
[11]
[12]
[13]
[14] [15]
[16]
[17] [18] [19]
[20] [21]
[22]
P. Taroni, A. Pifferi, E. Salvagnini, L. Spinelli, A. Torricelli, R. Cubeddu, Seven-wavelength time-resolved optical mammography extending beyond 1000 nm for breast collagen quantification, Opt. Express 17(2009) 15932-15946, https://doi.org/10.1364/OE.17.015932. P. Taroni, G. Danesini, A. Torricelli, A. Pifferi, L. Spinelli, R. Cubeddu, Clinical trial of time-resolved scanning optical mammography at 4 wavelengths between 683 and 975 nm, J. Biomed. Opt. 9(2004) 464–473, https://doi.org/10.1117/1.1695561. E.J. Walter, J.A. Knight, L. Lilge, A multi-wavelength, laser-based optical spectroscopy device for breast density and breast cancer risk pre-screening, J. Biophotonics 10(2017) 565-576, https://doi.org/10.1002/jbio.201600033. X. Li, Y. Wei, J. Xu, X. Feng, F. Wu, R. Zhou, J. Jin, K. Xu, X. Yu, Y. He, SSC and pH for sweet assessment and maturity classification of harvested cherry fruit based on NIR hyperspectral imaging technology, Postharvest Biol. Technol. 143(2018) 112-118, https://doi.org/10.1016/j.postharvbio.2018.05.003. P. Tsakanikas, D. Pavlidis, E. Panagou, G. Nychas, Exploiting multispectral imaging for non-invasive contamination assessment and mapping of meat samples, Talanta 161(2016) 606-614, https://doi.org/10.1016/j.talanta.2016.09.019. H.Y. Cen, R.F. Lu, D.P. Ariana, F. Mendoza, Hyperspectral Imaging-Based Classification and Wavebands Selection for Internal Defect Detection of Pickling Cucumbers, Food Bioprocess Technol. 7(2014) 1689-1700, https://doi.org/10.1007/s11947-013-1177-6. M.Z. Poh, D.J. McDuff, R.W. Picard, Non-contact, automated cardiac pulse measurements using video imaging and blind source separation, Opt. Express. 18(2010) 10762-10774, https://doi.org/10.1364/OE.18.010762. G. Li, C. Xiong, H.Q. Wang, L. Lin, B.J. Zhang, Y. Tong, Single-Trial Estimation of Dynamic Spectrum, Spectrosc. Spect. Anal. 31(2011) 1857-1861, https://doi.org/10.3964/j.issn.1000-0593(2011)07-1857-05. J. Yu, G. Li, S. Wang, L. Lin, Employment of the appropriate range of sawtooth-shaped-function illumination intensity to improve the image quality, OPTIK. 175(2018) 189-196, https://doi.org/10.1016/j.ijleo.2018.08.136. X. Yang, Y. Hu, G. Li, L. Lin, Optimized lighting method of applying shaped-function signal for increasing the dynamic range of LED-multispectral imaging system, Rev. Sci. Instrum. 89(2018) 025104, https://doi.org/10.1063/1.5022700. E.R. Fossum, D.B. Hondongwa, A Review of the Pinned Photodiode for CCD and CMOS Image Sensors, IEEE J. Electron Devices Soc. 2(2014) 33-43, https://doi.org/10.1109/JEDS.2014.2306412. J. Nakamura, Image Sensors and Signal Processing for Digital Still Cameras, first ed., CRC Press, 2005. S. Srinivasan, B.W. Pogue, S.D. Jiang, H. Dehghani, C. Kogel, S. Soho, J.J. Gibson, T.D. Tosteson, S.P. Poplack, K.D. Paulsen, Interpreting hemoglobin and. water concentration, oxygen saturation, and scattering measured in vivo by near-infrared breast tomography, PNAS. 100(2003) 12349-12354, https://doi.org/10.1073/pnas.2032822100. J.M. Roth, T.E. Murphy, C. Xu, Ultrasensitive and high-dynamic-range two-photon absorption in a GaAs photomultiplier tube, Opt. Lett. 27(2002) 2076-2078, https://dx.doi.org/10.1364/OL.27.002076. J.P. Carrere, S. Place, J.P. Oddou, D. Benoit, F. Roy, CMOS Image Sensor: Process impact on Dark current, IEEE International Reliability Physics Symposium (IRPS) 2014, https://doi.org/10.1109/IRPS.2014.6860620. V. Kamble, K.M. Bhurchandi, No-reference image quality assessment algorithms: A survey, OPTIK. 126(2015) 1080-1097, https://doi.org/10.1016/j.ijleo.2015.02.093.