- Email: [email protected]
Recovering the absolute phase maps of three selected spatial-frequency fringes with multi-color channels
ARTICLE IN PRESS
JID: NEUCOM
[m5G;April 14, 2017;22:0]
Neurocomputing 0 0 0 (2017) 1–7
Contents lists available at ScienceDirect
Neurocomputing journal homepage: www.elsevier.com/locate/neucom
Recovering the absolute phase maps of three selected spatial-frequency fringes with multi-color channels Yi Ding a, Jiangtao Xi b, Yanguang Yu b, Fuqin Deng c, Jun Cheng d,∗ a
School of Remote Sensing and Information Engineering, Wuhan University, 299 Bayi Road, Wuhan, China School of Electrical Computer and Telecommunications Engineering, University of Wollongong, Wollongong, NSW, 2522, Australia c Han’s Motor, Headquarter, 9018 Beihuan Road, Shenzhen, China d Guangdong Provincial Key Laboratory of Robotics and Intelligent System, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, 1068 Xueyuan Avenue, Shenzhen, China b
a r t i c l e
i n f o
Article history: Received 23 October 2015 Revised 17 March 2016 Accepted 29 April 2016 Available online xxx Keywords: Instrumentation Measurement and metrology Three-dimensional image acquisition Phase measurements Color channels
a b s t r a c t In a recent published work, we developed the technique to enhance the reliability of absolute phase maps by using the fringes of three spatial frequencies. However, it is time-consuming to capture the fringe images of three spatial frequencies with single channel in time sequence. To increase the efficiency of our proposed three-frequency technology, in this paper we propose a method to capture the fringe images of three spatial frequencies with multi-color channels and 3CCD camera. The projected spatial frequencies can be selected to guarantee the correctness of recovered fringe orders and avoid the chromatic aberration effect on frequency distortion. The cross talk among color channels can be eliminated effectively and the measured object can be reconstructed with high accuracy. The effectiveness of this method is verified by experimental results. © 2017 Elsevier B.V. All rights reserved.
1. Introduction Fringe projection profilometry (FPP) has become one of the most promising technologies for non-contact 3D shape measurement. The methods based on the phase maps of fringe patterns are the most widely utilized. A challenging task associated with existing phase measurement technique in FPP is phase unwrapping operation, which aims to recover the absolute phase maps from the wrapped phase maps. Existing phase unwrapping methods include spatial [1], temporal [2], and period coding [3]. However, recovery of absolute phase maps is still a challenging task when the wrapped phase maps contain noise, sharp changes or discontinuities. To achieve reliable and accurate phase unwrapping for FPP, a variety of temporal phase unwrapping approaches have been proposed. Huntley and Saldner [2] employ the multiple fringe patterns which are projected onto the object in time sequence, phase unwrapping can be carried out by comparing the wrapped phases of adjacent frequencies in order to avoid noise or boundaries and thus achieving correct recovery of the absolute phase map. While the method proposed in [2] is demonstrated to be effective for accurate phase unwrapping, it also suffers from the drawback of requiring many intermediate phase patterns, which is obviously not ∗
Corresponding author. E-mail address: [email protected] (J. Cheng).
suitable for fast or real-time measurement. In order to increase the efficiency, Zhao, et al. [4] propose to use two image patterns, one of which has a very low spatial frequency in contrast to the other. In particular, the low spatial frequency pattern only has a single fringe. Such a pattern has its absolute phase value falling within the range (−π , π ), and hence can be used as a reference to calculate the fringe number of the other fringe pattern, thus yielding its absolute phase map. Li, et al. [5] also employ the phase map of single fringe pattern as reference to unwrap high spatial frequency fringe patterns, and it is shown that the spatial frequency of the pattern to be unwrapped is determined by the level of noise. Following the same method in [4], Liu, et al. [6] project a single fringe pattern and a high frequency pattern in one shot to accelerate the speed of 3D measurement. These method works well in principle, but the gap between two spatial frequencies should be restricted within a range based on the noise level or steps in the low frequency phase maps. As the accuracy performance of FPP requires the use of high frequency fringe patterns, these methods may not work well when the phase maps are noisy. Consequently, multiple intermediate image patterns are still required in order to reduce the frequency gaps among adjacent patterns. Saldner and Huntley [7,8] study the multiple intermediate image patterns, showing that to unwrap a phase map of frequency f, log2 f + 1 sets of fringe patterns are required. A similar result is also reached by Zhang [9,10], indicating that the spatial frequency can be increased by a factor of 2 between two adjacent patterns.
http://dx.doi.org/10.1016/j.neucom.2016.04.074 0925-2312/© 2017 Elsevier B.V. All rights reserved.
Please cite this article as: Y. Ding et al., Recovering the absolute phase maps of three selected spatial-frequency fringes with multi-color channels, Neurocomputing (2017), http://dx.doi.org/10.1016/j.neucom.2016.04.074
JID: NEUCOM 2
ARTICLE IN PRESS
[m5G;April 14, 2017;22:0]
Y. Ding et al. / Neurocomputing 000 (2017) 1–7
Taking a typical FPP arrangement as an example where the image pattern has 16 fringes, 5 image patterns are still required with this approach. To reduce the frequencies used for temporal phase unwrapping, Zhong, et al. [11] also constructed a look-up table to unwrap the absolute phase maps for multiple-spatial-frequency fringes. This look-up table denotes the corresponding relationship from a pair of fringe orders at two spatial frequencies (f1 , f2 ) to [ f2 φ1 (x ) − f1 φ2 (x )]/2π . However, when the spatial frequencies f1 and f2 are large values, one value in [ f2 φ1 (x ) − f1 φ2 (x )]/2π may correspond to two or more pairs of fringe orders, thus the fringe orders cannot be determined uniquely. To make sure the values of [ f2 φ1 (x ) − f1 φ2 (x )]/2π unique, Zhong [12] proposed to generate the two relatively irrational spatial frequencies fringes by changing the projection angle of the grating. However, the minimal value gap of [ f2 φ1 (x ) − f1 φ2 (x )]/2π of two irrational frequencies is always smaller than the two rational frequencies [11,12], which may yield mistakes in determining fringe order pairs. In order to guarantee the uniqueness of recovered fringe orders, we have developed a temporal phase unwrapping technique based on the use of two fringe images with two selected frequencies [13]. When the two normalized spatial frequencies f1 and f2 are coprime, there exists a one-to-one map from [ f2 φ1 (x ) − f1 φ2 (x )]/2π to their fringe orders, where φ 1 (x), φ 2 (x) are the wrapped phase maps. The performance of the proposed method in [13] is limited by phase error tolerance bound, π /( f1 + f2 ) [14]. If the phase error of wrapped phase maps is larger than the phase error bound, errors may occur in the recovery of the absolute phase maps. In order to enhance the reliability of the recovered absolute phase maps, we propose to increase the phase error tolerance bound by the use of three fringe patterns with selected frequencies. The minimal value gap concerning the values of [ f2 φ1 (x ) − f1 φ2 (x )]/2π is increased significantly, resulting in a higher phase error tolerance bound [15]. The case of employing multiple spatial-frequency fringes is generalized for high accurate reconstruction [16]. However, the experiments in [15,16] are implemented by monochrome fringe patterns, at least 18 images are needed if we employ the six-step PSP (Phase-Shifting-Profilometry) to obtain the wrapped phase of three spatial-frequency fringes, which is not time-efficient for various applications. With the development of digital fringe projection system, multi-color fringe patterns have been employed for shape measurement in FPP. Huang et al. [17] project the color fringe patterns in 2π /3 phase shift at the same spatial frequencies onto the object and compensate the color channel coupling effect by intensity modulation parameter to obtain wrapped phase map. Skydan et al. [18] propose to use the colored fringe patterns at the same spatial frequencies from different angles to overcome the shadow problem of measured objects. Hu et al. [19] develop a blind approach to calculate the de-mixing matrix that no extra images are required for color calibration. However, these techniques only use the wrapped phase of single spatial frequency, which may lead to phase ambiguity for the objects with sharp changes or discontinuities. Pfortner et al. [20] use 3CCD camera to capture the interference patterns at three wavelengths generated by three laser sources simultaneously for distance measurement, while this method is more suitable to solve the problem of interferometry rather than fringe projection profilometry. Kinell [21] uses three color channels to carry three spatial frequencies to implement temporal phase unwrapping, in each channel the fringe images are projected and captured in phase shift sequences. The normalized three selected frequencies of three color channels are 14, 15, 16, errors occur in the recovered fringe orders due to the phase noise and spatial frequency selection. Zhang et al. [22] use the three color channels to carry the three normalized spatial frequencies 100, 99, 90 to implement temporal phase unwrapping. While these very high spatial frequencies will distort as the chromatic aberration effect, which
will lead to the errors in recovered fringe orders. Thus the actual projected frequencies should be adjusted according to the application scenarios, the actual spatial frequencies used in [22] are 100.196, 99 and 89.846. The estimation of proper actual spatial frequencies should be implemented before object measurement, which is not convenient for various applications. To accelerate the fringe image acquisition of our previous work in [15], in this paper we employ the color channels of projectors and 3CCD camera to capture the fringe patterns in a more time-efficient manner. Each channel carries the fringe pattern of one spatial frequency. Three spatial frequencies are selected based on the principles developed in [15], which yields high reliability for temporal phase unwrapping. Since the selected spatial frequencies are not very high, the frequency distortion caused by chromatic aberration is negligible, the actual projected frequencies need not to recalibrate before fringe projection. The experimental results validate the effectiveness of the proposed method, which enables our proposed three-frequency technology for time efficient applications. This paper is organized as follows. In Section 2 we revisit the technique to recover the absolute phase maps with three selected spatial-frequency fringes. In Section 3, we describe the multicolor channel scheme and strategies to eliminate the cross talk. In Section 4, experiments are presented to show the effectiveness of proposed method. Section 5 concludes the whole paper. 2. Absolute phase maps recovery with three frequency fringe patterns 2.1. Three -frequency technique Let us consider a FPP system, with which image patterns of three spatial frequencies are projected onto the object surface respectively. The image patterns are characterized by fringe structure where the light intensity is constant in y-axis and varies sinusoidally in x-axis. The normalized spatial frequencies of the multiple patterns are fi (i=1,2,3) referring to the total number of fringes on the respective patterns. Let us use i (x) (i=1,2,3) and φ i (x) (i=1,2,3) to denote respectively the absolute phase maps and the corresponding wrapped phase maps of the fringe patterns. Taking the central vertical line of images as the reference, the value of the wrapped phase map is limited by −π ≤ φi (x ) ≤ π (i=1,2,3), and the value of the absolute phase maps should fall into the following:
− fi π < i (x ) < fi π
(1)
Hence the absolute and wrapped phase maps are related by the following:
i (x ) = 2π mi (x ) + φi (x )
(2)
where mi (x) (i=1,2,3) are referred to as fringe numbers or indices. They are integers and − fi < mi (x ) < fi . Obviously, the absolute phases can be recovered if mi (x) are determined. In order to achieve this, the following relationships hold:
f j i (x ) = fi j (x )
(3)
where i = j. Combining Eqs. (2) and (3), we have:
f j φi ( x ) − f i φ j ( x ) = m j ( x ) f i − mi ( x ) f j 2π
(4)
Similar to the method employed in [15], an intermediate variable 0 (x) is introduced, which increases monotonically from −π to π with respect to x and defined as follows:
0 (x ) =
1 (x ) f1
=
2 (x ) f2
=
3 (x ) f3
(5)
Please cite this article as: Y. Ding et al., Recovering the absolute phase maps of three selected spatial-frequency fringes with multi-color channels, Neurocomputing (2017), http://dx.doi.org/10.1016/j.neucom.2016.04.074
ARTICLE IN PRESS
JID: NEUCOM
[m5G;April 14, 2017;22:0]
Y. Ding et al. / Neurocomputing 000 (2017) 1–7 Table 1. Mapping from 0 (x) [ f 3 φ2 (x ) − f 2 φ3 (x )]/2π ).
to
(m1 (x),
m2 (x),
0 ( x )
m1 (x), m2 (x), m3 (x)
11π /12 ≤ 0 (x) < π 9π /10 ≤ 0 (x) < 11π /12 13π /15 ≤ 0 (x) < 9π /10 9π /12 ≤ 0 (x) < 13π /15 11π /15 ≤ 0 (x) < 9π /12 7π /10 ≤ 0 (x) < 11π /15 9π /15 ≤ 0 (x) < 7π /10 7π /12 ≤ 0 (x) < 9π /15 5π /10 ≤ 0 (x) < 7π /12 7π /15 ≤ 0 (x) < 5π /10 5π /12 ≤ 0 (x) < 7π /15 5π /15 ≤ 0 (x) < 5π /12 3π /10 ≤ 0 (x) < 5π /15 3π /12 ≤ 0 (x) < 3π /10 3π /15 ≤ 0 (x) < 3π /12 π /10 ≤ 0 (x) < 3π /15 π /12 ≤ 0 (x) < π /10 π /15 ≤ 0 (x) < π /12 −π /15 < 0 (x ) < π /15 −π /12 < 0 (x ) ≤ −π /15 −π /10 < 0 (x ) ≤ −π /12 −3π /15 < 0 (x ) ≤ −π /10 −3π /12 < 0 (x ) ≤ −3π /15 −3π /10 < 0 (x ) ≤ −3π /12 −5π /15 < 0 (x ) ≤ −3π /10 −5π /12 < 0 (x ) ≤ −5π /15 −7π /15 < 0 (x ) ≤ −5π /12 −5π /10 < 0 (x ) ≤ −7π /15 −7π /12 < 0 (x ) ≤ −5π /10 −9π /15 < 0 (x ) ≤ −7π /12 −7π /10 < 0 (x ) ≤ −9π /15 −11π /15 < 0 (x ) ≤ −7π /10 −9π /12 < 0 (x ) ≤ −11π /15 −13π /15 < 0 (x ) ≤ −9π /12 −9π /10 < 0 (x ) ≤ −13π /15 −11π /12 < 0 (x ) ≤ −9π /10 −π < 0 (x ) ≤ −11π /12
5,6,7 5,5,7 4,5,7 4,5,6 4,4,6 4,4,5 3,4,5 3,4,4 3,3,4 2,3,4 2,3,3 2,2,3 2,2,2 1,2,2 1,1,2 1,1,1 0,1,1 0,0,1 0,0,0 0,0,−1 0,−1,−1 −1,−1,−1 −1,−1,−2 −1,−2,−2 −2,−2,−2 −2,−2,−3 −2,−3,−3 −2,−3,−4 −3,−3,−4 −3,−4,−4 −3,−4,−5 −4,−4,−5 −4,−4,−6 −4,−5,−6 −4,−5,−7 −5,−5,−7 −5,−6,−7
Considering 0 (x ) = 1 (x )/ f1 and taking account of Eq. (1), mi (x) (i=1,2,3) can be determined by the value of 0 (x) as follows:
mi ( x ) =
⎧ fi /2 ⎪ ⎪ ⎪ ... ⎪ ⎪ ⎪ ⎨ 1 ..
⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩
0 −1 ... − fi /2
m3 (x))
[ fi − ( fi mod 2 + 1 )]π / fi ≤ 0 (x ) < π ... π / fi ≤ 0 (x ) < 3π / fi −π / fi < 0 (x ) < π / fi −3π / fi ≤ 0 (x ) < −π / fi ... −π < 0 (x ) ≤ −[ fi − ( fi mod 2 + 1 )]π / fi (6)
where x denotes the operation of taking the largest integer not greater than x. From [15], we know that when the normalized three spatial frequencies do not have the common factor larger than 1, there exists f φ ( x )− f φ ( x ) an unique mapping from (m1 (x), m2 (x), m3 (x)) to ( 1 2 2π 2 1 , f 2 φ3 ( x ) − f 3 φ2 ( x ) ). 2π
If (f1 , f2 ) have the common factor larger than 1 and (f2 , f3 ) have the common factor larger than 1, the phase error bound of three frequency technique is significantly increased compared to the two frequency technique in [14]. The errors in fringe number can be eliminated effectively. Thus, we have the following steps to obtain the absolute phase maps of the fringes at three spatial frequencies: 1. Select multiple frequencies (f1 , f2 , f3 ) to make sure a unique mapping from (m1 (x ) f2 − m2 (x ) f1 , m1 (x ) f3 − m3 (x ) f1 ) to (m1 (x), m2 (x), m3 (x));
and
3
([ f 3 φ1 (x ) − f 1 φ3 (x )]/2π ,
f 3 φ1 (x )− f 1 φ3 (x ) 2π
f 3 φ2 (x )− f 2 φ3 (x ) 2π
−5 −5 10 0 0 −10 5 −5 −5 10 0 0 −10 5 5 −5 10 10 0 −10 −10 5 −5 −5 10 0 0 −10 5 5 −5 10 0 0 −10 5 5
−6 9 9 −3 12 0 0 −12 3 3 −9 6 −6 −6 9 −3 −3 12 0 −12 3 3 −9 6 6 −6 9 −3 −3 12 0 0 −12 3 −9 −9 6
2. Project onto the object with three fringe patterns of spatial frequencies (f1 , f2 , f3 ) respectively, acquiring multiple wrapped phase maps by a phase detection algorithm; 3. Calculate the terms mi (x ) f j − m j (x ) fi , round their values into the closest integers, denoted as Mi . Look up the mapping relationship from (m1 (x ) f2 − m2 (x ) f1 , m1 (x ) f3 − m3 (x ) f1 ) to (m1 (x), m2 (x), m3 (x)), find the entries whose values of mi (x ) f j − m j (x ) fi are the closest to Mi . 4. Record the corresponding mi (x) (i=1,2,3) and reconstruct the absolute phase maps by Eq. (2) using mi (x). As an example, we select three spatial frequencies as f 1 = 10, f2 = 12, f3 = 15 to construct the look-up table. The frequency pairs (10,15) and (12,15) has common factor as 5 and 3, respectively, which are the largest common factors in this three frequency f φ ( x )− f φ ( x ) f φ ( x )− f φ ( x ) group. The look-up table from ( 3 1 2π 1 3 , 3 2 2π 2 3 ) to (m1 (x), m2 (x), m3 (x)) are given in Table 1: f φ ( x )− f φ ( x ) From the table we can see, the mapping from ( 3 1 2π 1 3 , f 3 φ2 ( x ) − f 2 φ3 ( x ) ) 2π
to (m1 (x), m2 (x), m3 (x)) is unique, which means we could obtain (m1 (x), m2 (x), m3 (x)) by calculating f φ ( x )− f φ ( x ) f φ ( x )− f φ ( x ) the ( 3 1 2π 1 3 , 3 2 2π 2 3 ). The minimal value gaps of f 3 φ1 ( x ) − f 1 φ3 ( x ) 2π
f φ ( x )− f φ ( x )
and 3 2 2π 2 3 are 5 and 3, respectively. According to [15], the phase error tolerance bound of the frequency pairs (fi , fj ) with maximal common factor tij is:
0 ≤ φmax <
ti j π fi + f j
(7)
Please cite this article as: Y. Ding et al., Recovering the absolute phase maps of three selected spatial-frequency fringes with multi-color channels, Neurocomputing (2017), http://dx.doi.org/10.1016/j.neucom.2016.04.074
JID: NEUCOM 4
ARTICLE IN PRESS
[m5G;April 14, 2017;22:0]
Y. Ding et al. / Neurocomputing 000 (2017) 1–7
Fig. 1. The scheme of color-channel projection.
This means that when the actual phase error is less than this bound, the recovered fringe orders will be correct, when the actual phase error is larger than this bound, the recovered fringe orders may be wrong. For the amplitude-limited phase error, when the frequencies are appropriately selected, the correctness of the recovered fringe order can be guaranteed. This property is very used in multi-color channel scheme, we can select the spatial frequencies based on Eq. (7) to avoid the errors in recovered fringe orders for accurate three dimensional reconstruction.
mate the coupling matrix. For example, the red channel, three pure fringe images with 2π /3 shift are projected onto a white reference plane and captured by 3CCD camera, the initial phases of three phase shifted images are (0, 2π /3, 4π /3). We define the three phase shifted images as:
S1 = A + B cos[], S2 = A + B cos[ + 2π /3], S3 = A + B cos[ + 4π /3]. average
A=
intensity
2 1/2
3. Time efficient projection scheme and coupling elimination In the experiments of [15,16], the fringe images of three spatial frequencies are acquired in time series respectively. To accelerate the procedure of fringe image acquisition, we employ three color channel of camera and 3CCD camera to capture the different spatial frequency fringes. Fig. 1 illustrates the procedure: Each channel carries the fringe pattern of one spatial frequencies. In each channel the initial phase is shifted by 2π /3 for three times, thus the initial phases of the fringe patterns in the same channel are 0,2π /3, 4π /3. In this scheme, we can capture the shift fringe images of three spatial frequencies in color channel by three times rather than the eighteen times with single channel as [15,16], the time consumption in image capture has been greatly reduced. At the same time, the multi-color capture scheme brings the coupling effects among the fringe patterns in different channels, to reconstruct the object with high accuracy, it is necessary to eliminate the coupling effects to correct the distorted fringe patterns. The methods to eliminate the coupling effects can be divided into two categories, one kind of method is based on estimating of the intensity of coupling effects among color channels and correcting the distorted fringe patterns, another kind of method is based on the color filter in front of 3CCD camera to reduce the coupling effects at source. The method to estimate the coupling intensity has been established in [17], which uses a coupling matrix to describe the coupling effects. Here we implement the similar procedure to esti-
S1 +S2 +S3 , 3
modulation
intensity
B=
[3 (S2 −S3 )2 +(2S1 −S2 −S3 ) ] 3
. For three color channels, we have nine phase shifted images. Then we can separate the RGB components of each color image, yielding twenty-seven gray images. By these separated images, we can use the coupling intensity to describe the magnitude of coupling effect, the coupling intensity Iij is calculated by the ratio of intensity modulation B between channels. The coupling matrix among multi-color channels is constructed as:
Irr Igr Ibr
Irg Igg Ibg
Irb Igb Ibb
where Iij denotes the coupling intensity from channel i to channel j, the elements on the main diagonal are normalized as 100, that is Irr = Igg = Ibb . By calculating this matrix, the elements can be used to compensate the amplitude distortion of fringe patterns in each channel for high accurate measurement. In compensation processing, the cross talk components are subtracted to reserve the original color fringes, the detailed equations can refer to the equations (13)–(15) in [17]. The coupling matrix based method can reduce the linear distortion caused by multi-color channels. For the nonlinear distortion, Hu et al. [19] propose to use the blind separation filtering method to recover the deformed fringe patterns. However, this algorithm is time consuming, which is not suitable for time efficient measurement. It is more effective to reduce the nonlinear distortion among multi-color channels with hardware. Color dielectric filters in front of camera can eliminate the cross talk at source significantly, the computational burden of noise elimination can be greatly reduced.
Please cite this article as: Y. Ding et al., Recovering the absolute phase maps of three selected spatial-frequency fringes with multi-color channels, Neurocomputing (2017), http://dx.doi.org/10.1016/j.neucom.2016.04.074
ARTICLE IN PRESS
JID: NEUCOM
[m5G;April 14, 2017;22:0]
Y. Ding et al. / Neurocomputing 000 (2017) 1–7
5
Fig. 2. Coupling effects among color channels when extra filters used. The three rows correspond to the coupling effects of red, green and blue channels. In each row, the first image is the captured color image and the others are the corresponding RGB components. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
4. Experiment In order to verify the performance of the proposed technique, we implement experiment with multi-color projection. The camera in the experiment is DaHeng industry HV1351 high resolution 3CCD camera, the lens is Computar industry 16 mm lens, the projector is LG HW300 Multimedia DLP Projector. In the experiment, firstly, the cross talk among color channels are evaluated and compensated on white reference plane to eliminate the error in wrapped phase for high quality three dimensional reconstructions. Then the maximal error in wrapped phase is measured on white reference plane to test whether the errors exceed the error tolerance bound of selected spatial frequencies, if the errors exceed the bound, the spatial frequencies should be changed to increase the error tolerance bound to guarantee the correctness of recovered fringe orders. Thirdly, the fringes at selected spatial frequencies on three color channels are projected onto the measured object for three dimensional reconstructions. These steps are explained in details as follows. In the first step of experiment, we use 3CCD camera to capture color fringe images projected onto a white reference plane when the extra filters (Comar product numbers 465 1k 50, 540 1B 50 and 650 1Y50) are used. The aim of this step is to estimate the coupling matrix of three color channels. The captured color fringe images are separated into RGB components as shown in Fig. 2: Based on these RGB components, we calculate the elements of coupling matrix as:
100.0 15.5 6.8
1.5 100.0 3.1
1.2 3.6 100.0
From the matrix we can see most of the coupling effect is eliminated by extra filters. The coupling effects of multi-color channels are determined by the hardware of DLP projector, thus the estimation of coupling effect at different spatial frequencies is similar. It is not necessary to recalibrate the coupling effect with the fringe patterns at other spatial frequencies. The residue of coupling effects can be compensated by the method in [17].
In the second step of experiment, we select the spatial frequencies based on the phase error in three channels to guarantee the correctness of recovered fringe orders. In the single channels experiment in [15,16], the phase error tolerance bounds of spatial frequency group (10, 12, 15) are much larger than the maximal phase error and guarantee the accurate three dimensional reconstruction. We can firstly select this spatial frequency group, test this group whether its phase error bounds larger than the phase error in each channel. If it is, we can use this group for three dimensional reconstructions, if it is not, we can change frequency group till its phase error bounds are larger than the phase error in each channel. In order to obtain the maximum absolute value of phase error, we project the red, green, blue color fringes of spatial frequency 10, 12, 15 onto a white reference plane and obtain the wrapped phase maps of these three frequencies. Then we generate the ideal wrapped phase maps of spatial frequencies 10, 12, 15 by computer. The phase errors of three wrapped phase maps can be obtained by the subtraction between actual and ideal wrapped phase maps of each channel. The cross talk changes the amplitude and phase of original fringe in each channel, this error in wrapped phase calculated by phase shifting algorithm is amplitude-limited [23], thus the maximal error in wrapped phase caused by cross talk could be estimated. Among three color channels, the maximal absolute value of error in wrapped phase maps is 0.1675. Comparing with the phase error bound given by Eq. (7), we know that the maximum absolute value of phase error is smaller than the phase error bound, 0.1675 < 5π /(10 + 15 ) = π /5 and 0.1675 < 3π /(12 + 15 ) = π /9. In this case, the absolute phase maps should be recovered correctly. In another aspect, the chromatic aberration may lead to the distortion of high spatial frequencies, which may yield errors in three dimensional reconstructions. In [22], the phase map of unit frequency is obtained by heterodyning the wrapped phase maps at frequencies 100 and 99, the wrapped phase map of frequency 10 is obtained by heterodyning the wrapped phase maps at frequencies 100 and 90. The chromatic aberration would make the frequencies 100, 99, 90 distorted, thus the resulted phase maps will not correspond to the assumed spatial frequencies, the recovered absolute phase will be wrong.
Please cite this article as: Y. Ding et al., Recovering the absolute phase maps of three selected spatial-frequency fringes with multi-color channels, Neurocomputing (2017), http://dx.doi.org/10.1016/j.neucom.2016.04.074
JID: NEUCOM 6
ARTICLE IN PRESS
[m5G;April 14, 2017;22:0]
Y. Ding et al. / Neurocomputing 000 (2017) 1–7
Based on these considerations, we select three frequencies as (10, 12, 15), these spatial frequencies can be achieved by controlling the projector pitches in our experiments. Compared with [21], the three spatial frequencies of three color channels are selected as (14, 15, 16). However, these three spatial frequencies will yield a much smaller phase error tolerance bound than our selection, which can be easily exceeded by phase error. The experiment results in [21] show the reconstruction errors due to the spatial frequency selection. The chromatic aberration is negligible in our selected spatial frequencies, the actual projected frequencies need not to recalibrate before measurements. Thus, this spatial-frequency selection enables us with high accurate and efficient three-dimensional reconstruction. In the third step of experiment, we project three spatial frequencies 10, 12, 15 with red, green, blue channels onto the measured object. The measured object is a plaster toy, which is characterized by complex surface such as sharp changes and discontinuities around head and ears. Similar surfaces can be found in many engineering applications, thus it is used to demonstrate the effectiveness of proposed method. The length (from head to foot) of the test object is 151.2 mm, the width (from left ear to right ear) is 125.7 mm. The toy with color fringe patterns is shown in Fig. 3a. The recovered absolute phase map of toy at frequencies 15 is shown in Fig. 3b. Three dimensional reconstructed result is depicted in Fig. 3c. From Fig. 3 and the original plaster toy, we can see the absolute phase is correctly recovered, the measured small toy is reconstructed accurately. Since the maximal error in phase is 0.1675, the maximal error in height is 0.013 mm. By selecting the spatial frequencies appropriately, our method improves the reliability of recovered fringe orders and avoids the frequency distortion effect caused by chromatic aberration. Thus it is suitable for timeefficient and accurate measurement. Compared with the method in [22], we avoid the accumulated error introduced by the heterodyning among the fringe images of different spatial frequencies. Due to the proper spatial frequency selection, the chromatic aberration effect is not obvious in our work. Thus we need not to calibrate the spatial frequency before measurement, the process of measurement with multi-color channels can be obviously simplified. In future, we can explore the techniques to synthesis the multi-view results together for complete three dimensional reconstructions [24].
a
b
c
5. Conclusion This paper develops a time efficient temporal phase unwrapping scheme based on the projection of three-frequency fringe images with three color channels. The color channels accelerate the fringe image capture significantly, compared to single channel, only one third fringe images should be sampled. Based on the frequency selection principle developed in [15], we can select proper spatial frequencies for three color channels to improve the reliability of recovered fringe orders and avoid the problem of chromatic aberration. The cross talk among different color channels is eliminated by the coupling matrix based technique and filters. Thus the proposed time efficient three frequency technique can be employed for high speed and accurate three dimensional data acquisition. Acknowledgment This work is partially supported by Science and Technology Service Network, Chinese Academy of Sciences (KFJEW-STS-035/01), the National Natural Science Foundation of China (61403365), CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology (2014DP173025), Guangdong Technology Project
Fig. 3. Experiment results when f 1 = 10, f 2 = 12, f 3 = 15 on red, green, blue channels. (a) is the deformed composite color fringe pattern; (b) is the recovered phase maps at frequency f 3 = 15; (c) is three dimensional reconstruction result at frequency f 3 = 15. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
Please cite this article as: Y. Ding et al., Recovering the absolute phase maps of three selected spatial-frequency fringes with multi-color channels, Neurocomputing (2017), http://dx.doi.org/10.1016/j.neucom.2016.04.074
JID: NEUCOM
ARTICLE IN PRESS
[m5G;April 14, 2017;22:0]
Y. Ding et al. / Neurocomputing 000 (2017) 1–7
(2016B010108010, 2016B010125003) and Shenzhen Technology Project (JSGG20160331185256983, JSGG20160229115709109). References [1] S. Zhang, X. Li, S. Yau, Multilevel quality-guided phase unwrapping algorithm for real-time dimensional shape reconstruction, Appl. Opt 46 (2007) 50–57. [2] J.M. Huntley, H.O. Saldner, Temporal phase-unwrapping algorithm for automated interferogram analysis, Appl. Opt. 32 (1993) 3047–3052. [3] H.J. Chen, J. Zhang, D.J. Lv, J. Fang, 3-D shape measurement by composite pattern projection and hybrid processing, Opt. Express 15 (2007) 12318–12330. [4] H. Zhao, W. Chen, Y. Tan, Phase-unwrapping algorithm for the measurement of three-dimensional object shapes, Appl. Opt. 33 (1994) 4450–4497. [5] J. Li, L.G. Hassebrook, C. Guan, Optimized two-frequency phase-measuring-profilometry light-sensor temporal-noise sensitivity, J. Opt. Soc. Am. A 20 (2003) 106–115. [6] K. Liu, Y. Wang, D.L. Lau, Q. Hao, L.G. Hassebrook, Dual-frequency pattern scheme for high-speed 3-D shape measurement, Opt. Express 18 (2010) 5229–5244. [7] J.M. Huntley, H.O. Saldner, Error-reduction methods for shape measurement by temporal phase unwrapping, J. Opt. Soc. Am. A 14 (1997) 3188–3196. [8] H.O. Saldner, J.M. Huntley, Temporal phase unwrapping: application to surface profiling of discontinuous objects, Appl. Opt. 36 (1997) 2770–2775. [9] S. Zhang, Digital multiple-wavelength phase-shifting algorithm, SPIE Proceedings 7432 (2009) 74320N. [10] S. Zhang, Phase unwrapping error reduction framework for a multiple-wavelength phase-shifting algorithm, Opt. Eng. 48 (2009). [11] J. Zhong, M. Wang, Phase unwrapping by a lookup table method: application to phase maps with singular points, Opt. Eng. 38 (1999) 2075–2080. [12] J. Zhong, Linear integer unconcerned phase-map profilometry by changing the projection angle of the grating, Opt. Eng. 40 (2001) 1377–1382. [13] Y. Ding, J. Xi, Y. Yu, J. Chicharo, Recovering the absolute phase maps of two fringe patterns with selected frequencies, Opt. Lett. 36 (2011) 2518–2520. [14] Y. Ding, J. Xi, Y. Yu, W. Cheng, S. Wang, J. Chicharo, Frequency selection in absolute phase maps recovery with two frequency projection fringes, Opt. Express 20 (2012) 13228–13251. [15] Y. Ding, J. Xi, Y. Yu, F. Deng, Absolute phase recovery of three fringe patterns with selected spatial frequencies, Opt. Lasers Eng. 70 (2015) 18–25. [16] Y. Ding, J. Xi, Y. Yu, F. Deng, J. Cheng, Multiple spatial-frequency fringes selection for absolute phase recovery, Surf. Topography: Metrology Prop. 4 (2016) 015004. [17] P. Huang, Q. Hu, F. Jin, F. Chiang, Color-encoded digital fringe projection technique for high-speed three-dimensional surface contouring, Opt. Eng. 38 (1999) 1065–1071. [18] O.A. Skydan, M.J. Lalor, D.R. Burton, Technique for phase measurement and surface reconstruction by use of colored structured light, Appl. Optics 41 (2002) 6104–6107. [19] Y. Hu, J. Xi, J. Chicharo, Z. Yang, Blind color isolation for color-channel-based fringe pattern profilometry using digital projection, J. Opt. Soc. Am. A 24 (2007) 2372–2382. [20] A. Pfortner, J. Schwider, Red-green-blue interferometer for the metrology of discontinuous structures, Appl. Optics 42 (2003) 667–673. [21] L. Kinell, Multichannel method for asolute shape measurement using projected fringes, Opt. Lasers Eng. 41 (2004) 57–71. [22] Z. Zhang, C.E. Towers, D.P. Towers, Color fringe projection system based on optimum frequency selection, SPIE Proceedings 6292 (2006) 62920S. [23] B. Pan, K. Qian, L. Huang, A. Asundi, Phase error analysis and compensation for nonsinusoidal waveforms in phase-shifting digital fringe projection profilometry, Opt. Lett. 34 (2009) 416–418. [24] Y. Yang, X. Wang, Q. Liu, M. Xu, A bundled-optimization on model of multiview dense depth map synthesis for dynamic scene reconstruction, Inf. Sci. 320 (2015) 306–319.
7 Yi Ding is now the assistant Professor at School of Remote Sensing and Information Engineering, Wuhan University. He received his Ph.D. Degree in Electronic Engineering from Huazhong University of Science and Technology, China, 2013. During his PhD study, he has been the visiting PhD in School of ECTE, University of Wollongong, Australia from 2009 to 2011. He has worked as Research Associate in Department of Electrical and Electronic Engineering, University of Hong Kong from 2013 to 2014. His current research interests include 3D data acquisition and machine vision.
Jiangtao Xi is now the full professor and head of School of ECTE, University of Wollongong. Professor Xi has found the Optoelectronics Signal Processing Laboratory in School ECTE, University of Wollongong. His current research interests include 3D data acquisition, Optoelectronics signal processing and signal processing in wireless communications.
Yanguang Yu is now the associate professor in School of ECTE, University of Wollongong. Her current research interests include Optoelectronics signal processing and 3D data acquisition.
Fuqin Deng is now the technical director in Han’s motor S&T Co. LTD., Shenzhen. He received his Ph.D. Degree in Electronic Engineering from Department of Electrical and Electronic Engineering, University of Hong Kong, 2014. His current research interests include 3D data acquisition, computer vision and machine vision applications.
Jun Cheng is now the professor and director of Laboratory for Human Machine Control in Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences. His current research interests include computer vision, human compter interface and robotics.
Please cite this article as: Y. Ding et al., Recovering the absolute phase maps of three selected spatial-frequency fringes with multi-color channels, Neurocomputing (2017), http://dx.doi.org/10.1016/j.neucom.2016.04.074
JID: NEUCOM
[m5G;April 14, 2017;22:0]
Neurocomputing 0 0 0 (2017) 1–7
Contents lists available at ScienceDirect
Neurocomputing journal homepage: www.elsevier.com/locate/neucom
Recovering the absolute phase maps of three selected spatial-frequency fringes with multi-color channels Yi Ding a, Jiangtao Xi b, Yanguang Yu b, Fuqin Deng c, Jun Cheng d,∗ a
School of Remote Sensing and Information Engineering, Wuhan University, 299 Bayi Road, Wuhan, China School of Electrical Computer and Telecommunications Engineering, University of Wollongong, Wollongong, NSW, 2522, Australia c Han’s Motor, Headquarter, 9018 Beihuan Road, Shenzhen, China d Guangdong Provincial Key Laboratory of Robotics and Intelligent System, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, 1068 Xueyuan Avenue, Shenzhen, China b
a r t i c l e
i n f o
Article history: Received 23 October 2015 Revised 17 March 2016 Accepted 29 April 2016 Available online xxx Keywords: Instrumentation Measurement and metrology Three-dimensional image acquisition Phase measurements Color channels
a b s t r a c t In a recent published work, we developed the technique to enhance the reliability of absolute phase maps by using the fringes of three spatial frequencies. However, it is time-consuming to capture the fringe images of three spatial frequencies with single channel in time sequence. To increase the efficiency of our proposed three-frequency technology, in this paper we propose a method to capture the fringe images of three spatial frequencies with multi-color channels and 3CCD camera. The projected spatial frequencies can be selected to guarantee the correctness of recovered fringe orders and avoid the chromatic aberration effect on frequency distortion. The cross talk among color channels can be eliminated effectively and the measured object can be reconstructed with high accuracy. The effectiveness of this method is verified by experimental results. © 2017 Elsevier B.V. All rights reserved.
1. Introduction Fringe projection profilometry (FPP) has become one of the most promising technologies for non-contact 3D shape measurement. The methods based on the phase maps of fringe patterns are the most widely utilized. A challenging task associated with existing phase measurement technique in FPP is phase unwrapping operation, which aims to recover the absolute phase maps from the wrapped phase maps. Existing phase unwrapping methods include spatial [1], temporal [2], and period coding [3]. However, recovery of absolute phase maps is still a challenging task when the wrapped phase maps contain noise, sharp changes or discontinuities. To achieve reliable and accurate phase unwrapping for FPP, a variety of temporal phase unwrapping approaches have been proposed. Huntley and Saldner [2] employ the multiple fringe patterns which are projected onto the object in time sequence, phase unwrapping can be carried out by comparing the wrapped phases of adjacent frequencies in order to avoid noise or boundaries and thus achieving correct recovery of the absolute phase map. While the method proposed in [2] is demonstrated to be effective for accurate phase unwrapping, it also suffers from the drawback of requiring many intermediate phase patterns, which is obviously not ∗
Corresponding author. E-mail address: [email protected] (J. Cheng).
suitable for fast or real-time measurement. In order to increase the efficiency, Zhao, et al. [4] propose to use two image patterns, one of which has a very low spatial frequency in contrast to the other. In particular, the low spatial frequency pattern only has a single fringe. Such a pattern has its absolute phase value falling within the range (−π , π ), and hence can be used as a reference to calculate the fringe number of the other fringe pattern, thus yielding its absolute phase map. Li, et al. [5] also employ the phase map of single fringe pattern as reference to unwrap high spatial frequency fringe patterns, and it is shown that the spatial frequency of the pattern to be unwrapped is determined by the level of noise. Following the same method in [4], Liu, et al. [6] project a single fringe pattern and a high frequency pattern in one shot to accelerate the speed of 3D measurement. These method works well in principle, but the gap between two spatial frequencies should be restricted within a range based on the noise level or steps in the low frequency phase maps. As the accuracy performance of FPP requires the use of high frequency fringe patterns, these methods may not work well when the phase maps are noisy. Consequently, multiple intermediate image patterns are still required in order to reduce the frequency gaps among adjacent patterns. Saldner and Huntley [7,8] study the multiple intermediate image patterns, showing that to unwrap a phase map of frequency f, log2 f + 1 sets of fringe patterns are required. A similar result is also reached by Zhang [9,10], indicating that the spatial frequency can be increased by a factor of 2 between two adjacent patterns.
http://dx.doi.org/10.1016/j.neucom.2016.04.074 0925-2312/© 2017 Elsevier B.V. All rights reserved.
Please cite this article as: Y. Ding et al., Recovering the absolute phase maps of three selected spatial-frequency fringes with multi-color channels, Neurocomputing (2017), http://dx.doi.org/10.1016/j.neucom.2016.04.074
JID: NEUCOM 2
ARTICLE IN PRESS
[m5G;April 14, 2017;22:0]
Y. Ding et al. / Neurocomputing 000 (2017) 1–7
Taking a typical FPP arrangement as an example where the image pattern has 16 fringes, 5 image patterns are still required with this approach. To reduce the frequencies used for temporal phase unwrapping, Zhong, et al. [11] also constructed a look-up table to unwrap the absolute phase maps for multiple-spatial-frequency fringes. This look-up table denotes the corresponding relationship from a pair of fringe orders at two spatial frequencies (f1 , f2 ) to [ f2 φ1 (x ) − f1 φ2 (x )]/2π . However, when the spatial frequencies f1 and f2 are large values, one value in [ f2 φ1 (x ) − f1 φ2 (x )]/2π may correspond to two or more pairs of fringe orders, thus the fringe orders cannot be determined uniquely. To make sure the values of [ f2 φ1 (x ) − f1 φ2 (x )]/2π unique, Zhong [12] proposed to generate the two relatively irrational spatial frequencies fringes by changing the projection angle of the grating. However, the minimal value gap of [ f2 φ1 (x ) − f1 φ2 (x )]/2π of two irrational frequencies is always smaller than the two rational frequencies [11,12], which may yield mistakes in determining fringe order pairs. In order to guarantee the uniqueness of recovered fringe orders, we have developed a temporal phase unwrapping technique based on the use of two fringe images with two selected frequencies [13]. When the two normalized spatial frequencies f1 and f2 are coprime, there exists a one-to-one map from [ f2 φ1 (x ) − f1 φ2 (x )]/2π to their fringe orders, where φ 1 (x), φ 2 (x) are the wrapped phase maps. The performance of the proposed method in [13] is limited by phase error tolerance bound, π /( f1 + f2 ) [14]. If the phase error of wrapped phase maps is larger than the phase error bound, errors may occur in the recovery of the absolute phase maps. In order to enhance the reliability of the recovered absolute phase maps, we propose to increase the phase error tolerance bound by the use of three fringe patterns with selected frequencies. The minimal value gap concerning the values of [ f2 φ1 (x ) − f1 φ2 (x )]/2π is increased significantly, resulting in a higher phase error tolerance bound [15]. The case of employing multiple spatial-frequency fringes is generalized for high accurate reconstruction [16]. However, the experiments in [15,16] are implemented by monochrome fringe patterns, at least 18 images are needed if we employ the six-step PSP (Phase-Shifting-Profilometry) to obtain the wrapped phase of three spatial-frequency fringes, which is not time-efficient for various applications. With the development of digital fringe projection system, multi-color fringe patterns have been employed for shape measurement in FPP. Huang et al. [17] project the color fringe patterns in 2π /3 phase shift at the same spatial frequencies onto the object and compensate the color channel coupling effect by intensity modulation parameter to obtain wrapped phase map. Skydan et al. [18] propose to use the colored fringe patterns at the same spatial frequencies from different angles to overcome the shadow problem of measured objects. Hu et al. [19] develop a blind approach to calculate the de-mixing matrix that no extra images are required for color calibration. However, these techniques only use the wrapped phase of single spatial frequency, which may lead to phase ambiguity for the objects with sharp changes or discontinuities. Pfortner et al. [20] use 3CCD camera to capture the interference patterns at three wavelengths generated by three laser sources simultaneously for distance measurement, while this method is more suitable to solve the problem of interferometry rather than fringe projection profilometry. Kinell [21] uses three color channels to carry three spatial frequencies to implement temporal phase unwrapping, in each channel the fringe images are projected and captured in phase shift sequences. The normalized three selected frequencies of three color channels are 14, 15, 16, errors occur in the recovered fringe orders due to the phase noise and spatial frequency selection. Zhang et al. [22] use the three color channels to carry the three normalized spatial frequencies 100, 99, 90 to implement temporal phase unwrapping. While these very high spatial frequencies will distort as the chromatic aberration effect, which
will lead to the errors in recovered fringe orders. Thus the actual projected frequencies should be adjusted according to the application scenarios, the actual spatial frequencies used in [22] are 100.196, 99 and 89.846. The estimation of proper actual spatial frequencies should be implemented before object measurement, which is not convenient for various applications. To accelerate the fringe image acquisition of our previous work in [15], in this paper we employ the color channels of projectors and 3CCD camera to capture the fringe patterns in a more time-efficient manner. Each channel carries the fringe pattern of one spatial frequency. Three spatial frequencies are selected based on the principles developed in [15], which yields high reliability for temporal phase unwrapping. Since the selected spatial frequencies are not very high, the frequency distortion caused by chromatic aberration is negligible, the actual projected frequencies need not to recalibrate before fringe projection. The experimental results validate the effectiveness of the proposed method, which enables our proposed three-frequency technology for time efficient applications. This paper is organized as follows. In Section 2 we revisit the technique to recover the absolute phase maps with three selected spatial-frequency fringes. In Section 3, we describe the multicolor channel scheme and strategies to eliminate the cross talk. In Section 4, experiments are presented to show the effectiveness of proposed method. Section 5 concludes the whole paper. 2. Absolute phase maps recovery with three frequency fringe patterns 2.1. Three -frequency technique Let us consider a FPP system, with which image patterns of three spatial frequencies are projected onto the object surface respectively. The image patterns are characterized by fringe structure where the light intensity is constant in y-axis and varies sinusoidally in x-axis. The normalized spatial frequencies of the multiple patterns are fi (i=1,2,3) referring to the total number of fringes on the respective patterns. Let us use i (x) (i=1,2,3) and φ i (x) (i=1,2,3) to denote respectively the absolute phase maps and the corresponding wrapped phase maps of the fringe patterns. Taking the central vertical line of images as the reference, the value of the wrapped phase map is limited by −π ≤ φi (x ) ≤ π (i=1,2,3), and the value of the absolute phase maps should fall into the following:
− fi π < i (x ) < fi π
(1)
Hence the absolute and wrapped phase maps are related by the following:
i (x ) = 2π mi (x ) + φi (x )
(2)
where mi (x) (i=1,2,3) are referred to as fringe numbers or indices. They are integers and − fi < mi (x ) < fi . Obviously, the absolute phases can be recovered if mi (x) are determined. In order to achieve this, the following relationships hold:
f j i (x ) = fi j (x )
(3)
where i = j. Combining Eqs. (2) and (3), we have:
f j φi ( x ) − f i φ j ( x ) = m j ( x ) f i − mi ( x ) f j 2π
(4)
Similar to the method employed in [15], an intermediate variable 0 (x) is introduced, which increases monotonically from −π to π with respect to x and defined as follows:
0 (x ) =
1 (x ) f1
=
2 (x ) f2
=
3 (x ) f3
(5)
Please cite this article as: Y. Ding et al., Recovering the absolute phase maps of three selected spatial-frequency fringes with multi-color channels, Neurocomputing (2017), http://dx.doi.org/10.1016/j.neucom.2016.04.074
ARTICLE IN PRESS
JID: NEUCOM
[m5G;April 14, 2017;22:0]
Y. Ding et al. / Neurocomputing 000 (2017) 1–7 Table 1. Mapping from 0 (x) [ f 3 φ2 (x ) − f 2 φ3 (x )]/2π ).
to
(m1 (x),
m2 (x),
0 ( x )
m1 (x), m2 (x), m3 (x)
11π /12 ≤ 0 (x) < π 9π /10 ≤ 0 (x) < 11π /12 13π /15 ≤ 0 (x) < 9π /10 9π /12 ≤ 0 (x) < 13π /15 11π /15 ≤ 0 (x) < 9π /12 7π /10 ≤ 0 (x) < 11π /15 9π /15 ≤ 0 (x) < 7π /10 7π /12 ≤ 0 (x) < 9π /15 5π /10 ≤ 0 (x) < 7π /12 7π /15 ≤ 0 (x) < 5π /10 5π /12 ≤ 0 (x) < 7π /15 5π /15 ≤ 0 (x) < 5π /12 3π /10 ≤ 0 (x) < 5π /15 3π /12 ≤ 0 (x) < 3π /10 3π /15 ≤ 0 (x) < 3π /12 π /10 ≤ 0 (x) < 3π /15 π /12 ≤ 0 (x) < π /10 π /15 ≤ 0 (x) < π /12 −π /15 < 0 (x ) < π /15 −π /12 < 0 (x ) ≤ −π /15 −π /10 < 0 (x ) ≤ −π /12 −3π /15 < 0 (x ) ≤ −π /10 −3π /12 < 0 (x ) ≤ −3π /15 −3π /10 < 0 (x ) ≤ −3π /12 −5π /15 < 0 (x ) ≤ −3π /10 −5π /12 < 0 (x ) ≤ −5π /15 −7π /15 < 0 (x ) ≤ −5π /12 −5π /10 < 0 (x ) ≤ −7π /15 −7π /12 < 0 (x ) ≤ −5π /10 −9π /15 < 0 (x ) ≤ −7π /12 −7π /10 < 0 (x ) ≤ −9π /15 −11π /15 < 0 (x ) ≤ −7π /10 −9π /12 < 0 (x ) ≤ −11π /15 −13π /15 < 0 (x ) ≤ −9π /12 −9π /10 < 0 (x ) ≤ −13π /15 −11π /12 < 0 (x ) ≤ −9π /10 −π < 0 (x ) ≤ −11π /12
5,6,7 5,5,7 4,5,7 4,5,6 4,4,6 4,4,5 3,4,5 3,4,4 3,3,4 2,3,4 2,3,3 2,2,3 2,2,2 1,2,2 1,1,2 1,1,1 0,1,1 0,0,1 0,0,0 0,0,−1 0,−1,−1 −1,−1,−1 −1,−1,−2 −1,−2,−2 −2,−2,−2 −2,−2,−3 −2,−3,−3 −2,−3,−4 −3,−3,−4 −3,−4,−4 −3,−4,−5 −4,−4,−5 −4,−4,−6 −4,−5,−6 −4,−5,−7 −5,−5,−7 −5,−6,−7
Considering 0 (x ) = 1 (x )/ f1 and taking account of Eq. (1), mi (x) (i=1,2,3) can be determined by the value of 0 (x) as follows:
mi ( x ) =
⎧ fi /2 ⎪ ⎪ ⎪ ... ⎪ ⎪ ⎪ ⎨ 1 ..
⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩
0 −1 ... − fi /2
m3 (x))
[ fi − ( fi mod 2 + 1 )]π / fi ≤ 0 (x ) < π ... π / fi ≤ 0 (x ) < 3π / fi −π / fi < 0 (x ) < π / fi −3π / fi ≤ 0 (x ) < −π / fi ... −π < 0 (x ) ≤ −[ fi − ( fi mod 2 + 1 )]π / fi (6)
where x denotes the operation of taking the largest integer not greater than x. From [15], we know that when the normalized three spatial frequencies do not have the common factor larger than 1, there exists f φ ( x )− f φ ( x ) an unique mapping from (m1 (x), m2 (x), m3 (x)) to ( 1 2 2π 2 1 , f 2 φ3 ( x ) − f 3 φ2 ( x ) ). 2π
If (f1 , f2 ) have the common factor larger than 1 and (f2 , f3 ) have the common factor larger than 1, the phase error bound of three frequency technique is significantly increased compared to the two frequency technique in [14]. The errors in fringe number can be eliminated effectively. Thus, we have the following steps to obtain the absolute phase maps of the fringes at three spatial frequencies: 1. Select multiple frequencies (f1 , f2 , f3 ) to make sure a unique mapping from (m1 (x ) f2 − m2 (x ) f1 , m1 (x ) f3 − m3 (x ) f1 ) to (m1 (x), m2 (x), m3 (x));
and
3
([ f 3 φ1 (x ) − f 1 φ3 (x )]/2π ,
f 3 φ1 (x )− f 1 φ3 (x ) 2π
f 3 φ2 (x )− f 2 φ3 (x ) 2π
−5 −5 10 0 0 −10 5 −5 −5 10 0 0 −10 5 5 −5 10 10 0 −10 −10 5 −5 −5 10 0 0 −10 5 5 −5 10 0 0 −10 5 5
−6 9 9 −3 12 0 0 −12 3 3 −9 6 −6 −6 9 −3 −3 12 0 −12 3 3 −9 6 6 −6 9 −3 −3 12 0 0 −12 3 −9 −9 6
2. Project onto the object with three fringe patterns of spatial frequencies (f1 , f2 , f3 ) respectively, acquiring multiple wrapped phase maps by a phase detection algorithm; 3. Calculate the terms mi (x ) f j − m j (x ) fi , round their values into the closest integers, denoted as Mi . Look up the mapping relationship from (m1 (x ) f2 − m2 (x ) f1 , m1 (x ) f3 − m3 (x ) f1 ) to (m1 (x), m2 (x), m3 (x)), find the entries whose values of mi (x ) f j − m j (x ) fi are the closest to Mi . 4. Record the corresponding mi (x) (i=1,2,3) and reconstruct the absolute phase maps by Eq. (2) using mi (x). As an example, we select three spatial frequencies as f 1 = 10, f2 = 12, f3 = 15 to construct the look-up table. The frequency pairs (10,15) and (12,15) has common factor as 5 and 3, respectively, which are the largest common factors in this three frequency f φ ( x )− f φ ( x ) f φ ( x )− f φ ( x ) group. The look-up table from ( 3 1 2π 1 3 , 3 2 2π 2 3 ) to (m1 (x), m2 (x), m3 (x)) are given in Table 1: f φ ( x )− f φ ( x ) From the table we can see, the mapping from ( 3 1 2π 1 3 , f 3 φ2 ( x ) − f 2 φ3 ( x ) ) 2π
to (m1 (x), m2 (x), m3 (x)) is unique, which means we could obtain (m1 (x), m2 (x), m3 (x)) by calculating f φ ( x )− f φ ( x ) f φ ( x )− f φ ( x ) the ( 3 1 2π 1 3 , 3 2 2π 2 3 ). The minimal value gaps of f 3 φ1 ( x ) − f 1 φ3 ( x ) 2π
f φ ( x )− f φ ( x )
and 3 2 2π 2 3 are 5 and 3, respectively. According to [15], the phase error tolerance bound of the frequency pairs (fi , fj ) with maximal common factor tij is:
0 ≤ φmax <
ti j π fi + f j
(7)
Please cite this article as: Y. Ding et al., Recovering the absolute phase maps of three selected spatial-frequency fringes with multi-color channels, Neurocomputing (2017), http://dx.doi.org/10.1016/j.neucom.2016.04.074
JID: NEUCOM 4
ARTICLE IN PRESS
[m5G;April 14, 2017;22:0]
Y. Ding et al. / Neurocomputing 000 (2017) 1–7
Fig. 1. The scheme of color-channel projection.
This means that when the actual phase error is less than this bound, the recovered fringe orders will be correct, when the actual phase error is larger than this bound, the recovered fringe orders may be wrong. For the amplitude-limited phase error, when the frequencies are appropriately selected, the correctness of the recovered fringe order can be guaranteed. This property is very used in multi-color channel scheme, we can select the spatial frequencies based on Eq. (7) to avoid the errors in recovered fringe orders for accurate three dimensional reconstruction.
mate the coupling matrix. For example, the red channel, three pure fringe images with 2π /3 shift are projected onto a white reference plane and captured by 3CCD camera, the initial phases of three phase shifted images are (0, 2π /3, 4π /3). We define the three phase shifted images as:
S1 = A + B cos[], S2 = A + B cos[ + 2π /3], S3 = A + B cos[ + 4π /3]. average
A=
intensity
2 1/2
3. Time efficient projection scheme and coupling elimination In the experiments of [15,16], the fringe images of three spatial frequencies are acquired in time series respectively. To accelerate the procedure of fringe image acquisition, we employ three color channel of camera and 3CCD camera to capture the different spatial frequency fringes. Fig. 1 illustrates the procedure: Each channel carries the fringe pattern of one spatial frequencies. In each channel the initial phase is shifted by 2π /3 for three times, thus the initial phases of the fringe patterns in the same channel are 0,2π /3, 4π /3. In this scheme, we can capture the shift fringe images of three spatial frequencies in color channel by three times rather than the eighteen times with single channel as [15,16], the time consumption in image capture has been greatly reduced. At the same time, the multi-color capture scheme brings the coupling effects among the fringe patterns in different channels, to reconstruct the object with high accuracy, it is necessary to eliminate the coupling effects to correct the distorted fringe patterns. The methods to eliminate the coupling effects can be divided into two categories, one kind of method is based on estimating of the intensity of coupling effects among color channels and correcting the distorted fringe patterns, another kind of method is based on the color filter in front of 3CCD camera to reduce the coupling effects at source. The method to estimate the coupling intensity has been established in [17], which uses a coupling matrix to describe the coupling effects. Here we implement the similar procedure to esti-
S1 +S2 +S3 , 3
modulation
intensity
B=
[3 (S2 −S3 )2 +(2S1 −S2 −S3 ) ] 3
. For three color channels, we have nine phase shifted images. Then we can separate the RGB components of each color image, yielding twenty-seven gray images. By these separated images, we can use the coupling intensity to describe the magnitude of coupling effect, the coupling intensity Iij is calculated by the ratio of intensity modulation B between channels. The coupling matrix among multi-color channels is constructed as:
Irr Igr Ibr
Irg Igg Ibg
Irb Igb Ibb
where Iij denotes the coupling intensity from channel i to channel j, the elements on the main diagonal are normalized as 100, that is Irr = Igg = Ibb . By calculating this matrix, the elements can be used to compensate the amplitude distortion of fringe patterns in each channel for high accurate measurement. In compensation processing, the cross talk components are subtracted to reserve the original color fringes, the detailed equations can refer to the equations (13)–(15) in [17]. The coupling matrix based method can reduce the linear distortion caused by multi-color channels. For the nonlinear distortion, Hu et al. [19] propose to use the blind separation filtering method to recover the deformed fringe patterns. However, this algorithm is time consuming, which is not suitable for time efficient measurement. It is more effective to reduce the nonlinear distortion among multi-color channels with hardware. Color dielectric filters in front of camera can eliminate the cross talk at source significantly, the computational burden of noise elimination can be greatly reduced.
Please cite this article as: Y. Ding et al., Recovering the absolute phase maps of three selected spatial-frequency fringes with multi-color channels, Neurocomputing (2017), http://dx.doi.org/10.1016/j.neucom.2016.04.074
ARTICLE IN PRESS
JID: NEUCOM
[m5G;April 14, 2017;22:0]
Y. Ding et al. / Neurocomputing 000 (2017) 1–7
5
Fig. 2. Coupling effects among color channels when extra filters used. The three rows correspond to the coupling effects of red, green and blue channels. In each row, the first image is the captured color image and the others are the corresponding RGB components. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
4. Experiment In order to verify the performance of the proposed technique, we implement experiment with multi-color projection. The camera in the experiment is DaHeng industry HV1351 high resolution 3CCD camera, the lens is Computar industry 16 mm lens, the projector is LG HW300 Multimedia DLP Projector. In the experiment, firstly, the cross talk among color channels are evaluated and compensated on white reference plane to eliminate the error in wrapped phase for high quality three dimensional reconstructions. Then the maximal error in wrapped phase is measured on white reference plane to test whether the errors exceed the error tolerance bound of selected spatial frequencies, if the errors exceed the bound, the spatial frequencies should be changed to increase the error tolerance bound to guarantee the correctness of recovered fringe orders. Thirdly, the fringes at selected spatial frequencies on three color channels are projected onto the measured object for three dimensional reconstructions. These steps are explained in details as follows. In the first step of experiment, we use 3CCD camera to capture color fringe images projected onto a white reference plane when the extra filters (Comar product numbers 465 1k 50, 540 1B 50 and 650 1Y50) are used. The aim of this step is to estimate the coupling matrix of three color channels. The captured color fringe images are separated into RGB components as shown in Fig. 2: Based on these RGB components, we calculate the elements of coupling matrix as:
100.0 15.5 6.8
1.5 100.0 3.1
1.2 3.6 100.0
From the matrix we can see most of the coupling effect is eliminated by extra filters. The coupling effects of multi-color channels are determined by the hardware of DLP projector, thus the estimation of coupling effect at different spatial frequencies is similar. It is not necessary to recalibrate the coupling effect with the fringe patterns at other spatial frequencies. The residue of coupling effects can be compensated by the method in [17].
In the second step of experiment, we select the spatial frequencies based on the phase error in three channels to guarantee the correctness of recovered fringe orders. In the single channels experiment in [15,16], the phase error tolerance bounds of spatial frequency group (10, 12, 15) are much larger than the maximal phase error and guarantee the accurate three dimensional reconstruction. We can firstly select this spatial frequency group, test this group whether its phase error bounds larger than the phase error in each channel. If it is, we can use this group for three dimensional reconstructions, if it is not, we can change frequency group till its phase error bounds are larger than the phase error in each channel. In order to obtain the maximum absolute value of phase error, we project the red, green, blue color fringes of spatial frequency 10, 12, 15 onto a white reference plane and obtain the wrapped phase maps of these three frequencies. Then we generate the ideal wrapped phase maps of spatial frequencies 10, 12, 15 by computer. The phase errors of three wrapped phase maps can be obtained by the subtraction between actual and ideal wrapped phase maps of each channel. The cross talk changes the amplitude and phase of original fringe in each channel, this error in wrapped phase calculated by phase shifting algorithm is amplitude-limited [23], thus the maximal error in wrapped phase caused by cross talk could be estimated. Among three color channels, the maximal absolute value of error in wrapped phase maps is 0.1675. Comparing with the phase error bound given by Eq. (7), we know that the maximum absolute value of phase error is smaller than the phase error bound, 0.1675 < 5π /(10 + 15 ) = π /5 and 0.1675 < 3π /(12 + 15 ) = π /9. In this case, the absolute phase maps should be recovered correctly. In another aspect, the chromatic aberration may lead to the distortion of high spatial frequencies, which may yield errors in three dimensional reconstructions. In [22], the phase map of unit frequency is obtained by heterodyning the wrapped phase maps at frequencies 100 and 99, the wrapped phase map of frequency 10 is obtained by heterodyning the wrapped phase maps at frequencies 100 and 90. The chromatic aberration would make the frequencies 100, 99, 90 distorted, thus the resulted phase maps will not correspond to the assumed spatial frequencies, the recovered absolute phase will be wrong.
Please cite this article as: Y. Ding et al., Recovering the absolute phase maps of three selected spatial-frequency fringes with multi-color channels, Neurocomputing (2017), http://dx.doi.org/10.1016/j.neucom.2016.04.074
JID: NEUCOM 6
ARTICLE IN PRESS
[m5G;April 14, 2017;22:0]
Y. Ding et al. / Neurocomputing 000 (2017) 1–7
Based on these considerations, we select three frequencies as (10, 12, 15), these spatial frequencies can be achieved by controlling the projector pitches in our experiments. Compared with [21], the three spatial frequencies of three color channels are selected as (14, 15, 16). However, these three spatial frequencies will yield a much smaller phase error tolerance bound than our selection, which can be easily exceeded by phase error. The experiment results in [21] show the reconstruction errors due to the spatial frequency selection. The chromatic aberration is negligible in our selected spatial frequencies, the actual projected frequencies need not to recalibrate before measurements. Thus, this spatial-frequency selection enables us with high accurate and efficient three-dimensional reconstruction. In the third step of experiment, we project three spatial frequencies 10, 12, 15 with red, green, blue channels onto the measured object. The measured object is a plaster toy, which is characterized by complex surface such as sharp changes and discontinuities around head and ears. Similar surfaces can be found in many engineering applications, thus it is used to demonstrate the effectiveness of proposed method. The length (from head to foot) of the test object is 151.2 mm, the width (from left ear to right ear) is 125.7 mm. The toy with color fringe patterns is shown in Fig. 3a. The recovered absolute phase map of toy at frequencies 15 is shown in Fig. 3b. Three dimensional reconstructed result is depicted in Fig. 3c. From Fig. 3 and the original plaster toy, we can see the absolute phase is correctly recovered, the measured small toy is reconstructed accurately. Since the maximal error in phase is 0.1675, the maximal error in height is 0.013 mm. By selecting the spatial frequencies appropriately, our method improves the reliability of recovered fringe orders and avoids the frequency distortion effect caused by chromatic aberration. Thus it is suitable for timeefficient and accurate measurement. Compared with the method in [22], we avoid the accumulated error introduced by the heterodyning among the fringe images of different spatial frequencies. Due to the proper spatial frequency selection, the chromatic aberration effect is not obvious in our work. Thus we need not to calibrate the spatial frequency before measurement, the process of measurement with multi-color channels can be obviously simplified. In future, we can explore the techniques to synthesis the multi-view results together for complete three dimensional reconstructions [24].
a
b
c
5. Conclusion This paper develops a time efficient temporal phase unwrapping scheme based on the projection of three-frequency fringe images with three color channels. The color channels accelerate the fringe image capture significantly, compared to single channel, only one third fringe images should be sampled. Based on the frequency selection principle developed in [15], we can select proper spatial frequencies for three color channels to improve the reliability of recovered fringe orders and avoid the problem of chromatic aberration. The cross talk among different color channels is eliminated by the coupling matrix based technique and filters. Thus the proposed time efficient three frequency technique can be employed for high speed and accurate three dimensional data acquisition. Acknowledgment This work is partially supported by Science and Technology Service Network, Chinese Academy of Sciences (KFJEW-STS-035/01), the National Natural Science Foundation of China (61403365), CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology (2014DP173025), Guangdong Technology Project
Fig. 3. Experiment results when f 1 = 10, f 2 = 12, f 3 = 15 on red, green, blue channels. (a) is the deformed composite color fringe pattern; (b) is the recovered phase maps at frequency f 3 = 15; (c) is three dimensional reconstruction result at frequency f 3 = 15. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
Please cite this article as: Y. Ding et al., Recovering the absolute phase maps of three selected spatial-frequency fringes with multi-color channels, Neurocomputing (2017), http://dx.doi.org/10.1016/j.neucom.2016.04.074
JID: NEUCOM
ARTICLE IN PRESS
[m5G;April 14, 2017;22:0]
Y. Ding et al. / Neurocomputing 000 (2017) 1–7
(2016B010108010, 2016B010125003) and Shenzhen Technology Project (JSGG20160331185256983, JSGG20160229115709109). References [1] S. Zhang, X. Li, S. Yau, Multilevel quality-guided phase unwrapping algorithm for real-time dimensional shape reconstruction, Appl. Opt 46 (2007) 50–57. [2] J.M. Huntley, H.O. Saldner, Temporal phase-unwrapping algorithm for automated interferogram analysis, Appl. Opt. 32 (1993) 3047–3052. [3] H.J. Chen, J. Zhang, D.J. Lv, J. Fang, 3-D shape measurement by composite pattern projection and hybrid processing, Opt. Express 15 (2007) 12318–12330. [4] H. Zhao, W. Chen, Y. Tan, Phase-unwrapping algorithm for the measurement of three-dimensional object shapes, Appl. Opt. 33 (1994) 4450–4497. [5] J. Li, L.G. Hassebrook, C. Guan, Optimized two-frequency phase-measuring-profilometry light-sensor temporal-noise sensitivity, J. Opt. Soc. Am. A 20 (2003) 106–115. [6] K. Liu, Y. Wang, D.L. Lau, Q. Hao, L.G. Hassebrook, Dual-frequency pattern scheme for high-speed 3-D shape measurement, Opt. Express 18 (2010) 5229–5244. [7] J.M. Huntley, H.O. Saldner, Error-reduction methods for shape measurement by temporal phase unwrapping, J. Opt. Soc. Am. A 14 (1997) 3188–3196. [8] H.O. Saldner, J.M. Huntley, Temporal phase unwrapping: application to surface profiling of discontinuous objects, Appl. Opt. 36 (1997) 2770–2775. [9] S. Zhang, Digital multiple-wavelength phase-shifting algorithm, SPIE Proceedings 7432 (2009) 74320N. [10] S. Zhang, Phase unwrapping error reduction framework for a multiple-wavelength phase-shifting algorithm, Opt. Eng. 48 (2009). [11] J. Zhong, M. Wang, Phase unwrapping by a lookup table method: application to phase maps with singular points, Opt. Eng. 38 (1999) 2075–2080. [12] J. Zhong, Linear integer unconcerned phase-map profilometry by changing the projection angle of the grating, Opt. Eng. 40 (2001) 1377–1382. [13] Y. Ding, J. Xi, Y. Yu, J. Chicharo, Recovering the absolute phase maps of two fringe patterns with selected frequencies, Opt. Lett. 36 (2011) 2518–2520. [14] Y. Ding, J. Xi, Y. Yu, W. Cheng, S. Wang, J. Chicharo, Frequency selection in absolute phase maps recovery with two frequency projection fringes, Opt. Express 20 (2012) 13228–13251. [15] Y. Ding, J. Xi, Y. Yu, F. Deng, Absolute phase recovery of three fringe patterns with selected spatial frequencies, Opt. Lasers Eng. 70 (2015) 18–25. [16] Y. Ding, J. Xi, Y. Yu, F. Deng, J. Cheng, Multiple spatial-frequency fringes selection for absolute phase recovery, Surf. Topography: Metrology Prop. 4 (2016) 015004. [17] P. Huang, Q. Hu, F. Jin, F. Chiang, Color-encoded digital fringe projection technique for high-speed three-dimensional surface contouring, Opt. Eng. 38 (1999) 1065–1071. [18] O.A. Skydan, M.J. Lalor, D.R. Burton, Technique for phase measurement and surface reconstruction by use of colored structured light, Appl. Optics 41 (2002) 6104–6107. [19] Y. Hu, J. Xi, J. Chicharo, Z. Yang, Blind color isolation for color-channel-based fringe pattern profilometry using digital projection, J. Opt. Soc. Am. A 24 (2007) 2372–2382. [20] A. Pfortner, J. Schwider, Red-green-blue interferometer for the metrology of discontinuous structures, Appl. Optics 42 (2003) 667–673. [21] L. Kinell, Multichannel method for asolute shape measurement using projected fringes, Opt. Lasers Eng. 41 (2004) 57–71. [22] Z. Zhang, C.E. Towers, D.P. Towers, Color fringe projection system based on optimum frequency selection, SPIE Proceedings 6292 (2006) 62920S. [23] B. Pan, K. Qian, L. Huang, A. Asundi, Phase error analysis and compensation for nonsinusoidal waveforms in phase-shifting digital fringe projection profilometry, Opt. Lett. 34 (2009) 416–418. [24] Y. Yang, X. Wang, Q. Liu, M. Xu, A bundled-optimization on model of multiview dense depth map synthesis for dynamic scene reconstruction, Inf. Sci. 320 (2015) 306–319.
7 Yi Ding is now the assistant Professor at School of Remote Sensing and Information Engineering, Wuhan University. He received his Ph.D. Degree in Electronic Engineering from Huazhong University of Science and Technology, China, 2013. During his PhD study, he has been the visiting PhD in School of ECTE, University of Wollongong, Australia from 2009 to 2011. He has worked as Research Associate in Department of Electrical and Electronic Engineering, University of Hong Kong from 2013 to 2014. His current research interests include 3D data acquisition and machine vision.
Jiangtao Xi is now the full professor and head of School of ECTE, University of Wollongong. Professor Xi has found the Optoelectronics Signal Processing Laboratory in School ECTE, University of Wollongong. His current research interests include 3D data acquisition, Optoelectronics signal processing and signal processing in wireless communications.
Yanguang Yu is now the associate professor in School of ECTE, University of Wollongong. Her current research interests include Optoelectronics signal processing and 3D data acquisition.
Fuqin Deng is now the technical director in Han’s motor S&T Co. LTD., Shenzhen. He received his Ph.D. Degree in Electronic Engineering from Department of Electrical and Electronic Engineering, University of Hong Kong, 2014. His current research interests include 3D data acquisition, computer vision and machine vision applications.
Jun Cheng is now the professor and director of Laboratory for Human Machine Control in Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences. His current research interests include computer vision, human compter interface and robotics.
Please cite this article as: Y. Ding et al., Recovering the absolute phase maps of three selected spatial-frequency fringes with multi-color channels, Neurocomputing (2017), http://dx.doi.org/10.1016/j.neucom.2016.04.074