- Email: [email protected]
Implementation of a simulated display for hexagonal image processing
Displays 50 (2017) 63–69
Contents lists available at ScienceDirect
Displays journal homepage: www.elsevier.com/locate/displa
Implementation of a simulated display for hexagonal image processing
MARK
Xiangguo Li College of Information Science and Engineering, Henan University of Technology, Zhengzhou, Henan 450001, PR China
A R T I C L E I N F O
A B S T R A C T
Keywords: Hexagonal image processing Sampling lattice Simulated display Voronoi cell
Hexagonal image processing is theoretically superior to that based on the common square lattice, and due to the lack of practical imaging devices, this paper intends to implement a simulated display. The paper investigates the sampling lattices used in the hexagonal image processing, and finds that variable lattices occur in hexagonal discrete Fourier transform (HDFT), while the most commonly used “hyperpel” approach, due to its fixed cell pattern, cannot handle such displaying tasks. Then, the paper proposes to represent each pixel with the exact Voronoi cell (VC) according to the sampling lattice. In the paper, a simple algorithm is presented to compute the VC vertices, and each simulated pixel is constructed with the VC filled with its intensity value, and then the simulated display is implemented by the tessellation of each simulated pixel on the correct lattice position. Finally, experimental results show that the proposed simulated display can display data without geometric distortion.
1. Introduction Practical digital signals are mainly obtained by sampling the analog ones. For images, due to the 2-D nature, the sampling points lie on a 2-D plane and form a regular 2-D lattice. According to the sampling theory, the hexagonal lattice is the optimal sampling scheme for circularly band-limited analog images [1]. Since most optical systems are circularly symmetric and thus exhibit circularly low-pass nature, the hexagonal lattice is actually the optimal sampling scheme for sampled imaging systems. Compared with the commonly used square lattice, the hexagonal lattice can provide 13.4% fewer samples [1], and it is also superior with respect to its geometric properties, such as higher degree of symmetry, equal distance and uniform connectivity with its neighbors, as shown in Fig. 1. In recent years, especially with the rise of biologically inspired image processing, hexagonal image processing has spread to applications such as edge detection [2,3], hexagonal Gabor filtering [4], ultrasound image processing [5], and adaptive beamforming [6]. Besides, hexagonal lattice is also adopted in advanced imaging techniques, such as 3-D display system [7], 3-D endoscopy [8], and integral imaging system [9]. However, people are more familiar with the Cartesian orthogonal coordinate system as well as the square lattice. Also, the square lattice has the advantage of simplicity for practical data storage and addressing [10], thus it predominates in the commercial imaging devices, including image sensors (CCD and CMOS) and displays. Therefore, for the hexagonal image processing research at the present time, it is natural to adopt the simulation approach. For the data acquisition, lattice
conversion can be applied by means of interpolation under the resampling framework [11–13]. For the data displaying, each pixel can be simulated by a proper cell and the simulated display can be constructed based on current displays. This paper concerns the data displaying task and proposes to implement a simulated display for hexagonal image processing. Consider a common square lattice image displayed on a practical square lattice display. If we zoom in the image, it is easy to recognize that each pixel is presented with a small square cell, as illustrated in Fig. 2. This has motivated the construction of simulated display for hexagonal lattice data. Firstly, Lester and Sandor [14] proposed the “brick wall” technique, in which each hexagonal pixel was approximated with four (2 × 2) square pixels and even rows stagger odd rows with one square pixel pitch, as shown in Fig. 3(a). Obviously, the “brick wall” approach is coarse and can incur geometric distortion. Later, Wüthrich and Stucki [15] proposed the “hyperpel” technique, in which a more fine cell was used for each hexagonal pixel, as shown in Fig. 3(b). In the regular hexagonal lattice, the vertical and horizontal pitch ratio is , and in the “brick wall” approach, the ratio is 1:1, while in the “hyperpel” approach, the ratio is 7:8 = 0.8750. Therefore, the “hyperpel” approach can be treated as a good approximation and it is the most common displaying approach used in current hexagonal image research [2,3,16–18]. We notice that, when dealing with samples in the frequency domain, the sampling lattices will vary with several factors (fully discussed in Section 2). Then, for the displaying task of the discrete spectrum, the “hyperpel” approach will fail to provide good
E-mail address: [email protected]. http://dx.doi.org/10.1016/j.displa.2017.09.005 Received 11 February 2017; Received in revised form 1 August 2017; Accepted 18 September 2017 Available online 21 September 2017 0141-9382/ © 2017 Elsevier B.V. All rights reserved.
Displays 50 (2017) 63–69
X. Li
need of interpolating the image data. This paper will continue to apply this motivation into the general 2-D sampling lattice, i.e., the proposed simulated display will be based on the exact VC shape with regard to the given sampling lattice and it will be able to cope with the need of data displaying tasks in the hexagonal image processing. VC is an essential property of lattice. In digital image processing, VC is a fundamental tool in the sampling and reconstruction analysis. VC has also been used in hex-splines [22], Voronoi splines [23], screen reconstruction analysis [24], etc. However, computing the VC of a lattice, i.e., determining the vertices, is not direct and the task may be quite complicated for general lattices [25,26]. This paper concerns the 2-D sampling lattices in the hexagonal image processing, and owing to the regular nature, only the VC vertices with regard to one lattice point is needed. In the paper, an intuitive algorithm is presented to compute the VC vertices, and then each pixel cell as well as the whole simulated display can be constructed. The remainder of the paper is organized as follows. For the better understanding the motivation, Section 2 discusses the sampling lattices used in the hexagonal image processing. Section 3 presents the algorithm to compute the VC vertices. Then, Section 4 implements a simulated display. Finally, Section 5 summarizes this paper.
Fig. 1. Comparison of geometric properties between the square lattice (left) and the regular hexagonal lattice (right).
Fig. 2. Illustrations of displaying a square lattice image on the square lattice display and the partial enlarged views.
2. Lattices in hexagonal image processing
(a)
First of all, the (virtually) hexagonally sampled digital images correspond to a common regular hexagonal lattice in the spatial domain. Meanwhile, the periodic sampling process in the spatial domain causes the original spectrum to be periodically replicated in the frequency domain, for which the extension pattern is defined by the reciprocal lattice that is also a regular hexagonal lattice but with different orientation. Given the sampling matrix V for a given spatial sampling lattice, and the corresponding reciprocal lattice U is given by U = 2π V−T . Specifically, these two basic lattices in the hexagonal sampling are shown in Fig. 5, in which V = [v1|v2] = [1,−0.5;0, 3 /2] and U = [u1|u2] = 2π [1,0;1/ 3 ,2/ 3 ]. On the other hand, spectrum analysis and frequency domain image processing are fundamental from both the theoretical and the practical perspectives. However, although a digital image can be obtained from either practical imaging sensor or any resampling approach, the spectrum of the obtained digital image is still continuous because it is simply the periodic extension of the original continuous spectrum. To enable the digital processing techniques in the frequency domain, it is often assumed to construct the periodic sequence in the spatial domain, which corresponds to sampling the periodic spectrum in the frequency domain. This is the basis for the discrete Fourier series (DFS) as well as the discrete Fourier transform (DFT). With the spatial sampling matrix V and the spatial periodic extension pattern N , the frequency domain sampling matrix is given as S = 2π (VN)−T [27]. In the commonly used square lattice case, the spatial sampling matrix V is diagonal, and the extension pattern N is usually selected as
(b)
Fig. 3. Illustration of the techniques used for simulated hexagonal display. (a) the “brick wall” and (b) the “hyperpel”.
approximation and due to its fixed cell pattern it can not be easily adapted either. For this reason, this paper intends to implement a simulated display that can deal with the general data displaying tasks used in hexagonal image processing. This is the main starting point of this paper. Actually, the small squares in Fig. 2 serve as the zero-order hold reconstruction filter and the small square area is the Voronoi cell (VC) of the square lattice. Accordingly, each pixel of the hexagonal lattice data can also be represented with the VC of the hexagonal lattice [11,19–21], as shown in Fig. 4. This approach, instead of directly concerning the discrete nature of practical displays as in both the “brick wall” and the “hyperpel” approaches, is based on exact VC shape of the hexagonal lattice. Owing to the exact VC shape, the image data can be displayed without geometric distortion, and by simply linearly adjusting the VC values, the image can also be easily zoomed without
Fig. 4. Illustrations of displaying the hexagonal lattice image data based on the VC of the hexagonal lattice.
Fig. 5. Illustration of the hexagonal sampling lattice in the spatial domain (left) and the corresponding reciprocal lattice in the frequency domain (right).
64
Displays 50 (2017) 63–69
X. Li
Fig. 6. Illustration of the spatial extension pattern N and the corresponding frequency sampling lattice S of the HDFTs. (a) is for N = [3,0;0,4] and (c) is for N = [3,2;0,4], while (b) and (d) are their frequency samples, respectively.
in both the “brick wall” and the “hyperpel” approaches can not fit the variable nature of the S , this paper proposes to implement a simulated display with the exact VC shape of the corresponding sampling lattice. In the following sections, regarding to a general 2-D sampling lattice, the paper firstly presents a simple algorithm to compute the VC vertices, and then implements the simulated display accordingly.
diagonal matrix, thus the frequency sampling matrix S is also a diagonal matrix. This is a main advantage of the square lattice image processing. In contrast, the spatial sampling matrix V is non-diagonal in the hexagonal lattice case. Moreover, unlike the square lattice DFT, flexible choices of image shape and the extension pattern N are common in the hexagonal discrete Fourier transform (HDFT). In practice, over the past decades, various types of hexagonal fast Fourier transform (HFFT) algorithms have been developed [1,28–33]. Then, although the spatial sampling matrix V is generally constant, the frequency sampling matrix S still varies with the choice of extension pattern N together with the image shape. Even one maybe always use the preferred extension pattern, the shape of image will still affect N and thus S . Therefore, the frequency sampling matrix S may be neither diagonal nor regular hexagonal, and it will become a general one for the HDFT in hexagonal image processing. To better demonstrate the frequency sampling lattice S with regard to the spatial domain extension pattern N , an example is presented in Fig. 6. Consider to compute the HDFT of a hexagonally sampled sequence with the finite support 3 × 4 , the most natural choice of N is through the diagonal pattern as shown in Fig. 6(a), i.e., N = [3,0;0,4] in this case, and the frequency sampling matrix S = 2π [1/3,0;1/3 3 ,1/2 3 ] as shown in Fig. 6(b). It is obvious that the frequency sampling lattice is neither orthogonal nor regular hexagonal. Since the extension pattern N is not unique, another one with the orthogonal extension pattern [29] is presented in Fig. 6(c). In this case, N = [3,2;0,4] and S = 2π [1/3,0;0,1/2 3 ], as shown in Fig. 6(d). It is clear that Fig. 6(b) and (d) have completely different sampling patterns. Then, for the purpose of accurate spectrum analysis, the spectrum needs to be properly displayed. Due to the reasons above, the sampling lattice S needs to be considered for each specific case, and the simulated display should present each pixel with its accurate position together with the exact VC cell shape. Otherwise the spectrum may be displayed with geometric distortion, which will severely degrade the visual effectiveness and thus affect the judgement. Since the fixed cell patterns
3. Computing the VC vertices 3.1. Definitions Given a 2-D sampling lattice defined by the sampling matrix
V = [v1|v2],
(1)
in which v1 and v2 are the two non-collinear sampling vectors. Then, the position of any lattice point can be determined by Vn with n = [n1,n2]T the corresponding integer coordinate vector. Let P = {Pn,n ∈ 2} denote the collection of all the lattice points, and the VC of the point Pi can be defined as [34]:
VC (Pi ) = {x ∈ 2: ‖x−Pi ‖ ⩽ ‖x−Pj ‖ ,∀ i ≠ j},
(2)
in which ∥·∥ is the distance norm. For a regular lattice, the VC shapes are same for all the lattice points, and only the fundamental VC with regard to the origin P0 is needed. The VC shapes of the square lattice, the quincunx lattice, and the regular hexagonal lattice shown in Fig. 7, respectively. 3.2. Determining the neighbors There are eight candidates for the VC neighbors, as shown in Fig. 8(a), in which they are marked as P 0,P1,P 2 , P 3,P 4,P5,P 6, and P7 consecutively. From the typical VC shapes in Fig. 7, there are two typical numbers for neighbor and vertex, i.e., four and six, and this is the cue to follow. 65
Displays 50 (2017) 63–69
X. Li
adjacent neighbor pair. Algorithm 1. VC vertices of a 2-D sampling lattice
1: 2: 3: 4:
Fig. 7. Illustrations of the VCs of the square lattice (left), the quincunx lattice (middle), and the hexagonal lattice (right).
5: 6: 7: 8: 9:
When v1 and v2 are perpendicular to each other, as in the square and quincunx lattices, there are four cases where the intersection points are coincident, and the valid number reduces to four and the neighbors are {P 0,P 2,P 4,P 6} . Otherwise, two points will be removed from the neighbor candidates and the valid number reduces to six; furthermore, P1 and P5 will be removed if the angle between v1 and v2 is acute, or P3 and P7 will be removed if the angle is obtuse.
10: 11: 12: 13: 14: 15: 16: 17:
3.3. Intersection of two perpendicular bisectors Given the origin O and two adjacent neighbors P1 and P2 , for which the two vectors are u1 and u2 , respectively, as shown in Fig. 8(b). First, with the Euclidean norm, the boundary between O and P1 is their perpendicular bisector, and the corresponding vector x on this line must meet
u1T x = 0.5u1T u1.
Input: sampling matrix V = [v1|v2] Const: array {P 0,P1,P 2,P 3,P 4,P5,P 6,P 7} Outputs: number num and the vertices vertices [] function VORONOICELLVERTICES (V) if (v1T v2 = 0) then num ← 4 neighbors [] ← {P 0,P 2,P 4,P 6} else num ← 6 if (v1T v2 > 0) then neighbors [] ← {P 0,P 2,P 3,P 4,P 6,P 7} else neighbors [] ← {P 0,P1,P 2,P 4,P5,P 6} end if end if for (i ≔ 0 to num−1) do p0 ← neighbors [i]
p1 ← neighbors [(i + 1)%num] 18: 19: vertices [i] ← Solve (Vp0,Vp1) 20: end for 21: return num and vertices [] 22: end function
(3)
Similarly, the perpendicular bisector equation for O and P2 is given as:
u 2T x = 0.5u 2T u2.
(4)
3.5. Evaluation and analysis
Then, the intersection point is by solving the simultaneous equations: T T ⎡ u1 ⎤ x = 0.5 ⎡ u1 u1 ⎤. ⎢uTu ⎥ ⎢uT ⎥ ⎣ 2 2⎦ ⎣ 2⎦
Before applying the algorithm, we performed experiments to check the effectiveness. Firstly, we computed on the three typical lattices in Fig. 7, for which their VCs are apparent, and the results were correct. Then, we checked the two HDFTs illustrated in Fig. 6. For Fig. 6(b), num = 6 (0.1389,0.1443) , and the VC vertices were (−0.1389,0.1443),(−0.1944,−0.0481),(−0.1389,−0.1443),(0.1389,−0.1443), and (0.1944,0.0481) . For Fig. 6(d), num = 4 with the VC vertices (0.1667,0.1443),(−0.1667,0.1443),(−0.1667,−0.1443) , and (0.1667,−0.1443) . These results were plotted in Fig. 9, and we could verify the correctness according to the definition of VC. For a given sampling lattice, if V is a valid sampling matrix, with any integer unimodular matrix E (i.e., |detE| = 1), EV is also a valid sampling matrix [27]. Then, since the algorithm is based on the eight candidate points shown in Fig. 8(a), while the actual candidates are closely related to the sampling matrix, the algorithm would fail if any actual candidates escape out of these eight candidate points. In practice, the sampling matrices always use the natural manner, and thus the presented algorithm can work correctly.
(5)
Since u1 and u2 are noncollinear, the coefficient matrix is nonsingular and invertible, and the solution is unique. 3.4. Computing the VC vertices Based on the analysis above, we compute the VC of a 2-D sampling lattice. The input is the sampling matrix V , and the outputs are the vertex number num together with the coordinate array of the vertices vertices []. We firstly determine the num value by checking whether the angle between v1 and v2 is right. Furthermore, to judge the angle be acute, right, or obtuse, we judge whether the sign of the inner product is positive, zero, or negative, respectively. Then, after assigning the correct neighbors, we can compute each vertex by applying Eq. (5) on each
Fig. 8. Illustration of the eight neighbor candidates (a) and two perpendicular bisectors together with their intersection point (b).
66
Displays 50 (2017) 63–69
X. Li
Fig. 9. Illustration of VCs regarding to the sampling lattices in Fig. 6. (a) is for the diagonal pattern N = [3,0;0,4], and (b) is for the orthogonal pattern N = [3,2;0,4].
4. The simulated display
the same image shape but with different sizes, while the IM077 has rectangular shape and thus has completely different shapes and sizes from the other two. The test images and the magnitudes of the different DFTs were displayed in Fig. 10, in which the corresponding VC shapes were also accompanied. Since people are more familiar with the square lattice, the common square lattice DFT was used for reference purpose and its magnitudes served as the ground truth for visual judgement. Also, since different spatial sampling lattices correspond to different spectrum extension patterns, to facilitate the visual judgement, the shifted magnitudes of the square lattice DFT were also provided in Fig. 10, and the spectrum extension patterns together with the fundamental period shapes were illustrated in Fig. 11. Although the two HDFTs have the same spectrum extension pattern from the same spatial sampling lattice, they still have different sampling patterns as well as fundamental period shapes on the frequency plane. Note that, in the common Cartesian coordinate system, the y axis (the Ω2 axis in Fig. 11) is on the upwards direction, while in the display coordinate system, the y axis is generally on the downwards direction. This is the reason why the two fundamental period shapes of the HDFT with the diagonal extension in Figs. 10 and 11 are mirrored with each other. For the test image House, the original size is 512 × 512 , and after the square to hexagonal lattice conversion, the size becomes 512 × 443; for the test image Pentagon, the two sizes are 1024 × 1024 and 1024 × 886, respectively. It is obvious that the two HDFTs have different VC shapes, and due to the same image shape, House and Pentagon correspond to the same VC shapes in the three DFTs, respectively. For the test image IM077, its original size is 720 × 540 , and it becomes 720 × 467 after the square to hexagonal lattice conversion. It is clear that image shape affects the sampling lattice of the DFT apparently. Note that the digital frequency of the DFT has been normalized, thus even the image shape is not square, the spectrum support is still in square shape. Finally, we turn to the fundamental matter of the simulated display. Consider an analog image, we perform 2-D sampling to obtain the digital image. Then, if we perform DFT for spectrum analysis, we expect that, no matter which lattice pattern is used in the spatial domain sampling or which lattice pattern is used in the frequency domain sampling, the digital spectrum should be consistent with the original analog spectrum, and the digital spectra from different sampling patterns and DFTs should be visually similar to each other. Accordingly, the digital spectrum should be properly displayed for visual judgement. Then, referring to the spectrum extension patterns in Fig. 11, we can compare the spectra between the different DFTs and check whether the magnitudes are visually consistent with each other. For example, if we choose any visually apparent magnitude contents along a given direction and compare among the three DFTs, we can find that they are in parallel with each other (note that the displaying of the diagonal
Based on the motivation of the VC-based pixel simulation and the simple algorithm for VC vertices above, the proposed simulated display can be implemented as follows. Given a data array together with the corresponding sampling matrix V , each pixel needs to be simulated with a small VC area on proper position, for which the sampling matrix V is a fundamental tool. Since the VC shapes are the same for all the pixels, we first determine the VC vertices from the sampling matrix V , and we can construct each simulated pixel by drawing the VC area filled with the corresponding intensity value. Then, by means of the sampling matrix V , the position of each pixel can be determined by Vn , where n is the corresponding integer coordinate vector. Finally, the whole simulated display can be constructed by the tessellation of each simulated pixel on its correct lattice position. Also, in this simulated display, an extra parameter is used to control the zooming factor. Regarding to the purpose of this simulated display, i.e., to display the variable lattice data without geometric distortion, we would perform experiments to check whether it could meet the expectations. In the experiments, image data with typical lattices would be displayed on the simulated display, and we could evaluate the effectiveness via visual judgement. In the introduction, a hexagonal lattice image has been displayed on the proposed simulated display as in Fig. 4. If we compare with the original image in Fig. 2, we could find that there is no geometric distortion. Note that the converted hexagonal lattice image has different size from the original image. For the displaying of hexagonal image data as in Fig. 4, the sampling lattice is regular hexagonal, thus the VC vertices are constant and need to be determined only once. Therefore, to better demonstrate the motivation of the proposed simulated display, we would continue to show more examples with variable sampling lattices. As described in Section 2, variable sampling lattices mainly occur in the frequency domain of the HDFTs. In the following, three types of DFTs would be performed, the one was the common square lattice DFT, and the other two were the HDFT with the diagonal extension pattern and the HDFT with the orthogonal extension pattern, as illustrated in Fig. 6. Also, since the frequency domain sampling matrix S could be affected by the image size and shape, we would choose test images with different sizes and shapes. Note that, the square to hexagonal lattice conversion would be performed to obtain the hexagonal lattice data for the two HDFTs. For the test images, in addition to the House shown in Figs. 2 and 4, two other test images were selected, the one was the Pentagon from the USC-SIPI image database [35], and the other was IM077 from Laurent Condat’s image database [36], as shown in Fig. 10. All these three test images were specially chosen for their magnitudes, i.e., their magnitude spectra present strong directionality, and this is helpful for better visual judgement. Among these three test images, House and Pentagon have 67
Displays 50 (2017) 63–69
X. Li
(a)
(b)
(c) Fig. 10. Illustrations of displaying the test images and the spectrum magnitudes with different types of DFTs. For each displaying item, the corresponding VC shape is accompanied underneath. For each row, from left to right: the test image, the magnitude of the common square lattice DFT, the shifted magnitude of the common square lattice DFT, the magnitude of the HDFT with the diagonal extension pattern, and the magnitude of the HDFT with the orthogonal extension pattern.
hexagonal image processing research is mainly based on the simulation approach. This paper has presented a simulated display that could handle the displaying tasks in hexagonal image processing. The paper carefully investigated the most commonly used “hyperpel” approach, and found that, due to its fixed cell pattern, it could not deal with the variable lattices in the frequency domain sampling. Then, the paper proposed to apply the VC-based cell simulation to the general 2-D sampling lattice, for which a simple algorithm was presented to compute the VC vertices. Experimental results have showed that the proposed simulated display could display the data without geometric
extension HDFT has been scaled down for the equal height arrangement reason), i.e., the magnitudes of the two HDFTs have been displayed without geometric distortion. That is, the proposed simulated display can correctly reflect the contents with regard to the corresponding sampling lattice. 5. Conclusion Hexagonal image processing is superior to the commonly used square lattice, but due to the lack of practical hardware devices, current
(a)
(b)
(c)
(d)
Fig. 11. Illustrations of the spectrum extension patterns together with the fundamental period shapes. (a) is for common square lattice DFT, (b) is for the shifted version of the common square lattice DFT, (c) is for the diagonal pattern HDFT, and (d) is for the orthogonal pattern HDFT, respectively.
68
Displays 50 (2017) 63–69
X. Li
[16] S. Bobe, G. Schaefer, saLib - a toolbox and visualisation tool for image processing on spiral architecture, in: 19th Eur. Conf. Model. Simul., 2005, pp. 277–282. [17] X. He, J. Li, D. Wei, W. Jia, Q. Wu, Canny edge detection on a virtual hexagonal image structure, in: 2009 Jt. Conf. Pervasive Comput., 2009, pp. 167–172. [18] W. Wen, S. Khatibi, Back to basics: towards novel computation and arrangement of spatial sensory in images, Acta Polytech. 56 (2016) 409–416. [19] J. Rosenthal, Filters and filterbanks for hexagonally sampled signals, PhD Thesis, Georgia Institute of Technology, 2001. [20] L. Condat, D. Van De Ville, Quasi-interpolating spline models for hexagonallysampled data, IEEE Trans. Image Process. 16 (2007) 1195–1206. [21] V. Argyriou, Sub-hexagonal phase correlation for motion estimation, IEEE Trans. Image Process. 20 (2011) 110–120. [22] D. Van De Ville, T. Blu, M. Unser, W. Philips, I. Lemahieu, R. Van de Walle, Hexsplines: a novel spline family for hexagonal lattices, IEEE Trans. Image Process. 13 (2004) 758–772. [23] M. Mirzargar, A. Entezari, Voronoi splines, IEEE Trans. Signal Process. 58 (2010) 4572–4582. [24] R. Bregović, P.T. Kovács, A. Gotchev, Optimization of light field display-camera configuration based on display properties in spectral domain, Opt. Exp. 24 (2016) 3067–3088. [25] D. Micciancio, P. Voulgaris, A deterministic single exponential time algorithm for most lattice problems based on Voronoi cell computations, SIAM J. Comput. 42 (2013) 1364–1391. [26] M.D. Sikirić, A. Schürmann, F. Vallentin, Complexity and algorithms for computing Voronoi cells of lattices, Math. Comput. 78 (2009) 1713–1731. [27] R.M. Mersereau, T.C. Speake, The processing of periodically sampled multidimensional signals, IEEE Trans. Acoust., Speech, Signal Process. 31 (1983) 188–194. [28] A. Guessoum, R.M. Mersereau, Fast algorithms for the multidimensional discrete Fourier transform, IEEE Trans. Acoust., Speech, Signal Process. 34 (1986) 937–943. [29] J.C. Ehrhardt, Hexagonal fast Fourier transform with rectangular output, IEEE Trans. Signal Process. 41 (1993) 1469–1472. [30] J.L. Zapata, G.X. Ritter, Fast Fourier transform for hexagonal aggregates, J. Math. Imag. Vis. 12 (2000) 183–197. [31] A. Vince, X. Zheng, Computing the discrete Fourier transform on a hexagonal lattice, J. Math. Imag. Vis. 28 (2007) 125–133. [32] P. Sheridan, A method to perform a fast Fourier transform with primitive image transformations, IEEE Trans. Image Process. 16 (2007) 1355–1369. [33] J.B. Birdsong, N.I. Rummelt, The hexagonal fast Fourier transform, in: 2016 IEEE Int. Conf. Image Process., 2016, pp. 1809–1812. [34] M. Deza, M.D. Sikirić, Voronoi polytopes for polyhedral norms on lattices, Discret. Appl. Math. 197 (2015) 42–52. [35] The USC-SIPI Image Database, 2017. (accessed 2017.07.14). [36] Laurent Condat’s Image Database, 2017. (accessed 2017.07.14).
distortion. In the future, we will explore more efficient implementations and make the simulated display to be more practical. Also, current simulation is based on the zero-order hold reconstruction filter, and we expect to exploit high-order reconstruction filters for better visual performance. References [1] R.M. Mersereau, The processing of hexagonally sampled two-dimensional signals, Proc. IEEE 67 (1979) 930–949. [2] K. Mostafa, J.Y. Chiang, I. Her, Edge-detection method using binary morphology on hexagonal images, Imag. Sci. J. 63 (2015) 168–173. [3] B. Gardiner, S. Coleman, B. Scotney, Multiscale edge detection using a finite element framework for hexagonal pixel-based images, IEEE Trans. Image Process. 25 (2016) 1849–1861. [4] S. Veni, K.A. Narayanankutty, Vision-based hexagonal image processing using HexGabor, Signal, Image Video Process. 8 (2014) 317–326. [5] S.H. Contreras-Ortiz, M.D. Fox, Hexagonal filters for ultrasound images, J. Electron. Imag. 23 (2014) 043022. [6] J.F. de Andrade, M.L.R. de Campos, J.A. Apolinário, L1-constrained normalized LMS algorithms for adaptive beamforming, IEEE Trans. Signal Process. 63 (2015) 6524–6539. [7] D.-H. Kim, M.-U. Erdenebat, K.-C. Kwon, J.-S. Jeong, J.-W. Lee, K.-A. Kim, N. Kim, K.-H. Yoo, Real-time 3D display system based on computer-generated integral imaging technique using enhanced ISPP for hexagonal lens array, Appl. Opt. 52 (2013) 8411–8418. [8] A. Hassanfiroozi, Y.-P. Huang, B. Javidi, H.-P.D. Shieh, Hexagonal liquid crystal lens array for 3D endoscopy, Opt. Exp. 23 (2015) 971–981. [9] A. Karimzadeh, Analysis of the depth of field in hexagonal array integral imaging systems based on modulation transfer function and Strehl ratio, Appl. Opt. 55 (2016) 3045–3050. [10] X. Li, Storage and addressing scheme for practical hexagonal image processing, J. Electron. Imag. 22 (2013) 010502. [11] L. Condat, D. Van De Ville, B. Forster-Heinlein, Reversible, fast, and high-quality grid conversions, IEEE Trans. Image Process. 17 (2008) 679–693. [12] B. Gardiner, S. Coleman, B. Scotney, Comparing hexagonal image resampling techniques with respect to feature extraction, in: 14th Int. Mach. Vis. Image Process. Conf., 2011, pp. 102–115. [13] X. Li, B. Gardiner, S.A. Coleman, Square to hexagonal lattice conversion based on one-dimensional interpolation, in: 2016 Sixth Int. Conf. Image Process. Theory, Tools Appl. (IPTA’16), 2016, pp. 1–6. [14] L.N. Lester, J. Sandor, Computer graphics on a hexagonal grid, Comput. Graph. 8 (1984) 401–409. [15] C.A. Wüthrich, P. Stucki, An algorithmic comparison between square- and hexagonal-based grids, CVGIP Graph. Model. Image Process. 53 (1991) 324–339.
69
Contents lists available at ScienceDirect
Displays journal homepage: www.elsevier.com/locate/displa
Implementation of a simulated display for hexagonal image processing
MARK
Xiangguo Li College of Information Science and Engineering, Henan University of Technology, Zhengzhou, Henan 450001, PR China
A R T I C L E I N F O
A B S T R A C T
Keywords: Hexagonal image processing Sampling lattice Simulated display Voronoi cell
Hexagonal image processing is theoretically superior to that based on the common square lattice, and due to the lack of practical imaging devices, this paper intends to implement a simulated display. The paper investigates the sampling lattices used in the hexagonal image processing, and finds that variable lattices occur in hexagonal discrete Fourier transform (HDFT), while the most commonly used “hyperpel” approach, due to its fixed cell pattern, cannot handle such displaying tasks. Then, the paper proposes to represent each pixel with the exact Voronoi cell (VC) according to the sampling lattice. In the paper, a simple algorithm is presented to compute the VC vertices, and each simulated pixel is constructed with the VC filled with its intensity value, and then the simulated display is implemented by the tessellation of each simulated pixel on the correct lattice position. Finally, experimental results show that the proposed simulated display can display data without geometric distortion.
1. Introduction Practical digital signals are mainly obtained by sampling the analog ones. For images, due to the 2-D nature, the sampling points lie on a 2-D plane and form a regular 2-D lattice. According to the sampling theory, the hexagonal lattice is the optimal sampling scheme for circularly band-limited analog images [1]. Since most optical systems are circularly symmetric and thus exhibit circularly low-pass nature, the hexagonal lattice is actually the optimal sampling scheme for sampled imaging systems. Compared with the commonly used square lattice, the hexagonal lattice can provide 13.4% fewer samples [1], and it is also superior with respect to its geometric properties, such as higher degree of symmetry, equal distance and uniform connectivity with its neighbors, as shown in Fig. 1. In recent years, especially with the rise of biologically inspired image processing, hexagonal image processing has spread to applications such as edge detection [2,3], hexagonal Gabor filtering [4], ultrasound image processing [5], and adaptive beamforming [6]. Besides, hexagonal lattice is also adopted in advanced imaging techniques, such as 3-D display system [7], 3-D endoscopy [8], and integral imaging system [9]. However, people are more familiar with the Cartesian orthogonal coordinate system as well as the square lattice. Also, the square lattice has the advantage of simplicity for practical data storage and addressing [10], thus it predominates in the commercial imaging devices, including image sensors (CCD and CMOS) and displays. Therefore, for the hexagonal image processing research at the present time, it is natural to adopt the simulation approach. For the data acquisition, lattice
conversion can be applied by means of interpolation under the resampling framework [11–13]. For the data displaying, each pixel can be simulated by a proper cell and the simulated display can be constructed based on current displays. This paper concerns the data displaying task and proposes to implement a simulated display for hexagonal image processing. Consider a common square lattice image displayed on a practical square lattice display. If we zoom in the image, it is easy to recognize that each pixel is presented with a small square cell, as illustrated in Fig. 2. This has motivated the construction of simulated display for hexagonal lattice data. Firstly, Lester and Sandor [14] proposed the “brick wall” technique, in which each hexagonal pixel was approximated with four (2 × 2) square pixels and even rows stagger odd rows with one square pixel pitch, as shown in Fig. 3(a). Obviously, the “brick wall” approach is coarse and can incur geometric distortion. Later, Wüthrich and Stucki [15] proposed the “hyperpel” technique, in which a more fine cell was used for each hexagonal pixel, as shown in Fig. 3(b). In the regular hexagonal lattice, the vertical and horizontal pitch ratio is , and in the “brick wall” approach, the ratio is 1:1, while in the “hyperpel” approach, the ratio is 7:8 = 0.8750. Therefore, the “hyperpel” approach can be treated as a good approximation and it is the most common displaying approach used in current hexagonal image research [2,3,16–18]. We notice that, when dealing with samples in the frequency domain, the sampling lattices will vary with several factors (fully discussed in Section 2). Then, for the displaying task of the discrete spectrum, the “hyperpel” approach will fail to provide good
E-mail address: [email protected]. http://dx.doi.org/10.1016/j.displa.2017.09.005 Received 11 February 2017; Received in revised form 1 August 2017; Accepted 18 September 2017 Available online 21 September 2017 0141-9382/ © 2017 Elsevier B.V. All rights reserved.
Displays 50 (2017) 63–69
X. Li
need of interpolating the image data. This paper will continue to apply this motivation into the general 2-D sampling lattice, i.e., the proposed simulated display will be based on the exact VC shape with regard to the given sampling lattice and it will be able to cope with the need of data displaying tasks in the hexagonal image processing. VC is an essential property of lattice. In digital image processing, VC is a fundamental tool in the sampling and reconstruction analysis. VC has also been used in hex-splines [22], Voronoi splines [23], screen reconstruction analysis [24], etc. However, computing the VC of a lattice, i.e., determining the vertices, is not direct and the task may be quite complicated for general lattices [25,26]. This paper concerns the 2-D sampling lattices in the hexagonal image processing, and owing to the regular nature, only the VC vertices with regard to one lattice point is needed. In the paper, an intuitive algorithm is presented to compute the VC vertices, and then each pixel cell as well as the whole simulated display can be constructed. The remainder of the paper is organized as follows. For the better understanding the motivation, Section 2 discusses the sampling lattices used in the hexagonal image processing. Section 3 presents the algorithm to compute the VC vertices. Then, Section 4 implements a simulated display. Finally, Section 5 summarizes this paper.
Fig. 1. Comparison of geometric properties between the square lattice (left) and the regular hexagonal lattice (right).
Fig. 2. Illustrations of displaying a square lattice image on the square lattice display and the partial enlarged views.
2. Lattices in hexagonal image processing
(a)
First of all, the (virtually) hexagonally sampled digital images correspond to a common regular hexagonal lattice in the spatial domain. Meanwhile, the periodic sampling process in the spatial domain causes the original spectrum to be periodically replicated in the frequency domain, for which the extension pattern is defined by the reciprocal lattice that is also a regular hexagonal lattice but with different orientation. Given the sampling matrix V for a given spatial sampling lattice, and the corresponding reciprocal lattice U is given by U = 2π V−T . Specifically, these two basic lattices in the hexagonal sampling are shown in Fig. 5, in which V = [v1|v2] = [1,−0.5;0, 3 /2] and U = [u1|u2] = 2π [1,0;1/ 3 ,2/ 3 ]. On the other hand, spectrum analysis and frequency domain image processing are fundamental from both the theoretical and the practical perspectives. However, although a digital image can be obtained from either practical imaging sensor or any resampling approach, the spectrum of the obtained digital image is still continuous because it is simply the periodic extension of the original continuous spectrum. To enable the digital processing techniques in the frequency domain, it is often assumed to construct the periodic sequence in the spatial domain, which corresponds to sampling the periodic spectrum in the frequency domain. This is the basis for the discrete Fourier series (DFS) as well as the discrete Fourier transform (DFT). With the spatial sampling matrix V and the spatial periodic extension pattern N , the frequency domain sampling matrix is given as S = 2π (VN)−T [27]. In the commonly used square lattice case, the spatial sampling matrix V is diagonal, and the extension pattern N is usually selected as
(b)
Fig. 3. Illustration of the techniques used for simulated hexagonal display. (a) the “brick wall” and (b) the “hyperpel”.
approximation and due to its fixed cell pattern it can not be easily adapted either. For this reason, this paper intends to implement a simulated display that can deal with the general data displaying tasks used in hexagonal image processing. This is the main starting point of this paper. Actually, the small squares in Fig. 2 serve as the zero-order hold reconstruction filter and the small square area is the Voronoi cell (VC) of the square lattice. Accordingly, each pixel of the hexagonal lattice data can also be represented with the VC of the hexagonal lattice [11,19–21], as shown in Fig. 4. This approach, instead of directly concerning the discrete nature of practical displays as in both the “brick wall” and the “hyperpel” approaches, is based on exact VC shape of the hexagonal lattice. Owing to the exact VC shape, the image data can be displayed without geometric distortion, and by simply linearly adjusting the VC values, the image can also be easily zoomed without
Fig. 4. Illustrations of displaying the hexagonal lattice image data based on the VC of the hexagonal lattice.
Fig. 5. Illustration of the hexagonal sampling lattice in the spatial domain (left) and the corresponding reciprocal lattice in the frequency domain (right).
64
Displays 50 (2017) 63–69
X. Li
Fig. 6. Illustration of the spatial extension pattern N and the corresponding frequency sampling lattice S of the HDFTs. (a) is for N = [3,0;0,4] and (c) is for N = [3,2;0,4], while (b) and (d) are their frequency samples, respectively.
in both the “brick wall” and the “hyperpel” approaches can not fit the variable nature of the S , this paper proposes to implement a simulated display with the exact VC shape of the corresponding sampling lattice. In the following sections, regarding to a general 2-D sampling lattice, the paper firstly presents a simple algorithm to compute the VC vertices, and then implements the simulated display accordingly.
diagonal matrix, thus the frequency sampling matrix S is also a diagonal matrix. This is a main advantage of the square lattice image processing. In contrast, the spatial sampling matrix V is non-diagonal in the hexagonal lattice case. Moreover, unlike the square lattice DFT, flexible choices of image shape and the extension pattern N are common in the hexagonal discrete Fourier transform (HDFT). In practice, over the past decades, various types of hexagonal fast Fourier transform (HFFT) algorithms have been developed [1,28–33]. Then, although the spatial sampling matrix V is generally constant, the frequency sampling matrix S still varies with the choice of extension pattern N together with the image shape. Even one maybe always use the preferred extension pattern, the shape of image will still affect N and thus S . Therefore, the frequency sampling matrix S may be neither diagonal nor regular hexagonal, and it will become a general one for the HDFT in hexagonal image processing. To better demonstrate the frequency sampling lattice S with regard to the spatial domain extension pattern N , an example is presented in Fig. 6. Consider to compute the HDFT of a hexagonally sampled sequence with the finite support 3 × 4 , the most natural choice of N is through the diagonal pattern as shown in Fig. 6(a), i.e., N = [3,0;0,4] in this case, and the frequency sampling matrix S = 2π [1/3,0;1/3 3 ,1/2 3 ] as shown in Fig. 6(b). It is obvious that the frequency sampling lattice is neither orthogonal nor regular hexagonal. Since the extension pattern N is not unique, another one with the orthogonal extension pattern [29] is presented in Fig. 6(c). In this case, N = [3,2;0,4] and S = 2π [1/3,0;0,1/2 3 ], as shown in Fig. 6(d). It is clear that Fig. 6(b) and (d) have completely different sampling patterns. Then, for the purpose of accurate spectrum analysis, the spectrum needs to be properly displayed. Due to the reasons above, the sampling lattice S needs to be considered for each specific case, and the simulated display should present each pixel with its accurate position together with the exact VC cell shape. Otherwise the spectrum may be displayed with geometric distortion, which will severely degrade the visual effectiveness and thus affect the judgement. Since the fixed cell patterns
3. Computing the VC vertices 3.1. Definitions Given a 2-D sampling lattice defined by the sampling matrix
V = [v1|v2],
(1)
in which v1 and v2 are the two non-collinear sampling vectors. Then, the position of any lattice point can be determined by Vn with n = [n1,n2]T the corresponding integer coordinate vector. Let P = {Pn,n ∈ 2} denote the collection of all the lattice points, and the VC of the point Pi can be defined as [34]:
VC (Pi ) = {x ∈ 2: ‖x−Pi ‖ ⩽ ‖x−Pj ‖ ,∀ i ≠ j},
(2)
in which ∥·∥ is the distance norm. For a regular lattice, the VC shapes are same for all the lattice points, and only the fundamental VC with regard to the origin P0 is needed. The VC shapes of the square lattice, the quincunx lattice, and the regular hexagonal lattice shown in Fig. 7, respectively. 3.2. Determining the neighbors There are eight candidates for the VC neighbors, as shown in Fig. 8(a), in which they are marked as P 0,P1,P 2 , P 3,P 4,P5,P 6, and P7 consecutively. From the typical VC shapes in Fig. 7, there are two typical numbers for neighbor and vertex, i.e., four and six, and this is the cue to follow. 65
Displays 50 (2017) 63–69
X. Li
adjacent neighbor pair. Algorithm 1. VC vertices of a 2-D sampling lattice
1: 2: 3: 4:
Fig. 7. Illustrations of the VCs of the square lattice (left), the quincunx lattice (middle), and the hexagonal lattice (right).
5: 6: 7: 8: 9:
When v1 and v2 are perpendicular to each other, as in the square and quincunx lattices, there are four cases where the intersection points are coincident, and the valid number reduces to four and the neighbors are {P 0,P 2,P 4,P 6} . Otherwise, two points will be removed from the neighbor candidates and the valid number reduces to six; furthermore, P1 and P5 will be removed if the angle between v1 and v2 is acute, or P3 and P7 will be removed if the angle is obtuse.
10: 11: 12: 13: 14: 15: 16: 17:
3.3. Intersection of two perpendicular bisectors Given the origin O and two adjacent neighbors P1 and P2 , for which the two vectors are u1 and u2 , respectively, as shown in Fig. 8(b). First, with the Euclidean norm, the boundary between O and P1 is their perpendicular bisector, and the corresponding vector x on this line must meet
u1T x = 0.5u1T u1.
Input: sampling matrix V = [v1|v2] Const: array {P 0,P1,P 2,P 3,P 4,P5,P 6,P 7} Outputs: number num and the vertices vertices [] function VORONOICELLVERTICES (V) if (v1T v2 = 0) then num ← 4 neighbors [] ← {P 0,P 2,P 4,P 6} else num ← 6 if (v1T v2 > 0) then neighbors [] ← {P 0,P 2,P 3,P 4,P 6,P 7} else neighbors [] ← {P 0,P1,P 2,P 4,P5,P 6} end if end if for (i ≔ 0 to num−1) do p0 ← neighbors [i]
p1 ← neighbors [(i + 1)%num] 18: 19: vertices [i] ← Solve (Vp0,Vp1) 20: end for 21: return num and vertices [] 22: end function
(3)
Similarly, the perpendicular bisector equation for O and P2 is given as:
u 2T x = 0.5u 2T u2.
(4)
3.5. Evaluation and analysis
Then, the intersection point is by solving the simultaneous equations: T T ⎡ u1 ⎤ x = 0.5 ⎡ u1 u1 ⎤. ⎢uTu ⎥ ⎢uT ⎥ ⎣ 2 2⎦ ⎣ 2⎦
Before applying the algorithm, we performed experiments to check the effectiveness. Firstly, we computed on the three typical lattices in Fig. 7, for which their VCs are apparent, and the results were correct. Then, we checked the two HDFTs illustrated in Fig. 6. For Fig. 6(b), num = 6 (0.1389,0.1443) , and the VC vertices were (−0.1389,0.1443),(−0.1944,−0.0481),(−0.1389,−0.1443),(0.1389,−0.1443), and (0.1944,0.0481) . For Fig. 6(d), num = 4 with the VC vertices (0.1667,0.1443),(−0.1667,0.1443),(−0.1667,−0.1443) , and (0.1667,−0.1443) . These results were plotted in Fig. 9, and we could verify the correctness according to the definition of VC. For a given sampling lattice, if V is a valid sampling matrix, with any integer unimodular matrix E (i.e., |detE| = 1), EV is also a valid sampling matrix [27]. Then, since the algorithm is based on the eight candidate points shown in Fig. 8(a), while the actual candidates are closely related to the sampling matrix, the algorithm would fail if any actual candidates escape out of these eight candidate points. In practice, the sampling matrices always use the natural manner, and thus the presented algorithm can work correctly.
(5)
Since u1 and u2 are noncollinear, the coefficient matrix is nonsingular and invertible, and the solution is unique. 3.4. Computing the VC vertices Based on the analysis above, we compute the VC of a 2-D sampling lattice. The input is the sampling matrix V , and the outputs are the vertex number num together with the coordinate array of the vertices vertices []. We firstly determine the num value by checking whether the angle between v1 and v2 is right. Furthermore, to judge the angle be acute, right, or obtuse, we judge whether the sign of the inner product is positive, zero, or negative, respectively. Then, after assigning the correct neighbors, we can compute each vertex by applying Eq. (5) on each
Fig. 8. Illustration of the eight neighbor candidates (a) and two perpendicular bisectors together with their intersection point (b).
66
Displays 50 (2017) 63–69
X. Li
Fig. 9. Illustration of VCs regarding to the sampling lattices in Fig. 6. (a) is for the diagonal pattern N = [3,0;0,4], and (b) is for the orthogonal pattern N = [3,2;0,4].
4. The simulated display
the same image shape but with different sizes, while the IM077 has rectangular shape and thus has completely different shapes and sizes from the other two. The test images and the magnitudes of the different DFTs were displayed in Fig. 10, in which the corresponding VC shapes were also accompanied. Since people are more familiar with the square lattice, the common square lattice DFT was used for reference purpose and its magnitudes served as the ground truth for visual judgement. Also, since different spatial sampling lattices correspond to different spectrum extension patterns, to facilitate the visual judgement, the shifted magnitudes of the square lattice DFT were also provided in Fig. 10, and the spectrum extension patterns together with the fundamental period shapes were illustrated in Fig. 11. Although the two HDFTs have the same spectrum extension pattern from the same spatial sampling lattice, they still have different sampling patterns as well as fundamental period shapes on the frequency plane. Note that, in the common Cartesian coordinate system, the y axis (the Ω2 axis in Fig. 11) is on the upwards direction, while in the display coordinate system, the y axis is generally on the downwards direction. This is the reason why the two fundamental period shapes of the HDFT with the diagonal extension in Figs. 10 and 11 are mirrored with each other. For the test image House, the original size is 512 × 512 , and after the square to hexagonal lattice conversion, the size becomes 512 × 443; for the test image Pentagon, the two sizes are 1024 × 1024 and 1024 × 886, respectively. It is obvious that the two HDFTs have different VC shapes, and due to the same image shape, House and Pentagon correspond to the same VC shapes in the three DFTs, respectively. For the test image IM077, its original size is 720 × 540 , and it becomes 720 × 467 after the square to hexagonal lattice conversion. It is clear that image shape affects the sampling lattice of the DFT apparently. Note that the digital frequency of the DFT has been normalized, thus even the image shape is not square, the spectrum support is still in square shape. Finally, we turn to the fundamental matter of the simulated display. Consider an analog image, we perform 2-D sampling to obtain the digital image. Then, if we perform DFT for spectrum analysis, we expect that, no matter which lattice pattern is used in the spatial domain sampling or which lattice pattern is used in the frequency domain sampling, the digital spectrum should be consistent with the original analog spectrum, and the digital spectra from different sampling patterns and DFTs should be visually similar to each other. Accordingly, the digital spectrum should be properly displayed for visual judgement. Then, referring to the spectrum extension patterns in Fig. 11, we can compare the spectra between the different DFTs and check whether the magnitudes are visually consistent with each other. For example, if we choose any visually apparent magnitude contents along a given direction and compare among the three DFTs, we can find that they are in parallel with each other (note that the displaying of the diagonal
Based on the motivation of the VC-based pixel simulation and the simple algorithm for VC vertices above, the proposed simulated display can be implemented as follows. Given a data array together with the corresponding sampling matrix V , each pixel needs to be simulated with a small VC area on proper position, for which the sampling matrix V is a fundamental tool. Since the VC shapes are the same for all the pixels, we first determine the VC vertices from the sampling matrix V , and we can construct each simulated pixel by drawing the VC area filled with the corresponding intensity value. Then, by means of the sampling matrix V , the position of each pixel can be determined by Vn , where n is the corresponding integer coordinate vector. Finally, the whole simulated display can be constructed by the tessellation of each simulated pixel on its correct lattice position. Also, in this simulated display, an extra parameter is used to control the zooming factor. Regarding to the purpose of this simulated display, i.e., to display the variable lattice data without geometric distortion, we would perform experiments to check whether it could meet the expectations. In the experiments, image data with typical lattices would be displayed on the simulated display, and we could evaluate the effectiveness via visual judgement. In the introduction, a hexagonal lattice image has been displayed on the proposed simulated display as in Fig. 4. If we compare with the original image in Fig. 2, we could find that there is no geometric distortion. Note that the converted hexagonal lattice image has different size from the original image. For the displaying of hexagonal image data as in Fig. 4, the sampling lattice is regular hexagonal, thus the VC vertices are constant and need to be determined only once. Therefore, to better demonstrate the motivation of the proposed simulated display, we would continue to show more examples with variable sampling lattices. As described in Section 2, variable sampling lattices mainly occur in the frequency domain of the HDFTs. In the following, three types of DFTs would be performed, the one was the common square lattice DFT, and the other two were the HDFT with the diagonal extension pattern and the HDFT with the orthogonal extension pattern, as illustrated in Fig. 6. Also, since the frequency domain sampling matrix S could be affected by the image size and shape, we would choose test images with different sizes and shapes. Note that, the square to hexagonal lattice conversion would be performed to obtain the hexagonal lattice data for the two HDFTs. For the test images, in addition to the House shown in Figs. 2 and 4, two other test images were selected, the one was the Pentagon from the USC-SIPI image database [35], and the other was IM077 from Laurent Condat’s image database [36], as shown in Fig. 10. All these three test images were specially chosen for their magnitudes, i.e., their magnitude spectra present strong directionality, and this is helpful for better visual judgement. Among these three test images, House and Pentagon have 67
Displays 50 (2017) 63–69
X. Li
(a)
(b)
(c) Fig. 10. Illustrations of displaying the test images and the spectrum magnitudes with different types of DFTs. For each displaying item, the corresponding VC shape is accompanied underneath. For each row, from left to right: the test image, the magnitude of the common square lattice DFT, the shifted magnitude of the common square lattice DFT, the magnitude of the HDFT with the diagonal extension pattern, and the magnitude of the HDFT with the orthogonal extension pattern.
hexagonal image processing research is mainly based on the simulation approach. This paper has presented a simulated display that could handle the displaying tasks in hexagonal image processing. The paper carefully investigated the most commonly used “hyperpel” approach, and found that, due to its fixed cell pattern, it could not deal with the variable lattices in the frequency domain sampling. Then, the paper proposed to apply the VC-based cell simulation to the general 2-D sampling lattice, for which a simple algorithm was presented to compute the VC vertices. Experimental results have showed that the proposed simulated display could display the data without geometric
extension HDFT has been scaled down for the equal height arrangement reason), i.e., the magnitudes of the two HDFTs have been displayed without geometric distortion. That is, the proposed simulated display can correctly reflect the contents with regard to the corresponding sampling lattice. 5. Conclusion Hexagonal image processing is superior to the commonly used square lattice, but due to the lack of practical hardware devices, current
(a)
(b)
(c)
(d)
Fig. 11. Illustrations of the spectrum extension patterns together with the fundamental period shapes. (a) is for common square lattice DFT, (b) is for the shifted version of the common square lattice DFT, (c) is for the diagonal pattern HDFT, and (d) is for the orthogonal pattern HDFT, respectively.
68
Displays 50 (2017) 63–69
X. Li
[16] S. Bobe, G. Schaefer, saLib - a toolbox and visualisation tool for image processing on spiral architecture, in: 19th Eur. Conf. Model. Simul., 2005, pp. 277–282. [17] X. He, J. Li, D. Wei, W. Jia, Q. Wu, Canny edge detection on a virtual hexagonal image structure, in: 2009 Jt. Conf. Pervasive Comput., 2009, pp. 167–172. [18] W. Wen, S. Khatibi, Back to basics: towards novel computation and arrangement of spatial sensory in images, Acta Polytech. 56 (2016) 409–416. [19] J. Rosenthal, Filters and filterbanks for hexagonally sampled signals, PhD Thesis, Georgia Institute of Technology, 2001. [20] L. Condat, D. Van De Ville, Quasi-interpolating spline models for hexagonallysampled data, IEEE Trans. Image Process. 16 (2007) 1195–1206. [21] V. Argyriou, Sub-hexagonal phase correlation for motion estimation, IEEE Trans. Image Process. 20 (2011) 110–120. [22] D. Van De Ville, T. Blu, M. Unser, W. Philips, I. Lemahieu, R. Van de Walle, Hexsplines: a novel spline family for hexagonal lattices, IEEE Trans. Image Process. 13 (2004) 758–772. [23] M. Mirzargar, A. Entezari, Voronoi splines, IEEE Trans. Signal Process. 58 (2010) 4572–4582. [24] R. Bregović, P.T. Kovács, A. Gotchev, Optimization of light field display-camera configuration based on display properties in spectral domain, Opt. Exp. 24 (2016) 3067–3088. [25] D. Micciancio, P. Voulgaris, A deterministic single exponential time algorithm for most lattice problems based on Voronoi cell computations, SIAM J. Comput. 42 (2013) 1364–1391. [26] M.D. Sikirić, A. Schürmann, F. Vallentin, Complexity and algorithms for computing Voronoi cells of lattices, Math. Comput. 78 (2009) 1713–1731. [27] R.M. Mersereau, T.C. Speake, The processing of periodically sampled multidimensional signals, IEEE Trans. Acoust., Speech, Signal Process. 31 (1983) 188–194. [28] A. Guessoum, R.M. Mersereau, Fast algorithms for the multidimensional discrete Fourier transform, IEEE Trans. Acoust., Speech, Signal Process. 34 (1986) 937–943. [29] J.C. Ehrhardt, Hexagonal fast Fourier transform with rectangular output, IEEE Trans. Signal Process. 41 (1993) 1469–1472. [30] J.L. Zapata, G.X. Ritter, Fast Fourier transform for hexagonal aggregates, J. Math. Imag. Vis. 12 (2000) 183–197. [31] A. Vince, X. Zheng, Computing the discrete Fourier transform on a hexagonal lattice, J. Math. Imag. Vis. 28 (2007) 125–133. [32] P. Sheridan, A method to perform a fast Fourier transform with primitive image transformations, IEEE Trans. Image Process. 16 (2007) 1355–1369. [33] J.B. Birdsong, N.I. Rummelt, The hexagonal fast Fourier transform, in: 2016 IEEE Int. Conf. Image Process., 2016, pp. 1809–1812. [34] M. Deza, M.D. Sikirić, Voronoi polytopes for polyhedral norms on lattices, Discret. Appl. Math. 197 (2015) 42–52. [35] The USC-SIPI Image Database, 2017.
distortion. In the future, we will explore more efficient implementations and make the simulated display to be more practical. Also, current simulation is based on the zero-order hold reconstruction filter, and we expect to exploit high-order reconstruction filters for better visual performance. References [1] R.M. Mersereau, The processing of hexagonally sampled two-dimensional signals, Proc. IEEE 67 (1979) 930–949. [2] K. Mostafa, J.Y. Chiang, I. Her, Edge-detection method using binary morphology on hexagonal images, Imag. Sci. J. 63 (2015) 168–173. [3] B. Gardiner, S. Coleman, B. Scotney, Multiscale edge detection using a finite element framework for hexagonal pixel-based images, IEEE Trans. Image Process. 25 (2016) 1849–1861. [4] S. Veni, K.A. Narayanankutty, Vision-based hexagonal image processing using HexGabor, Signal, Image Video Process. 8 (2014) 317–326. [5] S.H. Contreras-Ortiz, M.D. Fox, Hexagonal filters for ultrasound images, J. Electron. Imag. 23 (2014) 043022. [6] J.F. de Andrade, M.L.R. de Campos, J.A. Apolinário, L1-constrained normalized LMS algorithms for adaptive beamforming, IEEE Trans. Signal Process. 63 (2015) 6524–6539. [7] D.-H. Kim, M.-U. Erdenebat, K.-C. Kwon, J.-S. Jeong, J.-W. Lee, K.-A. Kim, N. Kim, K.-H. Yoo, Real-time 3D display system based on computer-generated integral imaging technique using enhanced ISPP for hexagonal lens array, Appl. Opt. 52 (2013) 8411–8418. [8] A. Hassanfiroozi, Y.-P. Huang, B. Javidi, H.-P.D. Shieh, Hexagonal liquid crystal lens array for 3D endoscopy, Opt. Exp. 23 (2015) 971–981. [9] A. Karimzadeh, Analysis of the depth of field in hexagonal array integral imaging systems based on modulation transfer function and Strehl ratio, Appl. Opt. 55 (2016) 3045–3050. [10] X. Li, Storage and addressing scheme for practical hexagonal image processing, J. Electron. Imag. 22 (2013) 010502. [11] L. Condat, D. Van De Ville, B. Forster-Heinlein, Reversible, fast, and high-quality grid conversions, IEEE Trans. Image Process. 17 (2008) 679–693. [12] B. Gardiner, S. Coleman, B. Scotney, Comparing hexagonal image resampling techniques with respect to feature extraction, in: 14th Int. Mach. Vis. Image Process. Conf., 2011, pp. 102–115. [13] X. Li, B. Gardiner, S.A. Coleman, Square to hexagonal lattice conversion based on one-dimensional interpolation, in: 2016 Sixth Int. Conf. Image Process. Theory, Tools Appl. (IPTA’16), 2016, pp. 1–6. [14] L.N. Lester, J. Sandor, Computer graphics on a hexagonal grid, Comput. Graph. 8 (1984) 401–409. [15] C.A. Wüthrich, P. Stucki, An algorithmic comparison between square- and hexagonal-based grids, CVGIP Graph. Model. Image Process. 53 (1991) 324–339.
69