- Email: [email protected]
Description of the cardiac movement using hexagonal image structures
Computerized Medical Imaging and Graphics 30 (2006) 377–382
Description of the cardiac movement using hexagonal image structures Xiangjian He ∗,1 , Wenjing Jia 2 , Qiang Wu 2 , Tom Hintz 2 Computer Vision Research Group, University of Technology, Sydney, P.O. Box 123, Broadway 2007, NSW, Australia
Abstract The most notable characteristic of the heart is its movement. Detection of dynamic information describing cardiac movement such as amplitude, speed and acceleration facilitates interpretation of normal and abnormal function. In recent years, the Omni-directional M-mode Echocardiography System (OMES) has been developed as a process that builds moving information from a sequence of echocardiography image frames. OMES detects cardiac movement through construction and analysis of Position–Time Grey Waveform (PTGW) images on some feature points of the boundaries of the ventricles. Image edge detection plays an important role in determining the feature boundary points and their moving directions as the basis for extraction of PTGW images—Spiral Architecture (SA) has proved efficient for image edge detection. SA is a hexagonal image structure in which an image is represented as a collection of hexagonal pixels. There are two operations called spiral addition and spiral multiplication defined on SA. They correspond to image translation and rotation, respectively. In this paper, we perform ventricle boundary detection based on SA using various defined chain codes. The gradient direction of each boundary point is determined at the same time. PTGW images at each boundary point are obtained through a series of spiral additions according to the directions of boundary points. Unlike the OMES system, our new approach is no longer affected by the translation movement of the heart. As its result, three curves representing the amplitude, speed and acceleration of cardiac movement can be easily drawn from the PTGW images obtained. Our approach is more efficient and accurate than OMES, and our results contain a more robust and complete description of cardiac motion. © 2006 Elsevier Ltd. All rights reserved. Keywords: Edge detection; Hexagonal image structure; Image analysis; Omni-directional M-mode Echocardiography System; Spiral Architecture
1. Introduction Sequential image frames captured by a camera contain not only information about the objects in each individual frame but also about the motion of the objects. The optical flow information detection method [1] used to analyze the velocity field of each pixel in an image has such limitations as high computation load for tracking deformable objects and sensitive to noise when dealing with deformable objects. In order to overcome this problem and explore motion information of objects from sequential images, Lin et al. [2] proposed the Omni-directional M-mode Echocardiography System (OMES). OMES is a system that builds a position–time function (called Position–Time Grey Waveform, PTGW) for each pixel on the boundaries of cardiac ventricles. The PTGW extracts motion information of moving objects or moving characteristic points such as edge
∗
Corresponding author. E-mail addresses: [email protected] (X. He), [email protected] (W. Jia), [email protected] (Q. Wu), [email protected] (T. Hintz). 1 Senior Member of IEEE. 2 Members of IEEE. 0895-6111/$ – see front matter © 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.compmedimag.2006.09.001
points on images in certain directions. The motion information contains velocity, acceleration and amplitude. The next position of a moving pixel (or point) on a directional line is predicted. The movement of cardiac ventricles is complex, multidirectional and rotational. Hence, it is difficult to determine the direction of motion for each pixel on the boundary. There have been two schemes for setting the moving direction as shown in Ref. [2]: (1) fixed direction setting, (2) variable direction setting. For fixed direction setting, the direction lines are invariable to the frames of sequential images. For variable direction setting, the direction lines change according to the pixel location in every frame. Lin et al. in Ref. [2] demonstrated a method of fixed direction setting by considering only the motion in the plane of the ultrasound image. Their method simply selects 12 direction lines projected in every 30◦ from a common central point looking like a clock (see Fig. 1). The PTGW formed on each of the 12 direction lines depends on the location of the selected central point.
378
X. He et al. / Computerized Medical Imaging and Graphics 30 (2006) 377–382
Fig. 1. Omni-directional PTGW of cardiac ventricles [2].
Fig. 3. A 49 hexagonal pixels labeled by spiral addresses.
This method does not work well when the movement is acentric or when the centre of the object is moving. An example of acentric movement is the movement of cardiac ventricles especially when the cardiac image is represented in short axis [2]. In this paper, we will present another method based on Spiral Architecture (SA) [3], a relatively new image structure. The moving direction will be determined dynamically. For each image frame, the direction of an edge pixel is uniquely defined by the gradient direction of the edge pixel. The paper is organized as follows. Section 2 reviews Spiral Architecture. A contour extraction technique on SA is introduced in Section 3. This is followed by the construction of PTGW for detecting the movement of cardiac ventricles in Section 4. Experimental results are demonstrated in Section 5. We conclude in Section 6.
putation abilities. In SA, an image is represented as a collection of hexagonal pixels. Apparently, the hexagonal pixels cannot be labeled in normal column–row order as in conventional square grids. In order to properly address and store hexagonal images data, a one-dimensional addressing system, called spiral addressing, was defined. In the spiral addressing scheme, each pixel is assigned an integer of base 7 starting from 0 (see Fig. 3). The assigned integers are called spiral addresses. Based on this addressing scheme, another concept called spiral counting was defined [4]. Spiral counting can be considered as a spiral movement from a hexagonal pixel to another hexagonal pixel. Any hexagonal pixel in an image can be reached by spiral counting from any other given hexagon in the same image. When applying spiral counting, it is strictly dependent on a pre-determined key defined by Sheridan in Ref. [4]. A key is the first hexagon to be reached in an instance of a spiral counting, which determines two important parameters: the distance and the orientation. For instance, given a spiral address 15, the key of 15 can determine two values. One is the distance between the given hexagon 15 and the hexagon 0; the other is the orientation of hexagon 15 from hexagon 0. Spiral counting is used to define two operations in the SA, which are spiral addition and spiral multiplication [4]. Let a and b be the spiral addresses of two arbitrarily chosen hexagons in SA.
2. Spiral Architecture Spiral Architecture [3] is inspired from anatomical considerations of primate vision. The distribution of cones on the retina (see Fig. 2) provides the basis of the Spiral Architecture. In the case of the human eye, these elements would represent the relative positions of the rods and cones on the retina [4]. Research on the geometry of the visual cones of the primate retina concludes that the pattern of cone distribution determines its powerful com-
Definition 1. Spiral addition of a and b [4], denoted by a ⊕ b, is the spiral address of the hexagon found by spiral counting b hexagons in the key of spiral address 1 from the hexagon with spiral address a. For example, 3 ⊕ 2 = 26. Definition 2. Spiral multiplication of a and b [4], denoted by a ⊗ b, is the spiral address of the hexagon found by spiral counting b hexagons in the key of spiral address a from the hexagon with spiral address 0. For example, 15 ⊗ 2 = 26. Fig. 2. Distribution of cones on the retina [4].
It is important to note that each of these two operations takes two spiral addresses and maps them to another spiral address.
X. He et al. / Computerized Medical Imaging and Graphics 30 (2006) 377–382
379
They can then be used to define two transformations on the spiral address space [5]: (1) image translation (by spiral addition) and (2) image rotation with scaling (by spiral multiplication). Any movement from one pixel to another can be easily done through a spiral addition. For example, in order to move the pixel with spiral address 4 to the pixel location with address 31, we can simply perform a spiral addition, 4 ⊕ 3. In SA, we can perform the extraction of object boundary points together with their gradient directions in the next section. 3. Contour extraction The contour of an object contains relevant feature information about that object. Contour based technique has been proven to be an effective approach for object recognition in the area of computer vision [6]. Most of the recognition schemes assume that a closed object boundary is obtained and the recognition algorithms are developed based on the detected closed boundary. Accordingly, closed contour detection is a crucial and the first step in object recognition. Traditional contour detection algorithms suffer from the limitation [7,8] that a closed contour is not guaranteed. Many attempts have been made for the detection of closed contours. Some examples are an algorithm for adapting a geometrical model [9], an algorithm with which some contour closure requirements are set up [10], and the enhanced snakes algorithm [8]. In this section, an approach to the object contour extraction based on SA is presented [11]. In order to extract contours from pre-detected edge pixels, a pixel is defined as an object pixel if the pixel is on the object of interest (e.g., cardiac ventricles). An object pixel is called a contour pixel if and only if there exist at least two adjacent pixels and at least one non-object pixel next to it. As an illustration, in Fig. 4, pixel A is a contour pixel. The definition of contour pixels guarantees that any contour pixel is on a closed contour and contour pixels form closed contours and an object. A contour tangent line is a line segment connecting adjacent contour pixels. The direction of the tangent line connecting contour pixels A and B is called from A to B if there is an object pixel next to A and B, and on the right hand side of this oriented line. For example, the direction of the segment AB in Fig. 4 is from A to B since the object pixel C is on the right hand side of the tangent line. When the tangent line connecting contour pixels A and B has the direction from A to B, the direction is called a tangent direction of pixel A. A contour has at least one tangent direction and at most two tangent directions that are opposite. For example, in Fig. 4, pixel A has tangent directions from A to B and from A to D.
Fig. 4. Contour pixel, object pixel and non-object pixel.
Fig. 5. Chain code of different contour points.
In order to extract object contours, contour pixels are classified into six classes of which each is given a class code called chain code number from 0 to 5 as shown in Fig. 5. Fig. 5 also shows the tangent direction of each contour pixel as indicated by arrows. The chain code defined in Fig. 5 implies that outermost contours have clockwise direction. Based on the chain code, the method for contour extraction can be described as follows: (1) Determine the object mask: The object mask consists of selected object pixels from which the object contours can be extracted. (2) Edge detection: Choose an edge detector to obtain an edge map of the sample image. (3) Get initial contour pixels: Initial contour pixels are obtained according to the definition of contour pixels as shown in Fig. 3. (4) Include more contour pixels: Any contour pixel has a direction as shown in Fig. 4 and should be connected to an adjacent pixel indicated by this direction. We include these adjacent pixels and record them as contour pixels. (5) Get remaining contour pixels: We repeat the previous step until no more contour pixels can be found. (6) Find the outermost contours: The outermost contours are obtained by clockwise direction of these contours. 4. Construction of PTGW We now come to the stages for construction of Position–Time Grey Waveform for the ultrasound images of cardiac ventricles. We follow the steps shown in the previous section for contour extraction of ventricles. The object mask is obtained as follows. The movement of the ventricle wall is caused by blood filling and ejection, and the contractile and relaxation force of the cardiac muscle. When the incident ultrasound wave hits the interface of the blood and the ventricular wall of the heart, the echo wave forms a grey level image, of which the intensity of the grey
380
X. He et al. / Computerized Medical Imaging and Graphics 30 (2006) 377–382
Fig. 7. An example of a position–time Graph.
Fig. 6. A cardiac ultrasound image showing a cluster of bright pixels.
level at each pixel corresponds to the acoustic impedance and the ultrasonic attenuation properties along the propagation path of the ultrasound pulse. Because the echo is strong around the area close to the interface of the blood and the ventricle wall, the ultrasound image will show a cluster of bright pixels in this area (see Fig. 6). These bright pixels are collected to form the object mask of cardiac ventricles. After the object mask is obtained, the contour of each ventricle is extracted. The moving direction of each contour pixel is also determined perpendicular to the tangent direction at this pixel. The following steps show how the PTGW is constructed. (1) Choose a set of contour pixels on the first image from sequential image frames. For example, we can choose a contour pixel at set intervals (e.g., 10) of consecutive contour pixels. We call these pixels reference pixels. (2) Record the positions of these reference pixels. Corresponding to each reference pixel, create a position–time graph (PTG). Each PTG has time as its x-axis and position as its yaxis. The position is the distance between the reference pixel and the contour pixel (target pixel) in the moving direction of the reference pixel. The value representing the position (or the distance) can be either positive or negative depending on whether the target pixel is in the positive or negative gradient direction relative to the reference pixel. The position value of the reference pixel is 0. Plot this position on the PTG. (3) Move to next image in the sequential image frames. Starting from the reference pixel location, along the moving direction of the reference pixel, search for the first contour pixel (a target pixel). Moving one pixel to another pixel along a certain direction can be easily done through a spiral addition. Compute the position value of this target pixel and plot the position on the PTG. The assumption made here for searching for the target pixel from the know reference pixel is that the movement of every boundary pixel is along its gradient direction. Therefore, the target pixel must be in the direction of the gradient of the reference pixel.
Fig. 8. An example of a direction–time Graph.
(4) Move to next image in the sequential image frames. Starting from the location of the target pixel just found, along the moving direction of this target pixel, search for the first contour pixel (a new target pixel). Compute the position value of this new target pixel and plot the position on the PTG. (5) Repeat the previous steps until no more images remain in the sequential image frames. While constructing the PTG for each reference pixel, if we also record the grey levels of the reference pixel and all the target pixels, and display the PTG as a grey picture using grey levels, then the PTG becomes a PTGW. Furthermore, if we also record the moving directions of the reference pixel and the target pixels and represent them using the six integers3 from 0 to 5, then we can construct another graph that shows the relationship between the time and moving direction. We may call this graph a direction–time graph (DTG). Figs. 7 and 8 give examples of a PTG and a DTG, respectively. As the result, three curves representing the amplitude, speed and acceleration of cardiac movement, respectively, can be easily drawn from the PTGW/PTG images obtained. Meanwhile, the DTG images show clearly the changes of boundary moving directions of cardiac ventricles (Fig. 8).
3 Note that there are only six different moving directions on Spiral Architecture as displayed in Fig. 4.
X. He et al. / Computerized Medical Imaging and Graphics 30 (2006) 377–382
Fig. 9. The original image represented in a Spiral Architecture.
381
Fig. 11. Object mask obtained from Fig. 9.
5. Experimental results We demonstrate some experimental results in this section. The original image represented in traditional square image structure is shown in Fig. 6. The dimension of the image in traditional square image structure is 760 pixels wide and 560 pixels high. The same image is represented in a mimic Spiral Architecture as shown in Fig. 9. In the mimic Spiral Architecture, we use four square pixels to form one hexagonal pixel as shown in Ref. [3]. Fig. 10 gives an illustration of such mimic scheme. Image area in Spiral Architecture, as shown in Figs. 9 and 11, contains 76 = 117649 hexagonal grids. The distribution of the hexagonal grids is shown in Fig. 3. Based on our experiments and observation, the grey value of 240 is empirically set as the threshold to form a binary image. The pixel with a grey value ≥240 is recorded as an object (interest) pixel, and the pixel whose grey value <240 is recorded as a non-object pixel. Please note that for a better estimation of the threshold to use, a learning process can be applied using a number of sample images. The object mask is shown in Fig. 11. Using the method shown in Ref. [3] for contour extraction, the contour of the object along the boundary of the endocardium is demonstrated in Fig. 12, and the corresponding interest boundary pixels are displayed in Fig. 13. Note that, we used four square pixels to mimic one single hexagonal pixel as shown in Figs. 10 and 12 shows only one square pixel for each hexago-
Fig. 10. A cluster of seven mimic hexagons.
Fig. 12. Contour of interest object close to the cardiac boundary.
nal pixel on the contour, hence we see the gaps between adjacent contour pixels. These gaps will disappear if all four square pixels are used for each hexagonal pixel on this closed contour. In Fig. 13, there are 48 interest boundary pixels in total. We have recorded that their direction chain codes are either 1 or 2, except for the first pixel whose chain code is 0. The graph showing the directions of interest pixels is in Fig. 14.
Fig. 13. The interest pixels of cardiac boundary obtained from the contour shown in Fig. 12.
382
X. He et al. / Computerized Medical Imaging and Graphics 30 (2006) 377–382
Fig. 14. Graph showing the directions of the interest pixels shown in Fig. 13.
We can obtain interest boundary pixels and their directions (chain codes) for each image frame. The moving directions of the boundary pixels can then be determined from their chain codes. The PTG and DTG can hence be easily constructed. 6. Conclusion In this paper, we have performed a ventricle boundary detection scheme based on SA using various defined chain codes. The gradient or the moving direction of each boundary pixel is determined at the same time. PTGW together with PTG and DTG images at each boundary point can be obtained at the same time through a series of spiral additions according to the directions of boundary pixels. Unlike the OMES system, our new approach no longer requires the determination of the central pixel related to the boundary (or contour). Our approach is a new method for dynamic and variable direction line setting and is not affected by the translation movement of the heart. Three curves representing the amplitude, speed and acceleration of cardiac movement can be easily drawn from the PTGW/PTG images obtained. Our approach also shows the changes of the moving directions in terms of time from the DTG. It is more efficient and accurate than OMES [2], and contains a more robust and complete description of cardiac motion. The movement of the boundary interface between the blood and the ventricular wall directly reflects the movement of the wall of the heart. This movement is caused by blood filling and the contractile and relaxation force of the cardiac muscle. Hence, the kinetic detail described using PTG/PTGW and DTG provides important haemodynamic information that can be used to distinguish healthy and diseased cardiac function. Acknowledgment This paper is partly supported by the Readership Allowance of the Faculty of Information Technology, University of Technology, Sydney. References [1] Horn BK, Schunck BG. Determining optical flow. Artif Intell 1981;17: 185–204.
[2] Lin Q, Jia W, Yang X. A method for mining data of sequential images— rebuilding of gray (position)–time function on arbitrary direction lines. In: Proceedings of the international conference on imaging science, systems, and technology (CISST’02). 2002. p. 3–6. [3] He X, Hintz T, Szewcow U. Replicated shared object model for parallel edge detection algorithm based on spiral architecture. Fut Gen Comput Syst J Elsevier 1998;14:341–50. [4] Sheridan P. Sprial architecture for machine vision. PhD thesis. Sydney: University of Technology, 1996. [5] Wu Q, He X, Hintz T. Distributed image processing on spiral architecture. In: Proceedings of the fifth international conference on algorithm and architectures for parallel processing (IEEE). 2002. p. 84–91. [6] Mokhtarian F. Silhouette-based object recognition through curvature scale space. IEEE Trans Pattern Anal Mach Intell 1995;17(5):539–44. [7] Haralick RM, Shapiro LG. Computer and robot vision. Addison-Wesley Publishing, Co.; 1992. [8] Yuen PC, Wong YY, Tong CS. Enhanced snakes algorithm for contour detection. In: Proceedings of the IEEE Southwest symposium on image analysis and interpretation. 1996. p. 54–9. [9] Iannizzotto G, Vita L. A fast, accurate method to segment and retrieve object contours in real images. In: Proceedings of the IEEE international conference on image processing. 1996. p. 841–3. [10] Nabout A, Nour Eldin HA. The topological contour closure requirement for object extraction from 2D-digital images. In: Proceedings of the IEEE international conference on systems, man and cybernetics. 1993. p. 120–5. [11] He X, Hintz T. A parallel algorithm for object contour extraction within a spiral architecture. In: Proceedings of the international conference on parallel and distributed processing techniques and applications. 1999. p. 1462–8. Xiangjian He Associate Professor, received his BSc degree from Xiamen University, and MSc degree from Fuzhou University, China in 1982 and 1985 respectively. He received his PhD degree in the area of Computer Vision from the University of Technology, Sydney (UTS) in 1999. He is the Deputy Director of the UTS university-level Computer Vision Research Group, and a Senior Member of IEEE. He has been carrying out research in the areas of computer vision and image processing based on the Spiral Architecture since 1996. Wenjing Jia received her Bachelor degree from Changchun Institute of Posts and Telecommunications, China in 1999 and her Master degree from Fuzhou University, China in 2002. She has completed her PhD study in 2006. She was an associate lecture in Fuzhou University from 1999 to 2003. Currently, she is a Research Assistant in Faculty of IT at University of Technology, Sydney. Her current research interests include image processing, pattern recognition, object detection and recognition, and color image/video retrieval. She is a member of IEEE. Dr Qiang Wu received his PhD degree in 2004 from University of Technology, Sydney (UTS). He is currently a lecturer and the Coordinator of Computer Vision and Image Processing Lab at UTS. Dr Wu has been working in the field of computer vision and image processing since 2000. His research interests include image processing, pattern recognition, spiral architecture, and radar image processing. He is a member of IEEE. Tom Hintz (Associate Professor) received his Doctor of Science degree from New Mexico State University. He has been a consultant to NASA on the first manned flight to the moon. He was a Specialist Member of the Technical Staff of Bell Laboratories. He has been Chair of the Computer Science Department at New Mexico Tech, Head of the Computer Systems Department at UTS and has been Associate Dean (Research, Policy and Planning) in the Faculty of Information Technology since 2003. His research is in parallel image processing and image compression. He is a member of IEEE.
Description of the cardiac movement using hexagonal image structures Xiangjian He ∗,1 , Wenjing Jia 2 , Qiang Wu 2 , Tom Hintz 2 Computer Vision Research Group, University of Technology, Sydney, P.O. Box 123, Broadway 2007, NSW, Australia
Abstract The most notable characteristic of the heart is its movement. Detection of dynamic information describing cardiac movement such as amplitude, speed and acceleration facilitates interpretation of normal and abnormal function. In recent years, the Omni-directional M-mode Echocardiography System (OMES) has been developed as a process that builds moving information from a sequence of echocardiography image frames. OMES detects cardiac movement through construction and analysis of Position–Time Grey Waveform (PTGW) images on some feature points of the boundaries of the ventricles. Image edge detection plays an important role in determining the feature boundary points and their moving directions as the basis for extraction of PTGW images—Spiral Architecture (SA) has proved efficient for image edge detection. SA is a hexagonal image structure in which an image is represented as a collection of hexagonal pixels. There are two operations called spiral addition and spiral multiplication defined on SA. They correspond to image translation and rotation, respectively. In this paper, we perform ventricle boundary detection based on SA using various defined chain codes. The gradient direction of each boundary point is determined at the same time. PTGW images at each boundary point are obtained through a series of spiral additions according to the directions of boundary points. Unlike the OMES system, our new approach is no longer affected by the translation movement of the heart. As its result, three curves representing the amplitude, speed and acceleration of cardiac movement can be easily drawn from the PTGW images obtained. Our approach is more efficient and accurate than OMES, and our results contain a more robust and complete description of cardiac motion. © 2006 Elsevier Ltd. All rights reserved. Keywords: Edge detection; Hexagonal image structure; Image analysis; Omni-directional M-mode Echocardiography System; Spiral Architecture
1. Introduction Sequential image frames captured by a camera contain not only information about the objects in each individual frame but also about the motion of the objects. The optical flow information detection method [1] used to analyze the velocity field of each pixel in an image has such limitations as high computation load for tracking deformable objects and sensitive to noise when dealing with deformable objects. In order to overcome this problem and explore motion information of objects from sequential images, Lin et al. [2] proposed the Omni-directional M-mode Echocardiography System (OMES). OMES is a system that builds a position–time function (called Position–Time Grey Waveform, PTGW) for each pixel on the boundaries of cardiac ventricles. The PTGW extracts motion information of moving objects or moving characteristic points such as edge
∗
Corresponding author. E-mail addresses: [email protected] (X. He), [email protected] (W. Jia), [email protected] (Q. Wu), [email protected] (T. Hintz). 1 Senior Member of IEEE. 2 Members of IEEE. 0895-6111/$ – see front matter © 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.compmedimag.2006.09.001
points on images in certain directions. The motion information contains velocity, acceleration and amplitude. The next position of a moving pixel (or point) on a directional line is predicted. The movement of cardiac ventricles is complex, multidirectional and rotational. Hence, it is difficult to determine the direction of motion for each pixel on the boundary. There have been two schemes for setting the moving direction as shown in Ref. [2]: (1) fixed direction setting, (2) variable direction setting. For fixed direction setting, the direction lines are invariable to the frames of sequential images. For variable direction setting, the direction lines change according to the pixel location in every frame. Lin et al. in Ref. [2] demonstrated a method of fixed direction setting by considering only the motion in the plane of the ultrasound image. Their method simply selects 12 direction lines projected in every 30◦ from a common central point looking like a clock (see Fig. 1). The PTGW formed on each of the 12 direction lines depends on the location of the selected central point.
378
X. He et al. / Computerized Medical Imaging and Graphics 30 (2006) 377–382
Fig. 1. Omni-directional PTGW of cardiac ventricles [2].
Fig. 3. A 49 hexagonal pixels labeled by spiral addresses.
This method does not work well when the movement is acentric or when the centre of the object is moving. An example of acentric movement is the movement of cardiac ventricles especially when the cardiac image is represented in short axis [2]. In this paper, we will present another method based on Spiral Architecture (SA) [3], a relatively new image structure. The moving direction will be determined dynamically. For each image frame, the direction of an edge pixel is uniquely defined by the gradient direction of the edge pixel. The paper is organized as follows. Section 2 reviews Spiral Architecture. A contour extraction technique on SA is introduced in Section 3. This is followed by the construction of PTGW for detecting the movement of cardiac ventricles in Section 4. Experimental results are demonstrated in Section 5. We conclude in Section 6.
putation abilities. In SA, an image is represented as a collection of hexagonal pixels. Apparently, the hexagonal pixels cannot be labeled in normal column–row order as in conventional square grids. In order to properly address and store hexagonal images data, a one-dimensional addressing system, called spiral addressing, was defined. In the spiral addressing scheme, each pixel is assigned an integer of base 7 starting from 0 (see Fig. 3). The assigned integers are called spiral addresses. Based on this addressing scheme, another concept called spiral counting was defined [4]. Spiral counting can be considered as a spiral movement from a hexagonal pixel to another hexagonal pixel. Any hexagonal pixel in an image can be reached by spiral counting from any other given hexagon in the same image. When applying spiral counting, it is strictly dependent on a pre-determined key defined by Sheridan in Ref. [4]. A key is the first hexagon to be reached in an instance of a spiral counting, which determines two important parameters: the distance and the orientation. For instance, given a spiral address 15, the key of 15 can determine two values. One is the distance between the given hexagon 15 and the hexagon 0; the other is the orientation of hexagon 15 from hexagon 0. Spiral counting is used to define two operations in the SA, which are spiral addition and spiral multiplication [4]. Let a and b be the spiral addresses of two arbitrarily chosen hexagons in SA.
2. Spiral Architecture Spiral Architecture [3] is inspired from anatomical considerations of primate vision. The distribution of cones on the retina (see Fig. 2) provides the basis of the Spiral Architecture. In the case of the human eye, these elements would represent the relative positions of the rods and cones on the retina [4]. Research on the geometry of the visual cones of the primate retina concludes that the pattern of cone distribution determines its powerful com-
Definition 1. Spiral addition of a and b [4], denoted by a ⊕ b, is the spiral address of the hexagon found by spiral counting b hexagons in the key of spiral address 1 from the hexagon with spiral address a. For example, 3 ⊕ 2 = 26. Definition 2. Spiral multiplication of a and b [4], denoted by a ⊗ b, is the spiral address of the hexagon found by spiral counting b hexagons in the key of spiral address a from the hexagon with spiral address 0. For example, 15 ⊗ 2 = 26. Fig. 2. Distribution of cones on the retina [4].
It is important to note that each of these two operations takes two spiral addresses and maps them to another spiral address.
X. He et al. / Computerized Medical Imaging and Graphics 30 (2006) 377–382
379
They can then be used to define two transformations on the spiral address space [5]: (1) image translation (by spiral addition) and (2) image rotation with scaling (by spiral multiplication). Any movement from one pixel to another can be easily done through a spiral addition. For example, in order to move the pixel with spiral address 4 to the pixel location with address 31, we can simply perform a spiral addition, 4 ⊕ 3. In SA, we can perform the extraction of object boundary points together with their gradient directions in the next section. 3. Contour extraction The contour of an object contains relevant feature information about that object. Contour based technique has been proven to be an effective approach for object recognition in the area of computer vision [6]. Most of the recognition schemes assume that a closed object boundary is obtained and the recognition algorithms are developed based on the detected closed boundary. Accordingly, closed contour detection is a crucial and the first step in object recognition. Traditional contour detection algorithms suffer from the limitation [7,8] that a closed contour is not guaranteed. Many attempts have been made for the detection of closed contours. Some examples are an algorithm for adapting a geometrical model [9], an algorithm with which some contour closure requirements are set up [10], and the enhanced snakes algorithm [8]. In this section, an approach to the object contour extraction based on SA is presented [11]. In order to extract contours from pre-detected edge pixels, a pixel is defined as an object pixel if the pixel is on the object of interest (e.g., cardiac ventricles). An object pixel is called a contour pixel if and only if there exist at least two adjacent pixels and at least one non-object pixel next to it. As an illustration, in Fig. 4, pixel A is a contour pixel. The definition of contour pixels guarantees that any contour pixel is on a closed contour and contour pixels form closed contours and an object. A contour tangent line is a line segment connecting adjacent contour pixels. The direction of the tangent line connecting contour pixels A and B is called from A to B if there is an object pixel next to A and B, and on the right hand side of this oriented line. For example, the direction of the segment AB in Fig. 4 is from A to B since the object pixel C is on the right hand side of the tangent line. When the tangent line connecting contour pixels A and B has the direction from A to B, the direction is called a tangent direction of pixel A. A contour has at least one tangent direction and at most two tangent directions that are opposite. For example, in Fig. 4, pixel A has tangent directions from A to B and from A to D.
Fig. 4. Contour pixel, object pixel and non-object pixel.
Fig. 5. Chain code of different contour points.
In order to extract object contours, contour pixels are classified into six classes of which each is given a class code called chain code number from 0 to 5 as shown in Fig. 5. Fig. 5 also shows the tangent direction of each contour pixel as indicated by arrows. The chain code defined in Fig. 5 implies that outermost contours have clockwise direction. Based on the chain code, the method for contour extraction can be described as follows: (1) Determine the object mask: The object mask consists of selected object pixels from which the object contours can be extracted. (2) Edge detection: Choose an edge detector to obtain an edge map of the sample image. (3) Get initial contour pixels: Initial contour pixels are obtained according to the definition of contour pixels as shown in Fig. 3. (4) Include more contour pixels: Any contour pixel has a direction as shown in Fig. 4 and should be connected to an adjacent pixel indicated by this direction. We include these adjacent pixels and record them as contour pixels. (5) Get remaining contour pixels: We repeat the previous step until no more contour pixels can be found. (6) Find the outermost contours: The outermost contours are obtained by clockwise direction of these contours. 4. Construction of PTGW We now come to the stages for construction of Position–Time Grey Waveform for the ultrasound images of cardiac ventricles. We follow the steps shown in the previous section for contour extraction of ventricles. The object mask is obtained as follows. The movement of the ventricle wall is caused by blood filling and ejection, and the contractile and relaxation force of the cardiac muscle. When the incident ultrasound wave hits the interface of the blood and the ventricular wall of the heart, the echo wave forms a grey level image, of which the intensity of the grey
380
X. He et al. / Computerized Medical Imaging and Graphics 30 (2006) 377–382
Fig. 7. An example of a position–time Graph.
Fig. 6. A cardiac ultrasound image showing a cluster of bright pixels.
level at each pixel corresponds to the acoustic impedance and the ultrasonic attenuation properties along the propagation path of the ultrasound pulse. Because the echo is strong around the area close to the interface of the blood and the ventricle wall, the ultrasound image will show a cluster of bright pixels in this area (see Fig. 6). These bright pixels are collected to form the object mask of cardiac ventricles. After the object mask is obtained, the contour of each ventricle is extracted. The moving direction of each contour pixel is also determined perpendicular to the tangent direction at this pixel. The following steps show how the PTGW is constructed. (1) Choose a set of contour pixels on the first image from sequential image frames. For example, we can choose a contour pixel at set intervals (e.g., 10) of consecutive contour pixels. We call these pixels reference pixels. (2) Record the positions of these reference pixels. Corresponding to each reference pixel, create a position–time graph (PTG). Each PTG has time as its x-axis and position as its yaxis. The position is the distance between the reference pixel and the contour pixel (target pixel) in the moving direction of the reference pixel. The value representing the position (or the distance) can be either positive or negative depending on whether the target pixel is in the positive or negative gradient direction relative to the reference pixel. The position value of the reference pixel is 0. Plot this position on the PTG. (3) Move to next image in the sequential image frames. Starting from the reference pixel location, along the moving direction of the reference pixel, search for the first contour pixel (a target pixel). Moving one pixel to another pixel along a certain direction can be easily done through a spiral addition. Compute the position value of this target pixel and plot the position on the PTG. The assumption made here for searching for the target pixel from the know reference pixel is that the movement of every boundary pixel is along its gradient direction. Therefore, the target pixel must be in the direction of the gradient of the reference pixel.
Fig. 8. An example of a direction–time Graph.
(4) Move to next image in the sequential image frames. Starting from the location of the target pixel just found, along the moving direction of this target pixel, search for the first contour pixel (a new target pixel). Compute the position value of this new target pixel and plot the position on the PTG. (5) Repeat the previous steps until no more images remain in the sequential image frames. While constructing the PTG for each reference pixel, if we also record the grey levels of the reference pixel and all the target pixels, and display the PTG as a grey picture using grey levels, then the PTG becomes a PTGW. Furthermore, if we also record the moving directions of the reference pixel and the target pixels and represent them using the six integers3 from 0 to 5, then we can construct another graph that shows the relationship between the time and moving direction. We may call this graph a direction–time graph (DTG). Figs. 7 and 8 give examples of a PTG and a DTG, respectively. As the result, three curves representing the amplitude, speed and acceleration of cardiac movement, respectively, can be easily drawn from the PTGW/PTG images obtained. Meanwhile, the DTG images show clearly the changes of boundary moving directions of cardiac ventricles (Fig. 8).
3 Note that there are only six different moving directions on Spiral Architecture as displayed in Fig. 4.
X. He et al. / Computerized Medical Imaging and Graphics 30 (2006) 377–382
Fig. 9. The original image represented in a Spiral Architecture.
381
Fig. 11. Object mask obtained from Fig. 9.
5. Experimental results We demonstrate some experimental results in this section. The original image represented in traditional square image structure is shown in Fig. 6. The dimension of the image in traditional square image structure is 760 pixels wide and 560 pixels high. The same image is represented in a mimic Spiral Architecture as shown in Fig. 9. In the mimic Spiral Architecture, we use four square pixels to form one hexagonal pixel as shown in Ref. [3]. Fig. 10 gives an illustration of such mimic scheme. Image area in Spiral Architecture, as shown in Figs. 9 and 11, contains 76 = 117649 hexagonal grids. The distribution of the hexagonal grids is shown in Fig. 3. Based on our experiments and observation, the grey value of 240 is empirically set as the threshold to form a binary image. The pixel with a grey value ≥240 is recorded as an object (interest) pixel, and the pixel whose grey value <240 is recorded as a non-object pixel. Please note that for a better estimation of the threshold to use, a learning process can be applied using a number of sample images. The object mask is shown in Fig. 11. Using the method shown in Ref. [3] for contour extraction, the contour of the object along the boundary of the endocardium is demonstrated in Fig. 12, and the corresponding interest boundary pixels are displayed in Fig. 13. Note that, we used four square pixels to mimic one single hexagonal pixel as shown in Figs. 10 and 12 shows only one square pixel for each hexago-
Fig. 10. A cluster of seven mimic hexagons.
Fig. 12. Contour of interest object close to the cardiac boundary.
nal pixel on the contour, hence we see the gaps between adjacent contour pixels. These gaps will disappear if all four square pixels are used for each hexagonal pixel on this closed contour. In Fig. 13, there are 48 interest boundary pixels in total. We have recorded that their direction chain codes are either 1 or 2, except for the first pixel whose chain code is 0. The graph showing the directions of interest pixels is in Fig. 14.
Fig. 13. The interest pixels of cardiac boundary obtained from the contour shown in Fig. 12.
382
X. He et al. / Computerized Medical Imaging and Graphics 30 (2006) 377–382
Fig. 14. Graph showing the directions of the interest pixels shown in Fig. 13.
We can obtain interest boundary pixels and their directions (chain codes) for each image frame. The moving directions of the boundary pixels can then be determined from their chain codes. The PTG and DTG can hence be easily constructed. 6. Conclusion In this paper, we have performed a ventricle boundary detection scheme based on SA using various defined chain codes. The gradient or the moving direction of each boundary pixel is determined at the same time. PTGW together with PTG and DTG images at each boundary point can be obtained at the same time through a series of spiral additions according to the directions of boundary pixels. Unlike the OMES system, our new approach no longer requires the determination of the central pixel related to the boundary (or contour). Our approach is a new method for dynamic and variable direction line setting and is not affected by the translation movement of the heart. Three curves representing the amplitude, speed and acceleration of cardiac movement can be easily drawn from the PTGW/PTG images obtained. Our approach also shows the changes of the moving directions in terms of time from the DTG. It is more efficient and accurate than OMES [2], and contains a more robust and complete description of cardiac motion. The movement of the boundary interface between the blood and the ventricular wall directly reflects the movement of the wall of the heart. This movement is caused by blood filling and the contractile and relaxation force of the cardiac muscle. Hence, the kinetic detail described using PTG/PTGW and DTG provides important haemodynamic information that can be used to distinguish healthy and diseased cardiac function. Acknowledgment This paper is partly supported by the Readership Allowance of the Faculty of Information Technology, University of Technology, Sydney. References [1] Horn BK, Schunck BG. Determining optical flow. Artif Intell 1981;17: 185–204.
[2] Lin Q, Jia W, Yang X. A method for mining data of sequential images— rebuilding of gray (position)–time function on arbitrary direction lines. In: Proceedings of the international conference on imaging science, systems, and technology (CISST’02). 2002. p. 3–6. [3] He X, Hintz T, Szewcow U. Replicated shared object model for parallel edge detection algorithm based on spiral architecture. Fut Gen Comput Syst J Elsevier 1998;14:341–50. [4] Sheridan P. Sprial architecture for machine vision. PhD thesis. Sydney: University of Technology, 1996. [5] Wu Q, He X, Hintz T. Distributed image processing on spiral architecture. In: Proceedings of the fifth international conference on algorithm and architectures for parallel processing (IEEE). 2002. p. 84–91. [6] Mokhtarian F. Silhouette-based object recognition through curvature scale space. IEEE Trans Pattern Anal Mach Intell 1995;17(5):539–44. [7] Haralick RM, Shapiro LG. Computer and robot vision. Addison-Wesley Publishing, Co.; 1992. [8] Yuen PC, Wong YY, Tong CS. Enhanced snakes algorithm for contour detection. In: Proceedings of the IEEE Southwest symposium on image analysis and interpretation. 1996. p. 54–9. [9] Iannizzotto G, Vita L. A fast, accurate method to segment and retrieve object contours in real images. In: Proceedings of the IEEE international conference on image processing. 1996. p. 841–3. [10] Nabout A, Nour Eldin HA. The topological contour closure requirement for object extraction from 2D-digital images. In: Proceedings of the IEEE international conference on systems, man and cybernetics. 1993. p. 120–5. [11] He X, Hintz T. A parallel algorithm for object contour extraction within a spiral architecture. In: Proceedings of the international conference on parallel and distributed processing techniques and applications. 1999. p. 1462–8. Xiangjian He Associate Professor, received his BSc degree from Xiamen University, and MSc degree from Fuzhou University, China in 1982 and 1985 respectively. He received his PhD degree in the area of Computer Vision from the University of Technology, Sydney (UTS) in 1999. He is the Deputy Director of the UTS university-level Computer Vision Research Group, and a Senior Member of IEEE. He has been carrying out research in the areas of computer vision and image processing based on the Spiral Architecture since 1996. Wenjing Jia received her Bachelor degree from Changchun Institute of Posts and Telecommunications, China in 1999 and her Master degree from Fuzhou University, China in 2002. She has completed her PhD study in 2006. She was an associate lecture in Fuzhou University from 1999 to 2003. Currently, she is a Research Assistant in Faculty of IT at University of Technology, Sydney. Her current research interests include image processing, pattern recognition, object detection and recognition, and color image/video retrieval. She is a member of IEEE. Dr Qiang Wu received his PhD degree in 2004 from University of Technology, Sydney (UTS). He is currently a lecturer and the Coordinator of Computer Vision and Image Processing Lab at UTS. Dr Wu has been working in the field of computer vision and image processing since 2000. His research interests include image processing, pattern recognition, spiral architecture, and radar image processing. He is a member of IEEE. Tom Hintz (Associate Professor) received his Doctor of Science degree from New Mexico State University. He has been a consultant to NASA on the first manned flight to the moon. He was a Specialist Member of the Technical Staff of Bell Laboratories. He has been Chair of the Computer Science Department at New Mexico Tech, Head of the Computer Systems Department at UTS and has been Associate Dean (Research, Policy and Planning) in the Faculty of Information Technology since 2003. His research is in parallel image processing and image compression. He is a member of IEEE.