- Email: [email protected]
An improved method for generating virtual stretched view of stomach based on shape deformation
International Congress Series 1230 (2001) 447 – 453
An improved method for generating virtual stretched view of stomach based on shape deformation K. Moria,*, Y. Hoshinoa, Y. Suenagaa, J. Toriwakia, J. Hasegawab, K. Katadac a
Department of Computational Science and Engineering, Graduate School of Engineering, Nagoya University, Furo-cho, Chikusa-ku, Nagoya 464-8603, Japan b School of Computer and Cognitive Sciences, Chukyo University, 101, Kaizu-cho, Tokodachi, Toyota, Aichi 470-0393, Japan c School of Health Sciences, Fujita Health University, Dengakugakubo, Kutsukake, Toyoake, Aichi 470-1192, Japan
Abstract In this paper, we describe an improved method for generating a virtual stretched image of the stomach based on shape deformation. In observing large cavity organs, such as the stomach, a flattened view of the organs would be very useful in diagnosis. The proposed method generates a virtual stretched image by deforming a 3-D gray image. We generate a polygon model consisting of a set of triangular patches that approximates the global shape of the stomach (approximating shape). An original 3-D image is deformed by using the corresponding relationship between the cut and flattened shapes of the approximating shape. We can obtain a flattened view (virtual pathological specimen) from the deformed image. The proposed method was applied to eight cases of 3-D abdominal CT images. Experimental results showed that the proposed method can effectively generate a flattened view of the organ. D 2001 Elsevier Science B.V. All rights reserved. Keywords: Virtual pathology; Virtual stretched view; Unraveling; Stomach
1. Introduction The virtual endoscopy method (VE) is widely used for observing inside pipestructured organs [1,2]. The user of VE system (VES) can generate endoscopic views
*
Corresponding author. Tel.: +81-52-789-3310; fax: +81-52-789-3807. E-mail address: [email protected] (K. Mori).
0531-5131/01/$ – see front matter D 2001 Elsevier Science B.V. All rights reserved. PII: S 0 5 3 1 - 5 1 3 1 ( 0 1 ) 0 0 0 9 8 - X
448
K. Mori et al. / International Congress Series 1230 (2001) 447–453
with freely changing viewpoints and view directions. It is also possible to fly through the inside of the organ. VES is applicable to diagnosis, surgical planning, medical education, and informed consent. When we observe inside an organ that has large cavity, it is often necessary to change viewpoints and view directions to view the entire organ. This is a time-consuming task. In the pathological field, medical doctors usually make stretched specimens for diagnosing the details of diseases. In the case of the stomach, doctors measure the location of a stomach cancer on the stretched specimen after resection. If we could generate virtually stretched views of the organ, especially the stomach, this technique would be very useful for diagnose with 3-D X-ray CT images. The medical doctor could observe the entire organ by only one view. Many changes of the viewpoint and the view direction would not be necessary if we could observe a stretched view. In abnormal cases, the doctor could perform some quantitative measurements on the stretched views, such as measurements of the location and the size of a stomach cancer. We have already reported the development of a preliminary system for virtual stretching of the stomach [4]. In the previous method, we represented an approximated shape of the stomach by using B-spline curves. The approximated shape is indispensable for generating a virtual stretched view. However, it was a very time-consuming task to input B-spline curves manually. Also, in the previous method, we could not freely specify cutting lines on the stomach freely. The purpose of this paper is to present an improved method for generating virtual stretched views of the stomach from 3-D abdominal X-ray CT images. The method proposed here solves the above problems by generating an approximated shape directly from a stomach region extracted from a 3-D X-ray CT image. Morphological operations and decimation techniques of triangle meshes are employed here to generate the approximated shape. Some research groups have also reported methods for generating virtual pathological specimens from 3-D X-ray CT images [3]. The difference between their methods and our method is that our method directly cuts and stretches the organ. Methods of other research groups involve simple deformation of 3-D X-ray CT images. Their method is most often applied to flattening of the colon regions. In Section 2, we give an overview of the proposed method and the details of the processing procedures. Experimental results are shown in Section 3.
2. Method 2.1. Overview The method proposed here generates a virtual stretched image by deforming the neighboring regions of the organ’s wall into stretched regions. This process is performed by deforming original CT images. The deformation rule for CT images is determined by the relationship between the original shape of the stomach and its stretched shape. We use an approximated shape of the stomach where fold patterns are deleted in this cutting and stretching process. The entire procedure consists of four parts: (a) generation of the approximated shape of the stomach, (b) cutting and
K. Mori et al. / International Congress Series 1230 (2001) 447–453
449
stretching process, (c) deformation of the original X-ray CT image, and (d) re-extraction of inner wall (Fig. 1). 2.2. Generation of the approximated shape This process generates an approximated shape of the stomach where fold regions are deleted. If fold regions remain in the approximated shape, fold patterns are also stretched into a plane shape. First, we shrink an original X-ray CT image by using mathematical morphology operations (opening and closing operations). A stomach region is extracted from the filtered image. Triangular patches, which represent the stomach shape, are generated from the extracted region by employing the Marching Cubes method. A decimation technique, which reduces the number of triangular patches, is applied to the extracted patches. We modified the method described in Turk for this decimation. This method repeats the process of deleting the vertex that has the smallest feature value and then reconstructs the triangular patches. The area of a triangle and the length of an edge are added to the original method [5] as feature values for deletion. We call this set of decimated patches the ‘approximated shape’. 2.3. Cutting and stretching of the approximated shape 2.3.1. Specification of cutting line The approximated shape initially has a closed tube-like shape. We input a cutting line on the approximated shape by using a mouse (usually along the greater curvature of the stomach). It is possible to input a cutting line that avoids a cancer region. We also delete patches at each end of the stomach.
Fig. 1. Processing flow of the proposed method.
450
K. Mori et al. / International Congress Series 1230 (2001) 447–453
Fig. 2. Nodes and springs for node i.
2.3.2. Elastic modeling The method generates an elastic model of the approximated shape by converting it into the ‘node and spring model’. This elastic model is used in preserving the shape of each triangular patch during the stretching process and minimizing distortions of the stretched view. We allocate nodes on vertices and springs on the edges of triangular patches of the approximated shape. Additional springs are also allocated between a vertex and the center point of gravity of a triangular patch. The spring constant of the former type is k1 and that of the latter type is k2. 2.3.3. Stretching of the approximated shape The stretching is executed by moving nodes on the cutting line while adding the force Foi(n) to the nodes. The displacement caused by this movement is very small. We iterate this process during the stretching process. The force Fi(n) working on the node i at the n-th iteration step is caused by all springs connected to the node i (Fig. 2) and given by the following equation X X Fnij ðnÞ þ Fgik ðnÞ þ Foi ðnÞ; ð1Þ i ðnÞ ¼ F j2Ni k2Gi where Fnij(n) means the force working on the node i by the spring connecting the node i and j, Fgik(n) the force working on the node i by the spring connecting the node i and the gravity point k, and Foi(n) the force which we add to the node located on the cutting line for stretching. Fnij(n), Fgik(n), and Foi(n) are expressed as follows. Fnij ðnÞ ¼ k1 ðkRij ðnÞ lij kÞ
Rij ðnÞ kRij ðnÞk
Fgik ðnÞ ¼ k2 ðkRik ðnÞ lik kÞ Foi ðnÞ ¼ aTi ;
Rik ðnÞ kRik ðnÞk
ð2Þ
ð3Þ ð4Þ
K. Mori et al. / International Congress Series 1230 (2001) 447–453
451
Fig. 3. Stretching approximated shape onto stretching plane.
where lij means the natural length of the spring connecting nodes i and j and Rij(n) is the vector directed from node i to node j. T is computed as illustrated in Fig. 3. The displacement of each node is calculated as Ri ðn þ 1Þ ¼ Ri ðnÞ þ cFi ðnÞ:
ð5Þ
The method iterates this deformation process and then projects all points onto the stretching plane. After that, we execute this process without using the force Foi(n) to obtain the final stretched results of the approximated shape. 2.4. Deformation of original image We reconstruct the original CT image where the stomach region is stretched. This deformation is achieved by re-sampling the original image around the inner wall by using the relationship between the original approximated shape and its stretched shape. The detailed procedure is described in Ref. [4]. 2.5. Re-extraction of the inner wall The inner wall of the stomach is re-extracted from the reconstructed CT image by thresholding. We apply the Marching Cubes method to the extracted regions and obtain a set of triangular patches. The virtual stretched views are obtained by rendering the triangular patches. The volume rendering method is also applicable to directly render the reconstructed image.
3. Experimental results We have applied the proposed method to eight cases of 3-D abdominal X-ray CT images. Cutting lines were specified along the greater curvatures of the stomach.
452
K. Mori et al. / International Congress Series 1230 (2001) 447–453
Specifications of the images are: image size 512 512 pixels, number of slices: 150 –489 slices, X-ray beam width: 1 mm, reconstruction pitch: 2.0– 5.0 mm. Experimental results
Fig. 4. Experimental results. (a, c, e, g) Outside views of stomachs. (b, d, f, h) Stretched views.
K. Mori et al. / International Congress Series 1230 (2001) 447–453
453
showed that the proposed method could generate the stretched views of the stomach for all cases (Fig. 4). Fold regions were effectively reconstructed on stretched views satisfactory. Total processing time was much shorter than that of the previous methods. Furthermore, the user could intuitively specify cutting lines with using the mouse. These improvements were achieved by the methods introduced in this paper. We evaluated distortions of the stretched views by measuring the area of triangle patches before and after stretching. The results showed that the area of each patch was preserved in the stretchingprocess.
4. Conclusions This paper presented an improved method for generating virtual stretched images of the stomach based on deformation of the 3-D X-ray CT images. Eight cases of 3-D abdominal CT images were used to evaluate the performance of the proposed method. Experimental results showed that the improved method enabled us to generate stretched views easily and accurately. Direct cutting and stretching based on deformation of the triangle patch model greatly contributed to these improvements. Future work includes (a) evaluation of the method by using actual specimens of the stomach and (b) simulation of the elastic deformation of the actual stomach.
Acknowledgements The authors thank Dr. Shigenru Nawano, National Cancer Hospital East, for providing CT images and suggestions, and our colleagues for their useful suggestion and discussion. Parts of this research were supported by the Grant-In-Aid for Scientific Research from the Ministry of Education, the Grant-In-Aid for Scientific Research from Japan Society for Promotion of Science, and the Grant-In-Aid for Cancer Research from the Ministry Health and Welfare, of Japanese Government.
References [1] D.J. Vining, R.Y. Shitrin, E.F. Haponik, et al., Virtual bronchoscopy, Radiology 193 (P) (1994) 261, Supplement to Radiology (RSNA Scientific Program). [2] K. Mori, J. Hasegawa, J. Toriwaki, et al., Automated extraction and visualization of bronchus from 3-D CT images of lung, Proc. of CVRMed95 (Lecture Notes in Computer Science) 905 (1995) 542 – 548. [3] Ge. Wang, M.W. Vannier, E.G. MacFarland, et al., GI tract unraveling Volumetric CT, Proc. of VBC96 (Lecture Notes in Computer Science) 1131 (1996) 3 – 12. [4] K. Mori, A. Kushida, T. Saito, K. Katada, et al., A method for generating virtually stretched image of stomach using 3-D abdominal CT image, Proc. of CAR’98, 1998, pp. 112 – 117. [5] G. Turk, Re-tiling polygonal surfaces, Proc. of SIGGRAPH’92, Computer Graphics 26 (2) (1992) 55 – 64.
An improved method for generating virtual stretched view of stomach based on shape deformation K. Moria,*, Y. Hoshinoa, Y. Suenagaa, J. Toriwakia, J. Hasegawab, K. Katadac a
Department of Computational Science and Engineering, Graduate School of Engineering, Nagoya University, Furo-cho, Chikusa-ku, Nagoya 464-8603, Japan b School of Computer and Cognitive Sciences, Chukyo University, 101, Kaizu-cho, Tokodachi, Toyota, Aichi 470-0393, Japan c School of Health Sciences, Fujita Health University, Dengakugakubo, Kutsukake, Toyoake, Aichi 470-1192, Japan
Abstract In this paper, we describe an improved method for generating a virtual stretched image of the stomach based on shape deformation. In observing large cavity organs, such as the stomach, a flattened view of the organs would be very useful in diagnosis. The proposed method generates a virtual stretched image by deforming a 3-D gray image. We generate a polygon model consisting of a set of triangular patches that approximates the global shape of the stomach (approximating shape). An original 3-D image is deformed by using the corresponding relationship between the cut and flattened shapes of the approximating shape. We can obtain a flattened view (virtual pathological specimen) from the deformed image. The proposed method was applied to eight cases of 3-D abdominal CT images. Experimental results showed that the proposed method can effectively generate a flattened view of the organ. D 2001 Elsevier Science B.V. All rights reserved. Keywords: Virtual pathology; Virtual stretched view; Unraveling; Stomach
1. Introduction The virtual endoscopy method (VE) is widely used for observing inside pipestructured organs [1,2]. The user of VE system (VES) can generate endoscopic views
*
Corresponding author. Tel.: +81-52-789-3310; fax: +81-52-789-3807. E-mail address: [email protected] (K. Mori).
0531-5131/01/$ – see front matter D 2001 Elsevier Science B.V. All rights reserved. PII: S 0 5 3 1 - 5 1 3 1 ( 0 1 ) 0 0 0 9 8 - X
448
K. Mori et al. / International Congress Series 1230 (2001) 447–453
with freely changing viewpoints and view directions. It is also possible to fly through the inside of the organ. VES is applicable to diagnosis, surgical planning, medical education, and informed consent. When we observe inside an organ that has large cavity, it is often necessary to change viewpoints and view directions to view the entire organ. This is a time-consuming task. In the pathological field, medical doctors usually make stretched specimens for diagnosing the details of diseases. In the case of the stomach, doctors measure the location of a stomach cancer on the stretched specimen after resection. If we could generate virtually stretched views of the organ, especially the stomach, this technique would be very useful for diagnose with 3-D X-ray CT images. The medical doctor could observe the entire organ by only one view. Many changes of the viewpoint and the view direction would not be necessary if we could observe a stretched view. In abnormal cases, the doctor could perform some quantitative measurements on the stretched views, such as measurements of the location and the size of a stomach cancer. We have already reported the development of a preliminary system for virtual stretching of the stomach [4]. In the previous method, we represented an approximated shape of the stomach by using B-spline curves. The approximated shape is indispensable for generating a virtual stretched view. However, it was a very time-consuming task to input B-spline curves manually. Also, in the previous method, we could not freely specify cutting lines on the stomach freely. The purpose of this paper is to present an improved method for generating virtual stretched views of the stomach from 3-D abdominal X-ray CT images. The method proposed here solves the above problems by generating an approximated shape directly from a stomach region extracted from a 3-D X-ray CT image. Morphological operations and decimation techniques of triangle meshes are employed here to generate the approximated shape. Some research groups have also reported methods for generating virtual pathological specimens from 3-D X-ray CT images [3]. The difference between their methods and our method is that our method directly cuts and stretches the organ. Methods of other research groups involve simple deformation of 3-D X-ray CT images. Their method is most often applied to flattening of the colon regions. In Section 2, we give an overview of the proposed method and the details of the processing procedures. Experimental results are shown in Section 3.
2. Method 2.1. Overview The method proposed here generates a virtual stretched image by deforming the neighboring regions of the organ’s wall into stretched regions. This process is performed by deforming original CT images. The deformation rule for CT images is determined by the relationship between the original shape of the stomach and its stretched shape. We use an approximated shape of the stomach where fold patterns are deleted in this cutting and stretching process. The entire procedure consists of four parts: (a) generation of the approximated shape of the stomach, (b) cutting and
K. Mori et al. / International Congress Series 1230 (2001) 447–453
449
stretching process, (c) deformation of the original X-ray CT image, and (d) re-extraction of inner wall (Fig. 1). 2.2. Generation of the approximated shape This process generates an approximated shape of the stomach where fold regions are deleted. If fold regions remain in the approximated shape, fold patterns are also stretched into a plane shape. First, we shrink an original X-ray CT image by using mathematical morphology operations (opening and closing operations). A stomach region is extracted from the filtered image. Triangular patches, which represent the stomach shape, are generated from the extracted region by employing the Marching Cubes method. A decimation technique, which reduces the number of triangular patches, is applied to the extracted patches. We modified the method described in Turk for this decimation. This method repeats the process of deleting the vertex that has the smallest feature value and then reconstructs the triangular patches. The area of a triangle and the length of an edge are added to the original method [5] as feature values for deletion. We call this set of decimated patches the ‘approximated shape’. 2.3. Cutting and stretching of the approximated shape 2.3.1. Specification of cutting line The approximated shape initially has a closed tube-like shape. We input a cutting line on the approximated shape by using a mouse (usually along the greater curvature of the stomach). It is possible to input a cutting line that avoids a cancer region. We also delete patches at each end of the stomach.
Fig. 1. Processing flow of the proposed method.
450
K. Mori et al. / International Congress Series 1230 (2001) 447–453
Fig. 2. Nodes and springs for node i.
2.3.2. Elastic modeling The method generates an elastic model of the approximated shape by converting it into the ‘node and spring model’. This elastic model is used in preserving the shape of each triangular patch during the stretching process and minimizing distortions of the stretched view. We allocate nodes on vertices and springs on the edges of triangular patches of the approximated shape. Additional springs are also allocated between a vertex and the center point of gravity of a triangular patch. The spring constant of the former type is k1 and that of the latter type is k2. 2.3.3. Stretching of the approximated shape The stretching is executed by moving nodes on the cutting line while adding the force Foi(n) to the nodes. The displacement caused by this movement is very small. We iterate this process during the stretching process. The force Fi(n) working on the node i at the n-th iteration step is caused by all springs connected to the node i (Fig. 2) and given by the following equation X X Fnij ðnÞ þ Fgik ðnÞ þ Foi ðnÞ; ð1Þ i ðnÞ ¼ F j2Ni k2Gi where Fnij(n) means the force working on the node i by the spring connecting the node i and j, Fgik(n) the force working on the node i by the spring connecting the node i and the gravity point k, and Foi(n) the force which we add to the node located on the cutting line for stretching. Fnij(n), Fgik(n), and Foi(n) are expressed as follows. Fnij ðnÞ ¼ k1 ðkRij ðnÞ lij kÞ
Rij ðnÞ kRij ðnÞk
Fgik ðnÞ ¼ k2 ðkRik ðnÞ lik kÞ Foi ðnÞ ¼ aTi ;
Rik ðnÞ kRik ðnÞk
ð2Þ
ð3Þ ð4Þ
K. Mori et al. / International Congress Series 1230 (2001) 447–453
451
Fig. 3. Stretching approximated shape onto stretching plane.
where lij means the natural length of the spring connecting nodes i and j and Rij(n) is the vector directed from node i to node j. T is computed as illustrated in Fig. 3. The displacement of each node is calculated as Ri ðn þ 1Þ ¼ Ri ðnÞ þ cFi ðnÞ:
ð5Þ
The method iterates this deformation process and then projects all points onto the stretching plane. After that, we execute this process without using the force Foi(n) to obtain the final stretched results of the approximated shape. 2.4. Deformation of original image We reconstruct the original CT image where the stomach region is stretched. This deformation is achieved by re-sampling the original image around the inner wall by using the relationship between the original approximated shape and its stretched shape. The detailed procedure is described in Ref. [4]. 2.5. Re-extraction of the inner wall The inner wall of the stomach is re-extracted from the reconstructed CT image by thresholding. We apply the Marching Cubes method to the extracted regions and obtain a set of triangular patches. The virtual stretched views are obtained by rendering the triangular patches. The volume rendering method is also applicable to directly render the reconstructed image.
3. Experimental results We have applied the proposed method to eight cases of 3-D abdominal X-ray CT images. Cutting lines were specified along the greater curvatures of the stomach.
452
K. Mori et al. / International Congress Series 1230 (2001) 447–453
Specifications of the images are: image size 512 512 pixels, number of slices: 150 –489 slices, X-ray beam width: 1 mm, reconstruction pitch: 2.0– 5.0 mm. Experimental results
Fig. 4. Experimental results. (a, c, e, g) Outside views of stomachs. (b, d, f, h) Stretched views.
K. Mori et al. / International Congress Series 1230 (2001) 447–453
453
showed that the proposed method could generate the stretched views of the stomach for all cases (Fig. 4). Fold regions were effectively reconstructed on stretched views satisfactory. Total processing time was much shorter than that of the previous methods. Furthermore, the user could intuitively specify cutting lines with using the mouse. These improvements were achieved by the methods introduced in this paper. We evaluated distortions of the stretched views by measuring the area of triangle patches before and after stretching. The results showed that the area of each patch was preserved in the stretchingprocess.
4. Conclusions This paper presented an improved method for generating virtual stretched images of the stomach based on deformation of the 3-D X-ray CT images. Eight cases of 3-D abdominal CT images were used to evaluate the performance of the proposed method. Experimental results showed that the improved method enabled us to generate stretched views easily and accurately. Direct cutting and stretching based on deformation of the triangle patch model greatly contributed to these improvements. Future work includes (a) evaluation of the method by using actual specimens of the stomach and (b) simulation of the elastic deformation of the actual stomach.
Acknowledgements The authors thank Dr. Shigenru Nawano, National Cancer Hospital East, for providing CT images and suggestions, and our colleagues for their useful suggestion and discussion. Parts of this research were supported by the Grant-In-Aid for Scientific Research from the Ministry of Education, the Grant-In-Aid for Scientific Research from Japan Society for Promotion of Science, and the Grant-In-Aid for Cancer Research from the Ministry Health and Welfare, of Japanese Government.
References [1] D.J. Vining, R.Y. Shitrin, E.F. Haponik, et al., Virtual bronchoscopy, Radiology 193 (P) (1994) 261, Supplement to Radiology (RSNA Scientific Program). [2] K. Mori, J. Hasegawa, J. Toriwaki, et al., Automated extraction and visualization of bronchus from 3-D CT images of lung, Proc. of CVRMed95 (Lecture Notes in Computer Science) 905 (1995) 542 – 548. [3] Ge. Wang, M.W. Vannier, E.G. MacFarland, et al., GI tract unraveling Volumetric CT, Proc. of VBC96 (Lecture Notes in Computer Science) 1131 (1996) 3 – 12. [4] K. Mori, A. Kushida, T. Saito, K. Katada, et al., A method for generating virtually stretched image of stomach using 3-D abdominal CT image, Proc. of CAR’98, 1998, pp. 112 – 117. [5] G. Turk, Re-tiling polygonal surfaces, Proc. of SIGGRAPH’92, Computer Graphics 26 (2) (1992) 55 – 64.