- Email: [email protected]
3D building reconstruction from aerial CCD image and sparse laser sample data
ARTICLE IN PRESS
Optics and Lasers in Engineering 44 (2006) 555–566
3D building reconstruction from aerial CCD image and sparse laser sample data You Hongjiana,, Zhang Shiqiangb a
State Key Laboratory of Remote Sensing Science, Institute of Remote Sensing Applications, CAS, Beijing 100101, PR China b Beijing Geomatics Information Technology Ltd., 102600, PR China Received 1 January 2005; received in revised form 1 April 2005; accepted 1 June 2005 Available online 22 August 2005
Abstract An approach for 3D building reconstruction automatically based on aerial CCD image and sparse laser scanning sample points is presented in this paper. The geometry shape of a building is shown very clearly in the aerial high-resolution CCD image, so we use Laplacian sharpening operator and threshold segmentation to extract the edges of CCD image first, and then pixel connectivity is used to extract the linear features in the CCD image. Bi-direction projection histogram and line matching are proposed to extract the contours of buildings. The height of the building is determined from sparse laser sample points which are within the contour of the buildings extracted from CCD image; therefore the 3D information of each building is reconstructed. We reconstruct 3D buildings correctly by this approach using real aerial CCD and sparse laser rangefinder data. r 2005 Elsevier Ltd. All rights reserved. Keywords: CCD image; Laser scanning rangefinder; Contour extraction; 3D reconstruction
Corresponding author. Chinese Academy of Sciences (CAS), Institute of Electronics, No. 8, Research
Division, PO Box 2702, Beijing 100080, PR China. Tel.: +86 10 58887212; fax: +86 10 62535575. E-mail addresses: [email protected], [email protected] (Y. Hongjian), [email protected] (Z. Shiqiang). 0143-8166/$ - see front matter r 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.optlaseng.2005.06.004
ARTICLE IN PRESS 556
Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
1. Introduction Building is the key information of 3D city models, so extraction of buildings from remote sensing becomes an important step in order to build a digital city [1]. People propose some methods to extract buildings from remote sensing images because buildings often have a clear regular boundary, a height difference with the ground and a limited region in the remote-sensing image [2]. The traditional approach extracts buildings manually. Here operators interpret and sketch the buildings in the image, and it is both time-consuming and labor-consuming. Semiautomated extraction was proposed in the 1990s [3], but only initial positions or shapes of buildings that are given manually can buildings be extracted by approaching gradually. The accuracy of semi-automation is comparatively high if initially the position of the building is given correctly, and it is also time-consuming and labor-consuming. Automated reconstruction of buildings means extracting buildings by computer with little or no interaction and the aim is to realize this. The main data sources used to extract buildings are aerial photos, satellite highresolution images and airborne laser scanning rangefinder data. Therefore there are three types of methods to extract buildings automatically; i.e. image domain method [4], the digital surface model (DSM) domain method [5] and the combined image and DSM method [6,7]. The linear feature can be extracted automatically and effectively from the aerial image if the aerial image has high resolution, no geometry distortion and good contrast, but the method is weak in reconstructing the 3D information of buildings. The DSM used to extract buildings can be produced by stereo matching or from airborne laser scanning rangefinder data, but the result of stereo matching is not very good in urban areas. The airborne scanning laser rangefinder can acquire a high density of laser points to generate the DSM of a city [8]. The shortcoming of DSM domain extraction is that it cannot capture features like straight breaklines or ridges directly. In short, laser scanning data are 3D points, they have height information and precise position [9], but they have no geometry shape information. High-resolution image has spectral, texture and shape information. It is very beneficial to recognize a building by combining the complementary properties of laser scanning data and image, such as combining high-resolution satellite images with airborne laser scanning data to extract buildings [10], and fusion of LIDAR data and aerial imagery to automatically reconstruct the buildings [11]. An airborne laser scanning system and aerial CCD camera were developed successfully in China in the late 1990s. They can generate DSM and high-iresolution image in quasi-realtime. But the density of laser sample data is low because of the limited power of the laser and only one laser sample point per 3–4 m2 (4–5 points /m2 in other countries), while the resolution of the aerial CCD image can reach 18 cm in 1000 m height. Therefore, we proposed a new method to extract a building automatically by combining high-resolution aerial CCD image and sparse laser sample points.
ARTICLE IN PRESS Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
557
2. Data preprocessing Fig. 1 shows image captured by aerial CCD camera and Fig. 2 shows the distribution of laser sample points acquired by the airborne laser scanning system in the same area. We can see that the shape of buildings is clear in Fig. 1 and the laser sample points in Fig. 2 are sparse (the bottom is denser because of overlap of two neighboring strips), but the 3D position of each laser point can be generated by processing. Fig. 3 gives CCD image coupled with laser sample points, and there are
Fig. 1. Aerial CCD image.
Fig. 2. Airborne laser sample points.
ARTICLE IN PRESS 558
Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
Fig. 3. CCD image coupled with laser points.
Fig. 4. DSM interpolated by laser sample points.
always laser points in the roof of each building. But DSM generated based on laser sample points can only describe the surface of the city roughly due to the sparseness of laser sample points. Fig. 4 gives DSM generated by sparse laser points. We find that it is impossible to extract buildings automatically from DSM by comparing Fig. 1 with Fig. 4.The 3D buildings can be reconstructed if the contours of the buildings are extracted from the aerial CCD image and then the height of each building is calculated based on laser sample points in the roof of the buildings.
ARTICLE IN PRESS Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
559
3. Edge detecting from aerial CCD image Edge is the main feature of an image and we design an edge-detecting algorithm combining edge sharpening with the threshold segmentation. Edge sharpening can make the edge more clear and Laplacian sharpening is a good operator. We use the following Laplacian operator: 0
1
0
1
5
1
0
1
0
Fig. 5 shows the sharpened image. The edges in the sharpened image should be extracted in order to get the edges of buildings. There are two types of pixels in the sharpened image: one is edge pixel and the other is non-edge pixel. The histogram of sharpened image can show the cluster of edge pixels and non-edge pixels, therefore
Fig. 5. Laplacian-sharpened image.
ARTICLE IN PRESS 560
Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
threshold segmentation is used to segment the edge pixels based on the histogram of the sharpened image. Fig. 6 gives the segmented image and the black pixels are edge pixels. There are many non-edge pixels owing to the acute change of gray level. Some pixels are edges of buildings and some are not, like edges of roads or electrical wires and noise pixels in the segmented image, so we should extract the breaklines of buildings from the segmented image. The geometry feature of lines is that there is a connected path among pixels, so we use pixel connectivity to extract line segments. The algorithm can be expressed as follows: (1) Scanning non zero pixel in the segmented image in sequence, determining whether there is non zero pixel among current pixel’s eight neighboring pixels. Go to step (3) if there is no non zero, otherwise go to step (2). (2) The connected path is summed up if there is a non zero pixel among current pixel’s eight neighboring pixels. And take the first neighboring pixel as current pixel and continue step (1). (3) Determining the length of the connected path, only if the path is longer than the threshold is the line segment extracted, otherwise the path is not the segment of building.
Fig. 6. Threshold segmentation image of sharpened image.
ARTICLE IN PRESS Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
561
Fig. 7. Line segments extracted by pixel connectivity.
The extracted breaklines of buildings are shown in Fig. 7, as you can see that most edges of buildings are extracted correctly.
4. Building contour extraction based on bi-direction projection histogram In general, buildings have regular shapes and most of the edge lines of buildings are parallel or vertical, because they are man-made objects. So we design a method which uses bi-direction projection histogram to determine the corner points of building and then extract the contour of the building by searching and matching gradually. 4.1. Determining corner points based on bi-direction projection histogram The possible edge lines of one building are parallel or vertical in bi-level image, and the cross points of those edge lines are corner points of the building. Therefore we can determine the corner points using the cross points of vertical lines. The projection histogram in a direction shows the spatial distribution of pixels in that direction. The extremum points in two inter-vertical direction histograms can be used to extract the possible corner points of building. In Fig. 8 several extremum points appear in the X direction histogram and the two bigger extremum points are the X coordinates of the building, and a similar situation also appears in Y direction. The four corner points of the building can be extracted by combining the two directions.
ARTICLE IN PRESS 562
Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
Fig. 8. Edge of building and bi-direction projection histogram.
4.2. Contour recognition based on line matching There are many corner points extracted from bi-direction histogram and we cannot determine which corner points belong to one building, so searching and matching algorithm is used to recognize which corner points belong to the same building. During the searching process each polygon consisting of corner points is matched with the edge detection image, and the criterion is the fit ratio between polygon and the edge image [12]. We use the rectangle building as the example to demonstrate the algorithm. (1) Get a corner point in sequence from the possible corner point list of all buildings, and use this point as the top-left corner of one building. (2) Take another point from the left corner point in the list as the possible low-right corner of the building. The area of the rectangle consisting of top-left and lowright corner should be rational. (3) Use each possible low-right corner point and top-left corner point to form a rectangle and then matching the rectangle with detected edge lines. Fit ratio between the rectangle and the edge lines is used as the criterion. We think this rectangle can be used as the contour of the building if the fit ratio is bigger than the threshold. The building can be recognized correctly by this method, Figs. 9 and 10 show the recognized contour of buildings and the raw CCD image.
ARTICLE IN PRESS Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
563
Fig. 9. Extracted contour of buildings.
Fig. 10. Extracted contour of buildings coupled with image.
5. Extracting height from laser sample data Although the contour of each building can be extracted correctly, the height of the building cannot be determined from airborne CCD image. The airborne laser scanning system can give 3D coordinates of the ground laser sample points, and several laser sample points in the roof of each building can always be obtained.
ARTICLE IN PRESS 564
Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
Therefor, 3D information of building can be reconstructed by combining laser sample points in the roof of building with contour extracted from CCD. The processing steps are as follows: (1) Extract laser sample points which are in the contour of buildings. Fig. 11 shows the laser sample points in the contour of buildings and Fig. 12 shows the CCD image coupled with laser sample points in the roof of buildings.
Fig. 11. Contour of building and laser points within contour.
Fig. 12. Laser point in the roof of buildings.
ARTICLE IN PRESS Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
565
Fig. 13. 3D bird’s eye view of reconstructed buildings.
(2) 3D information of buildings can be determined by using the laser sample points in the roof to calculate the height of building. Fig. 13 gives a 3D bird’s eye view of reconstructed buildings. 6. Conclusions The method to reconstruct 3D information of buildings automatically based on airborne CCD image and laser sample points is presented in this paper. Both the character of high resolution of airborne CCD image and 3D character of laser sample points are considered in this method. The CCD images are used to extract the contours of buildings and laser sample points are used to extract the height of buildings. So the 3D information of the building can be reconstructed combining the contour and the height. The roof of building can be modeled with the density of laser sample points improved. Acknowledgment This work was sponsored by National Nature Science Foundation of China under Grant 40201035 as well as Key Laboratory of Information Engineering Surveying & Mapping and Remote Sensing under Grant wkl (02)0105. References [1] Gruen A, Wang X. CC-Modeler: a topology generator for 3D city models. ISPRS J.Photogramm. Remote Sensing 1998;53(4):286–95. [2] Baillard C. 3D reconstruction of urban scenes from aerial stereo imagery: a focus strategy. Comput. Vision Image Understanding 1999;76(3).
ARTICLE IN PRESS 566
Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
[3] Baillard C, Dissard O, Jamet O, et al. Extraction and textural characterization of above ground areas from aerial stereo pairs: a quality assessment. ISPRS J. Photogramm. Remote Sensing 1998; 53(3):130–41. [4] Kim JR, Muller J-.P. 3D reconstruction from very high resolution satellite stereo and its application to object identification. In: Proceedings of ISPRS technical Commission IV Symposium 2002 on Geospatial theory. Processing and Applications, Ottawa, Canada, July 9–12, 2002; 34(4). [5] Weidner U, Forstner W. Towards automation building extraction from high-resolution digital elevation models. ISPRS J. Photogramm. Remote Sensing 1995;50(4):38–49. [6] Haala N, Brenner C. Extraction of buildings and trees in urban environment. ISPRS J.Photogramm.Remote Sensing 1999;54(2–3). [7] Brunn A, Gulch E, Lang F, Forstner W. A hybrid concept for 3D building acquisition. ISPRS J. Photogramm.Remote Sensing 1998;53(3):119–29. [8] Maas H.-G. The potential of height texture measurement for the segmentation of airborne laserscanner data. In: Proceedings of fourth international airborne remote sensing conference and exhibition, Ottawa 21–25 June 1999,p. 154–161. [9] Masaharu H., Hasegawa H., Kamiya I. Extraction of building shapes from laser scanner data using region segmentation method. International workshop on vision-based techniques in visualization and animation, Oct.14–16, 1999, Onuma, Japan: p. 37–42. [10] Tao G., Yasuoka Y. Combining high resolution satellite imagery and airborne laser scanning data for generating bareland and DEM in urban areas. International workshop on visualization and animation of landscape, Kunming, China 26–28 February 2002; vol. XXX IV, part no. 5/W3. [11] Huber M., Schickler W., Hinz S., Baumgartner A. Fusion of LIDAR data and aerial imagery for automatic reconstruction of building surfaces, May 2003, Second joint workshop on remote sensing and data fusion over Urban areas, Berlin, Germany. [12] Ross J, Edward M. How easy is matching 2D mine models using local search. IEEE Trans.Pattern Anal.Machine Intel. 1997;19(6):564–79. You Hongjian was born in Jiangsu, PR China, on November 3, 1969. He received the BS degree from Wuhan Technical University of Surveying and Mapping in 1992, and MS from Tsinghua University in 1995 and Ph.D. degree from Chinese Academy of Sciences in 2001. Since 1995 he has been in the Institute of Remote Sensing Applications, Chinese Academy of Sciences. In 2000 he was promoted to Associate Professor. For the past several years he has been engaged in research on integration remote sensing technology, airborne remote sensing data processing and building extraction from remote sensing. Dr. You is a Member of Chinese Association for Remote Sensing.
Zhang Shiqiang was born in Shandong, PR China, on September 24, 1971. He received the BS degree from Wuhan Technical University of Surveying and Mapping in 1993. From 1993 to 1999 he has been in the Chinese Academy of Surveying and Mapping. Since 1999 work at Beijing Geomatics Information Technology Ltd. he has been engaged in research on remote sensing technology.
Optics and Lasers in Engineering 44 (2006) 555–566
3D building reconstruction from aerial CCD image and sparse laser sample data You Hongjiana,, Zhang Shiqiangb a
State Key Laboratory of Remote Sensing Science, Institute of Remote Sensing Applications, CAS, Beijing 100101, PR China b Beijing Geomatics Information Technology Ltd., 102600, PR China Received 1 January 2005; received in revised form 1 April 2005; accepted 1 June 2005 Available online 22 August 2005
Abstract An approach for 3D building reconstruction automatically based on aerial CCD image and sparse laser scanning sample points is presented in this paper. The geometry shape of a building is shown very clearly in the aerial high-resolution CCD image, so we use Laplacian sharpening operator and threshold segmentation to extract the edges of CCD image first, and then pixel connectivity is used to extract the linear features in the CCD image. Bi-direction projection histogram and line matching are proposed to extract the contours of buildings. The height of the building is determined from sparse laser sample points which are within the contour of the buildings extracted from CCD image; therefore the 3D information of each building is reconstructed. We reconstruct 3D buildings correctly by this approach using real aerial CCD and sparse laser rangefinder data. r 2005 Elsevier Ltd. All rights reserved. Keywords: CCD image; Laser scanning rangefinder; Contour extraction; 3D reconstruction
Corresponding author. Chinese Academy of Sciences (CAS), Institute of Electronics, No. 8, Research
Division, PO Box 2702, Beijing 100080, PR China. Tel.: +86 10 58887212; fax: +86 10 62535575. E-mail addresses: [email protected], [email protected] (Y. Hongjian), [email protected] (Z. Shiqiang). 0143-8166/$ - see front matter r 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.optlaseng.2005.06.004
ARTICLE IN PRESS 556
Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
1. Introduction Building is the key information of 3D city models, so extraction of buildings from remote sensing becomes an important step in order to build a digital city [1]. People propose some methods to extract buildings from remote sensing images because buildings often have a clear regular boundary, a height difference with the ground and a limited region in the remote-sensing image [2]. The traditional approach extracts buildings manually. Here operators interpret and sketch the buildings in the image, and it is both time-consuming and labor-consuming. Semiautomated extraction was proposed in the 1990s [3], but only initial positions or shapes of buildings that are given manually can buildings be extracted by approaching gradually. The accuracy of semi-automation is comparatively high if initially the position of the building is given correctly, and it is also time-consuming and labor-consuming. Automated reconstruction of buildings means extracting buildings by computer with little or no interaction and the aim is to realize this. The main data sources used to extract buildings are aerial photos, satellite highresolution images and airborne laser scanning rangefinder data. Therefore there are three types of methods to extract buildings automatically; i.e. image domain method [4], the digital surface model (DSM) domain method [5] and the combined image and DSM method [6,7]. The linear feature can be extracted automatically and effectively from the aerial image if the aerial image has high resolution, no geometry distortion and good contrast, but the method is weak in reconstructing the 3D information of buildings. The DSM used to extract buildings can be produced by stereo matching or from airborne laser scanning rangefinder data, but the result of stereo matching is not very good in urban areas. The airborne scanning laser rangefinder can acquire a high density of laser points to generate the DSM of a city [8]. The shortcoming of DSM domain extraction is that it cannot capture features like straight breaklines or ridges directly. In short, laser scanning data are 3D points, they have height information and precise position [9], but they have no geometry shape information. High-resolution image has spectral, texture and shape information. It is very beneficial to recognize a building by combining the complementary properties of laser scanning data and image, such as combining high-resolution satellite images with airborne laser scanning data to extract buildings [10], and fusion of LIDAR data and aerial imagery to automatically reconstruct the buildings [11]. An airborne laser scanning system and aerial CCD camera were developed successfully in China in the late 1990s. They can generate DSM and high-iresolution image in quasi-realtime. But the density of laser sample data is low because of the limited power of the laser and only one laser sample point per 3–4 m2 (4–5 points /m2 in other countries), while the resolution of the aerial CCD image can reach 18 cm in 1000 m height. Therefore, we proposed a new method to extract a building automatically by combining high-resolution aerial CCD image and sparse laser sample points.
ARTICLE IN PRESS Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
557
2. Data preprocessing Fig. 1 shows image captured by aerial CCD camera and Fig. 2 shows the distribution of laser sample points acquired by the airborne laser scanning system in the same area. We can see that the shape of buildings is clear in Fig. 1 and the laser sample points in Fig. 2 are sparse (the bottom is denser because of overlap of two neighboring strips), but the 3D position of each laser point can be generated by processing. Fig. 3 gives CCD image coupled with laser sample points, and there are
Fig. 1. Aerial CCD image.
Fig. 2. Airborne laser sample points.
ARTICLE IN PRESS 558
Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
Fig. 3. CCD image coupled with laser points.
Fig. 4. DSM interpolated by laser sample points.
always laser points in the roof of each building. But DSM generated based on laser sample points can only describe the surface of the city roughly due to the sparseness of laser sample points. Fig. 4 gives DSM generated by sparse laser points. We find that it is impossible to extract buildings automatically from DSM by comparing Fig. 1 with Fig. 4.The 3D buildings can be reconstructed if the contours of the buildings are extracted from the aerial CCD image and then the height of each building is calculated based on laser sample points in the roof of the buildings.
ARTICLE IN PRESS Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
559
3. Edge detecting from aerial CCD image Edge is the main feature of an image and we design an edge-detecting algorithm combining edge sharpening with the threshold segmentation. Edge sharpening can make the edge more clear and Laplacian sharpening is a good operator. We use the following Laplacian operator: 0
1
0
1
5
1
0
1
0
Fig. 5 shows the sharpened image. The edges in the sharpened image should be extracted in order to get the edges of buildings. There are two types of pixels in the sharpened image: one is edge pixel and the other is non-edge pixel. The histogram of sharpened image can show the cluster of edge pixels and non-edge pixels, therefore
Fig. 5. Laplacian-sharpened image.
ARTICLE IN PRESS 560
Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
threshold segmentation is used to segment the edge pixels based on the histogram of the sharpened image. Fig. 6 gives the segmented image and the black pixels are edge pixels. There are many non-edge pixels owing to the acute change of gray level. Some pixels are edges of buildings and some are not, like edges of roads or electrical wires and noise pixels in the segmented image, so we should extract the breaklines of buildings from the segmented image. The geometry feature of lines is that there is a connected path among pixels, so we use pixel connectivity to extract line segments. The algorithm can be expressed as follows: (1) Scanning non zero pixel in the segmented image in sequence, determining whether there is non zero pixel among current pixel’s eight neighboring pixels. Go to step (3) if there is no non zero, otherwise go to step (2). (2) The connected path is summed up if there is a non zero pixel among current pixel’s eight neighboring pixels. And take the first neighboring pixel as current pixel and continue step (1). (3) Determining the length of the connected path, only if the path is longer than the threshold is the line segment extracted, otherwise the path is not the segment of building.
Fig. 6. Threshold segmentation image of sharpened image.
ARTICLE IN PRESS Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
561
Fig. 7. Line segments extracted by pixel connectivity.
The extracted breaklines of buildings are shown in Fig. 7, as you can see that most edges of buildings are extracted correctly.
4. Building contour extraction based on bi-direction projection histogram In general, buildings have regular shapes and most of the edge lines of buildings are parallel or vertical, because they are man-made objects. So we design a method which uses bi-direction projection histogram to determine the corner points of building and then extract the contour of the building by searching and matching gradually. 4.1. Determining corner points based on bi-direction projection histogram The possible edge lines of one building are parallel or vertical in bi-level image, and the cross points of those edge lines are corner points of the building. Therefore we can determine the corner points using the cross points of vertical lines. The projection histogram in a direction shows the spatial distribution of pixels in that direction. The extremum points in two inter-vertical direction histograms can be used to extract the possible corner points of building. In Fig. 8 several extremum points appear in the X direction histogram and the two bigger extremum points are the X coordinates of the building, and a similar situation also appears in Y direction. The four corner points of the building can be extracted by combining the two directions.
ARTICLE IN PRESS 562
Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
Fig. 8. Edge of building and bi-direction projection histogram.
4.2. Contour recognition based on line matching There are many corner points extracted from bi-direction histogram and we cannot determine which corner points belong to one building, so searching and matching algorithm is used to recognize which corner points belong to the same building. During the searching process each polygon consisting of corner points is matched with the edge detection image, and the criterion is the fit ratio between polygon and the edge image [12]. We use the rectangle building as the example to demonstrate the algorithm. (1) Get a corner point in sequence from the possible corner point list of all buildings, and use this point as the top-left corner of one building. (2) Take another point from the left corner point in the list as the possible low-right corner of the building. The area of the rectangle consisting of top-left and lowright corner should be rational. (3) Use each possible low-right corner point and top-left corner point to form a rectangle and then matching the rectangle with detected edge lines. Fit ratio between the rectangle and the edge lines is used as the criterion. We think this rectangle can be used as the contour of the building if the fit ratio is bigger than the threshold. The building can be recognized correctly by this method, Figs. 9 and 10 show the recognized contour of buildings and the raw CCD image.
ARTICLE IN PRESS Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
563
Fig. 9. Extracted contour of buildings.
Fig. 10. Extracted contour of buildings coupled with image.
5. Extracting height from laser sample data Although the contour of each building can be extracted correctly, the height of the building cannot be determined from airborne CCD image. The airborne laser scanning system can give 3D coordinates of the ground laser sample points, and several laser sample points in the roof of each building can always be obtained.
ARTICLE IN PRESS 564
Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
Therefor, 3D information of building can be reconstructed by combining laser sample points in the roof of building with contour extracted from CCD. The processing steps are as follows: (1) Extract laser sample points which are in the contour of buildings. Fig. 11 shows the laser sample points in the contour of buildings and Fig. 12 shows the CCD image coupled with laser sample points in the roof of buildings.
Fig. 11. Contour of building and laser points within contour.
Fig. 12. Laser point in the roof of buildings.
ARTICLE IN PRESS Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
565
Fig. 13. 3D bird’s eye view of reconstructed buildings.
(2) 3D information of buildings can be determined by using the laser sample points in the roof to calculate the height of building. Fig. 13 gives a 3D bird’s eye view of reconstructed buildings. 6. Conclusions The method to reconstruct 3D information of buildings automatically based on airborne CCD image and laser sample points is presented in this paper. Both the character of high resolution of airborne CCD image and 3D character of laser sample points are considered in this method. The CCD images are used to extract the contours of buildings and laser sample points are used to extract the height of buildings. So the 3D information of the building can be reconstructed combining the contour and the height. The roof of building can be modeled with the density of laser sample points improved. Acknowledgment This work was sponsored by National Nature Science Foundation of China under Grant 40201035 as well as Key Laboratory of Information Engineering Surveying & Mapping and Remote Sensing under Grant wkl (02)0105. References [1] Gruen A, Wang X. CC-Modeler: a topology generator for 3D city models. ISPRS J.Photogramm. Remote Sensing 1998;53(4):286–95. [2] Baillard C. 3D reconstruction of urban scenes from aerial stereo imagery: a focus strategy. Comput. Vision Image Understanding 1999;76(3).
ARTICLE IN PRESS 566
Y. Hongjian, Z. Shiqiang / Optics and Lasers in Engineering 44 (2006) 555–566
[3] Baillard C, Dissard O, Jamet O, et al. Extraction and textural characterization of above ground areas from aerial stereo pairs: a quality assessment. ISPRS J. Photogramm. Remote Sensing 1998; 53(3):130–41. [4] Kim JR, Muller J-.P. 3D reconstruction from very high resolution satellite stereo and its application to object identification. In: Proceedings of ISPRS technical Commission IV Symposium 2002 on Geospatial theory. Processing and Applications, Ottawa, Canada, July 9–12, 2002; 34(4). [5] Weidner U, Forstner W. Towards automation building extraction from high-resolution digital elevation models. ISPRS J. Photogramm. Remote Sensing 1995;50(4):38–49. [6] Haala N, Brenner C. Extraction of buildings and trees in urban environment. ISPRS J.Photogramm.Remote Sensing 1999;54(2–3). [7] Brunn A, Gulch E, Lang F, Forstner W. A hybrid concept for 3D building acquisition. ISPRS J. Photogramm.Remote Sensing 1998;53(3):119–29. [8] Maas H.-G. The potential of height texture measurement for the segmentation of airborne laserscanner data. In: Proceedings of fourth international airborne remote sensing conference and exhibition, Ottawa 21–25 June 1999,p. 154–161. [9] Masaharu H., Hasegawa H., Kamiya I. Extraction of building shapes from laser scanner data using region segmentation method. International workshop on vision-based techniques in visualization and animation, Oct.14–16, 1999, Onuma, Japan: p. 37–42. [10] Tao G., Yasuoka Y. Combining high resolution satellite imagery and airborne laser scanning data for generating bareland and DEM in urban areas. International workshop on visualization and animation of landscape, Kunming, China 26–28 February 2002; vol. XXX IV, part no. 5/W3. [11] Huber M., Schickler W., Hinz S., Baumgartner A. Fusion of LIDAR data and aerial imagery for automatic reconstruction of building surfaces, May 2003, Second joint workshop on remote sensing and data fusion over Urban areas, Berlin, Germany. [12] Ross J, Edward M. How easy is matching 2D mine models using local search. IEEE Trans.Pattern Anal.Machine Intel. 1997;19(6):564–79. You Hongjian was born in Jiangsu, PR China, on November 3, 1969. He received the BS degree from Wuhan Technical University of Surveying and Mapping in 1992, and MS from Tsinghua University in 1995 and Ph.D. degree from Chinese Academy of Sciences in 2001. Since 1995 he has been in the Institute of Remote Sensing Applications, Chinese Academy of Sciences. In 2000 he was promoted to Associate Professor. For the past several years he has been engaged in research on integration remote sensing technology, airborne remote sensing data processing and building extraction from remote sensing. Dr. You is a Member of Chinese Association for Remote Sensing.
Zhang Shiqiang was born in Shandong, PR China, on September 24, 1971. He received the BS degree from Wuhan Technical University of Surveying and Mapping in 1993. From 1993 to 1999 he has been in the Chinese Academy of Surveying and Mapping. Since 1999 work at Beijing Geomatics Information Technology Ltd. he has been engaged in research on remote sensing technology.