- Email: [email protected]
Analytical reconstruction of three-dimensional weld pool surface in GTAW
Journal of Manufacturing Processes 15 (2013) 34–40
Contents lists available at SciVerse ScienceDirect
Journal of Manufacturing Processes journal homepage: www.elsevier.com/locate/manpro
Technical Paper
Analytical reconstruction of three-dimensional weld pool surface in GTAW Zhenzhou Wang a , YuMing Zhang a,∗ , Ruigang Yang b a b
Department of Electrical and Computer Engineering and Institute for Sustainable Manufacturing, University of Kentucky, Lexington, KY 40506, USA Department of Computer Science, University of Kentucky, Lexington, KY 40506, USA
a r t i c l e
i n f o
Article history: Received 24 August 2011 Received in revised form 9 August 2012 Accepted 9 August 2012 Available online 2 September 2012 Keywords: Mirror surface Specular surface Welding GTAW Laser Machine vision Reflection
a b s t r a c t The reflective characteristic of mirror surfaces such as a liquid pool surface in arc welding makes many traditional 3D measurement/reconstruction methods fail. The authors proposed to intercept, image, and measure two points in each laser ray reflected from a mirror surface with two diffusive planes and cameras to analytically calculate the equation of the ray. The samples of the points on the mirror surface where the incident laser rays are reflected can thus be analytically calculated as the intersections of the reflection rays with the corresponding incident rays. In this paper, the proposed method is applied to reconstruct the samples on the specular three-dimensional weld pool surface in GTAW (gas tungsten arc welding). Since two diffusive planes are used and must be placed with considerable distance to assure the accuracy of the calculated equations of the reflected rays, focusing reflected laser rays on these two planes becomes an issue. A trade-off among the size of the projected laser pattern, the distances of the arc light with the two diffusive planes, the focus range of the laser rays and the quality of the reflected laser dot images on the diffusive planes has been made to resolve this issue successfully. Further, calibration errors in the locations of the diffusive planes directly affect the accuracy of the calculated equations of reflected rays and an accurate calibration appears to impractical. To resolve this issue, the authors found the least deformation principle and successfully applied it to minimize the calculation errors through calibration rectification. Several weld pool surfaces have been sampled and reconstructed and experimental results verified the effectiveness of the proposed analytical method. © 2012 The Society of Manufacturing Engineers. Published by Elsevier Ltd. All rights reserved.
1. Introduction Many traditional 3D reconstruction methods will avoid reconstructing mirror surfaces because the reflection from mirror surfaces is view-angle dependent. It causes stereo vision methods to fail in finding correct correspondences between two images and also adds the ambiguity of normal and depth to the traditional structured light and phase shift methods [1]. In the case of sensing a weld pool surface [2–7], it is also interfered by the harsh welding environment. For instance, the work piece is melted by the arc and the bright arc light makes direct sensing of a weld pool surface difficult. To overcome these difficulties, laser patterns have been projected onto the weld pool surface and the reflections of the laser patterns from the weld pool surface are imaged on a diffusive plane. The images of reflected laser patterns on the diffusive plane are captured by a CCD camera and processed with image processing algorithms to calculate the distortion of laser patterns. From the distortion of the reflected laser patterns, the surface of the weld pool can be calculated [4–7]. The
∗ Corresponding author. E-mail address: [email protected] (Y. Zhang).
drawback of this method is that the reconstructed 3D surface is obtained by matching the calculated reflection pattern with the imaged reflection pattern and there is an iterative optimization process needed in order to achieve an acceptable matching. To avoid the iteration, direct analytical solutions are needed for sampled points on the mirror surface reflecting the projected laser pattern. Another method has thus been proposed in a previous study [8] that uses two diffusive screens to capture the reflected pattern twice such that the equation for the linear trajectory of each reflected ray can be analytically computed. With known incident rays’ equations and their corresponding reflected rays’ equations, their intersection points can be directly computed, therefore obtaining the sampled points of the 3D mirror surface where the incident rays are intercepted and reflected. It thus proposes a method that may be advantageous over the previous iteration based algorithm [7] in computation speed.
2. Principle Fig. 1 illustrates the principle of the previously proposed method for direction analytical solutions of the reflection rays [8]. As shown in Fig. 1, there are three cameras c1, c2 and c3 which aim at planes p1, p2 and p3, respectively. The origin of the world co-ordinate is
1526-6125/$ – see front matter © 2012 The Society of Manufacturing Engineers. Published by Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.jmapro.2012.08.002
Z. Wang et al. / Journal of Manufacturing Processes 15 (2013) 34–40
35
Fig. 1. System diagram.
defined at the point O. The equation of the plane p1 which is on the moving screen is defined as z = 0. The laser rays are projected onto the surface of the plane p1 from the projection center C. When the mirror reflected laser ray reaches the beam splitter, half power of it transmits through the splitter and hits on the semitransparent plane p2 while the other half of it is mirror reflected and hits on the plane p3. Both plane p2 and p3 are diffusive. Hence, the laser interception points on these two planes are imaged at camera c2 and c3, respectively. The symmetry plane of plane p3 with regard to the beam splitter is a virtual plane p4 as shown in Fig. 1. From the laser ray interception points with the plane p3, the laser interception points with the virtual plane p4 can be computed. Combined with the laser interception points on the plane p2, two points are known for each emergent laser ray. Hence, each ray is determined uniquely. The three-dimensional coordinates of the interception points on the plane p2 and the virtual plane p4 can apparently be directly computed with the equations of these two planes and their corresponding camera image plane coordinates of the interception laser dots on these two planes, respectively [8]. In this study, the maximum likelihood estimation [9] is applied to compute the homography between camera c2 and diffusive plane p2, camera c3 and diffusive plane p3, respectively, in order to improve the reconstruction accuracy. The projection center of the virtual camera is obtained by computing the point where all the incident laser rays intersect. To this end, each incident laser ray needs to be determined first. The method is to move the horizontal moving screens in the vertical direction and compute world coordinates of the interception points of incident laser rays with the horizontal screens. In theory, two points determine one line. However, in reality the noise affects the accuracy. Hence, six horizontal planes have been used to obtain six interception points for each incident ray. The equation for each incident laser ray is then computed by singular value decomposition algorithm as detailed in [8]. To reconstruct the mirror surface, the intersection point of the incident ray and its corresponding emergent ray is computed. In case they do not intersect exactly, the shortest line connecting the two rays is determined and its middle point is computed as the estimated intersection [8]. 3. Experimental system Fig. 2 shows the experimental system used to capture the weld pool reflected laser dots images. A 4 in. schedule 10 stainless steel pipe, the outside diameter is 114.3 mm (4.5 in.) and the wall
thickenss is 3.1 mm (0.12 in.) [10], is used as the workpiece for stationary GTAW. The used tungsten is 2% ceriated and its diameter is 3/32 in. The tip of the tungsten is shaped into a cone with cone angle 30◦ and its distance to the work piece is 6 mm. Pure argon is used as the shielding gas and the flow rate is 8.5 l/min (18 ft3 /h). The welding power supply mode is CC and the current is 70 A. The angle of the center incident ray with the horizontal plane is 44◦ . During the welding process, the laser rays are projected onto the weld pool and reflected by the specular surface onto the beam splitter. Half of the laser energy passes through the beam splitter and imaged on imaging plane 1, the backside of the beam splitter. Another half is reflected by the front-side surface of the beam splitter and intercepted and imaged on imaging plane 2. The front surface of the beam splitter is placed at y = −115 mm and the thickness of the beam splitter is 3 mm. Imaging plane 2 is placed at y = −30 mm. Two Dragonfly2 cameras synchronically record images on the two diffusive imaging planes at 60 frames/s. 4. Imaging The challenges are apparent: (1) the arc light radiation is strong and its power is much greater than that of a possible illumination laser; (2) two diffusive planes are needed and must be placed with a distance. Further, if the used laser is not “focusless”, the focus brings an additional issue. To overcome the first issue, spectral band-pass filter [4,6] has been used to decrease the effect of arc light and imaging plane 1 is placed with a sufficient distance such that the arc light intensity decays significantly when reaching imaging plane 1. To overcome the second issue, the authors had tried a “focusless” (Pico Laser) projector first. However, the manufacturer does not disclose the spectral data to customers such that the authors lack the knowledge to use an appropriate optical filter to reduce the arc light effect. As a result, a 660 nm 50 mW SNF laser with an adjustable focus is used in this study. To overcome the out-offocus problem, the authors increased the size of projected pattern to decrease the diffraction effect between adjacent rays. On the other hand, the size of the projected pattern should be as small as possible to obtain more weld pool reflected rays as long as it covers the entire weld pool surface. Hence, a tradeoff, among the size of the projected laser pattern, the distances of the diffusive planes from the arc light, and the focus range of the laser rays, is needed to optimize the image quality. Fig. 3(a)–(e) shows the captured images with (a)–(c) for system calibration. It is seen that the laser dots on any image are not perfectly clear because of the trade-off. However, the image quality appears to be adequate for image processing
36
Z. Wang et al. / Journal of Manufacturing Processes 15 (2013) 34–40
Fig. 2. Experimental system. The distance between the beam splitter reflection surface and imaging plane 2 (beam splitter back-side surface) used in the illustration is not proportional to the actual distance and used in order to demonstrate the image on imaging plane2.
Fig. 3. Captured images: (a) projected pattern on the diffusive calibration horizontal screen captured by cam1 shown in Fig. 1; (b) and (c) patterns reflected from the horizontal calibration mirror captured by cam2 and cam3, respectively; (d) and (e) an image of laser rays reflected from weld pool captured by cam2 and cam3, respectively.
algorithms to segment the laser dots from the background. Fig. 3(d) and (e) shows a pair of images of the laser rays reflected from the weld pool. As can be seen, the laser dots are clearly observable and can be extracted using an image processing algorithm. The images in (d) and (e) will be used to reconstruct the weld pool surface as the demonstration of the method proposed in this study.
5. Least deformation reconstruction The images in Fig. 3 are segmented [11–14] and the segmentation results are shown in Fig. 4. It is seen that some dots are missing in Fig. 4(a)–(c) due to the effect of the focus trade-off on the original images. Fortunately, calibration can be done using only some dots.
Z. Wang et al. / Journal of Manufacturing Processes 15 (2013) 34–40
37
Fig. 4. Segmentation of images in Fig. 3.
Fig. 5. Reconstruction result of a flat mirror surface: (a) comparison of points and (b) reconstruction errors.
In this study, the calibration that determines the equations of the two diffusive planes as detailed in [8] is done using seven lines of the laser dots. With the calibrated diffusive planes equations, the equation for each reflected ray can be determined using its corresponding dots in the images acquired by the two synchronized cameras. The sampled points on the weld pool surface that reflect the incident rays can then be calculated accordingly [8]. To test the reconstruction accuracy, a flat mirror surface is sampled and reconstructed with the previously proposed algorithm and maximum likelihood estimation (MLE) of homography computation [8,9,15–17]. The results are shown in Fig. 5(a) and (b) where (a) is the sampled against the originally computed points and (b) is the errors in z-direction for all points. It is seen that the reconstructed values are close to the original values. The computed reconstruction MSE (mean square errors) are 3.0188 e−004 mm in x-direction, 5.9231 e−004 mm in y-direction, and 0.0023 mm in z-direction. Please be noticed that all the dimensions of the reconstruction results hereafter in this paper are in millimeter. Since the sampled mirror plane is reconstructed with an adequate accuracy, the sampled weld pool surface whose images are given in Fig. 3(d) and (e) is reconstructed. The result is shown in
Fig. 6. From Fig. 6, it is seen that the reconstructed depth is increased greatly. Analysis shows that this increased deformation is caused by the calibration errors of the equations for the two diffusive planes p2
Fig. 6. Reconstructed weld pool sampled points.
38
Z. Wang et al. / Journal of Manufacturing Processes 15 (2013) 34–40
and p3 shown in Fig. 1. For any given differences between the actual positions and the calibrated nominal positions, the resultant intersections deviate from the actual positions in the depth (z-) direction in a definite direction (either up or down) despite the directions of the deflected rays. However, the amplitude of the deviation in the depth direction depends on the direction of the each reflected ray. The minimum/maximum deviation occurs when the reflected ray has the minimum/maximum angle with the normal of the diffusive planes. Hence, the calculated range of the depth of the weld pool surface must increase from its actual range when the calibration errors exist and the calculated range can never become smaller than the actual range by adjusting the nominal positions of the diffusive planes. That is, the minimum range of the calculated weld pool surface in the depth occurs when the calibration errors are zero. Hence, a least deformation method can be proposed to minimize the calibration errors and their effect on the results of the weld pool surface reconstruction. The least z range rectification method for the calibration is summarized as follows: Step 1 Reconstruct a simple shape (convex in our experiments) mirror surface with the experimental system. Step 2 Find the plane coefficient parameter vector ( = (a, b, c, d)) is the coefficient set in the plane equation (ax + by + cz = d) that makes the z range of the reconstructed mirror surface minimum, which is formulated by the following equation: ˆ = arg min( max Zir − min Zir )
i=1,...,N
(1)
After least z range rectification defined by Eq. (1), the coefficient set ˆ becomes: {a1 = −9.3174 e − 004, b1 = −0.0061, c 1 = 2.3832 e − 004, d1 = −0.0099, a2 = −0.0129, b2 = −0.0400, c 2 = 0.0062, d2 = −1.1729} ˆ the distorted weld pool With the rectified coefficient set , shown in Fig. 6 is reconstructed again and the reconstruction result is shown in Fig. 7. As can be seen, both the size and shape of the reconstructed weld pool become reasonable after coefficient rectification.
i=1,...,N
where Zir is the ith z coordinate for the reconstructed surface, i is the index of the interception (sampled) point and N is the total number of the interception (sampled) points. Step 3 Replace the previous plane coefficients with the found ones in Step 2 and reconstruct the weld pools. After system calibration the computed plane coefficients set are as follows: {a1 = −9.3174 e − 004, b1 = −0.0061, c 1 = 1.8318 e − 005, d1 = −0.0098, a2 = −0.0130, b2 = −0.0400, c 2 = 8.4368 e − 004, d2 = −1.1685}
Fig. 7. Reconstructed weld pool sampled points after least z range rectification.
6. Experimental results and discussion To verify the reconstruction reliability further, several weld pools are reconstructed in this section. The reconstruction results are shown in Figs. 8–12. From the reconstructed weld pool surfaces, it is seen that not all the laser lines projected onto the weld pool are reflected onto the diffusive planes. Some of the reflected laser lines are missing due to the orientation and limited size of the diffusive plane which will be addressed in the future. From all these reconstruction results, it is concluded that the proposed method is effective in reconstructing the sampled three-dimensional GTAW weld pool surfaces. Moreover, the reconstruction accuracy can still be improved by ray modeling with projective laser projectors which have been verified by our experiments.
Fig. 8. Reconstructed weld pool after coefficient rectification: (a) image captured by cam2; (b) image captured by cam3; (c) reconstructed 3D weld pool sampled points.
Z. Wang et al. / Journal of Manufacturing Processes 15 (2013) 34–40
39
Fig. 9. Reconstructed weld pool after coefficient rectification: (a) image captured by cam2; (b) image captured by cam3; (c) reconstructed 3D weld pool sampled points.
Fig. 10. Reconstructed weld pool after coefficient rectification: (a) image captured by cam2; (b) image captured by cam3; (c) reconstructed 3D weld pool sampled points.
Fig. 11. Reconstructed weld pool after coefficient rectification: (a) image captured by cam2; (b) image captured by cam3; (c) reconstructed 3D weld pool sampled points.
Fig. 12. Reconstructed weld pool after coefficient rectification: (a) image captured by cam2; (b) image captured by cam3; (c) reconstructed 3D weld pool sampled points.
40
Z. Wang et al. / Journal of Manufacturing Processes 15 (2013) 34–40
7. Conclusions and future work Weld pool surface in GTAW has been successfully sampled and reconstructed analytically through double intercepting/imaging the reflected laser rays. Imaging of reflected laser rays on two separate diffusive planes and the proposed least deformation principle play key roles toward this success. Future work includes a compact design of the double interception/imaging system and applications of this method in feedback control of weld pool surface. Acknowledgement This work is funded by the National Science Foundation under grant CMMI-0927707. References [1] Greivenkamp JE, Bruning JH. Phase shifting interferometry. In: Optical shop testing. 2nd ed., 1992. p. 501–98. [2] Mnich C, Al-Bayat F. In situ weld pool measurement using stereovision. In: Japan–USA Symposium on Flexible Automation. 2004. [3] Zhao DB, Yi JQ. Extraction of three-dimensional parameters for weld pool surface in pulsed GTAW with wire filler. Journal of Manufacturing Science and Engineering 2003;125:493–503. [4] Zhang YM, Song HS, Saeed G. Observation of a dynamic specular weld pool surface. Measurement Science and Technology 2006;17(6):L9–12.
[5] Song HS, Zhang YM. Measurement and analysis of three dimensional specular gas tungsten weld pool surface. Welding Journal 2008;87(4): 85s–95s. [6] Ma X, Zhang YM. Gas metal arc weld pool surface imaging: modeling and processing. Welding Journal 2011;90:85s–94s. [7] Song HS, Zhang YM. Three-dimensional reconstruction of specular surface for gas tungsten arc weld pool. Measurement Science and Technology 2007;18:3751–67. [8] Wang ZZ, Huang XY, Yang RG, Zhang YM. Measurement of mirror surfaces using specular reflection and analytical computation. Machine Vision and Applications 2012. [9] Agarwal A, Jawahar C, Narayanan P. A survey of planar homography estimation techniques. Technical Reports. Hyderabad: International Institute of Information Technology; 2005. [10] ASME/ANSI B36.10/19: pipe sizes, diameters, wall thickness, working pressures. [11] Wang ZZ, Zhang YM. Brightness based selection and edge detection based enhancement separation algorithm for low resolution metal transfer images. IEEE Transactions on Automation Science and Engineering 2009; 6(January (1)). [12] Wang ZZ, Zhang YM. Image processing algorithm for automated monitoring of metal transfer in double-electrode GMAW. Measurement Science and Technology 2007;18:2048–58. [13] Kovacevic R, Zhang YM. Real-time image processing for monitoring of free weld pool surface. Journal of Manufacturing Science and Engineering-Transactions of the ASME 1997;119(2):161–9. [14] Gonzalez RC, Woods RE. Digital image processing. 2nd ed; 1993. [15] Zhang ZY. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000;22(11):1330–4. [16] Hartley R, Zisserman A. Multi-view geometry in computer vision. United Kingdom: Cambridge University Press; March 2004. [17] Chum O, Pajdla T, Sturm P. The geometric error for homographies. Computer Vision and Image Understanding 2005;97(1):86–102.
Contents lists available at SciVerse ScienceDirect
Journal of Manufacturing Processes journal homepage: www.elsevier.com/locate/manpro
Technical Paper
Analytical reconstruction of three-dimensional weld pool surface in GTAW Zhenzhou Wang a , YuMing Zhang a,∗ , Ruigang Yang b a b
Department of Electrical and Computer Engineering and Institute for Sustainable Manufacturing, University of Kentucky, Lexington, KY 40506, USA Department of Computer Science, University of Kentucky, Lexington, KY 40506, USA
a r t i c l e
i n f o
Article history: Received 24 August 2011 Received in revised form 9 August 2012 Accepted 9 August 2012 Available online 2 September 2012 Keywords: Mirror surface Specular surface Welding GTAW Laser Machine vision Reflection
a b s t r a c t The reflective characteristic of mirror surfaces such as a liquid pool surface in arc welding makes many traditional 3D measurement/reconstruction methods fail. The authors proposed to intercept, image, and measure two points in each laser ray reflected from a mirror surface with two diffusive planes and cameras to analytically calculate the equation of the ray. The samples of the points on the mirror surface where the incident laser rays are reflected can thus be analytically calculated as the intersections of the reflection rays with the corresponding incident rays. In this paper, the proposed method is applied to reconstruct the samples on the specular three-dimensional weld pool surface in GTAW (gas tungsten arc welding). Since two diffusive planes are used and must be placed with considerable distance to assure the accuracy of the calculated equations of the reflected rays, focusing reflected laser rays on these two planes becomes an issue. A trade-off among the size of the projected laser pattern, the distances of the arc light with the two diffusive planes, the focus range of the laser rays and the quality of the reflected laser dot images on the diffusive planes has been made to resolve this issue successfully. Further, calibration errors in the locations of the diffusive planes directly affect the accuracy of the calculated equations of reflected rays and an accurate calibration appears to impractical. To resolve this issue, the authors found the least deformation principle and successfully applied it to minimize the calculation errors through calibration rectification. Several weld pool surfaces have been sampled and reconstructed and experimental results verified the effectiveness of the proposed analytical method. © 2012 The Society of Manufacturing Engineers. Published by Elsevier Ltd. All rights reserved.
1. Introduction Many traditional 3D reconstruction methods will avoid reconstructing mirror surfaces because the reflection from mirror surfaces is view-angle dependent. It causes stereo vision methods to fail in finding correct correspondences between two images and also adds the ambiguity of normal and depth to the traditional structured light and phase shift methods [1]. In the case of sensing a weld pool surface [2–7], it is also interfered by the harsh welding environment. For instance, the work piece is melted by the arc and the bright arc light makes direct sensing of a weld pool surface difficult. To overcome these difficulties, laser patterns have been projected onto the weld pool surface and the reflections of the laser patterns from the weld pool surface are imaged on a diffusive plane. The images of reflected laser patterns on the diffusive plane are captured by a CCD camera and processed with image processing algorithms to calculate the distortion of laser patterns. From the distortion of the reflected laser patterns, the surface of the weld pool can be calculated [4–7]. The
∗ Corresponding author. E-mail address: [email protected] (Y. Zhang).
drawback of this method is that the reconstructed 3D surface is obtained by matching the calculated reflection pattern with the imaged reflection pattern and there is an iterative optimization process needed in order to achieve an acceptable matching. To avoid the iteration, direct analytical solutions are needed for sampled points on the mirror surface reflecting the projected laser pattern. Another method has thus been proposed in a previous study [8] that uses two diffusive screens to capture the reflected pattern twice such that the equation for the linear trajectory of each reflected ray can be analytically computed. With known incident rays’ equations and their corresponding reflected rays’ equations, their intersection points can be directly computed, therefore obtaining the sampled points of the 3D mirror surface where the incident rays are intercepted and reflected. It thus proposes a method that may be advantageous over the previous iteration based algorithm [7] in computation speed.
2. Principle Fig. 1 illustrates the principle of the previously proposed method for direction analytical solutions of the reflection rays [8]. As shown in Fig. 1, there are three cameras c1, c2 and c3 which aim at planes p1, p2 and p3, respectively. The origin of the world co-ordinate is
1526-6125/$ – see front matter © 2012 The Society of Manufacturing Engineers. Published by Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.jmapro.2012.08.002
Z. Wang et al. / Journal of Manufacturing Processes 15 (2013) 34–40
35
Fig. 1. System diagram.
defined at the point O. The equation of the plane p1 which is on the moving screen is defined as z = 0. The laser rays are projected onto the surface of the plane p1 from the projection center C. When the mirror reflected laser ray reaches the beam splitter, half power of it transmits through the splitter and hits on the semitransparent plane p2 while the other half of it is mirror reflected and hits on the plane p3. Both plane p2 and p3 are diffusive. Hence, the laser interception points on these two planes are imaged at camera c2 and c3, respectively. The symmetry plane of plane p3 with regard to the beam splitter is a virtual plane p4 as shown in Fig. 1. From the laser ray interception points with the plane p3, the laser interception points with the virtual plane p4 can be computed. Combined with the laser interception points on the plane p2, two points are known for each emergent laser ray. Hence, each ray is determined uniquely. The three-dimensional coordinates of the interception points on the plane p2 and the virtual plane p4 can apparently be directly computed with the equations of these two planes and their corresponding camera image plane coordinates of the interception laser dots on these two planes, respectively [8]. In this study, the maximum likelihood estimation [9] is applied to compute the homography between camera c2 and diffusive plane p2, camera c3 and diffusive plane p3, respectively, in order to improve the reconstruction accuracy. The projection center of the virtual camera is obtained by computing the point where all the incident laser rays intersect. To this end, each incident laser ray needs to be determined first. The method is to move the horizontal moving screens in the vertical direction and compute world coordinates of the interception points of incident laser rays with the horizontal screens. In theory, two points determine one line. However, in reality the noise affects the accuracy. Hence, six horizontal planes have been used to obtain six interception points for each incident ray. The equation for each incident laser ray is then computed by singular value decomposition algorithm as detailed in [8]. To reconstruct the mirror surface, the intersection point of the incident ray and its corresponding emergent ray is computed. In case they do not intersect exactly, the shortest line connecting the two rays is determined and its middle point is computed as the estimated intersection [8]. 3. Experimental system Fig. 2 shows the experimental system used to capture the weld pool reflected laser dots images. A 4 in. schedule 10 stainless steel pipe, the outside diameter is 114.3 mm (4.5 in.) and the wall
thickenss is 3.1 mm (0.12 in.) [10], is used as the workpiece for stationary GTAW. The used tungsten is 2% ceriated and its diameter is 3/32 in. The tip of the tungsten is shaped into a cone with cone angle 30◦ and its distance to the work piece is 6 mm. Pure argon is used as the shielding gas and the flow rate is 8.5 l/min (18 ft3 /h). The welding power supply mode is CC and the current is 70 A. The angle of the center incident ray with the horizontal plane is 44◦ . During the welding process, the laser rays are projected onto the weld pool and reflected by the specular surface onto the beam splitter. Half of the laser energy passes through the beam splitter and imaged on imaging plane 1, the backside of the beam splitter. Another half is reflected by the front-side surface of the beam splitter and intercepted and imaged on imaging plane 2. The front surface of the beam splitter is placed at y = −115 mm and the thickness of the beam splitter is 3 mm. Imaging plane 2 is placed at y = −30 mm. Two Dragonfly2 cameras synchronically record images on the two diffusive imaging planes at 60 frames/s. 4. Imaging The challenges are apparent: (1) the arc light radiation is strong and its power is much greater than that of a possible illumination laser; (2) two diffusive planes are needed and must be placed with a distance. Further, if the used laser is not “focusless”, the focus brings an additional issue. To overcome the first issue, spectral band-pass filter [4,6] has been used to decrease the effect of arc light and imaging plane 1 is placed with a sufficient distance such that the arc light intensity decays significantly when reaching imaging plane 1. To overcome the second issue, the authors had tried a “focusless” (Pico Laser) projector first. However, the manufacturer does not disclose the spectral data to customers such that the authors lack the knowledge to use an appropriate optical filter to reduce the arc light effect. As a result, a 660 nm 50 mW SNF laser with an adjustable focus is used in this study. To overcome the out-offocus problem, the authors increased the size of projected pattern to decrease the diffraction effect between adjacent rays. On the other hand, the size of the projected pattern should be as small as possible to obtain more weld pool reflected rays as long as it covers the entire weld pool surface. Hence, a tradeoff, among the size of the projected laser pattern, the distances of the diffusive planes from the arc light, and the focus range of the laser rays, is needed to optimize the image quality. Fig. 3(a)–(e) shows the captured images with (a)–(c) for system calibration. It is seen that the laser dots on any image are not perfectly clear because of the trade-off. However, the image quality appears to be adequate for image processing
36
Z. Wang et al. / Journal of Manufacturing Processes 15 (2013) 34–40
Fig. 2. Experimental system. The distance between the beam splitter reflection surface and imaging plane 2 (beam splitter back-side surface) used in the illustration is not proportional to the actual distance and used in order to demonstrate the image on imaging plane2.
Fig. 3. Captured images: (a) projected pattern on the diffusive calibration horizontal screen captured by cam1 shown in Fig. 1; (b) and (c) patterns reflected from the horizontal calibration mirror captured by cam2 and cam3, respectively; (d) and (e) an image of laser rays reflected from weld pool captured by cam2 and cam3, respectively.
algorithms to segment the laser dots from the background. Fig. 3(d) and (e) shows a pair of images of the laser rays reflected from the weld pool. As can be seen, the laser dots are clearly observable and can be extracted using an image processing algorithm. The images in (d) and (e) will be used to reconstruct the weld pool surface as the demonstration of the method proposed in this study.
5. Least deformation reconstruction The images in Fig. 3 are segmented [11–14] and the segmentation results are shown in Fig. 4. It is seen that some dots are missing in Fig. 4(a)–(c) due to the effect of the focus trade-off on the original images. Fortunately, calibration can be done using only some dots.
Z. Wang et al. / Journal of Manufacturing Processes 15 (2013) 34–40
37
Fig. 4. Segmentation of images in Fig. 3.
Fig. 5. Reconstruction result of a flat mirror surface: (a) comparison of points and (b) reconstruction errors.
In this study, the calibration that determines the equations of the two diffusive planes as detailed in [8] is done using seven lines of the laser dots. With the calibrated diffusive planes equations, the equation for each reflected ray can be determined using its corresponding dots in the images acquired by the two synchronized cameras. The sampled points on the weld pool surface that reflect the incident rays can then be calculated accordingly [8]. To test the reconstruction accuracy, a flat mirror surface is sampled and reconstructed with the previously proposed algorithm and maximum likelihood estimation (MLE) of homography computation [8,9,15–17]. The results are shown in Fig. 5(a) and (b) where (a) is the sampled against the originally computed points and (b) is the errors in z-direction for all points. It is seen that the reconstructed values are close to the original values. The computed reconstruction MSE (mean square errors) are 3.0188 e−004 mm in x-direction, 5.9231 e−004 mm in y-direction, and 0.0023 mm in z-direction. Please be noticed that all the dimensions of the reconstruction results hereafter in this paper are in millimeter. Since the sampled mirror plane is reconstructed with an adequate accuracy, the sampled weld pool surface whose images are given in Fig. 3(d) and (e) is reconstructed. The result is shown in
Fig. 6. From Fig. 6, it is seen that the reconstructed depth is increased greatly. Analysis shows that this increased deformation is caused by the calibration errors of the equations for the two diffusive planes p2
Fig. 6. Reconstructed weld pool sampled points.
38
Z. Wang et al. / Journal of Manufacturing Processes 15 (2013) 34–40
and p3 shown in Fig. 1. For any given differences between the actual positions and the calibrated nominal positions, the resultant intersections deviate from the actual positions in the depth (z-) direction in a definite direction (either up or down) despite the directions of the deflected rays. However, the amplitude of the deviation in the depth direction depends on the direction of the each reflected ray. The minimum/maximum deviation occurs when the reflected ray has the minimum/maximum angle with the normal of the diffusive planes. Hence, the calculated range of the depth of the weld pool surface must increase from its actual range when the calibration errors exist and the calculated range can never become smaller than the actual range by adjusting the nominal positions of the diffusive planes. That is, the minimum range of the calculated weld pool surface in the depth occurs when the calibration errors are zero. Hence, a least deformation method can be proposed to minimize the calibration errors and their effect on the results of the weld pool surface reconstruction. The least z range rectification method for the calibration is summarized as follows: Step 1 Reconstruct a simple shape (convex in our experiments) mirror surface with the experimental system. Step 2 Find the plane coefficient parameter vector ( = (a, b, c, d)) is the coefficient set in the plane equation (ax + by + cz = d) that makes the z range of the reconstructed mirror surface minimum, which is formulated by the following equation: ˆ = arg min( max Zir − min Zir )
i=1,...,N
(1)
After least z range rectification defined by Eq. (1), the coefficient set ˆ becomes: {a1 = −9.3174 e − 004, b1 = −0.0061, c 1 = 2.3832 e − 004, d1 = −0.0099, a2 = −0.0129, b2 = −0.0400, c 2 = 0.0062, d2 = −1.1729} ˆ the distorted weld pool With the rectified coefficient set , shown in Fig. 6 is reconstructed again and the reconstruction result is shown in Fig. 7. As can be seen, both the size and shape of the reconstructed weld pool become reasonable after coefficient rectification.
i=1,...,N
where Zir is the ith z coordinate for the reconstructed surface, i is the index of the interception (sampled) point and N is the total number of the interception (sampled) points. Step 3 Replace the previous plane coefficients with the found ones in Step 2 and reconstruct the weld pools. After system calibration the computed plane coefficients set are as follows: {a1 = −9.3174 e − 004, b1 = −0.0061, c 1 = 1.8318 e − 005, d1 = −0.0098, a2 = −0.0130, b2 = −0.0400, c 2 = 8.4368 e − 004, d2 = −1.1685}
Fig. 7. Reconstructed weld pool sampled points after least z range rectification.
6. Experimental results and discussion To verify the reconstruction reliability further, several weld pools are reconstructed in this section. The reconstruction results are shown in Figs. 8–12. From the reconstructed weld pool surfaces, it is seen that not all the laser lines projected onto the weld pool are reflected onto the diffusive planes. Some of the reflected laser lines are missing due to the orientation and limited size of the diffusive plane which will be addressed in the future. From all these reconstruction results, it is concluded that the proposed method is effective in reconstructing the sampled three-dimensional GTAW weld pool surfaces. Moreover, the reconstruction accuracy can still be improved by ray modeling with projective laser projectors which have been verified by our experiments.
Fig. 8. Reconstructed weld pool after coefficient rectification: (a) image captured by cam2; (b) image captured by cam3; (c) reconstructed 3D weld pool sampled points.
Z. Wang et al. / Journal of Manufacturing Processes 15 (2013) 34–40
39
Fig. 9. Reconstructed weld pool after coefficient rectification: (a) image captured by cam2; (b) image captured by cam3; (c) reconstructed 3D weld pool sampled points.
Fig. 10. Reconstructed weld pool after coefficient rectification: (a) image captured by cam2; (b) image captured by cam3; (c) reconstructed 3D weld pool sampled points.
Fig. 11. Reconstructed weld pool after coefficient rectification: (a) image captured by cam2; (b) image captured by cam3; (c) reconstructed 3D weld pool sampled points.
Fig. 12. Reconstructed weld pool after coefficient rectification: (a) image captured by cam2; (b) image captured by cam3; (c) reconstructed 3D weld pool sampled points.
40
Z. Wang et al. / Journal of Manufacturing Processes 15 (2013) 34–40
7. Conclusions and future work Weld pool surface in GTAW has been successfully sampled and reconstructed analytically through double intercepting/imaging the reflected laser rays. Imaging of reflected laser rays on two separate diffusive planes and the proposed least deformation principle play key roles toward this success. Future work includes a compact design of the double interception/imaging system and applications of this method in feedback control of weld pool surface. Acknowledgement This work is funded by the National Science Foundation under grant CMMI-0927707. References [1] Greivenkamp JE, Bruning JH. Phase shifting interferometry. In: Optical shop testing. 2nd ed., 1992. p. 501–98. [2] Mnich C, Al-Bayat F. In situ weld pool measurement using stereovision. In: Japan–USA Symposium on Flexible Automation. 2004. [3] Zhao DB, Yi JQ. Extraction of three-dimensional parameters for weld pool surface in pulsed GTAW with wire filler. Journal of Manufacturing Science and Engineering 2003;125:493–503. [4] Zhang YM, Song HS, Saeed G. Observation of a dynamic specular weld pool surface. Measurement Science and Technology 2006;17(6):L9–12.
[5] Song HS, Zhang YM. Measurement and analysis of three dimensional specular gas tungsten weld pool surface. Welding Journal 2008;87(4): 85s–95s. [6] Ma X, Zhang YM. Gas metal arc weld pool surface imaging: modeling and processing. Welding Journal 2011;90:85s–94s. [7] Song HS, Zhang YM. Three-dimensional reconstruction of specular surface for gas tungsten arc weld pool. Measurement Science and Technology 2007;18:3751–67. [8] Wang ZZ, Huang XY, Yang RG, Zhang YM. Measurement of mirror surfaces using specular reflection and analytical computation. Machine Vision and Applications 2012. [9] Agarwal A, Jawahar C, Narayanan P. A survey of planar homography estimation techniques. Technical Reports. Hyderabad: International Institute of Information Technology; 2005. [10] ASME/ANSI B36.10/19: pipe sizes, diameters, wall thickness, working pressures. [11] Wang ZZ, Zhang YM. Brightness based selection and edge detection based enhancement separation algorithm for low resolution metal transfer images. IEEE Transactions on Automation Science and Engineering 2009; 6(January (1)). [12] Wang ZZ, Zhang YM. Image processing algorithm for automated monitoring of metal transfer in double-electrode GMAW. Measurement Science and Technology 2007;18:2048–58. [13] Kovacevic R, Zhang YM. Real-time image processing for monitoring of free weld pool surface. Journal of Manufacturing Science and Engineering-Transactions of the ASME 1997;119(2):161–9. [14] Gonzalez RC, Woods RE. Digital image processing. 2nd ed; 1993. [15] Zhang ZY. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000;22(11):1330–4. [16] Hartley R, Zisserman A. Multi-view geometry in computer vision. United Kingdom: Cambridge University Press; March 2004. [17] Chum O, Pajdla T, Sturm P. The geometric error for homographies. Computer Vision and Image Understanding 2005;97(1):86–102.