- Email: [email protected]
The modeling of welding pool surface reflectance of aluminum alloy pulse GTAW
Materials Science and Engineering A 394 (2005) 320–326
The modeling of welding pool surface reflectance of aluminum alloy pulse GTAW Li Laiping∗ , Chen Shanben, Lin Tao School of Materials Science and Engineering, Shanghai Jiaotong University, Shanghai 200030, PR China Received 16 August 2004; accepted 17 November 2004
Abstract The information of welding pool surface height plays an important role in the precise control of weld formation. The height measurement of welding pool surface is very difficult because there exists splash, fume, high temperature, electromagnetic disturbance, and so on during welding and welding pool have the characteristics such as small volume, short existence time in high temperature, high temperature, the flow in welding pool and concurrence of metal melting and freezing, etc. The application of aluminum alloy is wide because of low melting point, large conductivity of electric and heat, small density and so on. Welding is an important manufacture technology in aluminum alloy application where the quality control is too important. The color of aluminum alloy does not obviously change after melting which cause the difficulty to estimate welding estate. Visual sensing is a promising method to acquire the shape information of welding pool. The theory of surface height calculation of pulse GTAW welding pool based on SFS is to calculate the surface height from welding pool image whose key is to build up the surface reflectance model of welding pool. Based on the imaging characteristics of aluminum alloy pulse GTAW welding pool, the surface reflectance model is built after analyzing arc intensity, filter system, welding pool shape and reflectance characteristics. With smooth constraint condition of the welding pool surface and variable factor SOR method, the height of welding pool surface is calculated and error analyzed. © 2004 Published by Elsevier B.V. Keywords: Aluminum alloy pulse GTAW; Surface reflectance model of welding pool; Shape from shading; Surface height of welding pool
1. Introduction Aluminum alloy have the following characteristics, which make it have wide application, such as low melting temperature, large heat and electric conductivity, high specific heat and better anticorrosive, and so on. Welding is an important manufacturing technology in aluminum alloy application. Welding quality control is an important problem for welding automation, mechanization, robot welding and intelligent welding. Because of the characteristics of aluminum alloy and welding technology, the research of welding process control is lacking. According to relative theory, there are certain relations between the shape and size of welding pool surface, reinforce∗
Corresponding author. Tel.: +86 21 62932429; fax: +86 21 62932429. E-mail address: [email protected] (L. Laiping).
0921-5093/$ – see front matter © 2004 Published by Elsevier B.V. doi:10.1016/j.msea.2004.11.063
ment and weld penetration, that is to say the topside shape of welding pool can show the appearance of backside weld. Therefore the height measurement of welding pool topside surface is significant in precise control of appearance of weld, which is an index of welding quality. In real operation, the skilled welder observes the shape, size and dynamic change of welding pool, the shape and intensity of arc, the metal transfer model, etc. to feel the appearance of backside weld and adjust the welding parameters, the height and slant of welding torch so as to assure the formation quality of weld. There are fume, splash, high temperature and electromagnetic disturbance, and so on in welding process. The welding pool has the following characteristics, such as small volume, short stay in high temperature, high temperature, fast freezing, concurrence of melting and freezing process and dynamic flow. The above characteristics of welding process and welding pool make the measurement of welding
L. Laiping et al. / Materials Science and Engineering A 394 (2005) 320–326
pool surface height very difficult. The height of welding pool surface cannot be measured by mechanical method. The visual sensor simulates welder’s eye to check the status of welding process and acquire the characteristic information showing the welding process, which is used in weld size acquisition, seam tracking and arc spectrum analysis. R. Kovacevic calculated the surface height of welding pool by structure light. The calculated height is based on freezing weld, which make the control imperfect because the controlled parameters are under arc and the height information has lag [1]. The passive visual sensor is a promising sensor whose light source is arc. It is important in precise control of welding pool to acquire welding pool image and calculate the surface height from image. The characteristic of aluminum alloy welding is that there is not obviously color change after metal melting. The welding pool is flowing under electromagnetic force, surface tension, airflow forces and so on, which make the shape and size of welding pool change with time. Though there exists flow in welding process, the shape information of welding pool cannot be acquired from motion, optical flow and image serials. In the passive visual sensing technology, the arc cannot move to different places to acquire three images of the same welding pool, so surface height of welding pool cannot be calculated by photometric stereo method. In the welding process, the shape of welding pool cannot be acquired by stereo vision which use many cameras to calculate the object height because of the dynamic characteristic of welding pool, the cost of welding equipment, the volume and flexible of welding torch and the difficulty of stereo match. There may exist contour and texture in weld, but the information is not abundant to determine the shape of welding pool. The information lags because it is showed in freezing weld. Shape from shading is a method to calculate the object height from single image and other constraint conditions. It is a promising method to calculate the surface height of welding pool because of simple structure and low cost [2]. Shape from shading is a key problem in computer vision domain, which is based on the fact that the change of image gray shows the change of object shape [3]. The idea is to calculate the surface gradient and height from image gray. The key is to build the reflectance model of object surface, which describes the relation between the image gray and light source, shape and reflectance characteristics and camera. According to the imaging theory, the gray level at a pixel in image of 3D object depends on the shape, position, intensity of light source, the shape and reflectance characteristic of object and the characteristic of camera. Oren and Nayar analyzed the generalization form of Lambertian model and the application in the machine vision [4]. The specularity model developed by Phong represented the specular component of reflection as powers of the cosine of the angle between the perfect specular direction and the viewing direction [5]. The Torrance–Sparrow model assumed that a surface is composed of small, randomly oriented, mirror-like facets. It described
321
the specular brightness as the product of the following: energy of incident light, Fresnel coefficient, facet orientation distribution function and geometrical attenuation factor [6]. Healey and Binford derived a simplified model by using the Gaussian distribution as the facet orientation function and considering the other components as constants [7]. Nayar et al. proposed a reflectance model, which consists of three components: diffuse lobe, specular lobe and specular spike [8]. Nayar et al. analyzed the interreflection, which appear in concave surfaces or concavities result from multiple objects, and built the interreflection model [9]. Horn and Brooks proposed the shape from shading and analyzed the reflectance model under ideal imaging condition [10]. Lee and Kuo analyzed the effect of light source, reflectance model and projection on reflectance map model and built the reflectance model of real object under near point light source and calculated the surface height by photometric stereo method [11,12]. Because of the shortcoming of photometric stereo method, shape from a single image has more significant. Zhao et al. first introduced shape from shading into the height calculation of welding pool surface, but the complex of welding process and the characteristics of welding pool make application very difficult [2]. The imaging characteristics of welding aluminum alloy are analyzed. Based on the arc intensity, filter system, shape and reflectance characteristics of welding pool, and camera parameters, the reflectance model of welding pool is built. With the surface smooth constraint condition of welding pool and variable factor SOR method, the surface height of welding pool is calculated and the errors are analyzed in this paper.
2. Image analysis of aluminum alloy welding pool J.J. Wang analyzed the imaging characteristics of aluminum alloy pulse GTAW welding pool and acquired the welding pool image from side by wide filter method. Because of the limit of imaging position, nozzle occludes the viewer and the image is only half welding pool. The information of whole welding pool was acquired by symmetry method [13]. However, the welding pool is not symmetrical in welding pool when the assembly error or material characteristic causes the seam departure from the center. Thus the symmetrical method will come into being error. The welding pool image acquired from slant rear not only solve the nozzle occluding problem but also take into account the welding pool edge blur caused by the reflex intensity shortage because of large distance from rear. Fig. 1 shows the principle graphic of topside shape information acquisition of welding pool, where coordinate OXYZ is on the workpieces, origin under tungsten electrode, Y axis welding direction, X axis perpendicular to welding direction, Z axis point to tungsten electrode. Fig. 2 shows the welding pool image acquired from slant rear based on wide band filter proposed by Wang et al. [13]. The whole welding pool image region is composed of nozzle area, arc area, molten pool, solid metal and base metal. In the
322
L. Laiping et al. / Materials Science and Engineering A 394 (2005) 320–326
Fig. 1. Principle graphic of topside shape information acquisition of welding pool.
Fig. 2. Welding pool image of aluminum alloy pulse GTAW.
nozzle area, the reflex intensity is the lowest because of the nozzle occluding and the gray level at a pixel is the lowest. The gray level at a pixel of liquid metal area is lower, because the color of aluminum alloy does not change after melting. In the solid weld, its surface is relative rough and the reflectivity same in all kinds and the gray is low. Though the reflection of aluminum alloy is strong, the oxide make aluminum alloy lose metal polish, the gray level at a pixel is low too, while it is high near arc. The arc light intensity is high so the gray level is high. The center of welding pool is the inverted reflection of tungsten electrode and the gray level is very high. The contrast of every area is obvious in welding pool image.
3. Modeling of aluminum alloy welding pool surface reflectance To build precise reflectance model of object surface is the key of calculating object height from image gray. According to object imaging theory, the gray level at a pixel depends on the property of light source, the shape and reflection property of object surface, and camera parameters. The influence of welding arc, filter system, the reflection property and shape of welding pool, and camera on image gray in passive visual sensing is analyzed and the surface reflection model of welding pool is built. 3.1. Light source characteristics The imaging is based on reflecting arc light by the welding pool, weld and workpieces in passive visual sensing where
the arc light is imaging light source. The shape, position and angle of electrode are fixed, so the property of light source lies on the shape and intensity of arc. The shape of arc play an important role in welding pool imaging because the arc is a solid light source each part of which radiates. When the current is large, the arc is like a bell. The intensity of every part of arc is different, so the arc light source is anisotropic. The imaging quality is too bad caused by the shape of light source. When the current decreases, the volume of arc decrease till shrink to the butt of tungsten electrode. The arc light can be regarded as radiating from the butt of tungsten electrode and has the isotropic characteristic. Then the influence of arc shape can be ignored. According to the GTAW technology characteristics the distance between tungsten electrode and workpieces is only several millimeters in order to assure stable striking arc and pilot arc. Therefore, the imaging of welding pool is near point light source imaging. The light intensity reaching on welding pool surface is: E=
I0 max[0, iT n] r2
(1)
where E is the intensity reaching on the welding pool surface, I0 the arc radiation intensity, i the incident light direction, n the surface normal direction on point p in welding pool surface, r the distance between the butt of tungsten electrode and point p, iT n the cosine of angle between the surface normal and incident light. 3.2. complicated filter system The strong arc not only makes the image quality bad but also damages the CCD camera. The protection measure is imaging when low current and complicated filter system. The complicated filter system is composed of colorless glass, wide band filter and neutral density filter. The arc light reflected by welding pool, workpieces and weld receives camera target through glass, wide band filter and neutral density filter. The colorless glass prevents optical apparatus and lens from the splash and fume. The neutral density filter equally absorbs arc light of all wavelengths in order to decrease the arc intensity. In aluminum alloy pulse GTAW, arc spectrum is composed of continuous spectrum and line spectrum. The line
L. Laiping et al. / Materials Science and Engineering A 394 (2005) 320–326
spectrum of aluminum is 396 nm. From 380 to 760 nm, the distribution density of aluminum is less than that of argon. During 580 and 760 nm, the spectrum of aluminum is continuous spectrum while the other elements is very weak. Thus the wide filter system is based on continuous spectrum from 590 to 710 nm as light source and attenuating other spectrum, which is regarded as noise. The above filter system decrease the arc intensity that receive the lens. 3.3. The shape and reflectance characteristics of welding pool 3.3.1. Welding pool surface shape During the welding, the arc force makes the welding pool surface sink. The shape of welding pool is like an irregular elliptic hemispheroid. The change of shape of welding pool appears in the tail. When the welding pool transform from partial penetration to full penetration, depression of the foreside welding pool increase while the shape of rearward welding pool change from convex to concave. Under the electromagnetic force, the welding pool under the arc is concave. Either current is large or welding speed or wire feed rate is few, the rearward welding pool is concave, too. Contrarily, when welding current is few or welding speed or wire feed rate is large, the rearward welding pool is convex because of metal pileup. The existence time of welding pool is short in high temperature during welding. The melting of the filler metal or base metal and the freezing of liquid metal is concurrent. There is motion in welding pool. Thus the metal melts in the foreside and moves into the rearward and freezes in the rearward. The process of welding pool melting and freezing makes the edge of welding pool lack of smooth, that is to say the surface height of welding pool in edge change obviously. 3.3.2. Surface reflectance characteristics of welding pool According to the welding theory, the welding pool area is composed of liquid melting metal, solid freezing metal and base metal. Like all actual object surface, the welding pool surface reflection is hybrid, i.e. there exist both Lambertian reflection and specular reflection. Though the reflectance of aluminum alloy is strong, the oxide make the surface of aluminum alloy lose the glare of metal. The reflection intensity in all kinds is same and the workpieces surface can be regarded as ideal diffuse surface. The fact that the inverted image of tungsten electrode can be showed in the welding pool means that the welding pool has specular characteristics. However, aluminum alloy color does not obviously change after melting. Therefore the liquid welding pool can be regarded as part diffuse surface with weak specular components. The freezing weld similarly reflects the arc light in all directions and the surface is diffuse. In a word, the welding pool surface is hybrid reflectance surface. The bi-directional reflectance distribution function f(i, n, v), which formulates the relation of
323
incidence, surface normal and viewer of surface, is showed as follows: f (i, n, v) =
βd βs FDG + T π (i n)(vT n)
(2)
where the first term is Lambertian reflectance component, the second term is specular reflectance component, βd , βs the diffuse reflectance parameter and specular reflectance parameter, respectively, the function D is slope distribution function of micro-facets, D = exp(−k(cos−1 (hT n))2 ), v the viewer direction, h the specular reflectance direction, which is the bisector of i and v, h = (i + v)/||i + v||, and k the surface rough factor. The specular reflectance changes with incident angle and the refractive index of object, which is formulated by Fresnel function. Thus the function shows the influence of wavelength of light source and the material: 1 (g − c)2 (c(g + c) − 1)2 F= 1+ 2 (g + c)2 (c(g − c) + 1)2 where c = (iT h), g2 = η2 + c2 − 1, and η is the refractive index of aluminum alloy. In order to show the light intensity attenuation of specular components among micro-facets in welding pool, the geometrical attenuation factor G is introduced. Because the attenuation of reflection light intensity is caused by the occlusion among micro-facets, the factor G is the minimum of the following formula: 2(hT n)(vT n) 2(hT n)(iT n) G = min 1, , (iT h) (hT v) where the first term means there is no occlusion, the second term means the reflection light is occluded, and the third term means the incident light is occluded. Because the reflectivity of welding pool cannot be measured, it is estimated by compare method and does not change with metal temperature. Because the camera is not on the welding pool, the image welding pool may be occluded by nozzle. According to the characteristics of shape of welding pool and imaging, the welding pool surface maybe come forth interreflection and occluding. The interreflection in aluminum alloy welding pool is caused by liquid melting metal area. According to the reflectance characteristics of welding pool and simplification problem, the interreflection is regarded as diffuse reflectance. The interreflection characteristic is formulated as Einter : (−nTA r1 )(nTB r1 ) Einter = IB dB |r1T r1 |2 where nA is the surface normal at point A(x, y, z), nB the surface normal at point B(x1 , y1 , z1 ), r1 the distance vector from B to A, IB the incident light intensity of point B, and dB the element area of point B.
324
L. Laiping et al. / Materials Science and Engineering A 394 (2005) 320–326
In order to simplify the calculation, the light can receive the target of camera after one interreflection and all points where interreflection is visual: L=
I0 max[0, iT n]f (i, n, v) + Einter r2
(3)
where L is the light intensity that has reached camera target.
patches: Ek = Rk (z, zj , zl ) π d 2 T 4 I0 = b+g (vk nz ) max(iTk n) 4 f rk2 2 βs exp(−k[arccos(hTk n)] ) βd T + Einter (i n) + × π k (vTk n)
3.4. Camera parameters The camera parameters include the position, angle and optical parameters. The selection of position of camera takes into account the environment temperature caused by arc heat. The angle of camera need to think over the nozzle occlusion and the integrality of welding pool image. The optical parameters include electronic shuttle, focus and aperture. The electronic shuttle and aperture is used to adjust the light flux, while the focus is to adjust the depth of field. The optical parameters are used to adjust the light flux to acquire the clear welding pool image. These operations will influence the imaging intensity. In addition, only part of reflection light from welding pool arrive the camera target. The relation of the reflection intensity from welding pool surface and the image gray at a pixel is as follows: E=g
π d 2 T 4 (v nz ) L + b 4 f
The nonlinear reflectance map equation will make the calculation very complicated, linearization is needed to simplify calculation process. The above reflectance map equation can be resolved by functional minimum approaches where the reflectance map equation is formulated as bright energy function: εb =
E=R
π d 2 T 4 (v nz ) =g 4 f I0 βd βs DFG +b × 2 max[0, iT n] + T + E inter π (v n)(iT n) r (5)
where R is the reflectance map function, and E the image gray at a pixel.
4. The surface height calculation of welding pool The calculation of height from reflectance model is another key problem from shape from shading method. The triangular discrete method is used to plot the rectangle image region and the object surface, because any object surface can be approximated by a set of non-lapped triangular
(Ek − Rk )2 =
k=1
1 T z Ab z − b T z + c 2
where Rk ≈
Mn
wkm zm + ξk ,
m=1 n−1 ξk = Rk (zn−1 , zn−1 )− i j , zl
(4)
where nZ is the optical axis direction, d the diameter of lens, f the focus, g the gain of camera, b the camera constants, and vT nZ the cosine of the angle between light to lens and optical axis. In a word, the reflectance map equation of welding pool is
Mt
wkm
Mn
m=1
wkm zn−1 m ,
∂Rk (zi , zj , zl ) n−1 n−1 l−1 (zi , zj , zl ) = ∂zm 0
m = i, j, l, otherwise
where εb is the bright energy function, Mt the number of the triangle in rectangle image area, Ek the image gray at a pixel of a triangle, Rk the reflectance map function of triangular patch, Z the nodal variable, Ab the stiffness matrix, b the load vector, the element of Ab and b is ai,j = 2
Mt
wkm wkn ,
k=1
bi,j = 2
Mt
(Ek − ξk )wkm ,
1 ≤ m, n ≤ Mn
k=1
respectively, Mn is the number of triangle vertex. The stiffness matrix Ab is sparse, symmetrical and semidefinite, so the equation Ab z = b has not a unique solution. The smoothness constraint condition of welding pool is introduced to improve the ill-posed. The cost functional is as follows with the smoothness constraint condition: ε = εb + λεs
(6)
where εb is the bright cost functional, λ the smoothness constraint factor, 0 < λ < 1, εs the smoothness error cost functional, which can be formulated by the controlled-continuity
L. Laiping et al. / Materials Science and Engineering A 394 (2005) 320–326
325
stabilizer function proposed by Teropoulos [14]: 1 ρ(z2x + z2y ) + (1 − ρ)(z2xx + 2z2xy + z2yy ) dx dy εs = 2 Ω
where the first term, named membrane energy function, assure the recovered surface height continuity, which means the surface can have convex like spike, the second term, named thin plate energy function, assures second derivative of the surface continuity, which can assure the surface smooth, but oversmooth, ρ is smoothness factor, 0 < ρ < 1. The combining of membrane energy function and thin plate energy function can describe smoothness characteristics of actual surface. The smoothing constraint function can be formulated as matrix. The whole error cost functional can be formulated as ε = 21 zT Az − bT z + c where A = Ab + λB, B is the smoothness constraint matrix. The surface smoothness factor λ determines the calculation speed and precision. If λ is too large, the iteration number is too large and the calculation time too long. Otherwise the constraint role is decreased and the precision too bad and the algorithm even divergent. So λ changes with iteration process, large value to stabilize iteration process and small to speedup the convergence and decrease to zero to eliminate the effect of regularization terms. Similarly, ε plays an important role in iteration calculation, too large ε short calculation time and bad precision, while too small ε, long calculation time and good precision. Suitable ε can give attention to calculation precision and speed, i.e. large ε to accelerate the calculation process and small ε to assure the calculation precision. 4.1. Calculation result The welding parameters are following, material is LD10 whose size is 250 mm × 50 mm × 3 mm, arc length 5 mm, peak current 160 A, base current 30 A, welding speed 2.5 mm/s, tungsten diameter 3.2 mm, imaging 50 ms delay from descend step, imaging position slant rear. The camera parameters are as follows: position (−40, 210, 130), focus length 50 mm, CCD target size 4.8 mm × 4.8 mm, maximum size of image 512 pixel × 512 pixel. The height of object surface is based on camera coordinate in algorithm, while the measurement is based on workpieces coordinate. Therefore the coordinate transform is necessary to acquire the 3D shape information of welding pool. The parameters used in coordinate transform are showed in Fig. 1. The Fresnel function is 1. The surface reflectance parameters of liquid welding pool are βs = 0.8, βd = 0.2, k = 3.0, while those of the solid weld and base metal are βd = 1.0, βs = 0, the smoothness factor is ρ = 0.8. The initial λ is 0.5. The initial ε is 0.01 and end is 1 × e−6 . The image size is 64 pixel × 64 pixel, the length of triangle 4, the calculation time 78 ms. The initial height is Z(0) = 0.
Fig. 3. Calculation result: (a) height of welding pool; (b) height of center cross-section along x axis; (c) height of center cross-section along y axis.
The experiential knowledge is welding pool is symmetrical according to weld center. The welding pool is an ellipse semi-sphere on the workpieces. Fig. 3 shows the result, (a) the whole surface height calculated from Fig. 2 by SFS method, (b) the cross-section height along x direction in the center of welding pool, (c) the cross-section height along y direction in the welding pool center. From Fig. 3, the result shows the shape trend of aluminum alloy welding pool. Because the surface height of welding pool is difficult to measure, academic analysis is carried through in this paper. The reason of calculation error is as follows: (1) the arc light intensity is difficult to precise calculate; (2) the influence of filter system is difficult to confirm; (3) the reflectivity of welding pool is difficult to measure; (4) the color variable of aluminum alloy is not obvious after melting; (5) the calculation theory and calculation method of shape from shading are developing; (6) the arc disturbance and nozzle occlusion affect the calculation result.
5. Conclusion During welding, surface height of welding pool plays an important role in precise control of welding process. Based on imaging characteristics of welding pool, the surface reflectance model of welding pool was built after the analysis of arc intensity, filter system, shape and reflectance
326
L. Laiping et al. / Materials Science and Engineering A 394 (2005) 320–326
characteristics of welding pool and camera parameters. The surface height of welding pool was calculated with smoothness constraint condition of welding pool. The calculation result was analyzed in this paper.
Acknowledgements This work was supported by Sci. and Tech. Committee of Shanghai, China, no. 021111116 and Doctoral program foundation of Education Ministry of China.
References [1] R. Kovacevic, Y.M. Zhang, Three-dimension measurement of weld pool surface, in: Proceedings of the International Conference on Modeling and Control of Joining Processes, Oriado, FL, USA, 1993, pp. 600–607. [2] D.B. Zhao, J.Q. Yi, S.B. Chen, L. Wu, Lecture Notes in Control and Information Sciences, 2004.
[3] R. Zhang, P.-S. Tsai, J.E. Cryer, et al., IEEE Trans. Pattern Anal. Mach. Intell. 21 (8) (1999) 690–706. [4] M. Oren, S.K. Nayar, Int. J. Comput. Vision 14 (3) (1995) 227– 251. [5] B.T. Phong, Commun. ACM 18 (1975) 311–317. [6] L.B. Wolff, J. Opt. Soc. Am. A11 (1994) 2956–2968. [7] G. Healey, T.O. Binford, Local Shape from Specularity, Comput. Vision Graphics Image Process. 42 (1988) 62–86. [8] S.K. Nayar, K. Ikeuchi, T. Kanade, IEEE Trans. Pattern Anal. Mach. Intell. 13 (7) (1991) 611–634. [9] S.K. Nayar, K. Ikeuchi, T. Kanade, J. Comput. Vision 6 (3) (1991). [10] B.K.P. Horn, M.J. Brooks, Comput. Vision Graphics Image Process. 33 (1986) 174–208. [11] K.M. Lee, C.J. Kuo, Shape from shading with a generalized reflectance map model [J], Comput. Vision Image Process. 67 (2) (1997) 143–160. [12] K.M. Lee, C.J. Kuo, IEEE Trans. Pattern Anal. Mach. Intell. 15 (1993) 815–822. [13] J.J. Wang, T. Lin, S.B. Chen, Image taking and processing of molten pool characters during aluminum alloy TIG welding, Chin. J. Mech. Eng. 39 (5) (2003) 125–129 (in Chinese). [14] D. Terzopoulos, IEEE Trans. Pattern Anal. Mach. Intell. 10 (4) (1988) 417–438.
The modeling of welding pool surface reflectance of aluminum alloy pulse GTAW Li Laiping∗ , Chen Shanben, Lin Tao School of Materials Science and Engineering, Shanghai Jiaotong University, Shanghai 200030, PR China Received 16 August 2004; accepted 17 November 2004
Abstract The information of welding pool surface height plays an important role in the precise control of weld formation. The height measurement of welding pool surface is very difficult because there exists splash, fume, high temperature, electromagnetic disturbance, and so on during welding and welding pool have the characteristics such as small volume, short existence time in high temperature, high temperature, the flow in welding pool and concurrence of metal melting and freezing, etc. The application of aluminum alloy is wide because of low melting point, large conductivity of electric and heat, small density and so on. Welding is an important manufacture technology in aluminum alloy application where the quality control is too important. The color of aluminum alloy does not obviously change after melting which cause the difficulty to estimate welding estate. Visual sensing is a promising method to acquire the shape information of welding pool. The theory of surface height calculation of pulse GTAW welding pool based on SFS is to calculate the surface height from welding pool image whose key is to build up the surface reflectance model of welding pool. Based on the imaging characteristics of aluminum alloy pulse GTAW welding pool, the surface reflectance model is built after analyzing arc intensity, filter system, welding pool shape and reflectance characteristics. With smooth constraint condition of the welding pool surface and variable factor SOR method, the height of welding pool surface is calculated and error analyzed. © 2004 Published by Elsevier B.V. Keywords: Aluminum alloy pulse GTAW; Surface reflectance model of welding pool; Shape from shading; Surface height of welding pool
1. Introduction Aluminum alloy have the following characteristics, which make it have wide application, such as low melting temperature, large heat and electric conductivity, high specific heat and better anticorrosive, and so on. Welding is an important manufacturing technology in aluminum alloy application. Welding quality control is an important problem for welding automation, mechanization, robot welding and intelligent welding. Because of the characteristics of aluminum alloy and welding technology, the research of welding process control is lacking. According to relative theory, there are certain relations between the shape and size of welding pool surface, reinforce∗
Corresponding author. Tel.: +86 21 62932429; fax: +86 21 62932429. E-mail address: [email protected] (L. Laiping).
0921-5093/$ – see front matter © 2004 Published by Elsevier B.V. doi:10.1016/j.msea.2004.11.063
ment and weld penetration, that is to say the topside shape of welding pool can show the appearance of backside weld. Therefore the height measurement of welding pool topside surface is significant in precise control of appearance of weld, which is an index of welding quality. In real operation, the skilled welder observes the shape, size and dynamic change of welding pool, the shape and intensity of arc, the metal transfer model, etc. to feel the appearance of backside weld and adjust the welding parameters, the height and slant of welding torch so as to assure the formation quality of weld. There are fume, splash, high temperature and electromagnetic disturbance, and so on in welding process. The welding pool has the following characteristics, such as small volume, short stay in high temperature, high temperature, fast freezing, concurrence of melting and freezing process and dynamic flow. The above characteristics of welding process and welding pool make the measurement of welding
L. Laiping et al. / Materials Science and Engineering A 394 (2005) 320–326
pool surface height very difficult. The height of welding pool surface cannot be measured by mechanical method. The visual sensor simulates welder’s eye to check the status of welding process and acquire the characteristic information showing the welding process, which is used in weld size acquisition, seam tracking and arc spectrum analysis. R. Kovacevic calculated the surface height of welding pool by structure light. The calculated height is based on freezing weld, which make the control imperfect because the controlled parameters are under arc and the height information has lag [1]. The passive visual sensor is a promising sensor whose light source is arc. It is important in precise control of welding pool to acquire welding pool image and calculate the surface height from image. The characteristic of aluminum alloy welding is that there is not obviously color change after metal melting. The welding pool is flowing under electromagnetic force, surface tension, airflow forces and so on, which make the shape and size of welding pool change with time. Though there exists flow in welding process, the shape information of welding pool cannot be acquired from motion, optical flow and image serials. In the passive visual sensing technology, the arc cannot move to different places to acquire three images of the same welding pool, so surface height of welding pool cannot be calculated by photometric stereo method. In the welding process, the shape of welding pool cannot be acquired by stereo vision which use many cameras to calculate the object height because of the dynamic characteristic of welding pool, the cost of welding equipment, the volume and flexible of welding torch and the difficulty of stereo match. There may exist contour and texture in weld, but the information is not abundant to determine the shape of welding pool. The information lags because it is showed in freezing weld. Shape from shading is a method to calculate the object height from single image and other constraint conditions. It is a promising method to calculate the surface height of welding pool because of simple structure and low cost [2]. Shape from shading is a key problem in computer vision domain, which is based on the fact that the change of image gray shows the change of object shape [3]. The idea is to calculate the surface gradient and height from image gray. The key is to build the reflectance model of object surface, which describes the relation between the image gray and light source, shape and reflectance characteristics and camera. According to the imaging theory, the gray level at a pixel in image of 3D object depends on the shape, position, intensity of light source, the shape and reflectance characteristic of object and the characteristic of camera. Oren and Nayar analyzed the generalization form of Lambertian model and the application in the machine vision [4]. The specularity model developed by Phong represented the specular component of reflection as powers of the cosine of the angle between the perfect specular direction and the viewing direction [5]. The Torrance–Sparrow model assumed that a surface is composed of small, randomly oriented, mirror-like facets. It described
321
the specular brightness as the product of the following: energy of incident light, Fresnel coefficient, facet orientation distribution function and geometrical attenuation factor [6]. Healey and Binford derived a simplified model by using the Gaussian distribution as the facet orientation function and considering the other components as constants [7]. Nayar et al. proposed a reflectance model, which consists of three components: diffuse lobe, specular lobe and specular spike [8]. Nayar et al. analyzed the interreflection, which appear in concave surfaces or concavities result from multiple objects, and built the interreflection model [9]. Horn and Brooks proposed the shape from shading and analyzed the reflectance model under ideal imaging condition [10]. Lee and Kuo analyzed the effect of light source, reflectance model and projection on reflectance map model and built the reflectance model of real object under near point light source and calculated the surface height by photometric stereo method [11,12]. Because of the shortcoming of photometric stereo method, shape from a single image has more significant. Zhao et al. first introduced shape from shading into the height calculation of welding pool surface, but the complex of welding process and the characteristics of welding pool make application very difficult [2]. The imaging characteristics of welding aluminum alloy are analyzed. Based on the arc intensity, filter system, shape and reflectance characteristics of welding pool, and camera parameters, the reflectance model of welding pool is built. With the surface smooth constraint condition of welding pool and variable factor SOR method, the surface height of welding pool is calculated and the errors are analyzed in this paper.
2. Image analysis of aluminum alloy welding pool J.J. Wang analyzed the imaging characteristics of aluminum alloy pulse GTAW welding pool and acquired the welding pool image from side by wide filter method. Because of the limit of imaging position, nozzle occludes the viewer and the image is only half welding pool. The information of whole welding pool was acquired by symmetry method [13]. However, the welding pool is not symmetrical in welding pool when the assembly error or material characteristic causes the seam departure from the center. Thus the symmetrical method will come into being error. The welding pool image acquired from slant rear not only solve the nozzle occluding problem but also take into account the welding pool edge blur caused by the reflex intensity shortage because of large distance from rear. Fig. 1 shows the principle graphic of topside shape information acquisition of welding pool, where coordinate OXYZ is on the workpieces, origin under tungsten electrode, Y axis welding direction, X axis perpendicular to welding direction, Z axis point to tungsten electrode. Fig. 2 shows the welding pool image acquired from slant rear based on wide band filter proposed by Wang et al. [13]. The whole welding pool image region is composed of nozzle area, arc area, molten pool, solid metal and base metal. In the
322
L. Laiping et al. / Materials Science and Engineering A 394 (2005) 320–326
Fig. 1. Principle graphic of topside shape information acquisition of welding pool.
Fig. 2. Welding pool image of aluminum alloy pulse GTAW.
nozzle area, the reflex intensity is the lowest because of the nozzle occluding and the gray level at a pixel is the lowest. The gray level at a pixel of liquid metal area is lower, because the color of aluminum alloy does not change after melting. In the solid weld, its surface is relative rough and the reflectivity same in all kinds and the gray is low. Though the reflection of aluminum alloy is strong, the oxide make aluminum alloy lose metal polish, the gray level at a pixel is low too, while it is high near arc. The arc light intensity is high so the gray level is high. The center of welding pool is the inverted reflection of tungsten electrode and the gray level is very high. The contrast of every area is obvious in welding pool image.
3. Modeling of aluminum alloy welding pool surface reflectance To build precise reflectance model of object surface is the key of calculating object height from image gray. According to object imaging theory, the gray level at a pixel depends on the property of light source, the shape and reflection property of object surface, and camera parameters. The influence of welding arc, filter system, the reflection property and shape of welding pool, and camera on image gray in passive visual sensing is analyzed and the surface reflection model of welding pool is built. 3.1. Light source characteristics The imaging is based on reflecting arc light by the welding pool, weld and workpieces in passive visual sensing where
the arc light is imaging light source. The shape, position and angle of electrode are fixed, so the property of light source lies on the shape and intensity of arc. The shape of arc play an important role in welding pool imaging because the arc is a solid light source each part of which radiates. When the current is large, the arc is like a bell. The intensity of every part of arc is different, so the arc light source is anisotropic. The imaging quality is too bad caused by the shape of light source. When the current decreases, the volume of arc decrease till shrink to the butt of tungsten electrode. The arc light can be regarded as radiating from the butt of tungsten electrode and has the isotropic characteristic. Then the influence of arc shape can be ignored. According to the GTAW technology characteristics the distance between tungsten electrode and workpieces is only several millimeters in order to assure stable striking arc and pilot arc. Therefore, the imaging of welding pool is near point light source imaging. The light intensity reaching on welding pool surface is: E=
I0 max[0, iT n] r2
(1)
where E is the intensity reaching on the welding pool surface, I0 the arc radiation intensity, i the incident light direction, n the surface normal direction on point p in welding pool surface, r the distance between the butt of tungsten electrode and point p, iT n the cosine of angle between the surface normal and incident light. 3.2. complicated filter system The strong arc not only makes the image quality bad but also damages the CCD camera. The protection measure is imaging when low current and complicated filter system. The complicated filter system is composed of colorless glass, wide band filter and neutral density filter. The arc light reflected by welding pool, workpieces and weld receives camera target through glass, wide band filter and neutral density filter. The colorless glass prevents optical apparatus and lens from the splash and fume. The neutral density filter equally absorbs arc light of all wavelengths in order to decrease the arc intensity. In aluminum alloy pulse GTAW, arc spectrum is composed of continuous spectrum and line spectrum. The line
L. Laiping et al. / Materials Science and Engineering A 394 (2005) 320–326
spectrum of aluminum is 396 nm. From 380 to 760 nm, the distribution density of aluminum is less than that of argon. During 580 and 760 nm, the spectrum of aluminum is continuous spectrum while the other elements is very weak. Thus the wide filter system is based on continuous spectrum from 590 to 710 nm as light source and attenuating other spectrum, which is regarded as noise. The above filter system decrease the arc intensity that receive the lens. 3.3. The shape and reflectance characteristics of welding pool 3.3.1. Welding pool surface shape During the welding, the arc force makes the welding pool surface sink. The shape of welding pool is like an irregular elliptic hemispheroid. The change of shape of welding pool appears in the tail. When the welding pool transform from partial penetration to full penetration, depression of the foreside welding pool increase while the shape of rearward welding pool change from convex to concave. Under the electromagnetic force, the welding pool under the arc is concave. Either current is large or welding speed or wire feed rate is few, the rearward welding pool is concave, too. Contrarily, when welding current is few or welding speed or wire feed rate is large, the rearward welding pool is convex because of metal pileup. The existence time of welding pool is short in high temperature during welding. The melting of the filler metal or base metal and the freezing of liquid metal is concurrent. There is motion in welding pool. Thus the metal melts in the foreside and moves into the rearward and freezes in the rearward. The process of welding pool melting and freezing makes the edge of welding pool lack of smooth, that is to say the surface height of welding pool in edge change obviously. 3.3.2. Surface reflectance characteristics of welding pool According to the welding theory, the welding pool area is composed of liquid melting metal, solid freezing metal and base metal. Like all actual object surface, the welding pool surface reflection is hybrid, i.e. there exist both Lambertian reflection and specular reflection. Though the reflectance of aluminum alloy is strong, the oxide make the surface of aluminum alloy lose the glare of metal. The reflection intensity in all kinds is same and the workpieces surface can be regarded as ideal diffuse surface. The fact that the inverted image of tungsten electrode can be showed in the welding pool means that the welding pool has specular characteristics. However, aluminum alloy color does not obviously change after melting. Therefore the liquid welding pool can be regarded as part diffuse surface with weak specular components. The freezing weld similarly reflects the arc light in all directions and the surface is diffuse. In a word, the welding pool surface is hybrid reflectance surface. The bi-directional reflectance distribution function f(i, n, v), which formulates the relation of
323
incidence, surface normal and viewer of surface, is showed as follows: f (i, n, v) =
βd βs FDG + T π (i n)(vT n)
(2)
where the first term is Lambertian reflectance component, the second term is specular reflectance component, βd , βs the diffuse reflectance parameter and specular reflectance parameter, respectively, the function D is slope distribution function of micro-facets, D = exp(−k(cos−1 (hT n))2 ), v the viewer direction, h the specular reflectance direction, which is the bisector of i and v, h = (i + v)/||i + v||, and k the surface rough factor. The specular reflectance changes with incident angle and the refractive index of object, which is formulated by Fresnel function. Thus the function shows the influence of wavelength of light source and the material: 1 (g − c)2 (c(g + c) − 1)2 F= 1+ 2 (g + c)2 (c(g − c) + 1)2 where c = (iT h), g2 = η2 + c2 − 1, and η is the refractive index of aluminum alloy. In order to show the light intensity attenuation of specular components among micro-facets in welding pool, the geometrical attenuation factor G is introduced. Because the attenuation of reflection light intensity is caused by the occlusion among micro-facets, the factor G is the minimum of the following formula: 2(hT n)(vT n) 2(hT n)(iT n) G = min 1, , (iT h) (hT v) where the first term means there is no occlusion, the second term means the reflection light is occluded, and the third term means the incident light is occluded. Because the reflectivity of welding pool cannot be measured, it is estimated by compare method and does not change with metal temperature. Because the camera is not on the welding pool, the image welding pool may be occluded by nozzle. According to the characteristics of shape of welding pool and imaging, the welding pool surface maybe come forth interreflection and occluding. The interreflection in aluminum alloy welding pool is caused by liquid melting metal area. According to the reflectance characteristics of welding pool and simplification problem, the interreflection is regarded as diffuse reflectance. The interreflection characteristic is formulated as Einter : (−nTA r1 )(nTB r1 ) Einter = IB dB |r1T r1 |2 where nA is the surface normal at point A(x, y, z), nB the surface normal at point B(x1 , y1 , z1 ), r1 the distance vector from B to A, IB the incident light intensity of point B, and dB the element area of point B.
324
L. Laiping et al. / Materials Science and Engineering A 394 (2005) 320–326
In order to simplify the calculation, the light can receive the target of camera after one interreflection and all points where interreflection is visual: L=
I0 max[0, iT n]f (i, n, v) + Einter r2
(3)
where L is the light intensity that has reached camera target.
patches: Ek = Rk (z, zj , zl ) π d 2 T 4 I0 = b+g (vk nz ) max(iTk n) 4 f rk2 2 βs exp(−k[arccos(hTk n)] ) βd T + Einter (i n) + × π k (vTk n)
3.4. Camera parameters The camera parameters include the position, angle and optical parameters. The selection of position of camera takes into account the environment temperature caused by arc heat. The angle of camera need to think over the nozzle occlusion and the integrality of welding pool image. The optical parameters include electronic shuttle, focus and aperture. The electronic shuttle and aperture is used to adjust the light flux, while the focus is to adjust the depth of field. The optical parameters are used to adjust the light flux to acquire the clear welding pool image. These operations will influence the imaging intensity. In addition, only part of reflection light from welding pool arrive the camera target. The relation of the reflection intensity from welding pool surface and the image gray at a pixel is as follows: E=g
π d 2 T 4 (v nz ) L + b 4 f
The nonlinear reflectance map equation will make the calculation very complicated, linearization is needed to simplify calculation process. The above reflectance map equation can be resolved by functional minimum approaches where the reflectance map equation is formulated as bright energy function: εb =
E=R
π d 2 T 4 (v nz ) =g 4 f I0 βd βs DFG +b × 2 max[0, iT n] + T + E inter π (v n)(iT n) r (5)
where R is the reflectance map function, and E the image gray at a pixel.
4. The surface height calculation of welding pool The calculation of height from reflectance model is another key problem from shape from shading method. The triangular discrete method is used to plot the rectangle image region and the object surface, because any object surface can be approximated by a set of non-lapped triangular
(Ek − Rk )2 =
k=1
1 T z Ab z − b T z + c 2
where Rk ≈
Mn
wkm zm + ξk ,
m=1 n−1 ξk = Rk (zn−1 , zn−1 )− i j , zl
(4)
where nZ is the optical axis direction, d the diameter of lens, f the focus, g the gain of camera, b the camera constants, and vT nZ the cosine of the angle between light to lens and optical axis. In a word, the reflectance map equation of welding pool is
Mt
wkm
Mn
m=1
wkm zn−1 m ,
∂Rk (zi , zj , zl ) n−1 n−1 l−1 (zi , zj , zl ) = ∂zm 0
m = i, j, l, otherwise
where εb is the bright energy function, Mt the number of the triangle in rectangle image area, Ek the image gray at a pixel of a triangle, Rk the reflectance map function of triangular patch, Z the nodal variable, Ab the stiffness matrix, b the load vector, the element of Ab and b is ai,j = 2
Mt
wkm wkn ,
k=1
bi,j = 2
Mt
(Ek − ξk )wkm ,
1 ≤ m, n ≤ Mn
k=1
respectively, Mn is the number of triangle vertex. The stiffness matrix Ab is sparse, symmetrical and semidefinite, so the equation Ab z = b has not a unique solution. The smoothness constraint condition of welding pool is introduced to improve the ill-posed. The cost functional is as follows with the smoothness constraint condition: ε = εb + λεs
(6)
where εb is the bright cost functional, λ the smoothness constraint factor, 0 < λ < 1, εs the smoothness error cost functional, which can be formulated by the controlled-continuity
L. Laiping et al. / Materials Science and Engineering A 394 (2005) 320–326
325
stabilizer function proposed by Teropoulos [14]: 1 ρ(z2x + z2y ) + (1 − ρ)(z2xx + 2z2xy + z2yy ) dx dy εs = 2 Ω
where the first term, named membrane energy function, assure the recovered surface height continuity, which means the surface can have convex like spike, the second term, named thin plate energy function, assures second derivative of the surface continuity, which can assure the surface smooth, but oversmooth, ρ is smoothness factor, 0 < ρ < 1. The combining of membrane energy function and thin plate energy function can describe smoothness characteristics of actual surface. The smoothing constraint function can be formulated as matrix. The whole error cost functional can be formulated as ε = 21 zT Az − bT z + c where A = Ab + λB, B is the smoothness constraint matrix. The surface smoothness factor λ determines the calculation speed and precision. If λ is too large, the iteration number is too large and the calculation time too long. Otherwise the constraint role is decreased and the precision too bad and the algorithm even divergent. So λ changes with iteration process, large value to stabilize iteration process and small to speedup the convergence and decrease to zero to eliminate the effect of regularization terms. Similarly, ε plays an important role in iteration calculation, too large ε short calculation time and bad precision, while too small ε, long calculation time and good precision. Suitable ε can give attention to calculation precision and speed, i.e. large ε to accelerate the calculation process and small ε to assure the calculation precision. 4.1. Calculation result The welding parameters are following, material is LD10 whose size is 250 mm × 50 mm × 3 mm, arc length 5 mm, peak current 160 A, base current 30 A, welding speed 2.5 mm/s, tungsten diameter 3.2 mm, imaging 50 ms delay from descend step, imaging position slant rear. The camera parameters are as follows: position (−40, 210, 130), focus length 50 mm, CCD target size 4.8 mm × 4.8 mm, maximum size of image 512 pixel × 512 pixel. The height of object surface is based on camera coordinate in algorithm, while the measurement is based on workpieces coordinate. Therefore the coordinate transform is necessary to acquire the 3D shape information of welding pool. The parameters used in coordinate transform are showed in Fig. 1. The Fresnel function is 1. The surface reflectance parameters of liquid welding pool are βs = 0.8, βd = 0.2, k = 3.0, while those of the solid weld and base metal are βd = 1.0, βs = 0, the smoothness factor is ρ = 0.8. The initial λ is 0.5. The initial ε is 0.01 and end is 1 × e−6 . The image size is 64 pixel × 64 pixel, the length of triangle 4, the calculation time 78 ms. The initial height is Z(0) = 0.
Fig. 3. Calculation result: (a) height of welding pool; (b) height of center cross-section along x axis; (c) height of center cross-section along y axis.
The experiential knowledge is welding pool is symmetrical according to weld center. The welding pool is an ellipse semi-sphere on the workpieces. Fig. 3 shows the result, (a) the whole surface height calculated from Fig. 2 by SFS method, (b) the cross-section height along x direction in the center of welding pool, (c) the cross-section height along y direction in the welding pool center. From Fig. 3, the result shows the shape trend of aluminum alloy welding pool. Because the surface height of welding pool is difficult to measure, academic analysis is carried through in this paper. The reason of calculation error is as follows: (1) the arc light intensity is difficult to precise calculate; (2) the influence of filter system is difficult to confirm; (3) the reflectivity of welding pool is difficult to measure; (4) the color variable of aluminum alloy is not obvious after melting; (5) the calculation theory and calculation method of shape from shading are developing; (6) the arc disturbance and nozzle occlusion affect the calculation result.
5. Conclusion During welding, surface height of welding pool plays an important role in precise control of welding process. Based on imaging characteristics of welding pool, the surface reflectance model of welding pool was built after the analysis of arc intensity, filter system, shape and reflectance
326
L. Laiping et al. / Materials Science and Engineering A 394 (2005) 320–326
characteristics of welding pool and camera parameters. The surface height of welding pool was calculated with smoothness constraint condition of welding pool. The calculation result was analyzed in this paper.
Acknowledgements This work was supported by Sci. and Tech. Committee of Shanghai, China, no. 021111116 and Doctoral program foundation of Education Ministry of China.
References [1] R. Kovacevic, Y.M. Zhang, Three-dimension measurement of weld pool surface, in: Proceedings of the International Conference on Modeling and Control of Joining Processes, Oriado, FL, USA, 1993, pp. 600–607. [2] D.B. Zhao, J.Q. Yi, S.B. Chen, L. Wu, Lecture Notes in Control and Information Sciences, 2004.
[3] R. Zhang, P.-S. Tsai, J.E. Cryer, et al., IEEE Trans. Pattern Anal. Mach. Intell. 21 (8) (1999) 690–706. [4] M. Oren, S.K. Nayar, Int. J. Comput. Vision 14 (3) (1995) 227– 251. [5] B.T. Phong, Commun. ACM 18 (1975) 311–317. [6] L.B. Wolff, J. Opt. Soc. Am. A11 (1994) 2956–2968. [7] G. Healey, T.O. Binford, Local Shape from Specularity, Comput. Vision Graphics Image Process. 42 (1988) 62–86. [8] S.K. Nayar, K. Ikeuchi, T. Kanade, IEEE Trans. Pattern Anal. Mach. Intell. 13 (7) (1991) 611–634. [9] S.K. Nayar, K. Ikeuchi, T. Kanade, J. Comput. Vision 6 (3) (1991). [10] B.K.P. Horn, M.J. Brooks, Comput. Vision Graphics Image Process. 33 (1986) 174–208. [11] K.M. Lee, C.J. Kuo, Shape from shading with a generalized reflectance map model [J], Comput. Vision Image Process. 67 (2) (1997) 143–160. [12] K.M. Lee, C.J. Kuo, IEEE Trans. Pattern Anal. Mach. Intell. 15 (1993) 815–822. [13] J.J. Wang, T. Lin, S.B. Chen, Image taking and processing of molten pool characters during aluminum alloy TIG welding, Chin. J. Mech. Eng. 39 (5) (2003) 125–129 (in Chinese). [14] D. Terzopoulos, IEEE Trans. Pattern Anal. Mach. Intell. 10 (4) (1988) 417–438.