Quantitative determination of out-of-plane displacements by endoscopic electronic-speckle-pattern interferometry

1 July 2001

Optics Communications 194 (2001) 75±82

www.elsevier.com/locate/optcom

Quantitative determination of out-of-plane displacements by endoscopic electronic-speckle-pattern interferometry B. Kemper *, D. Dirksen, J. Kandulla, G. von Bally Laboratory of Biophysics, University of Muenster, Robert-Koch-Strasse 45, D-48129 Muenster, Germany Received 6 November 2000; received in revised form 12 February 2001; accepted 26 April 2001

Abstract The combination of endoscopes and electronic-speckle-pattern interferometry requires a more detailed consideration of their imaging properties with respect to interferometric fringe formation, if a quantitative analysis of the observed deformation is desired. Due to a relatively small distance between the illuminating and the imaging optical elements and to strongly divergent beams, the sensitivity vector may not be regarded as constant, and image distortions caused by the endoscope optics must be taken into account. A simpli®ed model that deals with the situation of a plane object perpendicular to the optical axes of the endoscope is presented and veri®ed by experimental results. Ó 2001 Elsevier Science B.V. All rights reserved. Keywords: Electronic-speckle-pattern interferometry; Endoscopy; Sensitivity vector; Image distortions; Imaging; Optical metrology

1. Introduction Endoscopy is a widespread intra-cavity observation technique routinely used in medical minimal-invasive diagnostics and industrial inspection. The combination of endoscopic imaging with holographic interferometric metrology allows the development of methods for non-destructive and non-contactive quantitative diagnostics within body cavities [1]. Earlier endoscopic investigations using holographic interferometry, Electronicspeckle-pattern interferometry (ESPI), and digital holography have been presented for detection of out-of-plane displacements and vibrations [2±4]. *

Corresponding author. Fax: +49-251-83-58536. E-mail addresses: [email protected], [email protected] (B. Kemper).

Usually in these studies, only the interference phase is considered and no quantitative values of the displacement are calculated. The endoscopic illumination and the large imaging angle for endoscopic optics cause serious image distortions and a non-constant sensitivity vector, which makes it more dicult to describe and verify the relation between the sensitivity vector and the deformation vector. In this paper we present a simpli®ed method for the reduction of radial distortions of the endoscopic imaging system, and for the correction of the non-constant sensitivity vector in case of out-of-plane displacements which allows to neglect the complex endoscopic imaging systems. Experimental results of measurements of out-of-plane displacements on plane technical specimen are shown, which demonstrate the improvements in accuracy obtained by this approach.

0030-4018/01/$ - see front matter Ó 2001 Elsevier Science B.V. All rights reserved. PII: S 0 0 3 0 - 4 0 1 8 ( 0 1 ) 0 1 2 7 2 - X

76

B. Kemper et al. / Optics Communications 194 (2001) 75±82

2. Theory 2.1. Calculation of the sensitivity vector In endoscopy it is necessary to arrange the imaging system as well as the light source for illumination (here: coherent laser light), which usually is a spherical wave, close to each other in the tip of the endoscope system. In general, therefore, an optical ®ber for illumination and the ®rst lens of the imaging system are arranged in two parallel tubes positioned in the tip of the endoscope. For the following calculation of the sensitivity vector it is furthermore assumed that the test object is plane and positioned perpendicularly to the optical axis of the imaging system. Fig. 1 illustrates the principle of arrangement for endoscopic illumination and the geometric parameters corresponding to the sensitivity vector. The origin of the coordinates system …x; y; z† is de®ned by the intersection of the object plane and the optical axis of the endoscopic imaging system. The distance between the origin of the point source for illumination and the optical axis of the endoscope imaging system is denoted by Dx. The parameters a and b describe the distance from the principle plane of the ®rst endoscope lens in the tip of the endoscope and the end of the optical ®ber for

object illumination to the object, respectively. The relation between the phase di€erence D/…x; y† at a point P …x; y; 0† on the object surface, the sensitivity vector S ˆ kQ ‡ kB

…1†

…kQ : illumination direction, kB : observation direction) and the vector of deformation of the object d is [5]: D/…x; y† ˆ

2p S
d: k

…2†

Assuming pure out-of-plane displacement d ˆ …0; 0; d† and S ˆ …sx ; sy ; sz † the interference phase in the object plane is: D/…x; y† ˆ

2p dsz ; with sz ˆ sz …x; y; a; b; Dx† k

…3†

with the light wavelength k for illumination. The component sz can be determined by geometric calculation from Fig. 1: sz ˆ kQ;z ‡ kB;z a b ˆ q ‡ p 2 2 x ‡ y 2 ‡ b2 …x Dx† ‡ y 2 ‡ a2

…4†

with sz …x; y; a; b; Dx†max ˆ 2. With Eq. (3) the corresponding deformation is: d…x; y† ˆ D/…x; y† 

0

k

1:

a b B C 2p@ q ‡ p A 2 2 ‡ b2 2 ‡ y x 2 2 …Dx x† ‡ y ‡ a

…5† Eq. (5) allows the determination of the displacement of a measured phase map for known parameters a, b, and Dx as well as the calculation of the phase distribution D/…x; y† for known parameters a, b, Dx, k and d. Fig. 1. Sketch of endoscopic illumination; EN: endoscope imaging system, F: endoscope illumination, O: object, P …x; y; 0†: point on object surface, kQ , kB : direction of illumination and endoscopic observation, S: sensitivity vector, d: vector of outof-plane deformation, Dx: distance between endoscope and illumination ®ber, a: distance from the tip of the endoscope to the object, b: distance from the end of the optical illumination ®ber to the object.

2.2. Compensation of image distortions In general, a precise quantitative geometrical evaluation of images requires the knowledge of all imaging parameters (of the underlying theoretical model), which are usually acquired by a calibra-

B. Kemper et al. / Optics Communications 194 (2001) 75±82

tion procedure [6]. However, compensation of perspective distortions needs a stereoscopic imaging system, which does not seem suitable for endoscopes. Furthermore, this interferometric measurements are mostly carried out with the optical axis adjusted nearly perpendicular to the investigated surface. Nevertheless, due to lens systems with a very short focal length, as it is the case in endoscopic imaging, radial distortions must be taken into account. Thus, to allow a quantitative comparison with Eq. (2), also a correction for spherical aberrations of the endoscopic imaging system had to be carried out. Therefore, the radial distortion dr of the endoscopic imaging system was assumed to be a series of odd powered terms [6]: dr ˆ K1 r3 ‡ K2 r5 ‡ K3 r7 ‡




…6†

In Eq. (6) the parameter r denotes the radius from the center of the image. The parameters Kx take into consideration the deviation from an ideal lens. With known parameters Kx the coordinates r0 of the image corrected with respect to the distortions can be obtained by 0

r ˆ r ‡ dr :

…7†

3. Experimental methods The experiments were carried out using an endoscope ESPI system based on optical ®bers, and a rigid endoscope according to Fig. 1 was used, which is described in detail in Ref. [3]. To detect the change of the speckle phase caused by displacement, a nearly constant spatial phase shift between object and reference wave is introduced in the plane of the CCD sensor. This is carried out by a lateral shift of the optical ®ber of the reference beam to the optical axis of the imaging system [7,8]. For all experiments described here, a spatial phase shift of 90° per pixel and a three-step algorithm for phase calculation (modulo 2p) [7] have been used. The procedure for the detection of displacements includes, in a ®rst step, the recording of the interference patterns of two states of displacements of the object on the CCD chip. In a second step, the corresponding absolute speckle phase is determined for each object state

77

and subtracted (mod 2p) to obtain the phase difference D/…x; y†. Prior to phase unwrapping, the obtained wrapped phase distributions are transformed to sine and cosine images and ®ltered twice by an 9  9 pixel average box ®lter. This is necessary, because of the noise from the large speckle size (£  60 lm), which was caused by the small aperture of the endoscopic imaging system (here £ ˆ 2 mm). To display the results on a computer monitor, the phase distributions were transformed to 256 gray levels (8-bit). In order to achieve a variable distance Dx; the light source for illumination was positioned on a translation stage that allowed de®ned changes along the x-direction. For all measurements a white painted metal plate was positioned perpendicular to the optical axis of the imaging system in front of the endoscope. Shifts and tilts within micrometer range were introduced by a calibrated piezo translator with an integrated deformation monitor based on a strain gauge. 4. Results and discussion Before carrying out the interferometric experiments, the parameters for correction of the distortions of the endoscopic imaging system had to be found. These were obtained by calibration of the optical system on a test target with a millimeter grating. For the compensation of the image distortions in a ®rst step, K1 was determined by comparing the number of image pixels for a known distance from the center to the border of the uncorrected image of the test target, with the pixel number for the same distance calculated from the number of Pixel per millimeter in the image center, where almost no distortion was observed. The correction parameter was determined to be K1  …4:0  0:6†  10 7 Pixel 2 . Fig. 2 shows the white light image of the millimeter grating as recorded by the endoscope imaging system and after the correction described above. Obviously, distortions of the imaging system within a range of 1 mm of the observed area appear, which must be taken into account. After the correction of image distortions, this behavior is strongly reduced. After correction of these image distortions, interferometric experiments for comparison of the

78

B. Kemper et al. / Optics Communications 194 (2001) 75±82

Fig. 2. Correction with respect to radial imaging distortions caused by the endoscope imaging system: (a) white light image of a test target with millimeter grating as recorded with the endoscope imaging system, (b) ®rst order correction of imaging distortions of the image in (a) by Eq. (5). The distortions are clearly reduced.

calculated data by Eq. (5) with experimental data were carried out. For this purpose, the analytical data for D/…x; y† obtained by Eqs. (3) and (4) were transformed into a wrapped phase image (mod 2p) with 256 gray levels (black and white correspond to D/ ˆ 0 and D/ ˆ 2p, respectively). Fig. 3a shows the analytical result for D/…x; y† with parameters a ˆ b ˆ 11 mm, Dx ˆ 3:5 mm and a pure out-of-plane displacement d…x; y† ˆ const ˆ 6 lm. Fig. 3b shows the corresponding experimental result for the phase map measured by spatial phase shifting ESPI as described above with the same

parameters a, b, Dx. The phase map was corrected with respect to the image distortions of the endoscope imaging system by Eq. (7). To introduce a pure out-of-plane displacement the white painted metal plate was shifted d…x; y† ˆ 6 lm in z-direction by a calibrated piezo translator. Quantitative comparison of analytical and experimental data is shown in Fig. 4 for the displacement d…x; y† ˆ 6 lm. The image size is in each case 782  580 pixels …11  8 mm2 †, which is the maximal achievable resolution of the used progressive scan camera (SONY XC-8500 CE). The unwrapped

Fig. 3. Wrapped phase distributions caused by a pure out-of-plane displacement. (a) Analytical results by Eq. (4), (b) experimental result of a piezo shifted metal plate …Dx ˆ 3, 5 mm, a ˆ b ˆ 11 mm, d…x; y† ˆ 6 lm). For further details see text.

B. Kemper et al. / Optics Communications 194 (2001) 75±82

79

phase distribution from the experiment as well as from the corresponding analytical data are plotted for the line in the center of horizontal direction (line 290  y ˆ 0 mm) for these parameters. For that purpose, analytical as well as measured phase data were normalized to the maximum of D/…x; y†. This was necessary, because in this con®guration the phase map of D/…x; y† can be measured only up to integral multiple of 2p. Fig. 5 shows the maximum position of D/…x; y† (see also the centers

of the fringes in Fig. 3) vs. Dx for the distance a ˆ b ˆ 11 mm respectively to the observed object. The dashed lines in Fig. 5 denote the corresponding position of the sensitivity maximum calculated from Eqs. (3) and (4). The maximum of the measured phase change corresponding to the maximum of sz and the position of the maximum agrees with the analytical data within the error range. Small deviations in Fig. 3 between analytical results and experiments can be explained by phase instabilities caused by the optical ®bers as well as by the in¯uence of speckle noise on the process of phase unwrapping. In conclusion, the analytical data are found to correspond with the measured results. In order to validate Eq. (3) for out-of-plane displacements, the painted metal plate was tilted by the calibrated piezo translator, and displacements were calculated from the resulting phase distributions with the parameters a, b and Dx as well as from the monitor control of the piezo translator. For determining absolute fringe orders, an arrangement was chosen in which the ®xed edge of the plate was visible in the image. Fig. 6 shows the result of the wrapped phase obtained from a tilt by the piezo translator of 1:7 lm corresponding to an imaged area of approximately 12  9 mm2 . Without the endoscope imaging system and with a constant sensitivity vector, a parallel

Fig. 5. Position of maximum sensitivity vs. Dx for analytical (sim) and experimental (exp) data for a ˆ b ˆ 11 mm.

Fig. 6. Displacement of a metal plate tilted by a piezo translator: phase distribution with 2p discontinuities (wrapped phase). The marked black line corresponds to the displacement data in Fig. 8.

Fig. 4. Comparison of analytical (sim) and experimental (exp) phase data for pure out-of-plane displacements for d…x; y† ˆ 6 lm.

80

B. Kemper et al. / Optics Communications 194 (2001) 75±82

fringe pattern should appear. The detected phase function obviously shows a di€erent behavior which proves that the factors in Eq. (5) must be taken into account. Fig. 7a shows the corresponding contour plot of the ®ltered and unwrapped phase which were corrected of image distortions by using Eq. (7). Here, a constant sensitivity vector of sz …x; y† ˆ Smax ˆ 2 (maximum sensitivity) in Eq. (5) was assumed. In Fig. 7b the same data were corrected additionally with respect to the nonconstant sensitivity by using Eq. (5). In both plots the distance between two contour lines is 0:25 lm.

Fig. 8. Displacement of a metal plate tilted by a piezo translator: quantitative evaluation of deformation along the line indicated in Fig. 6 of the unwrapped phase map with linear ®t for the determination of the displacement gradient.

In Fig. 8 data corrected for the inhomogeneous sensitivity and the uncorrected data were compared. For this purpose the deformation gradient along the corresponding black line in Fig. 6 was calculated as a linear ®t (dashed lines) to the experimental results (marked with dots at every 20th point) of the corrected and uncorrected data, respectively. The results of these experiments with maximum tilts in the range of 0:36±1:7 lm of the metal plate are shown in Table 1. The ESPI data corrected for non-constant sensitivity and the control data of the calibrated piezo translator overlap within the range of experimental error. The standard deviation of the linear functions

Table 1 Displacement detection of a piezo tilted metal plate: Comparison of the gradient of deformations by calibrated piezo translator and detected by spatial phase shifting endoscopic ESPI

Fig. 7. Displacement of a metal plate tilted by a piezo translator: (a) contour map of the uncorrected unwrapped phase distribution, (b) contour map of the corrected unwrapped phase distribution (distance between two contour lines: 0:25 lm).

Maximum displacement (lm)

Deformation gradient (piezo) (lm=mm)

Deformation gradient (endoscopic ESPI) (lm=mm)

0:36  0:06 0:5  0:1 0:9  0:1 1:0  0:1 1:4  0:1 1:7  0:1

0:032  0:006 0:065  0:008 0:10  0:01 0:13  0:01 0:15  0:01 0:19  0:02

0:030  0:007 0:06  0:01 0:10  0:01 0:12  0:01 0:16  0:01 0:19  0:01

B. Kemper et al. / Optics Communications 194 (2001) 75±82

81

Table 2 Displacement detection of a piezo tilted metal plate: Comparison of the standard deviation of the linear ®t for uncorrected and for sensitivity corrected data Maximum displacement by piezo translator (lm)

Standard deviation corrected data (lm/mm)

Standard deviation uncorrected data (lm/mm)

0:36  0:06 0:5  0:1 0:9  0:1 1:0  0:1 1:4  0:1 1:7  0:1

0.0056 0.0099 0.0108 0.0102 0.0096 0.0083

0.0066 0.0106 0.0117 0.0103 0.0109 0.0127

®tted to the corrected and the uncorrected data are displayed in Table 2. In all cases, the functions ®tted to the corrected data show a lower standard deviation. This indicates, that Eq. (5) minimizes the errors for calculation of the out-of-plane measurements. The quantitative non-linear in¯uence of the correction on the unwrapped phase data is illustrated in Fig. 9 by determination of the difference between the corrected and the uncorrected data in Fig. 8. Finally, the error of the endoscope ESPI system that may result from an additional in-plane displacement is quanti®ed by the relative phase error D/=D/max , where D/ denotes the phase di€erence obtained with the sensitivity sz in Eq. (3) and D/max corresponds to a homogenous sensitivity

Fig. 10. Relative error D/=D/max for several distances to the investigated object with Dx ˆ 4 mm and k ˆ 514:5 nm.

vector with sz ˆ Smax ˆ 2. Therefore D/=D/max is calculated for the distances a ˆ b ˆ 5, 10, 20 and 30 mm for Dx ˆ 4 mm and k ˆ 514:5 nm. The size of the investigated area is chosen in accordance to the data of the experimental setup. The results are shown in Fig. 10 where the relative error on the surface of the investigated object D/=D/max is plotted vs. x …y ˆ 0†. It can be seen that without any correction for the sensitivity vector, for small distances D/=D/max in the con®guration used for the experiments the error is nearly 50% at the image borders while the error is only about 10% in the image center. Here, the in¯uence of in-plane displacements cannot be neglected. For increasing distances D/=D/max decreases as well as the in¯uence of unwanted in-plane displacements. These circumstances have to be considered for the practical application of the endoscope ESPI system, especially when objects of high elasticity are investigated, i.e. biological tissues.

5. Conclusion

Fig. 9. Di€erence Dx between corrected and uncorrected data in Fig. 8.

The experiments demonstrate that the in¯uence of imaging distortions, divergent waves for object illumination, short observation distances, and large angles of object imaging must be taken into account when using endoscope ESPI systems. For correction of these e€ects a simple method of

82

B. Kemper et al. / Optics Communications 194 (2001) 75±82

calibration of endoscope ESPI systems is presented. Thus, this method allows to improve accuracy during endoscopic ESPI measurements of object displacements or deformations with a main component in out-of-plane direction. The procedure has also the advantage that it is not necessary to take into account the complex particular endoscopic imaging system and, thus, it may be applied to various endoscope systems for ESPI and holographic interferometry.

Acknowledgements Financial support by the German Ministry for Education and Research (BMBF) and Karl Storz GmbH, Tuttlingen, Germany, is gratefully acknowledged.

References [1] G. von Bally, H.I. Bjelkhagen, in: G. von Bally, S. Khanna (Eds.), Optics in Medicine, Biology and Environmental Research, Optics Within Life Sciences (OWLS I), Elsevier, Amsterdam, 1993. [2] M. Facchini, P. Zanetta, D. Paolotti, G. Scirripa Spagnolo, Optik 95 (1993) 27±30. [3] B. Kemper, D. Dirksen, W. Avenhaus, A. Merker, G. von Bally, Appl. Opt. 39 (2000) 3899±3905. [4] A. Schedin, G. Pedrini, H.J. Tiziani, F.M. Santoyo, Opt. Commun. 165 (1999) 183±188. [5] Y. Ostrovskij, V.P. Shchepinow, V.V. Ykolev, Holographic Interferometry in Experimental Mechanics, Springer, Berlin, 1991. [6] J.G. Fryer, in: K.B. Atkinson (Ed.), Close Range Photogrammetry and Machine Vision, Whittles Publishing, Caithness, 1996. [7] G. Pedrini, B. Pfeister, H.J. Tiziani, J. Mod. Opt. 40 (1993) 89±96. [8] T. Bothe, J. Burke, H. Helmers, Appl. Opt. 196 (1997) 849± 857.