- Email: [email protected]
Speckle pattern correlation by digital image processing
Speckle pattern correlation by digital image processing D i p l - p h y s R. H 6 f l i n g *
a n d D r rer n a t W . O s t e n t
*Academy of Sciences of the GDR, The Institute of Mechanics, POB 408, KarI-Marx-Stadt, 9010, GDR tCentral Institute of Cybernetics and Information Processes, Kurstrafle 33, Berlin, 1086, GDR Digital Speckle Pattern Interferometry (DSPI) has been implemented using a commercial image processing system. The pipeline image processor produces real-time correlation fringes and allows rapid automatic evaluation of correlograms. On-line operation gives quantitative results after 2-3 min. Keywords: Speckle pattern interferometry, digital imageprocessing,automatic evaluation, fringe skeleton 1 Introduction
The displacement fields between two states of a diffusely scattering object may be obtained from the degree of correlation between the speckle fields produced by the scattering surface (Pedersen, 1982; Yamagushi, 1985). The relations between the displacement vector field d and the cross-correlation function are well known and will be summarised in Section 2. The question remains as to how to measure that quantity. Different methods have been developed from several authors and fall into two categories: optical and electronic. In optical processing, the speckle fields of both objects states are recorded on the same photographic plate. Sharp structures of the speckle pattern itself or additional introduced speckle structuring (interference or shift) are exploited to detect regions of decorrelation (where the structures disappear) by an optical Fourier processing. Electronic processing allows bypassing the photochemical process by pattern recording with photoelectric detectors (TV pick-up tube, CCD matrix, photodiode array), thereby simplifying the procedure and making it more acceptable for application. The method is called Electronic Speckle Pattern Interferometry (ESPI). ESPI is a widely used tool in mapping vibration modes and displacement fields for scientific and engineering purposes (Butters, 1976; Lokberg, 1980). Visual fringe interpretation, however, often yields qualitative results only. The reduced volume and cost of memory circuits allow the video recording media used in ESPI to be substituted by more accurate digital image memories. Consequently, a computer may be used to perform both correlation of the stored speckle patterns and displacement extraction from the resulting correlograms (often called 'interferograms'). Some recent papers (Nakadate and Saito, 1985; Creath, 1985a, b; Creath and Slettmoen, 1985; Robinson and Williams, 1986) present this Digital Speckle Pattern lnterferometry (DSPI), and introduce phase-shifting techniques in order to get quantitative results. Digital pattern processing by today's microcomputer systems suffers from high processing time requirements for full TV image 30
operations (eg, 512 x 512 pixel). Off-line links to largescale computers (Nakadate and Saito, 1985) or reduced pixel numbers (Creath, 1985a, b; Creath and Slettmoen, 1985; Robinson and Williams, 1986) are reported to be reducing the time problem. We, instead, use the fast pipeline image processor that achieves image operations at video rates (40 ms) by virtue of the SIMD principle. It provides real-time correlograms as well as fast on-line evaluation.
2 Theory The basic relation in speckle pattern correlation measurements connects the cross-correlation function with the mutual intensity of the light field scattered from the undeformed and deformed object, respectively:
(AI1AIE) p,2
-
(11)(/2)
I(U*Uz)I 2 -
(1,)(I2)
...
(1)
and holds if the speckle pattern obeys Gaussian statistics (Goodman, 1975). I and U denote the intensity and amplitude field of the light and ( . . . ) is an ensemble mean. In the following, we consider SPI only and assume the speckle position to be fixed, ie, no speckle shift. Beyond this, the mean intensities (11) and (12) should be equal; thus we drop the index. The mutual intensity in this case becomes:
( U ' U 2 ) = (l)exp i ~ - (k - h ) . d
... (2)
where h and k are the unit vectors of the illumination and observation direction, respectively, d--(d,, d2, d3)r denotes the displacement vector field and 2 is the wavelength of the used laser light. As it is obvious from Eqn (2), the correlation function of a simple illuminated and observed object is a constant. In common SP! set-ups, two statistically independent speckle patterns U1 and U2 are superposed in order to produce interferometric sensitivity (Jones and Wykes, 1983). We select the case of maximal contrast of correlation fringes. This is achieved, if the mean intensities of the interfering speckle fields are equal: (11) = (12). With the abbreviation S for the sensitivity Measurement Vol 5 No 1, Jan-Mar 1987
HiSfling and Osten and from Eqn (4) we get:
vector we finally get: 27r p12 = ~(1 + c o s ( ~ - S . d ) )
... (4)
((lz-lz)2)=2(l)2sin2(;S'd)
S---h1 - k t-(h 2-k2)
... (5)
In order to give an example, Fig 1 shows the monitor print of two different simulated speckle patterns (H6fling and Osten, in prep) assuming a displacement of S.d ~ (k - ko) + (l - 1o) where (ko, lo) denotes pixel locations in the image centre. Clear correlation fringes are visible if the patterns are subtracted and squared according to Eqn (7).
With the proper choice of illumination and observation geometries, the sensitivity vector S allows the separation of in-plane and out-of-plane displacement components (Jones and Wykes, 1983).
3 Correlogram formation in speckle pattern interferometry
4 Experimental results in DSPI
The way of getting the desired correlation function depends on the type of experiment: dynamic or static load. We focus our interest on the static load case and investigate what happens in detail. The patterns of the initial and deformed state of the object are recorded successively and stored on a video tape or in a digital memory. After that, the squared difference of the patterns 11 and 12 is displayed. If we replace the ensemble mean by a spatial average (i e, by eye), we measure: ((It
--
12)2)
2(I)2(1
=
--
i012)
... (7)
(6)
. . .
A
The scheme of our speckle pattern interferometer is shown in Fig 2. The expanded and collimated light from a 30 mW He-Ne laser is directed to a beam splitter mirror, thus illuminating a 50 mm-spot on the diffusely scattering reference and object surfaces. One of the objects is a central loaded circular disc, sprayed white. The reflected light of both surfaces is collected again by the mirror and imaged on to the pick-up tube of a simple TV camera. A numerical aperture (NA) = 5.6 has been found sufficient for the camera lens, although the resulting speckles (typically less than 15pm in size) are not completely
B
c
Fig 1 Example of correlogram formation with synthetic patterns. (A) Pattern of the underformed state," (B) Pattern of the deformed state; (C) Squared difference (A-B) 2 IHE-NE-LASERi
Jv,o,oN \
_tY
\
II °
A ~.72
K 1600
'
MINICOMPUTER B U S ~ '
].,l_.._....~IFAST .~l
I IMAGE MEMOI~J Measurement Vol 5 No 1, Jan-Mar 1987
IMAGE [PROCESSOR I
Fig 2 Schematic diagram of the experimental set-up : M, mirror;L, beam expanding lenses; O, object; R, reference surface; BS, beam splitter 31
H6fling and Osten al, 1985; Saedler et al, 1986) and we apply it to the DSPI correlograms. It is suggested sometimes that automatic fringe centre determination is hard, if not impossible, in speckle pattern interferometry. We found the linethinning algorithm working quite well if careful preprocessing had been applied to the correlogram. Speckles carry the displacement information in DSPI but after the correlogram is obtained they represent pure noise. This is a main source of casual errors in DSPI (Nakadate and Saito, 1985) and some care should be taken to suppress it. We use mean filter algorithms as well as a special geometric filter (HBfling and Osten, in prep). After correlogram smoothing, a shading correction is recommended for equalisation of fringe modulation. This is done by dividing the correlogram by the mean intensity ( I ) which is obtained from the camera again. After the equalisation step, a global threshold may be found for image binarisation. Two examples are presented in Fig 4. Binary fringes are the starting point for line thinning algorithms that produce a fringe skeleton after 3-10s. Such a skeleton may be improved by both automatic filter algorithms (tip removement) or man-machine interactions (gap closing). Doing so, skeletons of proper accuracy have been extracted. For an example, the skeleton of Fig 4(A) is plotted in Fig 5(A). The displacement values have been evaluated giving a standard deviation from a fitted curve of % = 12° corresponding to a casual displacement error of 2ad = 43 nm. The displacement field calculation proceeds with semi-automatic fringe order assignment and interpolation of broken fringe orders on a regular mesh (32 x 32 nodal points). The final plotting of the experimental result (1024 points) appears after altogether 2-3 min and the pseudo-3D plot of the displacement field
resolved by our vidicon. The camera signal is fed to the image processing system and digitized to be stored in one of the four 512 x 512 pixel image memories, each being 8-bit deep. The squared and contrast-enhanced difference between a stored reference frame and the live camera image is presented on the image display at video rates, using the fast pipeline processor K2072 of the image processing system ROBOTRON A6472. If only one TV frame is used for the reference pattern, heavy electronic noise impairs the fringe contrast (Fig 3(A)). According to its origin, electronic noise may be reduced drastically by frame averaging. Mostly we use 12-20 TV frames for at least the reference pattern. The frame averaging takes us about 2s but produces much better correlograms almost free of electronic noise (Fig 3(B)). Up to 50 straight or 18 closed circular fringes have been counted by eye. Sometimes the interferometer head has been replaced by a double-beam illumination (Jones and Wykes, 1983) in order to investigate in-plane deformations. 5 Automatic correlograms
evaluation
of DSPI
From Eqn (7) it is obvious that displacement extraction from a correlogram may take advantage from extreme locations that coincide with: 2 S.d=N~ N=0, 1,2... (8) In order to get automatically all points satisfying Eqn (8), fringe centres (or skeletons) of the correlogram have to be found. Image processing software is available for this purpose in holographic interferometry (Eichhorn et
~~.~r.
".,
.~
A
.~;
:
13,
Fig 3 Influence of frame averaging. ( d ) Correlogram with high electronic noise (single frame); (B) Correlogram from frame averaged speckle patterns
Fig 4 Correlation fringes after preprocessing of correlograms. (A) Outof-plane deformation of a circular disc; (B) In-plane deformation of a tuning fork (static) 32
Measurement Vol 5 No 1, Jan-Mar 1987
HOfling and Osten
"r
A
8
Fig5 Evaluation example for the correlogram in Fig 4A. (A) Fringe skeleton; (B) Plot of the displacement fieM
corresponding to the skeleton in Fig 5(A) is printed in Fig 5(B).
References
Butters, J. N. 1976. 'Electronic speckle pattern interferometry: a general review background to subsequent papers', The engineering use of coherent optics (ed E. R. Robertson), 155-169. Creath, K. 1985a. 'Phase-shifting speckle interferometry', Appl Opt, 24, 3053-3058. Creath, K. 1985b. 'Speckle interferometry techniques for motion sensing', Topical Meeting on Machine Vision, THB4/1-4. Creath, K. and Slettmoen, G. A. 1985. 'Vibrationobservation techniques for digital speckle pattern interferometry', J Opt Soc Am, A2, 1629-1639. Eiehhorn, N. et al. 1985. 'Digital analysis of tyre interferograms', Acta Polytech Scand Appl Phys Ser, No PHI50, 88-91. Goodman, J. W. 1975. 'Statistical properties of laser
Measurement Vol 5 No 1, Jan-Mar 1987
speckle patterns', Laser and related phenomena ( ed J. C. Dainty), 9-77, Springer, Berlin. H6fling, R. and Osten, W. (In prep). 'Displacement measurement by image-processed speckle patterns' (to appear in Optica Acta). Jones, R. and Wykes, C. 1983. Holographic and speckle interferometry, Cambridge University Press, Cambridge Lokberg, O. J. 1980. 'Electronic speckle pattern interferometry', Phys Technol, 11, 16-22. Nakadate, S. and Saito, H. 1985. 'Fringe scanning speckle pattern interferometry', Appl Opt, 24, 2172-2180. Pedersen, H. M. 1982. 'Intensity correlation metrology: a comparative study', Optica Acta, 29, 105-118. Robinson, D. W. and Williams, D. C. 1986. 'Digital phase stepping speckle interferometry, Opt Commun, 57, 26-30 Saedler, J. et at 1986. 'Implentierte Algorithmen auf dem Displayprozessor K2027', Bild und Ton, 39, 140-144, 165-173. Yamagushi, I. 1985. 'Fringe formations in deformation and vibration measurements using laser light', Progress in Optics (ed E. Wolf), XXlI, 271-339.
33
a n d D r rer n a t W . O s t e n t
*Academy of Sciences of the GDR, The Institute of Mechanics, POB 408, KarI-Marx-Stadt, 9010, GDR tCentral Institute of Cybernetics and Information Processes, Kurstrafle 33, Berlin, 1086, GDR Digital Speckle Pattern Interferometry (DSPI) has been implemented using a commercial image processing system. The pipeline image processor produces real-time correlation fringes and allows rapid automatic evaluation of correlograms. On-line operation gives quantitative results after 2-3 min. Keywords: Speckle pattern interferometry, digital imageprocessing,automatic evaluation, fringe skeleton 1 Introduction
The displacement fields between two states of a diffusely scattering object may be obtained from the degree of correlation between the speckle fields produced by the scattering surface (Pedersen, 1982; Yamagushi, 1985). The relations between the displacement vector field d and the cross-correlation function are well known and will be summarised in Section 2. The question remains as to how to measure that quantity. Different methods have been developed from several authors and fall into two categories: optical and electronic. In optical processing, the speckle fields of both objects states are recorded on the same photographic plate. Sharp structures of the speckle pattern itself or additional introduced speckle structuring (interference or shift) are exploited to detect regions of decorrelation (where the structures disappear) by an optical Fourier processing. Electronic processing allows bypassing the photochemical process by pattern recording with photoelectric detectors (TV pick-up tube, CCD matrix, photodiode array), thereby simplifying the procedure and making it more acceptable for application. The method is called Electronic Speckle Pattern Interferometry (ESPI). ESPI is a widely used tool in mapping vibration modes and displacement fields for scientific and engineering purposes (Butters, 1976; Lokberg, 1980). Visual fringe interpretation, however, often yields qualitative results only. The reduced volume and cost of memory circuits allow the video recording media used in ESPI to be substituted by more accurate digital image memories. Consequently, a computer may be used to perform both correlation of the stored speckle patterns and displacement extraction from the resulting correlograms (often called 'interferograms'). Some recent papers (Nakadate and Saito, 1985; Creath, 1985a, b; Creath and Slettmoen, 1985; Robinson and Williams, 1986) present this Digital Speckle Pattern lnterferometry (DSPI), and introduce phase-shifting techniques in order to get quantitative results. Digital pattern processing by today's microcomputer systems suffers from high processing time requirements for full TV image 30
operations (eg, 512 x 512 pixel). Off-line links to largescale computers (Nakadate and Saito, 1985) or reduced pixel numbers (Creath, 1985a, b; Creath and Slettmoen, 1985; Robinson and Williams, 1986) are reported to be reducing the time problem. We, instead, use the fast pipeline image processor that achieves image operations at video rates (40 ms) by virtue of the SIMD principle. It provides real-time correlograms as well as fast on-line evaluation.
2 Theory The basic relation in speckle pattern correlation measurements connects the cross-correlation function with the mutual intensity of the light field scattered from the undeformed and deformed object, respectively:
(AI1AIE) p,2
-
(11)(/2)
I(U*Uz)I 2 -
(1,)(I2)
...
(1)
and holds if the speckle pattern obeys Gaussian statistics (Goodman, 1975). I and U denote the intensity and amplitude field of the light and ( . . . ) is an ensemble mean. In the following, we consider SPI only and assume the speckle position to be fixed, ie, no speckle shift. Beyond this, the mean intensities (11) and (12) should be equal; thus we drop the index. The mutual intensity in this case becomes:
( U ' U 2 ) = (l)exp i ~ - (k - h ) . d
... (2)
where h and k are the unit vectors of the illumination and observation direction, respectively, d--(d,, d2, d3)r denotes the displacement vector field and 2 is the wavelength of the used laser light. As it is obvious from Eqn (2), the correlation function of a simple illuminated and observed object is a constant. In common SP! set-ups, two statistically independent speckle patterns U1 and U2 are superposed in order to produce interferometric sensitivity (Jones and Wykes, 1983). We select the case of maximal contrast of correlation fringes. This is achieved, if the mean intensities of the interfering speckle fields are equal: (11) = (12). With the abbreviation S for the sensitivity Measurement Vol 5 No 1, Jan-Mar 1987
HiSfling and Osten and from Eqn (4) we get:
vector we finally get: 27r p12 = ~(1 + c o s ( ~ - S . d ) )
... (4)
((lz-lz)2)=2(l)2sin2(;S'd)
S---h1 - k t-(h 2-k2)
... (5)
In order to give an example, Fig 1 shows the monitor print of two different simulated speckle patterns (H6fling and Osten, in prep) assuming a displacement of S.d ~ (k - ko) + (l - 1o) where (ko, lo) denotes pixel locations in the image centre. Clear correlation fringes are visible if the patterns are subtracted and squared according to Eqn (7).
With the proper choice of illumination and observation geometries, the sensitivity vector S allows the separation of in-plane and out-of-plane displacement components (Jones and Wykes, 1983).
3 Correlogram formation in speckle pattern interferometry
4 Experimental results in DSPI
The way of getting the desired correlation function depends on the type of experiment: dynamic or static load. We focus our interest on the static load case and investigate what happens in detail. The patterns of the initial and deformed state of the object are recorded successively and stored on a video tape or in a digital memory. After that, the squared difference of the patterns 11 and 12 is displayed. If we replace the ensemble mean by a spatial average (i e, by eye), we measure: ((It
--
12)2)
2(I)2(1
=
--
i012)
... (7)
(6)
. . .
A
The scheme of our speckle pattern interferometer is shown in Fig 2. The expanded and collimated light from a 30 mW He-Ne laser is directed to a beam splitter mirror, thus illuminating a 50 mm-spot on the diffusely scattering reference and object surfaces. One of the objects is a central loaded circular disc, sprayed white. The reflected light of both surfaces is collected again by the mirror and imaged on to the pick-up tube of a simple TV camera. A numerical aperture (NA) = 5.6 has been found sufficient for the camera lens, although the resulting speckles (typically less than 15pm in size) are not completely
B
c
Fig 1 Example of correlogram formation with synthetic patterns. (A) Pattern of the underformed state," (B) Pattern of the deformed state; (C) Squared difference (A-B) 2 IHE-NE-LASERi
Jv,o,oN \
_tY
\
II °
A ~.72
K 1600
'
MINICOMPUTER B U S ~ '
].,l_.._....~IFAST .~l
I IMAGE MEMOI~J Measurement Vol 5 No 1, Jan-Mar 1987
IMAGE [PROCESSOR I
Fig 2 Schematic diagram of the experimental set-up : M, mirror;L, beam expanding lenses; O, object; R, reference surface; BS, beam splitter 31
H6fling and Osten al, 1985; Saedler et al, 1986) and we apply it to the DSPI correlograms. It is suggested sometimes that automatic fringe centre determination is hard, if not impossible, in speckle pattern interferometry. We found the linethinning algorithm working quite well if careful preprocessing had been applied to the correlogram. Speckles carry the displacement information in DSPI but after the correlogram is obtained they represent pure noise. This is a main source of casual errors in DSPI (Nakadate and Saito, 1985) and some care should be taken to suppress it. We use mean filter algorithms as well as a special geometric filter (HBfling and Osten, in prep). After correlogram smoothing, a shading correction is recommended for equalisation of fringe modulation. This is done by dividing the correlogram by the mean intensity ( I ) which is obtained from the camera again. After the equalisation step, a global threshold may be found for image binarisation. Two examples are presented in Fig 4. Binary fringes are the starting point for line thinning algorithms that produce a fringe skeleton after 3-10s. Such a skeleton may be improved by both automatic filter algorithms (tip removement) or man-machine interactions (gap closing). Doing so, skeletons of proper accuracy have been extracted. For an example, the skeleton of Fig 4(A) is plotted in Fig 5(A). The displacement values have been evaluated giving a standard deviation from a fitted curve of % = 12° corresponding to a casual displacement error of 2ad = 43 nm. The displacement field calculation proceeds with semi-automatic fringe order assignment and interpolation of broken fringe orders on a regular mesh (32 x 32 nodal points). The final plotting of the experimental result (1024 points) appears after altogether 2-3 min and the pseudo-3D plot of the displacement field
resolved by our vidicon. The camera signal is fed to the image processing system and digitized to be stored in one of the four 512 x 512 pixel image memories, each being 8-bit deep. The squared and contrast-enhanced difference between a stored reference frame and the live camera image is presented on the image display at video rates, using the fast pipeline processor K2072 of the image processing system ROBOTRON A6472. If only one TV frame is used for the reference pattern, heavy electronic noise impairs the fringe contrast (Fig 3(A)). According to its origin, electronic noise may be reduced drastically by frame averaging. Mostly we use 12-20 TV frames for at least the reference pattern. The frame averaging takes us about 2s but produces much better correlograms almost free of electronic noise (Fig 3(B)). Up to 50 straight or 18 closed circular fringes have been counted by eye. Sometimes the interferometer head has been replaced by a double-beam illumination (Jones and Wykes, 1983) in order to investigate in-plane deformations. 5 Automatic correlograms
evaluation
of DSPI
From Eqn (7) it is obvious that displacement extraction from a correlogram may take advantage from extreme locations that coincide with: 2 S.d=N~ N=0, 1,2... (8) In order to get automatically all points satisfying Eqn (8), fringe centres (or skeletons) of the correlogram have to be found. Image processing software is available for this purpose in holographic interferometry (Eichhorn et
~~.~r.
".,
.~
A
.~;
:
13,
Fig 3 Influence of frame averaging. ( d ) Correlogram with high electronic noise (single frame); (B) Correlogram from frame averaged speckle patterns
Fig 4 Correlation fringes after preprocessing of correlograms. (A) Outof-plane deformation of a circular disc; (B) In-plane deformation of a tuning fork (static) 32
Measurement Vol 5 No 1, Jan-Mar 1987
HOfling and Osten
"r
A
8
Fig5 Evaluation example for the correlogram in Fig 4A. (A) Fringe skeleton; (B) Plot of the displacement fieM
corresponding to the skeleton in Fig 5(A) is printed in Fig 5(B).
References
Butters, J. N. 1976. 'Electronic speckle pattern interferometry: a general review background to subsequent papers', The engineering use of coherent optics (ed E. R. Robertson), 155-169. Creath, K. 1985a. 'Phase-shifting speckle interferometry', Appl Opt, 24, 3053-3058. Creath, K. 1985b. 'Speckle interferometry techniques for motion sensing', Topical Meeting on Machine Vision, THB4/1-4. Creath, K. and Slettmoen, G. A. 1985. 'Vibrationobservation techniques for digital speckle pattern interferometry', J Opt Soc Am, A2, 1629-1639. Eiehhorn, N. et al. 1985. 'Digital analysis of tyre interferograms', Acta Polytech Scand Appl Phys Ser, No PHI50, 88-91. Goodman, J. W. 1975. 'Statistical properties of laser
Measurement Vol 5 No 1, Jan-Mar 1987
speckle patterns', Laser and related phenomena ( ed J. C. Dainty), 9-77, Springer, Berlin. H6fling, R. and Osten, W. (In prep). 'Displacement measurement by image-processed speckle patterns' (to appear in Optica Acta). Jones, R. and Wykes, C. 1983. Holographic and speckle interferometry, Cambridge University Press, Cambridge Lokberg, O. J. 1980. 'Electronic speckle pattern interferometry', Phys Technol, 11, 16-22. Nakadate, S. and Saito, H. 1985. 'Fringe scanning speckle pattern interferometry', Appl Opt, 24, 2172-2180. Pedersen, H. M. 1982. 'Intensity correlation metrology: a comparative study', Optica Acta, 29, 105-118. Robinson, D. W. and Williams, D. C. 1986. 'Digital phase stepping speckle interferometry, Opt Commun, 57, 26-30 Saedler, J. et at 1986. 'Implentierte Algorithmen auf dem Displayprozessor K2027', Bild und Ton, 39, 140-144, 165-173. Yamagushi, I. 1985. 'Fringe formations in deformation and vibration measurements using laser light', Progress in Optics (ed E. Wolf), XXlI, 271-339.
33